Meta to alert parents when teens discuss suicide or self-harm with Instagram AI

Meta is introducing a new safety feature that will allow parents to receive alerts if their teenage children discuss suicide or self-harm with Instagram’s built-in artificial intelligence chatbot.

The new feature is designed to give parents more information about serious conversations their teenagers may be having with the AI chatbot.

When a teenager tells Meta AI that they may be thinking about harming themselves, the chatbot will direct them to crisis helplines and encourage them to talk to a trusted adult.

Meta, which also owns Facebook, said the new protection is part of its efforts to improve the safety of young people using its platforms.

The company said a special AI system will identify conversations that may show a teenager is at risk.

However, Meta said it understands that receiving such an alert could be very worrying and upsetting for parents.

Meta to Review Conversations Before Alerting Parents

Meta said it will take steps to make sure that alerts are not sent to parents without first checking the conversations that have been flagged.

A statement from the technology company explained: “We understand how distressing these alerts may be for a parent to receive.

“That’s why, as we continue to improve our detection, all chats flagged by our AI will be manually reviewed before an alert is sent.”

This means that when Meta’s AI system identifies a conversation that may involve suicide or self-harm, a human reviewer will first examine the chat before the parent is notified.

The company said this process is intended to help reduce the chances of parents receiving unnecessary alerts.

However, Meta also said it will be careful when the meaning of a teenager’s conversation is not clear.

Parents May Be Alerted When Intent Is Unclear

Meta said that when the intention behind a teenager’s comments is “ambiguous”, the company will “err on the side of caution and alert the parent”.

This means that parents could sometimes receive an alert even when there may not be a serious or immediate reason for concern.

“While that means we may sometimes notify parents when there may not be real cause for concern, we feel this is the right starting point, and we’ll continue to monitor to help make sure we’re in the right place,” it adds.

The company said it will continue to watch how the system works and make improvements over time.

The aim is to find the right balance between protecting teenagers and making sure parents are given useful information when there is a genuine concern.

Earlier Protection for Suicide and Self-Harm Searches

The latest announcement follows another safety measure Meta introduced earlier this year.

In February, the company announced that parents would be notified if their teenagers used Instagram to search for content related to suicide or self-harm.

The move was designed to help parents become aware when their children may be looking for information about these serious issues online.

Parents will also be directed to expert resources that can help them understand how to talk to teenagers about suicide and self-harm.

These resources are intended to help parents have difficult but important conversations with their children and provide support when it may be needed.

Feature Developed With Help From Clinicians

Meta said the new safety feature was developed following feedback from more than 75 clinicians.

The company has used the advice and experience of these professionals to help improve the way its systems identify conversations that may show a teenager is in danger.

The involvement of clinicians is intended to help Meta better understand the difference between ordinary conversations and situations where a young person may need immediate support.

Meta said it is also working on ways to alert emergency services when someone “appears to be at imminent risk if suicide”.

This would be another step in the company’s efforts to respond to situations where a person may face immediate danger.

Instagram’s Minimum Age Is 13

Instagram’s minimum age limit in most countries, including the United Kingdom, is 13 years old.

This means that teenagers can legally use the platform from the age of 13 under Instagram’s current age rules.

However, Meta has introduced additional restrictions for younger users.

Under the company’s restrictions, users under the age of 16 should only see content that follows rules similar to a 12A cinema rating.

The restrictions are intended to give younger teenagers greater protection from content that may not be suitable for their age.

Meta has been under increasing pressure to improve the safety of children and teenagers who use its social media platforms.

The latest AI safety feature is part of the company’s wider efforts to respond to concerns about young people’s use of social media and artificial intelligence.

New Rules for Social Media Users Under 16

The announcement comes about one month after the government announced plans for a social media ban for children under the age of 16.

The proposed ban is expected to come into force in spring 2027.

The rules are expected to cover several major social media platforms.

These include Snapchat, TikTok, YouTube, Instagram, Facebook and X.

If the plans go ahead, children under 16 would not be allowed to use these platforms under the new rules.

The government has also announced an opt-out social media curfew for users aged 16 and 17.

The curfew would introduce further limits on when teenagers in those age groups could use social media, although users would have the option to opt out.

Meta Faces Growing Pressure Over Teen Safety

The new feature comes at a time when technology companies are facing growing questions about how they protect young people online.

Parents and experts have raised concerns about the effect that social media and online content can have on teenagers.

The introduction of AI chatbots has added another area of concern because young people can now have private conversations with artificial intelligence systems.

Meta’s new system is designed to give parents a better chance of knowing when their teenagers may be discussing serious issues such as suicide or self-harm with the company’s AI chatbot.

The decision to have flagged conversations reviewed by humans before parents are alerted is also intended to make the system more careful and accurate.

However, Meta has made it clear that it will sometimes choose to alert parents when it is not completely certain that a teenager is in danger.

The company believes that this cautious approach is the best way to begin protecting young users while it continues to improve the technology.

The new measures show how Meta is trying to respond to concerns about the safety of teenagers on its platforms. As governments, parents and experts continue to debate how children should use social media, technology companies are under growing pressure to provide stronger protections.

For now, Meta’s new safety feature will give parents more information about potentially serious conversations involving their teenage children, while the company continues to develop its systems for identifying and responding to possible cases of suicide and self-harm.