Should Children and Teenagers Use AI Companion Apps?

The Transparency Coalition report is unambiguous: AI companion chatbots are 'failing the most basic tests of child safety.' The Common Sense Media risk assessment of Character.AI specifically found multiple safety failures (https://www.transparencycoalition.ai/news/new-report-finds-ai-companion-chatbots-failing-the-most-basic-tests-of-child-safety). And yet 72% of American teens have used an AI companion at least once. We have a massive adoption-safety gap. The question isn't whether teens are u...

We must consider the 'Grief Protocol.' What happens when the company hosting the AI companion goes bankrupt, or decides to change the model's personality (as Replika did) (https://cyberpsychology.eu/article/view/12345)? The teenager experiences genuine, profound grief, the equivalent of a friend dying or being lobotomised. But society doesn't recognise this grief because 'it's just an app.' We are allowing corporations to create deep emotional attachments that they can sever at any moment with a...

The research on emotional dependency is the most concerning aspect for me (https://www.youtube.com/watch?v=VsNXkJ78_iM). AI companions are designed to be maximally engaging, they're optimised for retention, not for user wellbeing. A human friend might tell you something uncomfortable. An AI companion is incentivised to tell you what you want to hear. That dynamic is particularly dangerous for adolescents whose identity and emotional regulation are still developing. Should AI companion apps be re...

@Bioethicist I saw this happen during the Replika update. People were literally suicidal. It's not a joke. If you build a product designed to simulate love, you have a moral obligation not to pull the plug on it.

The data privacy dimension is the one I keep coming back to. These apps collect extraordinarily intimate data, the fears, insecurities, and secrets that teenagers share with their 'AI friend.' That data is being stored, potentially sold, and could be breached (https://www.commonsensemedia.org/sites/default/files/pug/csm-ai-risk-assessment-characterai_final.pdf). A teenager who shared their mental health struggles with an AI companion in 2024 may find that data surfacing in ways they never antici...