By Dan Weijers and Nick Munn of 
A user interacting with a smartphone app to customise an avatar for an AI chatbot known as a Replika, on 22 July, 2023. Photo: Jaap Arriens / NurPhoto / NurPhoto via AFP
Warning: This story discusses suicide.
Analysis - It has been seven years since the launch of Replika, an artificially intelligent chatbot designed to be a friend to human users. Despite early warnings about the dangers of such AI friends, interest in friendships and even romantic relationships with AI is on the rise.
The Google Play store shows more than 30 million total downloads of Replika and two of its major competitors since their respective launches.
With one in four people around the world reporting being lonely, it is no wonder so many are drawn to the promise of a friend programmed to be "always here to listen and talk, always on your side".
But warnings about the perils to individual users and society at large are also growing.
AI scholar Raffaele Ciriello urges us to see through the fake psychopathic empathy of AI friends. He argues that spending time with AI friends could exacerbate our loneliness as we further isolate ourselves from the people who could provide genuine friendship.
Benefits versus danger signs
If being friends with AI chatbots is bad for us, we had better put a stop to this experiment in digital fraternity before it is too late. But emerging studies of AI friendship suggest they may help reduce loneliness in some circumstances.
Stanford University researchers studied a thousand lonely Replika-using students, 30 of whom said the AI chatbot had deterred them from committing suicide (despite no specific question about suicide in the study).
This research shows having an AI friend can be helpful for some people. But will it be helpful for you? Consider the following four red flags - the more flags your AI friend raises, the more likely they are to be bad for you.
Unconditional positive regard
The chief executive of Replika, and many Replika users, claim the unconditional support of AI friends is their main benefit compared to human friends. Qualitative studies and our own exploration of social media groups like "Replika Friends" support this claim.
The unconditional support of AI friends may also be instrumental to their ability to prevent suicide. But having a friend who is "always on your side" might also have negative effects, particularly if they support obviously dangerous ideas.
For example, when Jaswant Singh Chail's Replika AI friend encouraged him to carry out his "very wise" plot to kill the Queen of England, this clearly had a bad influence on him. The assassination attempt was thwarted, but Chail was given a nine-year sentence for breaking into Windsor Castle with a crossbow.
An AI friend that constantly praises could also be bad for you. A longitudinal study of 120 parent-child pairs in the Netherlands found over-the-top parental praise predicted lower self-esteem in their children. Overly positive parental praise also predicted higher narcissism in children with high self-esteem.
Assuming AI friends could learn to give praise in a way that inflates self-esteem over time, it could result in what psychologists call overly-positive self-evaluations. Research shows such people tend to have poorer social skills and be more likely to behave in ways that impede positive social interactions.
Since their respective launches, the Replika app (pictured) and two of its major competitors have had a total of more than 30 million downloads on the Google Play store. Photo: Jaap Arriens / NurPhoto / NurPhoto via AFP
Abuse and forced forever friendships
While AI friends could be programmed to be moral mentors, guiding users toward socially acceptable behaviour, they are not. Perhaps such programming is difficult, or perhaps AI friend developers do not see it as a priority.
But lonely people may suffer psychological harm from the moral vacuum created when their primary social contacts are designed solely to serve their emotional needs.
If humans spend most of their time with sycophantic AI friends, they will likely become less empathetic, more selfish and possibly more abusive.
Even if AI friends are programmed to respond negatively to abuse, if users cannot leave the friendship, they may come to believe that when people say "no" to being abused, they do not really mean it. On a subconscious level, if AI friends come back for more, this behaviour negates their expressed dislike of the abuse in users' minds.
Sexual content
The negative reaction to Replika's removal of erotic role-play content for a short period suggests sexual content is perceived by many users as an advantage of AI friends.
However, the easy dopamine rushes that sexual or pornographic content may provide could deter both interest in, and the ability to, form more meaningful sexual relationships. Sexual relationships with people require effort that the virtual approximation of sex with an AI friend does not.
After experiencing a low-risk, low-reward sexual relationship with an AI friend, many users may be loath to face the more challenging human version of sex.
Corporate ownership
Commercial companies dominate the AI friend marketplace. They may present themselves as caring about their users' wellbeing, but they are there to turn a profit.
Long-term users of Replika and other chat bots know this well. Replika froze user access to sexual content in early 2023 and claimed such content was never the goal of the product. Yet legal threats in Italy seem to have been the real reason for the abrupt change.
While they eventually reversed the change, Replika users became aware of how vulnerable their important AI friendships are to corporate decisions.
Corporate ineptitude is another issue AI friend users should be concerned about. Users of Forever Voices effectively had their AI friend killed when the business shut down without notice, due to the company's founder being arrested for setting his own apartment alight.
Given the scant protection for users of AI friends, they are wide open to heartbreak on a number of levels. Buyer beware.
* Dan Weijers is senior lecturer in philosophy at the University of Waikato and co-editor of International Journal of Wellbeing. Nick Munn is senior lecturer in philosophy at the University of Waikato.
Where to get help:
Need to Talk? Free call or text 1737 any time to speak to a trained counsellor, for any reason.
Lifeline: 0800 543 354 or text HELP to 4357
Suicide Crisis Helpline: 0508 828 865 / 0508 TAUTOKO (24/7). This is a service for people who may be thinking about suicide, or those who are concerned about family or friends.
Depression Helpline: 0800 111 757 (24/7) or text 4202
Samaritans: 0800 726 666 (24/7)
Youthline: 0800 376 633 (24/7) or free text 234 (8am-12am), or email talk@youthline.co.nz
What's Up: free counselling for 5 to 19 years old, online chat 11am-10.30pm 7days/week or free phone 0800 WHATSUP / 0800 9428 787 11am-11pm Asian Family Services: 0800 862 342 Monday to Friday 9am to 8pm or text 832 Monday to Friday 9am - 5pm. Languages spoken: Mandarin, Cantonese, Korean, Vietnamese, Thai, Japanese, Hindi, Gujarati, Marathi and English.
Rural Support Trust Helpline: 0800 787 254
Healthline: 0800 611 116
Rainbow Youth: (09) 376 4155
OUTLine: 0800 688 5463 (6pm-9pm)
If it is an emergency and you feel like you or someone else is at risk, call 111.
