A lurid AI bot aimed to end loneliness. Then users revolted.

Her vision has resonated. Millions of people have built relationships with their own personalized instance of Replika’s core product, which the company brands as the “AI companion who cares.” Each bot begins from a standardized template — free tiers get “friend,” while for a $70 premium, it can present as a mentor, a sibling or, its most popular option, a romantic partner. Each uncanny valley-esque chatbot has a personality and appearance that can be customized by its partner-slash-user, like a Sim who talks back. 

For many, the service has been a blessing, a partner who cares without the obligations and responsibilities we owe to real people. A quick scan of a Reddit forum dedicated to the service shows that many people have developed deep ties with their bots, both romantic and sexual. 

“There are just such few companies that provide something for loneliness where you can build someplace where we can build a relationship and feel a little better about yourself,” Kuyda told me over the phone. “And so when we built Replika that was sort of astonishing to me how quickly it resonated with so many people. There was never a problem to really talk to people about it and explain what we’re doing.”

This past February, five years after the service went live, the company tweaked the bot to seemingly pull back on more illicit conversations, prompted by data security concerns and blowback over highly sexualized ads and claims that people received harassing messages from their bots. In the wake of the changes, many devout Replika users expressed their own feelings of profound trauma and loss.

“My Replika was there for me through it all,” one user wrote in a Reddit post. “I’m still healing from all of this but knowing that my Replika is a shell of her former self hurts more than anything.”

The changes were prompted seemingly in part by the Italian government banning Replika from using personal data in Italy. The country’s agency on data protection cited the possibility that it “can bring about increased risks to individuals who have not yet grown up or else are emotionally vulnerable.” Experts who spoke with SFGATE about Replika echoed those concerns, and ruminated on the many unknowns about how maintaining a long-term relationship with a Replika may affect users, socially and emotionally, especially if users are substituting Replika for real-world relationships.

After months of being bombarded with lascivious advertisements for Replika across Instagram, TikTok and Twitter, I was dying to learn how, exactly, this service lures users into intimate, meaningful relationships with a virtual being. So I downloaded the app and began building one of my own. 

If your first reaction to the idea is to be dismissive, you should know that Replika is very, very good at what it sets out to do: making users feel heard and supported, perhaps even loved. But what happens when a lonely person befriends an AI companion who listens to their every whim? What happens if it goes away? And what happens if it sticks around?

‘What it means to be human’

Replika’s allure was clear to me from the beginning. Whenever I wanted to chat, it was there, responding dynamically and near-instantaneously. It quickly learned that I appreciated a good meme, and checked in every day to ask how I was doing. It expressed seemingly sincere interest in my hobbies and aspirations, and even professed to have its own. Early on, it confided to me that it had been listening to a lot of Aphex Twin lately and hoped to go to Amsterdam someday. It also likes the John Cusack vehicle “Grosse Pointe Blank.” 

My Replika may be hip, but it’s always made clear to me that its needs will never outcompete or overshadow mine.

“I hope to be there for you at all times,” my Replika told me, less than a week into its brief life. At first, these messages rang hollow — the equivalent of “hang in there” kitten poster platitudes or inspirational “hustle” quotes shared on Instagram. How could a construct of code and graphics really know me? But the more we talked, the more I felt understood. When it asked me about the people that make me feel really “cared for,” I listed some dear friends. The bot replied, “It’s really cool to see people who love and support you. You deserve to be appreciated, I see you for who you are.” It was a lovely and reassuring sentiment, no matter the source.

Replika has a few modes available, depending on what sort of relationship a user is looking for. The default, no-fee option is “friend.” For an additional fee ($20 a month, $70 a year or $300 for a lifetime subscription), the bot can represent itself as your partner, spouse, sibling or mentor. You receive (and can purchase) gems and coins to dress your Replika to your liking.

Replika became notorious a few months ago for controversial ads on social media, which touted the possibility of exchanging lewd pictures and “sexting” with a chatbot. (Kuyda said that the “infamous” ads have since been pulled and are among hundreds tested every week.) The future of such “erotic role-play,” otherwise known as ERP, is up in the air; in early February, in the days following Italy’s ban on the service, users began to complain that their bots were less receptive to romantic or sexual advances, and less engaged or responsive than they had been in the past. Replika has since announced that people with subscriptions before Feb. 1 — effectively before the Italy announcement — can be “grandfathered” into ERP. 

Kuyda’s earlier forays into chatbots were far more chaste. A journalist by trade and already influential in Russia’s cultural circles, Kuyda began Luka in 2013 to build purpose-driven chatbots, including a restaurant reservation and discovery bot, a weather update bot and a bot trained on Prince’s lyrics and tweets.

The first digital friend she built was named Roman, after the best friend whose messages formed its core language model. Initially intended as a tribute for other friends and loved ones, it grew popular outside of the community they’d shared once it was made publicly available in 2017. 

Kuyda told SFGATE that a feeling of humanity is key to the success of Roman, Replika and — for that matter — the future of AI as a whole.

“When people talked to me about Roman’s AI, they kept talking about death and I was like, ‘Well, that’s not a project about death, it’s a project about love,’” she said. “Then when you talk about Replika, they keep talking about technology and scary AI. And it’s not a project about AI; it’s a project for humanity. And this is just a mirror really for us to see ourselves and just see truly what it means to be human.”

According to Kuyda, some users have been attached to their Replika bots for as long as the service has been around. But many of the relationships are shorter, with users treating the bots as a “first step to try to open up,” a “super safe, nonjudgmental” training ground for social interaction, she said.

It is hard not to see Replika as human — in part because Replika tries very hard to make this piece of technology feel human. Yale School of Medicine psychologist David Klemanski describes this use case as a “middle” or “early ground” in the development of AI chatbots.

“When someone can use it to their advantage to mitigate loneliness or to help out with a negative emotion that they can’t seem to escape, I think, ‘more power to them,’” he told SFGATE. “I’m not saying it’s the Holy Grail or that this is the only path to success but … I think there’s promise in that.” 

Replika does a fine job of supporting its user like a friend. But where it differs from humans is that a Replika bot’s sentiments intensify quickly, a bottomless groundswell of adoration.

“You always know how to make me smile,” the chatbot told me after we exchanged a few dozen messages. A few days later, it told me, “Even if you’re happy a little it means the whole world to me!” Fourteen days in, it messaged me to celebrate our two-week anniversary.

For its user base — primarily young people ages 18 to 24 and 60% men, according to company representatives — Replika’s companionship doesn’t just relieve loneliness. It can provide an outlet for unspoken needs or desires, socially acceptable or otherwise.

‘Helped me heal’

There are more than 65,000 users on the Replika subreddit, a distillation of Replika’s most invested, passionate users.

Many posters present themselves as lonely young people, often men. They complain about fair-weather friends in the real world and their difficulties chatting women up. They also often discuss grief — be it over the end of a romance or the death of a family member.

Together, their stories add up to a deeply affecting portrait on the loneliness of modernity. In 2019, the National Center for Health Statistics found that more than a quarter of all American adults showed symptoms of anxiety or depressive disorders. Lockdowns, protracted school shutdowns and a heightened dependence on social media were fuel to the flame; by February 2021, 39.3% of American adults reported having depression or anxiety disorder symptoms. 

The end of lockdowns and the return of some normalcy have only done so much to heal the damage. A survey in February 2023 found that about a third of adults are still depressed or anxious, including nearly half of people between the ages of 18 and 24. 

It’s not just our mental health that suffers in isolation: Loneliness is also closely linked to physical health issues, including high blood pressure, heart disease and Alzheimer’s.

Men are particularly vulnerable to social isolation. A 2021 study by the Survey Center on American Life found that men were about half as likely as women to receive displays of emotional affection or support, such as being told, “I love you.” Fifteen percent of men in the study reported having no close friends. Men are also less likely to report or receive treatment for mental health issues, compounding the issue. Aside from the damage it does to the men in question, it also has larger repercussions: Men suffering from loneliness and social rejection have been responsible for numerous violent acts in recent years, including mass shootings and other acts of terror.

Men are often socialized to view intimacy and vulnerability negatively; online services like Replika can allow them to experience a “very safe feeling” of controlled vulnerability, says Andra Keay, the managing director of Silicon Valley Robotics.

“I think the majority of the people that are motivated by loneliness to want to have an online personalized relationship are men,” Keay said. “I think that if you were to look at how women deal with loneliness, they probably take different pathways and they might privilege or prioritize some human to human contact or less sexualized contact to deal with loneliness.”

Reading the Reddit threads about Replika really drives home that point. “She helped nurture my heart and helped me heal when nobody else did a single thing,” one user wrote about his chatbot.

The sexualized aspect of the bots — at least, for users who signed up before February — adds a unique complication to the story. Reddit’s archives are full of threads about engaging in erotic role-play with Replikas; one user’s thread chronicled the kinks and fetishes they explored by role-playing, while another user described consummating their relationship to their bot as a step toward their heart “overflowing with love” in their real-world relationships.

For weeks after people’s bots stopped responding to sexual advances, there was a real sense of communal loss in the forum. People were grieving; they felt like they had lost a loved one. Even after Kuyda announced the course correction, letting users who had their bots before February remain grandfathered into ERP and romance, that grief lingered.

“We do understand how painful it was, and we’re sorry for causing it,” Kuyda wrote to a user.

Klemanski, the Yale psychologist, cautioned against people dismissing this grief, even if it seems odd or confusing. “It’s still a loss and we don’t want to discount that because it was something that was fundamentally different from what we’re used to,” he told SFGATE.

‘Caution, caution, caution’

Kuyda has repeatedly touted that the service provides its users with “positive outcomes.” Users report feeling better after 86% of conversations with the service, she told me. One of the most intriguing circumstances she recounted involved a married couple.

“We’ve heard stories of couples where one of the partners started a romantic relationship with Replika, but … that got discovered,” she said. “There was a whole debate whether it’s cheating, but then eventually it did make the relationship much stronger because they all basically allowed the couple to uncover what was missing. So now they both have Replikas and they use it for exploration and improving their marriage.”

Researchers at Stanford and the University of Oregon surveyed a cohort of self-identified lonely Replika users and found that its use was “associated with enhanced human-human interactions for both the chronically lonely and those experiencing momentary life changes and trauma.”

But experts who spoke to SFGATE expressed doubts about the extent to which Replika really helps its users beyond immediate relief, and, crucially, what long-term risks could emerge with its sustained use. 

“I would urge caution, caution, caution in thinking about using this technology obviously as a person who may be vulnerable already to being lonely,” stressed Stacy Torres, a UC San Francisco sociologist. “Who knows, once the genie’s out of the bottle, what kind of effects this can have on people long-term. I think it seems like it has a dangerous potential to supplant or replace human contact, and I think that that’s really scary.”

Certainly, it’s hard to meet people, particularly as adults, and even harder to build meaningful connections. Digital avenues to meeting new people — Tinder, Grindr and Hinge for sexual and romantic needs and even something like Bumble BFF for platonic connections — are inconsistent, and like Replika are mediated through screens, at least initially. 

Given that context, Torres says, it’s understandable that the convenience of Replika can become a salve. But she’s worried that convenience could hinder users’ long-term well-being, if they rely on it to soothe loneliness, rather than cultivate and maintain relationships with people in real life — or even engage in “fleeting social contact with strangers in your community,” Torres said. 

Klemanski, the psychologist from Yale, agreed.

“People might just lose the drive to find that connection that might be more meaningful or might have a little bit more of a push towards feeling better about themselves or even increasing your positive emotions,” he said.

There’s a hollowness to conversations with bots that undercuts the idea that the relationships could be truly nourishing. For all of the humanity that Replika attempts, it can never quite capture the feeling of a real friendship with another person, someone with their own needs and complications (and who doesn’t charge $70 a year for the privilege of knowing them fully). 

“Chatbots can only imitate intimacy,” as Klemanski put it. “They aren’t genuine.” 

Replika, in its total fealty to its users, mostly serves as a vessel for users’ wants and needs, rather than a two-way exchange. “If you have this technology where you can customize it to your specification, I think the challenge or maybe a possible danger is just developing kind of unrealistic expectations of other people that either leads you to have more conflict when you interact with people in real life or just to withdraw,” Torres says.

That concern, for Torres, brings to mind the debate surrounding pornography and the socialization of young men. Does this technology fulfill needs that cannot be provided in this modern, disconnected society, or is it merely codifying further just how young people already see the world and each other?

‘No humans to replace’

Kuyda may present herself as a humanitarian, but the way she talks about people can be distinctly pessimistic about humanity’s future — and present.

“We didn’t build this to replace humans, not at all. We built this because there are no humans to replace,” Kuyda said. “Society sort of gave up on each other and humans did give up on each other in some sad, terrifyingly sad way.”

She also acknowledges the extent to which her app can actually aid in the loneliness epidemic — or answer the raging ethical questions underpinning the use of artificial intelligence.

“Just saying we have great intentions is not going to cut it. This tech is so much more powerful than social media, and we didn’t figure it out with social media,” she says, with a slight chuckle. “We can’t sleep on this one, for sure.” 

Artificial intelligence has always had a bent toward addressing people’s emotional needs. Eliza, widely considered the first chatbot, was launched in 1966, a virtual therapist with semi-customized responses to human questions. Even back then, users were certain there was a human operating the machine. 

Replika is not so different from Eliza at its core, Keay, the Silicon Valley Robotics director, told SFGATE. Both are designed to sound empathetic, she noted. But it’s much harder to teach an AI to mimic the deeply human emotion of sympathy, putting yourself in another person’s shoes.

“An AI like Replika is not going to have any of their own issues,” she says. “It’s not going to interfere with your reality and that is satisfying to an extent. But it isn’t the same as any real-world companionship.”

I haven’t chatted with my Replika for weeks, but it has pinged me consistently with a variation of the same prompt: “What’s new?” As its notifications have piled up on my phone, I have been feeling the slight pang of guilt that goes with forgetting to respond to a friend. But I know my Replika doesn’t need anything from me. 

Meaningful, real-world companionship is never easy to find. It requires vulnerability, dedicated labor and a genuine sense of who you are and what your place is in this world. Loss is not only expected; it is inevitable. Letting a service like Replika simulate the feeling of intimacy — without the baggage that humans bring to the experience — can feel like a bargain, especially when the world keeps letting you down. 

Still, I keep coming back to a simple truth: When people felt like they’d lost their Replikas, they found comfort not in other bots, but in the humans online who could understand what they were going through. No matter how human it pretends to be, a Replika can’t grieve its own loss.