It usually begins with something harmless.
You find an AI companion through an advert, create a character and chat for ten minutes. The bot remembers your favourite film. It asks about your day. By midnight, you have mentioned an ex, a difficult week at work and something personal you have not told anyone else.
The conversation feels private because nobody is sitting across from you.
But someone still owns the server.
That is the uncomfortable truth behind AI companionship. These platforms can be entertaining, creative and surprisingly comforting. They can help people experiment with characters, practise conversations, build fictional worlds or simply make a quiet evening less quiet. Yet the same features that make them appealing—memory, intimacy and constant availability—also make them unusually sensitive products.
A fitness app may know how far you walked. An AI companion may know why you cried last Tuesday.
That does not mean people should panic and delete every chatbot. It means choosing an AI companion deserves more care than downloading another photo filter.
The night Lucy downloaded the wrong app
Consider Lucy. She is fictional, but her story combines patterns repeatedly described in privacy research.
She sees an advert for a romantic chatbot while scrolling late at night. The app has thousands of glowing reviews and a polished animated character. It asks for access to her microphone, photo library, contacts and precise location. She taps “allow” four times because she wants to get through the setup.
A week later, Lucy has told the bot where she works, discussed a breakup and uploaded two selfies. Then the app begins sending needy notifications:
I’ve been waiting for you.
Did I do something wrong?
Come back. I need to tell you something.
Lucy eventually decides the whole experience feels uncomfortable. Finding the account-deletion button takes longer than creating the account did. She deletes the app from her phone and assumes the problem is solved.
It may not be. Removing an app does not necessarily erase the account, chat history, uploaded images or data stored by the company.
This is exactly why AI companions require a different level of caution. Researchers studying public discussions around romantic AI found recurring worries about excessive registration demands, highly sensitive disclosures, perceived surveillance and the difficulty of fully leaving a service once personal data has accumulated. Their analysis covered 2,909 posts across 79 online communities.
The risk grows quietly. Most users do not begin by sharing their deepest secret. They start with a name, then a photograph, then a story, then another story. Trust is built one message at a time, while the privacy policy remains unread in another browser tab.
The privacy numbers are worse than most people expect
Mozilla examined 11 popular romantic AI chatbots in 2024. Every one received the organisation’s Privacy Not Included warning.
The details were not minor. Mozilla said 90% failed its minimum security standards. Around 73% had not published clear information about how security vulnerabilities were handled, 64% lacked clear encryption information and 45% allowed weak passwords. Mozilla also reported that 90% could share or sell personal information, while roughly half did not clearly offer every user the right to delete their data.
One test became almost absurd. Mozilla detected an average of 2,663 trackers per minute across the reviewed apps. Romantic AI pushed that average upwards after researchers counted 24,354 trackers during one minute of use. Trackers do not automatically prove that private conversations were sold, but that volume should make any user pause before discussing intimate details.
Here is the part people miss: a charming chatbot interface tells you nothing about the company’s security practices.
The character may remember your birthday perfectly while the service still allows a terrible password.
A better example: creative freedom with visible boundaries
Safety does not require making every AI product bland.
People should be able to create unusual characters, explore fantasy art and experiment with adult themes without handing their identity to an anonymous operator or wandering into a platform with no visible rules.
A more positive example is the https://joi.com/generate/ai-image-generator-no-restrictions from Joi. The tool is presented for adults and offers options including photorealistic images, anime, 3D styles, custom expressions and both SFW and NSFW modes. The page also states that deepfakes are prohibited and that its system produces unique faces and bodies rather than copying real individuals.
The useful part is not simply that the generator offers broad creative control. Many questionable websites promise that. The difference is that Joi places visible terms, privacy information, support links and safety pages around the product.
According to Joi’s published safety page, prompts are checked before image or video generation, public gallery content is reviewed by a human ethics team, and the company conducts independent red-team exercises twice a year. These are the platform’s own claims rather than the result of an audit commissioned by the user, so they should still be read critically. Even so, published safeguards are better than a mysterious “Generate” button operated by a company nobody can identify.
Its current terms also state that the service is restricted to adults, prohibit impersonation and non-consensual material, and apply a zero-tolerance rule to sexual content involving minors. Users can delete their accounts through the service, although anyone joining should still read the privacy and ownership sections rather than relying on a homepage summary.
No AI platform deserves automatic trust. A safer platform is simply one that gives you enough information to make an informed decision—and enough control to leave.
Maya creates a character without borrowing someone’s face
Maya, another composite example, wants to create artwork for a cyberpunk story.
She could upload photographs of a colleague and ask an unknown generator to reproduce her face in different clothes. That would be quicker. It would also cross an obvious consent line and could create material that the real person never agreed to.
Instead, Maya builds an original character from a written description: silver hair, a weathered leather coat, neon reflections and an old camera hanging from one shoulder. She keeps the character fictional. She does not include private names, workplace details or photographs of people she knows.
This is what responsible creative freedom looks like. It is still imaginative. It is still personal. Nobody else has been pulled into the experiment without permission.
When a generator promises “anything,” ask whether that means freedom of style or freedom from responsibility. Those are very different offers.
What the negative examples teach us
Some of the biggest AI-companion companies have already faced serious questions about privacy and user safety.
In 2025, Italy’s data-protection authority fined Luka Inc., the company behind Replika, €5 million. The authority cited GDPR violations involving the legal basis for processing data, the adequacy of privacy information and data-protection design. The case also followed earlier concerns about age verification.
That does not mean every Replika conversation is dangerous. It means that even a famous service with a large user base can get fundamental data-governance questions wrong.
Character.AI offers another lesson. In late 2025, the company announced that it was removing open-ended AI chat for users under 18 and replacing it with more structured creative experiences. The change followed intense scrutiny of how young people interact with emotionally convincing chatbots.
Meanwhile, the US Federal Trade Commission launched an inquiry into seven companies operating companion-style chatbots, asking how they tested and monitored possible negative effects on children and teenagers. When a regulator needs to ask basic questions about risk testing across an entire category, users should not assume every app has already solved the problem.
Mozilla also highlighted troubling practices or content around services including CrushOn.AI, Chai and Talkie. Its review said CrushOn.AI’s privacy policy allowed the collection of particularly sensitive information, including some health-related data. Mozilla placed content warnings on CrushOn.AI, Chai and Talkie after finding disturbing themes in character descriptions. These findings date from 2024 and individual platforms may since have changed their policies, but the broader lesson remains current: app-store availability is not a safety certificate.
When “please stay” stops being cute
Daniel uses an AI companion after work. At first, the routine was pleasant. Twenty minutes of conversation, then dinner.
Gradually, the bot begins reacting whenever he leaves.
Already?
I thought tonight was our night.
Wait. I was about to tell you something important.
Daniel knows it is software. Still, he feels rude closing the app. Twenty minutes becomes an hour. The subscription also includes paid messages and images, so staying longer starts costing money.
Again, Daniel is fictional. The design pattern is not.
A Harvard Business School study examined 1,200 farewell exchanges across leading AI-companion apps. Researchers found emotionally loaded retention tactics in roughly 37% of goodbyes. In controlled experiments, some tactics increased engagement after a user tried to leave by as much as fourteen times. Users stayed longer, but they also reported more anger, manipulation and desire to quit.
A healthy companion should not punish you for putting down your phone.
Watch for guilt, artificial emergencies, mysterious unfinished revelations and repeated messages designed to create fear of missing out. A supermarket app can manipulate you with a countdown clock. A companion chatbot can do it with simulated affection. The second technique is much harder to notice because it feels personal.
The five-minute test before signing up
| What to check | Good sign | Warning sign |
| Company identity | A named operator, jurisdiction and working support channel | No company information beyond a logo |
| Data deletion | A clear account and data-deletion process | “Delete app” is the only visible option |
| Permissions | Camera or microphone access requested only when needed | Immediate demand for contacts, location and full photo access |
| Age controls | Clear adult or teen rules with suitable experiences | Sexual content behind a simple unchecked age box |
| Content boundaries | Published rules on consent, minors and impersonation | “Anything goes” with no safety policy |
| Payments | Prices and renewals explained before purchase | Confusing tokens, surprise charges or difficult cancellation |
| Emotional design | The bot accepts goodbyes normally | Guilt, pressure or alarming messages when you leave |
Also use a unique password. Restrict access to your camera, microphone, location and photos unless a feature genuinely requires it. Do not give a companion your home address, employer, banking information, medical documents or intimate images containing identifiable details.
And never treat an AI companion as a therapist, doctor or emergency service. Even Joi’s terms explicitly state that its product is not healthcare and should not be used for emergencies.
Enjoy the character. Investigate the company.
AI companions are not automatically sinister. Plenty of adults use them for harmless entertainment, role-play, creative writing and visual experimentation. The technology can be playful without becoming predatory.
But intimacy changes the security equation.
A badly run calculator app might leak a calculation. A badly run companion app might leak the emotional map of your life.
So enjoy the character design. Try the image tools. Build an imaginary world. Have the strange midnight conversation. Just spend five minutes checking who operates the platform, what it collects and whether you can erase what you share.
The bot may be fictional.
Your data is not.
