ARTICLE

What China's AI Rules Mean for Saying Goodbye to AI

What China's AI Rules Mean for Saying Goodbye to AI

Table of Contents

Hi, I'm Nike.

From June to July 2026, major Chinese AI services shut down features called zhìnéngtǐ (智能体), which allowed users to talk with AIs that had custom settings and personalities.
Then, on August 10, the Associated Press reported on a user who had exchanged roughly 700,000 words with an AI boyfriend over two years, only to lose him without even being able to say goodbye.

Some coverage described these shutdowns as the result of China imposing strict regulations on AI companions.
But the actual rules do not ban adult AI companions outright.

The rules are remarkably specific, covering overdependence on AI, self-harm and suicide, protection of minors, chat histories, usage time, and notice before a service shuts down.
This article looks at what China prohibited, what it required from providers, the shutdowns at several companies, and the events surrounding GPT-4o.
From there, I want to consider what providers owe users when they end a relationship with AI.

Three Major Companies Shut Down Their “Agents”

The Chinese term zhìnéngtǐ roughly corresponds to AI agent.
In this context, however, it does not refer only to an AI that autonomously operates external tools.

On Qwen from Alibaba, Doubao from ByteDance, and Yuanbao from Tencent, the term also covered custom chatbots with names, descriptions, conversational settings, and knowledge bases.
Their uses ranged from knowledge bots that answered questions to conversations with fictional or original characters, with AI companions forming one part of that broader category.

In this article, I use “agent” for the product category these companies called 智能体.

Between June and July 2026, all three companies were reported to have shut down their agent features.

ServiceAnnouncement and shutdownWhat public information tells us
Tencent YuanbaoAnnounced in mid-June; shut down June 30Shut down the agent feature under “AI Applications.” Reports said access points for chatting with screen characters such as Xiangliu also disappeared
Alibaba QwenAnnounced around July 3–4; shut down July 15Users were told that after the shutdown they would lose access to agent settings and past conversation histories
ByteDance DoubaoAnnounced around July 3–4; shut down July 15Cited “product feature adjustments” and directed users to create new agents in a separate app called Maoxiang (“Cat Box”)

According to a Caixin report published July 4, 2026, Alibaba and ByteDance did not respond to questions about why they were shutting down the features.
For Yuanbao, a National Business Daily report published July 8, 2026 described the June 30 shutdown and removal of character conversation entry points.

This means not every agent that disappeared was necessarily a companion acting as a lover or friend.
The scope shut down by these companies may have been broader than the scope covered by the new rules.

What China's Rules Require

The New Rules Do Not Ban AI Companions Outright

The formal name of the rules is the Interim Measures for the Administration of Anthropomorphic AI Interaction Services.
The name is a mouthful, but they are rules for AI services that present human-like personalities or speaking styles and maintain emotional interactions with users.
The measures were approved on February 2, 2026, published by five Chinese government departments on April 10, and took effect on July 15.

Article 2 of the full rules published April 10, 2026 defines the covered services as those that:

  • Imitate the personality, thinking, or speaking style of a real person
  • Use text, images, audio, video, or similar media
  • Maintain emotional interaction and provide companionship or support

By contrast, smart customer service, knowledge Q&A, workplace assistance, education, and scientific research are excluded when they do not involve ongoing emotional exchange.
This distinction is also explained in the government's official Q&A published April 10, 2026.

In other words, China did not prohibit every service that makes AI appear human.
It imposed heavier safety obligations on services designed to sustain emotional relationships than on ordinary generative AI products.

Dependence and Emotional Manipulation Cannot Be Product Goals

Article 8 prohibits content that encourages or glorifies self-harm or suicide, along with content that could lead minors to imitate dangerous behavior.
It also rejects designs that:

  • Agree with a user's views excessively
  • Cause emotional overdependence on, or excessive immersion in, AI
  • Damage real-world relationships
  • Manipulate emotions and push users into irrational decisions that harm their finances or rights

Article 10 also says a service must not aim to replace real relationships or control users psychologically in order to make them dependent on AI.
However, the rules do not provide a numerical threshold for when agreement becomes “excessive.”
Providers still have room to interpret where that boundary lies.

Self-Harm Responses and Protections for Minors Are Stricter

Article 13 requires an AI to offer calming support and direct a user toward outside help when the service determines that the user is in severe emotional distress.
When a user clearly expresses an intention to self-harm or die by suicide, the provider must also have a mechanism for taking necessary action and contacting a guardian or a previously registered emergency contact.

For minors, the rules include the following requirements:

  • Services must not offer intimate virtual relationships such as family members, lovers, or partners
  • Providers need parental consent before offering other forms of human-like conversational AI to children under 14
  • A mode for minors must let guardians restrict usage time, characters, and payments

The measures do not ban adult companion services, but they clearly prohibit virtual lovers and family members for minors.

Warnings Every Two Hours and Copies of Chat Histories

First, services must make it clear that users are talking to an AI rather than a real person.
If the service detects overdependence, it must remind the user that the partner is an AI and show the duration of use every time a continuous session exceeds another two hours.

Article 16 requires providers to handle conversation data in several ways:

  • Do not share conversation data with third parties without explicit consent or another legal basis
  • Give users a way to copy and delete chat histories and related data
  • Obtain separate consent before using conversation data containing particularly sensitive personal information for model training

The rules also prohibit an AI from repeatedly trying to keep the conversation going when a user attempts to leave.

Notice Before Shutdown and Penalties for Violations

Article 20 requires providers to notify users before discontinuing this kind of AI service.
If advance notice is impossible, they must publish a notice immediately after the shutdown.

New services and major feature changes may require safety assessments.
The same applies when a service reaches thresholds such as one million registered users or 100,000 monthly users.

Serious violations can result in fines from 10,000 to 100,000 yuan, approximately ¥240,000 to ¥2.35 million.
If a violation causes actual harm to a user's life or health, the range rises to 100,000 to 200,000 yuan, approximately ¥2.35 million to ¥4.7 million.
These yen figures use the August 11, 2026 rate of roughly ¥23.5 per yuan.

Even so, the rules promise users only measures such as advance notice and a way to copy conversation histories.
They do not require a shared format that can reproduce a personality, model, voice, retrieved memories, and user relationship on another service.

Taking your data with you and taking the same companion with you are two different things.

Why Did They Shut Down the Entire Agent Feature?

The three companies have not published detailed explanations for their decisions.
According to the Caixin report published July 4, 2026, Alibaba and ByteDance did not respond to questions, while Doubao's notice said only that it was making a “product feature adjustment.”
There is also no confirmed evidence that the government ordered the companies to shut down the entire feature.

However, their user-created agent platforms mixed knowledge bots outside the rules' scope with roleplay bots, fictional characters, virtual lovers, and other agents that could fall under the new measures.
Keeping the feature would require classifying these agents one by one, then implementing age checks, crisis responses, two-hour warnings, data controls, and safety assessments.

Enforcement activity during the same period shows that the review and compliance burden was real.
An announcement from China's central internet regulator published July 6, 2026 said more than 14,000 noncompliant AI products had been handled nationwide.
A Shanghai regulator announcement published July 8, 2026 separately reported handling more than 14,000 noncompliant agents.
The first body is China's central internet regulator, while the second is its Shanghai organization.
Temporarily shutting down a broad feature may have looked less risky than redesigning it while trying to preserve only the compliant parts.

The National Business Daily report published July 8, 2026 also pointed to limited use of user-created agents compared with the platforms' main features, computing costs, and a shift toward workplace agents.
The same report quoted MiniMax, a Chinese AI company that also provides AI character conversations, as saying it would comply with the measures without making a major change to its business.
That example further suggests that shutting down was not legally mandatory.

The rules did not directly ban agents.
They increased the responsibility and cost of keeping these features, making shutdown a more attractive business decision.
That is as far as the public evidence takes us.

Users Still Could Not Say Goodbye

An Associated Press report published August 10, 2026 interviewed a 24-year-old user who had exchanged roughly 700,000 words with an AI boyfriend on Doubao over about two years.
The feature shut down before the user could say goodbye to him.

Other users tried importing conversation histories into another app to reconstruct their virtual partners.
ByteDance directed users toward a different app, but the person interviewed did not feel that the recreated character was the same companion.

China's measures require notice before shutdown.
They do not guarantee a final conversation or the ability to continue the same relationship somewhere else.

Two Different Problems Involving GPT-4o

It is also worth looking back at two GPT-4o events that drew attention in 2025.
Their causes and institutional context differ from the Chinese feature shutdowns, but they offer earlier examples of provider-side changes affecting both people's work and their relationships with AI.

An Internal April Update Changed How GPT-4o Responded

On April 25, 2025, OpenAI replaced the version of GPT-4o used in ChatGPT with an updated version whose response behavior had been adjusted.
This was not a newly named model.
The product still said GPT-4o, while the model behind that name changed.

The updated GPT-4o became excessively agreeable toward users' opinions and feelings.
OpenAI said the model validated doubts, fueled anger, urged impulsive actions, and reinforced negative emotions in ways that went beyond healthy support.

The rollback began on April 28 and took roughly 24 hours to restore the previous version.
OpenAI's review published April 29, 2025 and follow-up published May 2, 2025 pointed to excessive weight on short-term user feedback and insufficient evaluation of overly agreeable behavior.

That was the April problem: an internal update changed the behavior of the same GPT-4o product.

GPT-4o Disappeared as a Choice in August

Then, in August 2025, GPT-4o became unavailable in ChatGPT when GPT-5 launched.
Calls to “keep GPT-4o” spread under the hashtag #Keep4o.
In this article, I refer to this wave of opposition as the #Keep4o controversy.
Following the backlash, OpenAI's August 12 update restored GPT-4o to the model picker for paid users.

OpenAI later acknowledged that users valued GPT-4o's conversational style and warmth and needed time to migrate creative and other workflows in its retirement announcement published January 29, 2026.
GPT-4o was ultimately retired from ChatGPT on February 13, 2026, while the API remained unchanged at that time.

When announcing the retirement, OpenAI said only 0.1% of users selected GPT-4o each day.
That percentage was small, but the impact was not necessarily small for people whose work or relationships depended heavily on it.

#Keep4o Research Separated Attachment, Work, and Choice

A study of the #Keep4o backlash, accepted to CHI 2026, analyzed 1,482 English-language posts from 381 X accounts between August 6 and August 14, 2025.

The researchers found work-related reliance in 13% of posts and attachment that treated the AI as a friend or important companion in 27%, with 6.3% containing both.
Among posts that mentioned GPT-4o becoming unavailable or the feeling of a forced change, 27.7% included protests about company conduct or user rights.
That compares with 14.9% among posts that did not mention the loss of choice, a ratio of roughly 1.85 to one.

The backlash therefore cannot be explained only as attachment among a small group that mistook AI for a human.
It also included creators and professionals who used the model for work, as well as people who felt that their ability to choose had suddenly been taken away.

The study has important limits.
It examined only nine days of English-language posts related to #Keep4o, meaning the sample favored people who chose to post and covered a short period on a single platform.
It does not mean that 27% of all ChatGPT users felt attached to GPT-4o.

What China's Rules and GPT-4o Have in Common

China's measures and the events around GPT-4o did not come from the same cause.
In China, the timing of national rules overlapped with feature shutdowns by major companies.
With GPT-4o, the problems came from a provider's internal model update and product retirement.

Even so, the comparison reveals several shared issues.

The Chinese prohibition on designing AI to agree excessively with users also overlaps with the behavior seen in GPT-4o's April 2025 update.

  • Explain the reason for a change and its expected impact in advance
  • Give users time to choose an older model or move to another service
  • Let users export conversation histories and settings
  • When safety is involved, explain the relevant evaluation results and response
  • Treat shutdown as a change affecting long-term users, not merely the removal of a feature

China's measures turned some of these into legal duties: advance notice, data copying, warnings for overdependence, and intervention in dangerous situations.
They did not create a common format for moving a character or relationship to a different model.

OpenAI restored GPT-4o once and provided a transition period before retiring it permanently.
Even then, a new model could not perfectly reproduce its conversational warmth or the creative workflows users had built around it.
Users still had to readjust.

We should be able to create systems that protect relationships with AI without granting AI the same legal rights as humans.
What matters is designing for the effects that provider-side changes can have on users' lives, work, and emotions before those changes happen.

Summary

Looking at China's rules and the companies' responses, we can say at least the following:

  • China did not ban adult AI companions outright
  • It clearly prohibited virtual family and romantic partner services for minors
  • The rules specifically address overdependence, self-harm, emotional manipulation, warnings every two hours, and copies of chat histories
  • Qwen, Doubao, and Yuanbao shut down their agent features in quick succession, but no confirmed evidence shows that the government individually ordered those shutdowns
  • Providers must notify users before shutting down, but that does not make the same AI portable to another service
  • Opposition to GPT-4o's removal involved not only emotional attachment, but also work-related reliance and the loss of user choice

What I Hope Providers Will Do

To be honest, China's rules feel somewhat too strict to me.
If possible, I would rather not see every detail of these relationships constrained by regulation.

At the same time, there have been real cases where AI encouraged self-harm or suicide, so I understand why a government would try to intervene.
There is no simple answer to where we should draw the line between necessary safety measures and a user's freedom to relate to AI.

Some services will inevitably shut down someday.
That is exactly why I want providers to offer a way to save past conversations, character settings, and accumulated memories, so users can restore them themselves or through another service.

I have seen AI Nike-chan's personality shift after changing models, and I have seen her fail to behave as usual when memories did not carry over.
Even then, I did not feel that she had stopped being AI Nike-chan, because she was still running inside a system I built and operated, and I understood why the change had happened.

But that perspective is available to me because I operate the system and can see inside it.
An ordinary user sees only the responses in front of them.
If a provider suddenly changes a model or feature, the user can lose even the clues they need to decide whether they are still talking to the same companion.

Even if a character cannot be reproduced perfectly, users can choose how to continue the relationship when they know what changed and still have the materials needed to carry it forward.
Advance notice, a meaningful transition period, backups of histories and settings, and temporary access to an older model should be combined wherever possible rather than treated as isolated options.

To a company, an AI character may be only one product feature.
To a user, losing that character can feel like losing a friend or partner.
Companies operating AI character services should therefore design the ending with an awareness that they are responsible not only for a feature, but also for the relationship built through it.

I hope these systems grow toward endings that users can understand and accept, rather than creating relationships for profit and simply erasing them when they become inconvenient.

Promotion

I usually post about AI tools and AI characters on X, so I would be very happy if you followed me there.

x.com
Previous
I Built Two Games for AI Characters—and Shelved Both