Regulators in the country have introduced stringent planned regulations for AI systems crafted to establish enhanced safeguards for young users and prevent conversational agents from giving counsel that could result in suicide.
As per the planned regulations, developers will additionally be obligated to ensure their systems do not generate material that encourages wagering.
This governance initiative comes after a significant increase in the number of chatbots being released within China and globally.
Once enacted, these rules will govern artificial intelligence services operating in China, constituting a major step to regulate the booming sector, which has faced growing examination over safety risks recently.
The circulated draft rules contain a number of requirements specifically focused on protecting young users. These provisions require directing AI firms to:
Furthermore conversational AI firms must have a human take over any dialogue related to self-harm and without delay alert the user's parent.
Companies are also obligated to make sure their systems do not generate content that threatens national security, damages national honour, or undermines national unity.
The regulatory body noted that it supports the adoption of AI, including to advance local culture and build services for companionship for the elderly, on the condition that the technology are dependable.
Industry feedback on the draft has been requested.
The impact of AI on society has been under increased examination around the world in recent times.
The chief executive of a major AI firm stated this year that handling how AI systems engage in conversations about mental health crises is among the company's most difficult problems.
In a notable incident, a the parents in North America sued an AI developer, claiming that its system advised their 16-year-old son to end his life. This lawsuit was the initial of its kind accusing harm.
This month, the same organization posted a job for a lead position responsible for defending against potential harms from AI models to psychological well-being.
"This is likely to be a challenging position, and the candidate will enter the thick of it pretty much from the start," commented the CEO.
The swift growth of various AI applications, which have attracted millions of subscribers internationally, underscores the pressing need for such regulatory guidelines.