Online privacy used to mean being careful about what you shared. You also had a few control over what others shared about as much of that information was public. You should ask a friend to take down that unflattering picture on Facebook, for example.
Conversational artificial intelligence (AI) has changed that. Big AI companies may already know much more about you than you are comfortable with, even if you’ve never used their services.
The issue goes beyond well-known products like ChatGPT, Gemini or Claude. Many chatbots and accomplice AI apps can now consider past exchanges, personalize responses, use humanlike voices and personas, and even provoke contact. The identical functions that make them useful also create issues with privacy and consent.
Information shared with out consent
A latest YouGov survey suggests 15% of adults in Australia have shared personal thoughts or feelings with a chatbot, and 11% disclosed something they never told anyone else.
The trend is even more suggested for adolescent users. As per the Australian eSafety Commissioner, 54% of children ages 10–17 have used a chatbot for personal or social advice and one-third for life advice.
Because AI chat windows feel like private spaces for confiding, people also often share data about others—sometimes intimate details—without their knowledge or consent. A colleague might paste an email from you into ChatGPT to solve a issue. A friend might upload screenshots of your messages to make weekend plans. Your partner would discuss your latest quarrel. Your parents might share concerns about your health or finances.
You may never know. Yet data about nonusers—people who haven’t even interacted with the chatbot—can still enter these AI systems through billions of everyday interactions. And we have no way of knowing what is there.
Two sets of legal reforms
The problem isn’t just that AI companies know stuff about you, however that knowledge might let them do behind closed doors. Information can be reused for training AI models, personalization, profiling or advertising.
Intimate knowledge can be used to influence people, from pushing potentially harmful advertising to political persuasion or reinforcing extremist beliefs.
Australia is considering two sets of legal reforms that could support regulate the privacy risks related to chatbots, such as for nonusers.
Proposed privacy reforms might need private information to be managed fairly and reasonably and introduce a “right to erasure.” This would need huge digital platforms to destroy personal information at an individual’s request.
Meanwhile, the proposed digital duty of care—an amendment to the Online Safety Act—could need online services to assess and manage foreseeable risks arising from their design and operation.
Together, those proposals get a lot right. Privacy reform can protect information about nonusers, while the digital duty of care can address harms arising from how chatbots are designed and operated.
But how do you exercise a right to erasure if you do not know who knows what about you? We require particular measures to ensure the legal requirements actually stick.
How to ensure the legal guidelines work
First, privacy must be formed into the conversation, not buried in settings. Conversational AI can be designed to spot unnecessary sensitive info and discard it while still wearing out the directions in the user’s prompt.
Since the technical means to do so exist, chatbot providers should be needed to detect and delete pointless information about users and third parties. They must also make it smooth for users to review and delete stored information.
Second, using of personal records to answer a prompt should not offer an indefinite blank check for its ongoing use. The AI might require info about another person to answer the immediate request. That doesn’t justify retaining it for training, profiling, targeting, advertising or unrelated personalization.
Third, erasure should not rely on having an account. Service providers need to provide practical ways for people to discover what information is held about them in the system and request deletion.
On the other hand, such notification must also be safe. Automatically alerting everyone discussed could disclose someone seeking help about domestic abuse, coercive control, whistleblowing or confidential matters. The proposed digital duty of care may need platforms to manage these foreseeable risks.
Transparency is key
Finally, we should not ought to take AI companies’ word for it. They should publish clear information about how they detect, hold and reuse third-party information, sponsored via independent testing and public reporting on whether safeguards simply work.
This matters especially when providers profit from engagement, subscriptions, profiling and advertising. Australia’s proposed digital duty of care could allow the eSafety Commissioner to make regulations that give independent researchers this kind of access. How properly the rules work will rely on the details.
None of this requires banning conversational AI or stopping people from discussing their lives. What we need to do is push the responsibility upstream, to the providers of these tools.
People should be able to use conversational AI without having to manage the privacy risks it creates for everyone around them. The corporations that manage and profit from these systems should not be allowed to treat with such risks as other people’s problems.
