DFFH child protection report contained ChatGPT errors, data disclosed to OpenAI
A child protection worker at the Department of Families, Fairness and Housing used ChatGPT to draft a court report, resulting in inaccurate information that downplayed child protection risks and disclosed sensitive personal data to OpenAI. Victoria's Information Commissioner found the department failed to take reasonable steps to ensure accuracy and protect personal information, issuing a compliance notice requiring the blocking of ChatGPT use by child protection workers.
What happened
In December 2023, a child protection worker at the Department of Families, Fairness and Housing used ChatGPT to draft a Protection Application Report submitted to the Children's Court for a case involving a young child whose parents had been charged with sexual offences.
The worker entered personal and sensitive information into ChatGPT, including the child's name and risk assessment details, disclosing this information to OpenAI, an overseas company, outside DFFH's control.
The content generated by ChatGPT and used in the PA report contained inaccurate personal information that downplayed risks to the child.
Reported impact
The inaccurate information in the report could have affected the Children's Court's decision on whether the child required protection.
Index analysis
The Privacy and Data Protection Deputy Commissioner found that DFFH lacked sufficient controls to manage risks associated with using generative AI tools in a child protection context.
OVIC concluded that DFFH contravened Information Privacy Principles 3.1 and 4.1 by failing to take reasonable steps to ensure accuracy of personal information and to protect it from unauthorised disclosure.
Governance controls
OVIC issued a compliance notice requiring DFFH to implement six specific remedial actions, including blocking child protection workers from using ChatGPT and similar tools.
DFFH accepted the investigation findings and is required to implement the remedial actions.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.