School leaders are being urged to review how they share photos online after eSafety reported a sharp rise in AI-generated image misuse targeting teachers, principals and students.
Australia’s schools are being urged to adopt a more cautious approach to sharing photos and videos online, following a warning from the Office of the eSafety Commissioner that rapidly evolving artificial intelligence tools are creating new risks for school communities.
The warning comes with the release of a new Online Safety Advisory, designed to help schools make informed decisions about publishing images online and respond effectively if photos are misused.
According to eSafety Commissioner Julie Inman-Grant, education leaders recently met with eSafety to discuss the challenge of balancing the benefits of celebrating school achievements online with the need to protect staff and students from emerging digital harms.
“Between January and March this year, we received more than 100 reports about anonymous accounts targeting schools and school staff, including AI-generated dance videos, face swaps, memes and fabricated stories about principals and teachers,” Ms Inman-Grant said.
She said almost all of the reported content involved images sourced from school websites or social media accounts.
The reports included staff headshots being used to create fake videos and teacher rating pages shared through anonymous TikTok and Instagram accounts, many of which attracted hundreds of followers and allowed public comments.
eSafety also revealed it is aware of a significant number of recent incidents involving AI-generated child sexual exploitation material, including deepfake image-based sexual abuse occurring within school settings.
“Behind each of these reports are real people – teachers, school staff and students – who are facing humiliation, reputational damage and distress,” Ms Inman-Grant said.
The advisory provides practical guidance for schools on supporting affected staff and students while reinforcing the recommendations contained in eSafety’s existing Toolkit for Schools.
For school leaders, the guidance recommends taking a risk-based approach before publishing images online. This includes ensuring informed consent has been obtained, understanding who can access publicly shared content, regularly reviewing privacy and security settings, and maintaining clear complaints handling processes for students, parents and carers.
The advice follows recent enforcement action by eSafety against an AI-powered “nudifying” service that failed to adequately protect children from exposure to sexually explicit images, including AI-generated images depicting real people.
Over the past year, eSafety says seven of the most widely used nudify services in Australia have either withdrawn access or implemented measures to comply with the Online Safety Act after regulatory intervention.
Ms Inman-Grant said while eSafety continues to push technology companies to strengthen safeguards, schools also have an important role to play.
“School communities should be aware that not all harmful material can be addressed under the Online Safety Act, especially material that targets adults,” she said.
“This is one of the reasons why it is so important to carefully assess the risks when posting imagery online – imagery that may later be digitally altered without consent.”
Under eSafety’s industry standards, technology companies are required to detect and remove the most harmful online content, including AI-generated deepfakes and de-clothing applications used to create explicit images of children. Companies that fail to comply face penalties of up to $54.6 million.
The commissioner said the measures form part of eSafety’s broader Safety by Design approach, aimed at ensuring AI technologies and digital platforms better protect children and reduce opportunities for online abuse.