Content Regulation and Online Classification

1. Historical development of content regulation
2. Current regulatory framework
3. Content classification in Australia
   1. Classification Guidelines: R18+
   2. Classification Guidelines: X18+
   3. Classification Guidelines: Refused Classification (RC)
4. Online Content Scheme
5. Basic Online Safety Expectations
6. A duty of digital care in Australia
7. Section 313 of the Telecommunications Act 1997 (Cth)
8. Image-based abuse
   1. Social context and prevalence
      1. Key statistics
      2. Factors affecting reporting
      3. Social prevention and response
9. The Office of the e-Safety Commissioner
   1. Cyberbullying and hate speech
      1. Adult Cyber Abuse Scheme
      2. The Challenge of Online Abuse: Trolling
      3. Cyberbullying and the queer community
   2. Technology-facilitated abuse
      1. Tracking and location sharing
      2. The role of the eSafety Commissioner
10. Abhorrent Violent Material
    1. How Abhorrent Violent Material emerged
    2. Definition
    3. Criticisms of the AVM Act
       1. “As soon as reasonably possible”
    4. The Online Safety Act 2021 (Cth)
    5. Disguised Content and Evasion Techniques
       1. TikTok, YouTube and ‘Splicing’
       2. Disguised Content and the Online Safety Act
    6. Case Study: eSafety Commissioner v X Corp [2024] FCA 499
       1. Facts
       2. The Commissioner’s Response
       3. Proceedings
11. Social media minimum age
    1. Design of the scheme
    2. Evidence relied on
    3. Which services are covered
    4. Age assurance
    5. Criticism
       1. Proportionality and human rights
       2. Effect on vulnerable groups
12. Regulating content in other jurisdictions
    1. Social media regulation in Brazil
       1. Elon Musk and X under investigation
       2. Implications of the Brazilian decision
       3. United Kingdom
       4. Canada
       5. European Union
13. Other emerging issues
    1. Example: Unsolicited Dick Pics
    2. Deepfakes
       1. Deepfakes and Non-consensual Sexual Imagery
       2. Political deepfakes and disinformation
    3. AI voice agents and AI characters
       1. Social context and potential harms
       2. The Australian framework
       3. Open questions
       4. International approaches
    4. Misinformation and Disinformation

Historical development of content regulation

Video Overview of Online Content Regulation in Australia by Nicolas Suzor

The evolution of content regulation in Australia has been marked by several key developments:

• 1992: The Broadcasting Services Act 1992 established the first framework for content regulation in Australia, aimed at protecting public interests whilst balancing the freedoms of the broadcasting sector.

• 1997: The Australian Broadcasting Authority (ABA) investigated online content regulation and introduced 47 legislative principles for a National Framework, establishing the groundwork for industry codes of practice and the Platform for Internet Content Selection (PICS).

• 1999: Schedule 5 of the Broadcasting Services Act 1992 introduced a self-regulatory scheme with the ABA (later ACMA) as the enforcement agency. The scheme focused on ISP regulation through industry codes and complaints mechanisms.

• 2007: The National Filter Scheme (NetAlert Program) was introduced to provide internet safety education and filtering technology. It was discontinued in 2008 due to technical limitations and low take-up: around 144,000 filter products had been downloaded or ordered against a target of 2.5 million households, and the responsible department estimated that only about 29,000 of those remained in use.¹

• 2008: The Content Services Code 2008 provided clearer guidelines for content and hosting service providers and established self-regulatory content assessment regimes. The Code required providers to implement systems for restricting access to adult material, and set out procedures for responding to take-down notices issued by the ACMA.

• 2015: The Enhancing Online Safety for Children Act 2015 established the Office of the Children’s eSafety Commissioner. The office was expanded between 2017 and 2021 to cover online safety for all Australians and a broader range of harms, including image-based abuse and technology-facilitated domestic violence.²

• 2021: The Online Safety Act 2021 replaced previous frameworks with a comprehensive regulatory scheme administered by the eSafety Commissioner.

• 2024: The Online Safety Amendment (Social Media Minimum Age) Act 2024 (Cth) introduced a minimum age of 16 years for holding an account with an age-restricted social media platform. The obligation rests on the provider, which must take reasonable steps to prevent Australians under 16 from holding accounts. The civil penalty for failing to do so is 30,000 penalty units, which is multiplied by five for a body corporate — a maximum of 150,000 penalty units, or $54.6 million as at 1 July 2026.³ The requirement commenced in December 2025.

For more detailed information on earlier regulatory approaches, see the Code for Industry Co-Regulation in Areas of Mobile and Internet Content (2005).

Current regulatory framework

Australia has a co-regulatory content regulation scheme. Under a co-regulation model, an industry body (such as the Communications Alliance) usually develops a code of practice, which is then made binding on industry participants through a legislative mechanism. Co-regulation is a common form of regulation in Australian media law.

The Online Safety Act 2021 (Cth) sets out an expectation that industry bodies or associations will develop industry codes to regulate certain types of harmful online material. The Act provides for the eSafety Commissioner to register the codes if certain conditions are met. These include, among other things, that the Commissioner was consulted on the code and the Commissioner is satisfied that:

• The code was developed by a body or association that represents a particular section of the online industry, and the code deals with one or more matters relating to the online activities of those participants.

• To the extent to which the code deals with one or more matters of substantial relevance to the community—the code provides appropriate community safeguards for that matter or those matters.

• To the extent to which the code deals with one or more matters that are not of substantial relevance to the community—the code deals with that matter or those matters in an appropriate manner.

• The body or association published a draft of the code and invited members of the public and industry participations to make submissions, and gave consideration to any submissions that were received.

The Commissioner may also request that a particular body or association which represents a section of the online industry develop an industry code dealing with one or more specified matters relating to the online activities of those industry participants. In April 2022, the Commissioner issued such a request, seeking the development of codes relating to ‘class 1’ material by six industry associations. The associations submitted draft codes in November 2022. All six draft codes were rejected in February 2023 as they did not meet the standards for registration. The Commissioner believed that they did not provide appropriate community safeguards. Following resubmission, five of the draft codes were registered on 16 June 2023 and took effect six months later, on 16 December 2023. Those five codes apply to social media services, app distribution services, hosting services, internet carriage services, and equipment. A sixth code, applying to internet search engine services, was resubmitted to take account of developments in generative artificial intelligence; it was registered separately on 12 September 2023 and took effect on 12 March 2024.⁴

See the Online Content Scheme - Regulatory Guidance for further information.

Watch the following videos for background on online content regulation prior to the 2021 changes:

• Video Overview of Online Content Regulation in Australia by Nicolas Suzor

Content classification in Australia

Video overview of content classification and online safety in Australia by Jennifer O’Connor

The Department of Infrastructure, Transport, Regional Development, Communications and the Arts provides a quick guide to Australian classification ratings

The rules that apply to content depend upon the classification of the content. Australia has a national classification scheme for content (films, games, publications) likely to cause offence which was enacted in 1995 – the National Classification Scheme/Code. The Online Safety Act establishes an online content scheme which is partly dependent upon classification under the National Classification Code. As such, an overview of the basic features of the code supports an understanding of the Online Safety Act scheme.

The National Classification Code provides a statement of purpose that classification decisions are to give effect, as far as possible, to the following principles: (a) adults should be able to read, hear and see what they want; (b) minors should be protected from material likely to harm or disturb them; (c) everyone should be protected from exposure to unsolicited material that they find offensive; (d) the need to take account of community concerns about: (i) depictions that condone or incite violence, particularly sexual violence; and (ii) the portrayal of persons in a demeaning manner.

Publications, Films, and Computer games are rated by the Classification Board, according to the Classification Guidelines. Each State and Territory determines the consequences of classification. The ratings systems differ by media type:

• Films: G, PG, M, MA15+, R18+, X18+, RC
• Publications: Unrestricted, Unrestricted (M), Category 1 Restricted, Category 2 Restricted, RC
• Games: G, PG, M, MA15+, R18+, RC

Classification Guidelines: R18+

• High impact violence, simulated sex, drug use, nudity
• No restrictions on language

Classification Guidelines: X18+

• Real depictions of sexual intercourse and sexual activity between consenting adults
• No depiction of violence or sexual violence
• No sexually assaultive language
• No consensual activities that ‘demean’ one of the participants
• No fetishes (such as ‘body piercing’; candle wax; bondage; fisting; etc)
• No depictions of anyone under 18, or of adults who look under 18.

Classification Guidelines: Refused Classification (RC)

“Publications that appear to purposefully debase or abuse for the enjoyment of readers/viewers, and which lack moral, artistic or other values to the extent that they offend against generally accepted standards of morality, decency and propriety will be classified ‘RC’.”

For films, anything that exceeds X18+ is Refused Classification. For Games, anything that exceeds R18+ is RC (A new R18+ category was introduced for Games in 2012). Material classified RC may not lawfully be sold, hired or publicly exhibited anywhere in Australia.⁵

Classification Guidelines: RC (Films)

• Detailed instruction in crime or violence
• Descriptions or depictions of child sexual abuse or any other exploitative or offensive descriptions or depictions involving a person who is, or appears to be, a child under 18 years.
• Violence: Gratuitous, exploitative or offensive depictions of:
  • violence with a very high degree of impact or which are excessively frequent, prolonged or detailed;
  • cruelty or real violence which are very detailed or which have a high impact;
  • sexual violence.
• Sexual activity: “Gratuitous, exploitative or offensive depictions of:
  • activity accompanied by fetishes or practices which are offensive or abhorrent;
  • incest fantasies or other fantasies which are offensive or abhorrent.”
• Drug use:
  • Detailed instruction in the use of proscribed drugs.
  • Material promoting or encouraging proscribed drug use.

Online Content Scheme

The online content scheme under the Online Safety Act relates to two kinds of material: ‘class 1 material’ and ‘class 2 material’. Pursuant to s 106, class 1 material is material which is, or would likely be, classified as ‘RC’ by the Classification Board. Pursuant to s 107, class 2 material is material which is, or would likely be, classified as X 18+, R 18+, Category 2 or Category 1 restricted.

The Act provides for the notice and removal of class 1 material. Sections 109 and 110 provides that the Commissioner may give a notice to certain online service providers (including social media and hosting services) to remove or cease hosting material which the Commissioner is satisfied is class 1 material that can be accessed by end-users in Australia. It is not relevant where the service is provided from, or where the material is hosted – it merely needs to be accessible from Australia.

The notice may require the service provider to take all reasonable steps to remove the material from the service within 24 hours or such longer period specified by the Commissioner. Section 111 requires the service provider to comply with a removal notice to the extent they are capable of doing so.

The Act also provides for the notice and removal of certain class 2 material, namely material classified or likely classifiable as X 18+ or Category 2 restricted. The Commissioner may issue a notice to the relevant provider under ss 114 or 115. In this case, the location of the services or hosting is relevant. The Commissioner may only issue notices in relation to services provided from Australia, or content hosted within Australia. Pursuant to s 116, the provider must comply with the notice to the extent capable of doing so.

With respect to class 2 material which falls within the R 18+ or category 1 restricted classifications, the Commissioner has the power to give the provider a remedial notice under s 119. The notice may require the relevant provider to remove the material or ensure that the material is subject to a ‘restricted access system’. A restricted access system is an access-control system which the Commissioner declares to be a ‘restricted access system’. In essence, these are systems which limit the exposure of person under 18 to ‘age-inappropriate’ content online.

Under s 124, the Commissioner also has the power to issue notice to search engine providers requiring the provider to cease providing links to class 1 materials (a ‘link deletion notice’) in certain circumstances. Under s 128, the Commissioner may issue notice to an app distribution service provider to cease enabling end users in Australia to download an app that facilitates the posting of class 1 material (an ‘app removal notice’) in certain circumstances.

The online content scheme regulates material by reference to its classification. A separate part of the Online Safety Act 2021 (Cth) restricts who may hold an account with certain services, regardless of the material on them. That scheme is discussed at Social media minimum age below.

Basic Online Safety Expectations

The Online Safety Act provides for the Minister for Communications to make a determination (a form of legislative instrument) setting out basic online safety expectations. The basic online safety expectations are a set of minimum standards that online services are expected to meet in order to help keep their users safe.

The first determination was made in 2022. The Online Safety (Basic Online Safety Expectations) Determination 2022 specifies the basic online safety expectations for a social media service and other services that allow end users to access material using a carriage service or a service that delivers material by means of a carriage service.

The 2022 Determination was amended by the Online Safety (Basic Online Safety Expectations) Amendment Determination 2024, which was registered on 30 May 2024 and commenced the following day.⁶ The amendment responds to harms that have emerged since the first determination, including those associated with generative artificial intelligence, strengthens protections for children, and addresses gaps identified in the original determination. Among other changes, it inserted s 6(2A) into the 2022 Determination, adding an expectation that the provider of a service ‘will take reasonable steps to ensure that the best interests of the child are a primary consideration in the design and operation of any service that is likely to be accessed by children’.

Under s 49, the Commissioner may require the relevant providers to submit periodic reports on how they are meeting the expectations set out in the determination. These reports explain what steps the service has taken to protect users, such as blocking harmful content or improving safety tools. The Commissioner may also publish statements about the provider’s compliance or non-compliance with the expectations on its website.

A duty of digital care in Australia

Video overview of the proposed duty of care by Kaila Andrews

The schemes described above operate largely after the fact: they identify categories of material and provide for its removal once it has been posted and reported. An alternative approach places a positive, ongoing obligation on services to identify and mitigate the risks their systems create, enforced by a regulator against the service’s systems rather than against particular items of content.

The statutory review of the Online Safety Act 2021 (Cth) recommended that Australia adopt a duty of care of this kind, under which providers would be required to take reasonable steps to prevent foreseeable harms arising from the design and operation of their services.⁷ The Government indicated it would legislate a digital duty of care.⁸

The model is drawn from comparable schemes overseas — the systemic risk assessment obligations in the EU Digital Services Act, and the duties imposed on services by the Online Safety Act 2023 (UK), both discussed at Regulating content in other jurisdictions below.

A digital duty of care has been announced and recommended but, at the time of writing, has not been enacted. This section describes the proposal rather than operative law, and should be checked against the current legislative position.

Section 313 of the Telecommunications Act 1997 (Cth)

Video unavailable. Please help by creating a new video on Section 313.

In Australia, several different forms of pressure have been exercised in recent years to encourage intermediaries to take action to police the actions of their users. The most blunt is direct action by law enforcement agencies, who are empowered to make requests of telecommunications providers under s 313 of the Telecommunications Act. This provision requires carriers and carriage service providers to “do the carrier’s best or the provider’s best to prevent telecommunications networks and facilities from being used in, or in relation to, the commission of offences against the laws of the Commonwealth or of the States and Territories”, and to “give officers and authorities of the Commonwealth and of the States and Territories such help as is reasonably necessary” to enforce criminal law, impose pecuniary penalties, assist foreign law enforcement, protect the public revenue, and safeguard national security.

Video unavailable. Please help by creating a new video on How s 313 Is Used by Government Agencies to Block Websites.

Video unavailable. Please help by creating a new video on Explains.

The section essentially enables police and other law enforcement agencies to direct ISPs to hand over information about users and their communications. Increasingly, however, it is also apparently used by a number of government actors to require service providers to block access to content that appears to be unlawful, in cases ranging from the Australian Federal Police seeking to block access to child sexual abuse material to the Australian Securities and Investment Commission (ASIC) blocking access to phishing websites. Even the RSPCA is reported to have used the power, although the details of its request are not clear. There is significant concern over the lack of transparency around s 313(3) and lack of safeguards over its use.⁹ These came to the fore in 2013 when ASIC asked an ISP to block a particular IP address, not realising that the address was shared between up to 250,000 different websites, including the Melbourne Free University.

Image-based abuse

Video overview of image-based abuse laws by Danielle Harris

The non-consensual sharing of intimate images is often colloquially referred to as ‘revenge porn’. The term ‘image-based abuse’ is generally considered to be a better term because it avoids the victim-blaming connotations that the abuse is done in ‘revenge’ for some perceived wrong.

The National Statement of Principles Relating to the Criminalisation of the Non-consensual Sharing of Intimate Images encouraged each Australian jurisdiction to adopt nationally consistent criminal offences.

Under the Criminal Code Act 1995 (Cth), it is an offence to post, or threaten to post, non-consensual intimate images.¹⁰ Specifically, s 474.17 of the Criminal Code sets out an offence for the use of a carriage service in a way that reasonable persons would regard as being, in all the circumstances, menacing, harassing or offensive. Section 474.17A was substituted in 2024 and is now a standalone offence of using a carriage service to transmit sexual material depicting another person without that person’s consent; the definition of ‘private sexual material’, on which the former aggravated offence depended, was repealed at the same time. The 2024 offences are discussed at Criminal Code Amendment (Deepfake Sexual Material) Act 2024 (Cth) below.

Section 75 of the Online Safety Act prohibits the posting, or threatened posting, of an intimate image of another person without their consent. The prohibition applies where the person in the image or person posting the image are ordinarily resident in Australia. An ‘intimate image’ is defined to include images that depict genital or anal areas, a female, transgender or intersex person’s breasts, private activities such as showering, using the toiler or engaging in a sexual act not ordinarily done in public.

There is also a complaints-based system in the Online Safety Act, whereby the eSafety Commissioner may issue a removal notice or another civil remedy upon receipt of a victim’s complaint.

Queensland extended the definition of ‘intimate’ images to include original or photoshopped still or moving images of a person engaged in intimate sexual activity; a person’s bare genital or anal region; or a female, transgender or intersex person’s breasts.¹¹

The definition covers an image that has been altered to appear to show any of the above-mentioned things.

The State also introduced three new misdemeanours into their Criminal Code to broaden the scope of conduct which is captured under the offence. These include distributing intimate images without the consent of the person depicted,¹² observing or recording breaches of privacy,¹³ and distributing prohibited visual recordings.¹⁴

Social context and prevalence

Key statistics

The prevalence of image-based abuse was highlighted in a study conducted by the eSafety Commissioner in 2017 that found that 1 in 10 individuals experienced image-based abuse, with females aged between 15 to 17 years being most at risk. The report also found:

• 6 in 10 victims knew the perpetrator;
• The perpetrator was a friend that they knew offline (29%), an ex-partner (13%), a current partner (12%) or a family member (10%);
• Image-based abuse is more likely to occur on Facebook (53%); and
• Aboriginal and Torres Strait Islander people were more than twice as likely to have experienced image-based abuse as non-Indigenous people (25% compared with 11%).¹⁵

A separate survey by researchers at RMIT University and Monash University found that 1 in 2 Australians with a disability reported having been a victim of image-based abuse (56%, compared with 18% of respondents who did not report needing assistance with daily living, body movement or communication).¹⁶ Respondents with a disability were most likely to be targeted by someone they knew, and were more likely than other respondents to be targeted by a known person who was not a partner or former partner.

Factors affecting reporting

The eSafety Commissioner is empowered to investigate and make decisions regarding image-based abuse but this requires victims to report it. Studies have estimated that only 35% of cases of image-based abuse are reported. The factors influencing a victim not reporting can include:

• Negative stigma;
• Psychological barriers including victim blaming, humiliation and embarrassment;
• An unawareness of the severity of the incident;
• A fear of exacerbating or making it worse including attention to image-based abuse;
• Lack of confidence in law enforcement; and
• Unaware of the support services available.

Social prevention and response

Considering the social context of image-based abuse, there have been several actions taken by the Australian Government and other bodies to raise awareness and better educate individuals:

1. The Office of the eSafety Commissioner has developed a professional learning program for teachers and facilitators titled ‘Online Harmful Sexual Behaviours, Misinformation and Emerging Technology’. Its goal is to equip individuals with the necessary skills to identify and respond to incidences of image-based abuse and the role that coercion plays.

2. In 2020, the NSW Government launched a campaign to help prevent image-based abuse and educate individuals on the topic including information on where to seek help. This was largely due to the number of reports between 2019 to 2020, namely a 172% increase. The campaign also offered counselling to individuals and the removal of the content.

3. In 2022, the Australian Government responded to the report from the Senate titled ‘Phenomenon colloquially referred to as ‘revenge porn’’. The statement supported most of the recommendations proposed including that all police officers undertake mandatory training around image-based abuse.

Angelina Kardum explains: How the major social media platforms deal with image-based abuse

Most major social media sites now have policies against image-based abuse in their community guidelines or standards. Victims of image-based abuse can make a report directly to the site on which their intimate image was shared. This report is then assessed against the site’s community guidelines or standards and if it appears to be in violation of the community guidelines or standards, the image is generally removed within 24 hours. Whilst major social media services have taken positive steps towards tackling image-based abuse, such as developing reporting and take-down mechanisms, these mechanisms have their shortcomings. The two main issues with the current approaches taken by social media services are the delays associated with the assessment of reports and the heavy reliance on self-reporting.

The responses from online service providers to the issue of image-based abuse include:

1. In February 2015, Reddit updated its private policy to prohibit the publication of image-based abuse;
2. In March 2015, Twitter (now known as ‘X’) announced that they would immediately remove any link image-based abuse upon request; and
3. In June/July 2015, Google and Microsoft announced they would remove links upon request.

The Office of the e-Safety Commissioner

Lauren Trickey explains how to make a complaint to the eSafety Commissioner

The eSafety Commissioner is a statutory office which was first established by the Enhancing Online Safety Act 2015 (Cth) to promote and enhance online safety. The powers of the Commissioner were later enhanced in the Online Safety Act 2021 (Cth). While most of the Commissioner’s functions are contained in the Online Safety Act 2021, the Commissioner also has powers and functions under the Telecommunications Act 1997 (Cth) and the Criminal Code Act 1995 (Cth).

The Commissioner can receive reports for cyber-bulling, image-based abuse or offensive and illegal content.

Under s 30, complaints about cyberbullying of a child can be made by an Australian child or parent, guardian or person authorised by the child. An adult person can also make a complaint if they believe they were the target of cyberbullying material as a child, so long as the complaint is made within a reasonable time after they became aware and 6 months after they reached 18 years old. Cyberbullying refers to online material intended to seriously threaten, intimidate, harass or humiliate an Australian child.

The 2021 amendments introduced the world’s first legal scheme dealing with cyberbullying of adults. Under s 36, an Australian adult may make a complaint to the Commissioner about cyber-abuse material. Cyber-abuse material is material an ordinary reasonable person would conclude is likely intended to have an effect of causing serious harm to a particular Australian adult; and an ordinary reasonable person in the position of the Australian adult would regard the material as being, in all the circumstances, menacing, harassing or offensive.

Cyberbullying and hate speech

Young children and adolescents are being increasingly impacted by the high use of electronic devices and social media, resulting in bullying, exclusion and intimidation of young people.

Cyberbullying involves bullying online. It occurs where a perpetrator intentionally acts violently towards a victim repeatedly over a long period of time through a variety of social media platforms such as Facebook, Instagram, Snapchat or other online forums, often anonymously. It falls under the umbrella term ‘cyber hate’, which encompasses many types of harmful behaviours including hate speech, harassment, and discrimination targeting individuals based on their personal characteristics or identity.

The eSafety Commissioner’s Keeping Kids Safe Online survey, conducted between December 2024 and February 2025 with a nationally representative sample of 3,454 children aged 10 to 17, found that 53% had experienced cyberbullying at some point and 74% had seen or heard content associated with harm online. A quarter (25%) had experienced non-consensual tracking, monitoring or harassment, and 42% had seen or heard offensive or threatening material directed at others because of their identity. The survey also found that trans and gender-diverse children, and girls, were more likely than boys to have experienced cyberbullying and non-consensual tracking, monitoring or harassment.

Adult Cyber Abuse Scheme

Part 7 of the Online Safety Act 2021 (Cth) establishes an Adult Cyber Abuse Scheme, the first in the world. This scheme provides the eSafety Commissioner with the power to issue service providers with a formal notice to remove harmful content targeting an Australian adult within 24 hours. The provider could incur civil penalties and fines if they fail to remove the harmful content. Section 162 of the Online Safety Act gives the Commissioner the powers to seek these penalties. However, there is a high threshold to be satisfied before the eSafety Commissioner has the authority to act:

• Section 7(1)(c) requires that the material was “intended to have an effect of causing serious harm”; and
• Section 7(1)(d) requires that “an ordinary reasonable person in the position of the Australian adult would regard the material as being, in all the circumstances, menacing, harassing or offensive.”

There is question about what surpasses the threshold and what is of a ‘serious’ nature as the harm can be subjective and arbitrary.

The threshold was considered in X Corp and Elston v eSafety Commissioner, in which the Administrative Review Tribunal set aside a removal notice on the basis that the material was not cyber-abuse material within the meaning of s 7.¹⁷ The Tribunal held that material which is merely offensive or distressing does not meet the requirement that it be intended to cause serious harm, and cautioned against an interpretation of the provision that would capture lawful expression.

This is a recent first-instance merits-review decision and may be subject to appeal. It should not be treated as a settled statement of the law.

As outlined above, image based-abuse complaints can be made to the Commissioner. Pursuant to s 32, complaints can be made by the person in the intimate image, a person authorised to make a report or a parent or guardian of a child or a person who does not have capacity.

Australian residents can also report offensive or illegal content, which includes abhorrent violent material or material depicting illegal acts.

For each type of material an online form can be completed on the eSafety website. Each form requests information regarding what is contained or depicted in the material and where the material has been posted. After receiving a complaint, the Commissioner has the power to conduct an investigation (as the Commissioner thinks fit). The Commissioner assesses the material complained of to determine the appropriate course of action, which may include liaising with the relevant platform for the material to be removed.

The Challenge of Online Abuse: Trolling

What is Trolling?

Trolling takes many forms. Trolls typically respond to or post inflammatory, off-topic, or ludicrous material to generate an emotional response from users. This behaviour can lead to a pile-on effect, where others join in on the attack. Many Australian users experience online trolling. The most common platforms for such encounters are Instagram, YouTube, and Snapchat. On these platforms, trolling is often confused with cyberbullying.

In Australia, it is up to the individual to report to the online service first. This typically involves the user collecting evidence, such as screenshotting abusive comments and reporting the troll within the app. When the service does not remove the content within 48 hours, an individual can report to eSafety if their experience meets the legal threshold of serious cyberbullying. A key message to internet users on platforms is to not feed into the trolls and report the abuse within the used app.

Social media platforms can be reluctant to change bullying policies as they try to balance users’ freedom of speech, privacy, and protection. Private intermediaries that do not make in-app regulatory changes, such as social media platforms, continue to amplify the voices of trolls.

Trolling and the Law

Compared with other online antisocial behaviour, such as cyberbullying, trolling remains largely unregulated within the legal framework of Australia. There are fundamental differences between cyberbullying and trolling behaviours regarding form, content, intent, and consequence. These differences are not reflected in the Online Safety Act 2021 (Cth), which has formed a world-first cyber abuse scheme for adult Australians and introduced new basic online safety expectations to promote and improve online safety. Although researchers have deemed cyberbullying and trolling to be different behaviours, the Online Safety Act 2021 (Cth) does not specifically protect the safety of users from being trolled, and no other legislation exists that specifically targets trolling in Australia.

To have access to a legal remedy, Australian residents seeking justice for being trolled on platforms need to fall within legal provisions in the Online Safety Act 2021 (Cth) and Criminal Code Act 1995 (Cth) that regulate cyberbullying, harassment, image-based abuse, or offensive and illegal content. Trolling can, but doesn’t always, fall within these definitions.

If the complaint fits within the criteria, a complaint can be made to the Commissioner about the matter. Criteria includes:

• Under s 30 of the Online Safety Act, an Australian child who has reason to believe they are a target of cyber-bullying material on platforms is a justified matter;
• An Australian adult under s 36 can make a complaint if they believe they have been a target of cyber-abuse material; and
• Under s 474.17 of the Criminal Code Act, an offence is set out in the way a ‘reasonable person would regard as being, in all the circumstances, menacing, harassing or offensive’.

Both the Online Safety Act 2021 (Cth) and Criminal Code Act 1995 (Cth) address cyberbullying and harassment broadly and require modifications to address trolling effectively.

Social Media (Anti-Trolling) Bill

The federal government introduced an exposure draft on Social Media (Anti-Trolling) Bill 2021 (Cth) shortly after the decision in Fairfax Media Publications Pty Ltd v Voller. The High Court found that media companies can be held responsible for alleged defamatory third-party comments made on Facebook accounts of media companies. The Bill intended to address defamatory comments by exposing anonymous commenters through platforms obtaining their contact details. The Bill established a limited role on trolling issues and lapsed at the dissolution of Parliament in April 2022.

Self-help responses and their limits

Where the statutory thresholds are not met, targets of trolling are left to the remedies they can arrange themselves. The limits of that position were illustrated in 2025.

Identifying anonymous accounts: Indy Clinton (2025)

In June 2025, Australian content creator Indy Clinton engaged a licensed private investigator to identify anonymous accounts that had been targeting her over a period of years. The investigator produced a report identifying the people behind a number of the accounts, which Clinton discussed publicly.¹⁸ Many were found to be operated by people who were not anonymous strangers in any meaningful sense.

The investigator noted that a licensed investigator may lawfully conduct surveillance that would not be available to a private individual, and that identified account holders may expose themselves to liability, including in defamation.¹⁹

Reaction was divided: some commentators treated the exercise as a demonstration that anonymity online is weaker than users assume, while others raised concerns about a public figure directing attention towards identified individuals.

The case is instructive less for what it achieved than for what it reveals about the distribution of remedies. Clinton’s approach depended on the cost of engaging an investigator, the time to pursue it, and the existence of a potential cause of action in defamation — a combination available to very few people. The statutory schemes described above are addressed to material that meets a seriousness threshold; conduct falling below it, however sustained, is left to platform reporting tools and to whatever the target can arrange privately.²⁰

Cyberbullying and the queer community

LGBTQIA+ internet users experience online abuse at higher rates than cisgender and heterosexual users. In 2020 the eSafety Commissioner identified LGBTQIA+ people as a group at increased risk of online abuse, ranging from dismissive comments and scams through to severe harassment and sustained targeting.²¹ Low-level micro-aggressions directed at queer users frequently go unreported until they escalate.

The text alternative below reproduces the content of the infographic in full, so that the material is available to readers using a screen reader or a text-only view.

The second page of this infographic (a case study on dating app facilitated sexual violence) has been removed from this chapter: the graphic stated prevalence figures and an attribution that could not be verified against the research it cited. Its content, with corrected figures and sources, is incorporated in the text below.

Terms used

• Cyberbullying: the use of technology to bully a person with the intent to hurt or intimidate.
• Cyber abuse: behaviour that intentionally uses technology to threaten, harass or humiliate, with intent to cause social, psychological or physical harm.
• Cissexism: the belief that being cisgender — identifying with the gender assigned at birth — is natural and superior to other gender identities.
• Heterosexism: the belief that the world is heterosexual, and that social customs and behaviours should conform to that assumption.
• Unconscious bias: a belief held by an individual of which they are not personally aware.

Prevalence

Reported research indicates that LGBTQIA+ students experience cyberbullying at roughly twice the rate of their heterosexual peers (36.1% compared with 20.1%),²² and that 30% of LGBTQI adults were the target of online hate speech in the 12 months to August 2019, compared with a national average of 14% of adults aged 18 to 65.²³

The legislative framework

Australia (Commonwealth). The Online Safety Act 2021 (Cth) establishes the notice regime described earlier in this chapter, under which the eSafety Commissioner may require providers and end-users to remove cyberbullying material and, in some cases, restrict users. The Act protects Australian adults and children generally; it does not identify cissexism or heterosexism as specific harms.

The Sex Discrimination Act 1984 (Cth) prohibits discrimination on the basis of sexual orientation (s 5A) and gender identity (s 5B).²⁴ It is a general anti-discrimination framework and does not address cyberbullying directly.

Case study: dating app facilitated sexual violence

Dating app facilitated sexual violence

Dating app facilitated sexual violence refers to sexual aggression, harassment and other violence occurring through communication with users on dating apps, including violence arranged through an app and then carried out in person. It can include image-based abuse — unsolicited sexual images, pressure to share images, and sharing images with third parties without consent — abusive or threatening messages, and in-person abuse arranged through the app.

Research reports that around three in four Australians have experienced abuse on dating apps in the preceding five years,²⁵ and that prevalence of online dating app facilitated sexual violence differs sharply by gender and sexuality: LGB+ women reported the highest prevalence (86.6%), then non-binary respondents (84.5%) and LGB+ men (79.2%), followed by heterosexual women (79.1%) and heterosexual men (61.5%).²⁶ Among the 102 participants in a 2023 Australian focus group study, the most used dating apps were Tinder (27%), Bumble (21%), Grindr (8%), Hinge (6%), eHarmony (6%) and Plenty of Fish (2%).²⁷

New South Wales Police laid 39 charges in relation to 44 incidents reported to them from January 2024, in which victims were robbed, assaulted or extorted after meeting offenders through dating apps.²⁸

Victoria Police’s LGBTIQA+ Communities Portfolio Manager has described a pattern of ‘posting and boasting’, in which victims of assaults arranged through dating apps are filmed and the video posted to social media, the recording then being used in a shaming process that includes threatening to out the victim.²⁸

Industry response

In October 2024 the LGBTQIA+ organisation ACON and NSW Police jointly urged users to take precautions after a ‘high number’ of violent assaults across New South Wales arranged through gay dating and hook-up apps, with guidance on meeting someone for the first time.²⁹

In October 2024 a number of dating platforms, including Grindr, developed the Online Dating Code of Practice in consultation with the Australian Government.³⁰ The Code requires participating platforms to implement systems to detect dating app facilitated sexual violence, act against perpetrators, and improve complaint and reporting mechanisms. It became subject to enforcement in 2025.³¹ Two limitations are apparent: it is a voluntary industry code, which constrains its enforceability against platforms that do not participate, and it does not address the disproportionate effect of this violence on queer users.

Technology-facilitated abuse

Technology-facilitated abuse is the use of technology to harm or control another person. It arises most often in the context of domestic, family and sexual violence, and includes harassing or threatening a person online, sharing or threatening to share intimate images, cyberstalking, monitoring a person’s communications and movements, and restricting a person’s access to their accounts or devices.³²

Technology also enables coercive control — patterns of manipulation, pressure and fear used to control a partner or family member. In a technological setting this includes tracking and monitoring, cutting a person off from online support networks, and isolating or gaslighting them.³³

Technology-facilitated abuse is not a marginal phenomenon. In a 2020 survey of 442 domestic and family violence practitioners, 99.3% reported having clients who had experienced technology-facilitated stalking and abuse.³⁴ Reported prevalence increased with the shift to online communication during the COVID-19 pandemic. It is identified as a priority in the National Plan to End Violence against Women and Children 2022–2032.³⁵

The harm is unevenly distributed. Women experience technology-facilitated abuse at substantially higher rates than men, consistent with the gendered pattern of domestic and family violence generally.³⁶ The eSafety Commissioner’s research indicates that one in four children aged 10 to 17 has experienced non-consensual tracking, monitoring or harassment.³⁷

Tracking and location sharing

Consumer location-sharing features are a common vector. The eSafety Commissioner has observed that tracking another person is frequently framed as care but functions as control.³⁸

Inquest into the death of Lilie James (2023–25)

In October 2023, Lilie James was murdered at a school in Sydney by a man with whom she had had a brief relationship. Evidence before the inquest was that the perpetrator had used a social media application’s location-sharing feature to monitor her movements, and had become aware through it that she was at a gathering. He had a history of controlling behaviour towards a former partner, including anger at her refusal to share her location, and had shown intimate images of Ms James to others.³⁹

Expert evidence at the inquest characterised this conduct as technology-facilitated abuse, and described location monitoring as a recognised form of digital coercive control.⁴⁰

New South Wales. In December 2024, the definition of stalking in s 8 of the Crimes (Domestic and Personal Violence) Act 2007 (NSW) was expanded to include ‘the monitoring or tracking of a person’s activities, communications or movements’, whether by using technology or in another way. The offence of stalking or intimidation with intent to cause fear of physical or mental harm is in s 13.⁴¹

The role of the eSafety Commissioner

The eSafety Commissioner receives reports of adult cyber abuse and image-based abuse under the schemes described above, and can compel removal of material, publish the names of non-compliant services, and seek civil penalties.⁴² The Commissioner also supports online safety planning for people at risk of, or experiencing, technology-facilitated abuse.

The Commissioner’s role in this area has attracted criticism. Removal of individual items of content has been described as an incomplete response, on the basis that it addresses the artefact rather than the pattern of control and provides limited assistance to the person affected.⁴³ It has been argued that the stronger statutory powers available under the Online Safety Act 2021 (Cth) are used sparingly, and that the scheme does little to produce systemic change in platform design.⁴⁴ Research has also identified that frontline services do not consistently treat technology-facilitated abuse as a serious form of abuse.⁴⁵ Separate work has argued that responses to technology-facilitated abuse in Aboriginal and Torres Strait Islander communities require Indigenous-led research and design rather than the application of general frameworks.⁴⁶

Proposals for improvement have centred on education and resourcing: better training for frontline workers, and material directed at younger people addressing digital coercive control and expectations of privacy and consent within relationships.³⁸

Abhorrent Violent Material

See overview by Georgie Vine about the Criminal Code Amendment (Sharing of Abhorrent Violent Material) Act 2019

How Abhorrent Violent Material emerged

In response to the Christchurch massacre, the Australian government passed an amendment to the Criminal Code Act 1995 (Cth) in 2019 to regulate abhorrent material.

In Christchurch, New Zealand on 15 March 2019 an Australian gunman, Brenton Tarrant, entered two Mosques massacring 51 individuals and injuring an additional 49. Tarrant wore a GoPro attached to his head, recording and live streaming the entire massacre to his personal Facebook page. The live stream lasted 17 minutes and was initially viewed by 4,000 users but was later re-produced and shared in 1.5 million videos within 24 hours of the incident. It took approximately one hour from the start of the streaming for Facebook to remove the original video off the platform.

Definition

The Criminal Code Amendment (Sharing of Abhorrent Violent Material) Act 2019 creates new offences under the Criminal Code Act 1995 (Cth), effective from 6 April 2019.

Abhorrent violent material is defined by section 474.31 of the Criminal Code Amendment Act (Sharing of Abhorrent Violent Material) Act 2019 (Cth) as audio, visual, or audio-visual material that records or streams abhorrent violent conduct being engaged in by one or more persons. Section 474.32 classifies that a person engages in abhorrent violent conduct if the person:

a) engages in a terrorist act; b) murders another person; c) attempts to murder another person; d) tortures another person; e) rapes another person; or f) kidnaps another person.

These offences target content that is reasonably capable of being accessed within Australia, regardless of where the material was created or where the platform operator is located. The offences include substantial penalties if individuals and companies do not remove or report such classified material: fines up to 10% of annual global turnover for companies, and up to 3 years imprisonment for individuals.

There are a few defences to the new offences, including:

• material necessary for law enforcement,
• material distributed by journalists,
• material used for scientific, medical, academic or historical research, and
• the exhibition of artistic works.

Criticisms of the AVM Act

The AVM Act has been criticised for drafting deficiencies. The AVM Act came into effect on 6 April 2019, just under a month after the Christchurch Massacre. In the Second Reading of the AVM Bill, Mr Bandt, then leader of the Australian Greens, in highlighting the government’s haste to pass the Bill noted that it may undermine the legitimacy of the Bill further down the track and there could be unintended consequences. The President of the Law Council of Australia echoed this sentiment, describing the legislation “knee jerk reaction to tragic event”.

“As soon as reasonably possible”

Under section 474.33(c) of the AVM Act, it is an offence for a person who has reasonable grounds to believe that certain material is abhorrent violent material, and fails to report the details to the Australian Federal Police within a “reasonable time” after becoming aware of its existence. The AVM Act does not specify what will constitute a “reasonable time”. However, the Attorney-General deemed it “unacceptable” that the Christchurch Massacre livestream was available for over an hour before first attempts were taken to remove the content. This points to the conclusion that it is likely that the timeframes would be measured in hours and minutes rather than days. The United Nations has expressed concerns that the wording of “reasonable time” is ambiguous and in practice will cause hasty decisions made by content services or hosting services.

The Online Safety Act 2021 (Cth)

The eSafety Commissioner has wide-ranging powers under the Online Safety Act 2021 (Cth) to order the removal of certain online material.

Under section 109, the eSafety Commissioner can issue a removal notice when satisfied the material is or was class 1 material; the material can be accessed by end-users in Australia, and that once issued a removal notice the service provider must take ‘all reasonable steps’ to ensure removal of the material.

Under section 95, the eSafety Commissioner can issue a provider with a blocking request, requesting that they take steps to disable access to the material if the: a) material can be accessed using an internet carriage service supplied by an internet service provider; b) Commissioner is satisfied that the material that promotes, instructs, incites or depicts abhorrent violent conduct; and c) Commissioner is satisfied that the availability of the material online is likely to cause significant harm to the Australian community.

Disguised Content and Evasion Techniques

Needs review for fit and accuracy. I don’t think ‘disguised content’ or ‘splicing’ are known phrases, and evasion isn’t as simple as this section suggests – nic.

Despite content regulation efforts, some individuals and organisations have used coercive and nefarious tactics to share prohibited imagery and videos online, especially aimed at those most vulnerable, children. Disguised content refers to harmful content that seeks to evade online content regulation or filtering mechanisms.

TikTok, YouTube and ‘Splicing’

In January 2021, users of TikTok, one of the world’s most popular social media sites, were exposed to a viral video containing a graphic depiction of bodily mutilation and the recorded death of an individual. Although TikTok strictly prohibits content of this nature, the video was able to be shared globally due to the original poster ‘splicing’ the clip of the violent crime behind a video of a girl dancing — allowing the upload to ‘trick’ the application’s artificial intelligence algorithm into thinking that it was a normal video.

In 2019, parents became aware of a similar issue when YouTube hosted popular videos that at first showed children’s content such as Peppa Pig that were found to be spliced with videos of a disfigured humanoid character called ‘Momo’ repeating inappropriate phrases and showing videos of cartoon characters being tortured. Another example involving YouTube involved children being exposed to spliced videos that gave explicit instructions on how to inflict self-harm.

Other manners of disguising content have begun to emerge. In August 2024, a new trend arose that involved nefarious creators uploading videos where the first few seconds appear relatively normal and fun before they begin to video themselves exposing themselves on camera or showing disturbing content.

Disguised Content and the Online Safety Act

In Australia the primary instrument that instructs online content regulation is the Online Safety Act 2021 (Cth). This Act works to classify online content in conjunction with the National Classification Code into class 1 material and class 2 material under section 106 and 107. The Act utilises these classification systems as a way of directing enforcement, with sections 109-128 stipulating that the publication of harmful material against the regulations of the Act be followed by orders of deletion or the removal of content or platforms in extreme circumstances. However, these measures have been criticised for focusing too heavily on content removal rather than prevention of publication.

Disguised content is often class 1 material. Due to creators evading the regulatory algorithms on platforms like YouTube, these videos remain online until a substantial number of reports are made, and the videos are manually reviewed or flagged as class 1 material, by which point these videos have been widely circulated, causing harm. Exacerbating this issue is that the current laws are limited by geographical jurisdiction and therefore cannot effectively govern ‘cloud computing’ or the uploading or creation of disguised content from locations outside of Australia particularly with the more prevalent use of proxy servers such as VPNs that can hide the location of a poster.

Another issue around enforcement is that creators and uploaders of disguised content often are anonymous, undetectable and in viral circumstances, numerous due to re-uploads. This is also problematic for enforcing the new Criminal Code Amendment (Sharing of Abhorrent Violent Material) Act 2019 (Cth) provisions that criminalises the acts of uploading harmful material.

Case Study: eSafety Commissioner v X Corp [2024] FCA 499

The recent case of eSafety Commissioner v X Corp [2024] FCA 499 tested the powers and limits of the Online Safety Act in relation to the Commissioner’s powers under section 109.

Facts

On 15 April 2024, in the Sydney suburb of Wakeley, Bishop Mar Mari Emmanuel was attacked and stabbed while delivering a service that was being live streamed by the church. The Bishop, along with a priest, a member of the congregation, and the suspected attacker, all sustained injuries in the incident.

The footage that was captured through the live stream was available instantly. Following its online livestream, the footage was reduced to a shorter clip of roughly 11 seconds, which depicted the suspected attacker approaching the Bishop and striking him several times in a downward motion. No highly graphic details were visible (i.e., no blood or wounds), however the audio captured the sounds of the impacts between the weapon and the Bishop, and the shocked and distressed reactions of witnesses can be heard. The clip was subsequently shared across various mainstream social media sites, including X (formerly Twitter).

The Commissioner’s Response

The day following the incident, the eSafety Commissioner deemed the clip as ‘class 1 material’, and through invoking her powers pursuant to section 109 of the Act, issued a formal notice to X Corp requiring them to take all reasonable steps to remove the footage from their platforms. The notice did not apply to all copies of the footage, but rather only to a set of 65 links which contained footage of the incident and were posted to X. The justification being that the material was deemed to be of class 1 classification under the Act, which depicted ‘crime, cruelty and real violence’, such that it ‘offends against the standards of morality, decency and propriety generally accepted by reasonable adults’.

X subsequently responded to the notice by geo blocking each of the specified posts for Australian users; meaning that users who had their IP address set to an Australian location, were unable to access the content. However, the eSafety Commissioner was not satisfied that X’s response in geo blocking the material constituted compliance with the notice, as Australians could still access the URLs via a VPN.

Proceedings

On 22 April 2024, the Commissioner commenced proceedings in the Federal Court, seeking injunctive relief that would require X to remove the material from its platform or make them inaccessible to all users. The eSafety Commissioner was successful - at least temporarily - as the Federal Court ordered an interim injunction against X, which remained in effect until 13 May. The injunction required X to hide the material identified by the Commissioner behind a separate notice, such that X users would instead only see the notice blocking the material (rather than the material itself), which could not be removed.

On 13 May 2024, the Federal Court of Australia handed down its judgment. In delivering the judgment, his Honour Justice Kennett considered two key issues, including:

1. whether the removal notice was a valid exercise of the eSafety Commissioner’s power under section 109 of the Act; and
2. whether, given the notice only requires X to take “reasonable steps” to ensure the removal of the material, the proposed final injunction goes further than what is required for compliance with the notice.

In relation to the second issue, at the core of the dispute was that the eSafety Commissioner argued it was insufficient for X to simply ‘geo-block’ the material for Australian users, and that the 65 URLs should be removed from the platform altogether. The Commissioner argued that such action is within the “all reasonable steps” that the notice required to be taken. X argued that a requirement to remove the material worldwide, goes beyond what could be considered “reasonable”.

Justice Kennett held that, read in context, the ‘reasonable steps’ required by a removal notice issued under s 109 do not include the steps the Commissioner sought to compel — blocking access to the 65 URLs for all users of X worldwide — and that the Commissioner had therefore not established a prima facie case for the final injunction sought ([2024] FCA 499, [53]–[54]). The injunction was thus refused. Further observations were made that, had the injunction been ordered, it would have:

• Had a ‘global effect’, impacting X and many other organisations who have no real connection to Australia or its interests.
• Impacted the interests of individuals globally who have no connection to the proceedings.
• Been ineffective in preventing people from watching the video elsewhere.
• Been highly unlikely to be enforced by a US court.

On 5 June 2024, the eSafety Commissioner discontinued the proceedings in the Federal Court.

The Federal Court has deemed this case to be of ‘public interest’, meaning that an almost complete public record of the documentation can be accessed here.

Social media minimum age

Video overview of the social media minimum age scheme by Olivia Stanley

The Online Safety Amendment (Social Media Minimum Age) Act 2024 (Cth) inserted a minimum age requirement into the Online Safety Act 2021 (Cth). From 10 December 2025, providers of age-restricted social media platforms must take reasonable steps to prevent Australians under 16 from creating or holding an account.⁴⁷ It was the first measure of its kind in any jurisdiction.

Design of the scheme

The obligation is placed on the provider, not on the child or the parent. Neither a child who holds an account nor their parents commit an offence or incur a penalty. The eSafety Commissioner has characterised the measure as a delay rather than a prohibition, on the basis that it defers access rather than removing it permanently, and locates responsibility with platforms.⁴⁸

The stated rationale is protective. The Government presented the measure as a response to platforms that were designed for adults but are used by children, and to the accumulation of research on the effects of social media use on young people’s wellbeing.⁴⁹

The penalty for failing to take reasonable steps is set out at Current regulatory framework above.

Evidence relied on

eSafety research published in 2025 found that 96% of children aged 10 to 15 had used social media, that 71% had ever seen or heard content associated with harm online, and that more than half of children aged 13 to 15 had experienced cyberbullying. One in seven reported experiencing grooming-like behaviour from an adult or from a child at least four years older.⁵⁰

Separately, the Australian Institute of Health and Welfare reports higher levels of psychological distress among young Australians than among older cohorts, with the highest levels among young women.⁵¹

Which services are covered

A service is an age-restricted social media platform where its sole or a significant purpose is to enable online social interaction between two or more end-users, it allows users to link to or interact with other users, it allows users to post material, and it satisfies ‘such other conditions (if any) as are set out in the legislative rules’.⁵² From 26 March 2026 that fourth condition is populated: the service must have either or both of a ‘recommender feature’ — the ability to select material by reference to information associated with a user’s account and display it to that user — and a ‘logged-in feature’, being an endless-feed, feedback or time-limited feature that is available only to account holders.⁵³

In November 2025 the eSafety Commissioner published a list of ten services it considered were likely to be age-restricted social media platforms: Facebook, Instagram, Kick, Reddit, Snapchat, Threads, TikTok, Twitch, X and YouTube.⁵⁴

The Online Safety (Age-Restricted Social Media Platforms) Rules 2025 (Cth) also exclude a number of services, on the basis that young people require continued access to services supporting education, health and communication.⁵⁵ The exclusions are threshold-based rather than categorical: a service is excluded only where its ‘sole or primary purpose’ is messaging, email or voice or video calling; online games; sharing information such as reviews, technical support or advice about products or services; professional networking or development; supporting the education of end-users; or supporting the health of end-users. Two further classes are excluded where the service has a ‘significant purpose’ of facilitating communication between educational institutions and students or their families, or between health care providers and the people using their services.

Age assurance

Compliance requires some means of establishing age. ‘Age assurance’ covers a range of methods, from those producing a high degree of certainty to those producing only an estimate or a range.⁵⁶ Platforms have adopted a range of methods for users who are affected by the restriction but say they are 16 or older, including facial age estimation from a video selfie assessed by a third-party provider, photo identification, and bank-verified digital identity.⁵⁷

The Australian Government funded an Age Assurance Technology Trial in 2024–25. The trial concluded that age assurance can be implemented in Australia, while identifying significant limitations. There is no single technology that works across all cases; systems need fallback options, such as identity document checks, for cases where estimation fails. On demographic performance the trial reported broad consistency: the systems tested ‘performed broadly consistently across demographic groups assessed’, and, despite an acknowledged deficit of training data about Indigenous populations, the trial found ‘no substantial difference in the outcomes for First Nations and Torres Strait Islander Peoples and other multi-cultural communities’, with ‘variances across race’ not deviating ‘by more than recognised tolerances’. It recorded one qualified exception: while systems generally performed well across diverse user groups, ‘some showed reduced accuracy for older adults, non-Caucasian users and female-presenting individuals near policy thresholds’.⁵⁸

The trial’s finding of reduced accuracy in some systems, for some groups of users, near policy thresholds, is a deployment risk rather than a finding of systemic bias: where a particular system performs less well near a threshold, the burden of failed age estimation, and of falling back to identity document checks, will not fall evenly. That has consequences both for access to services and for the volume of identity information collected from affected users. The trial’s overall finding was one of broad demographic consistency, and the position should be checked against the outcomes of deployed systems as they are reported.

The Act restricts what platforms may do with information collected for age assurance: it must be destroyed once used for that purpose, and must not be used for any other purpose unless the user consents, with consent required to be current, informed, voluntary, specific and unambiguous.⁵⁹

Criticism

Proportionality and human rights

The Australian Human Rights Commission (AHRC) recommended that the Bill not pass in its then form.⁶⁰

Under human rights law, a limitation on a right must be necessary and proportionate to a legitimate aim, which the AHRC treats as requiring the least restrictive option capable of achieving the purpose.⁶¹ The AHRC accepted that protecting children from harm associated with social media is a legitimate aim, consistent with art 17 of the Convention on the Rights of the Child, which contemplates guidelines protecting children from material injurious to their wellbeing.⁶²

Its objection was to the means. The AHRC argued that social media provides young people with access to inclusion and to information that may be critical or life-saving, and that a blanket restriction is disproportionate to the aim. It identified tension with the rights recognised in arts 13 and 17 of the Convention — freedom of expression, and access to information from a diversity of sources. The UN Committee on the Rights of the Child has said that content moderation and controls should prevent harmful material reaching children rather than restrict children’s access to information in the digital environment.⁶³

Effect on vulnerable groups

The effect of exclusion is not uniform across the affected age group. For LGBTQIA+ young people in particular, online spaces provide connection and a means of exploring identity that may not be available offline, where they may face marginalisation or lack of acceptance in families, schools and communities.⁶⁴ Those spaces carry real risks, including the cyberbullying and abuse documented earlier in this chapter. The difficulty is that the same platforms are the source of both the risk and the support, and a measure directed at the first necessarily removes the second.

The scheme’s effects will not be known until it has operated for some time. The Act requires the Minister to cause an independent review of the operation of pt 4A to be conducted within two years after the minimum age obligation took effect.⁶⁵

Regulating content in other jurisdictions

Snoot Boot explains France and Germany’s online hate speech laws

Social media regulation in Brazil

The Civil Framework of the Internet governs internet regulation in Brazil. It outlines the fundamental rights that internet users have, including freedom of expression, privacy protection, and preservation of net neutrality.

The Brazilian Government has prioritised combatting misinformation and online extremism, which increased significantly during Bolsonaro’s administration, and in the lead up to the 2018 and 2022 presidential elections. This came to a head on 8 January 2023, when thousands of Bolsonaro’s supporters stormed various government buildings in Brasilia following President Lula’s victory in the 2022 federal election. Similar to the US Capitol riot in 2020, social media platforms were used to spread misinformation about a ‘stolen election’ and organise the attack. A study conducted by Ozowa, Lukito, Bailez, and Fakhouri found that Twitter and WhatsApp were heavily used by right-wing extremists to spread propaganda and conspiracy theories and instigate violence.

Following the attack, the Supreme Court began investigating the incident and putting more pressure on social media platforms to filter hate speech and misinformation. Part of this involved introducing a Bill to combat misinformation, which ultimately failed after several platforms campaigned against its introduction.

Elon Musk and X under investigation

As part of the investigation led by Supreme Court Justice Alexandre de Moraes, in early April 2024, X was ordered to block several accounts that were accused of spreading misinformation. Elon Musk refused to do so, and in a series of posts, accused Moraes and the Brazilian Government of censorship and threatened to lift all platform restrictions on X. Moraes then commenced Inquiry No. 4957 under article 12 of the Civil Framework of the Internet, which allows the court to investigate internet-based obstruction of justice and criminal acts. The investigation was also concerned with leaked internal emails from X, which contained criticisms of the Superior Electoral Court and its decisions.

On 17 August 2024, X closed its office in Brazil and removed its legal representative. This contravened article 1134 of the Civil Code, which requires a foreign company operating in the country to have a legal representative. As a result of the inquiry, Moraes issued a summons to X by posting it on the platform.

At page 23 of Moraes’s judgement, he found that ‘Musk confuses freedom of speech with a non-existent right to aggression, and deliberately confuses censorship with a constitutional prohibition against hate-speech and incitement of antidemocratic action’. Moraes also referenced the proceedings against X in Australia at pages 28-29 to find that X has a history of not cooperating with governments and judicial orders, and the platform is regularly involved in antidemocratic action.

Moraes ultimately ruled against X and ordered the platform to be blocked in Brazil due to its incitement of extremist activity, obstruction of justice, and lack of legal representation in the country. This decision was later affirmed by the other members of the Supreme Court. X has also been ordered to pay fines valuing at least 20 million Reais (roughly $5 million AUD at 2024 exchange rates), and Musk’s Starlink assets in the country were frozen.

Implications of the Brazilian decision

On 31 August 2024, X was blocked in Brazil and inaccessible to millions of users. Initially Moraes attempted to outlaw the use of VPNs in the country entirely, but instead the court ordered that users in Brazil who use a VPN specifically to access X face fines of up to 50,000 Reais (roughly $13,000 AUD at 2024 exchange rates). This has caused debate within the country, with activists calling for Moraes to reconsider fining users.

After X was blocked, rival platform BlueSky saw an increase of 2 million members in a week, and Meta’s Threads also saw a significant increase in activity.

Despite the verdict, there is still a possibility that X can be reinstated, provided they comply with court orders and pay the accumulated fines.

Both France and Germany have attempted stricter approaches to regulating online hate speech. In particular, Germany’s laws require social media companies to remove hate speech and report users to the police, or else face significant fines. In 2020, France proposed laws similar to Germany’s, but these laws were struck down by a French court, as the laws were unconstitutional and imposed an unreasonable burden on the freedom of speech because they incentivized over-censorship. The laws highlight a deeper tension between free speech and the need to regulate and censor hateful ideologies being spread online.

United Kingdom

The United Kingdom’s Online Safety Act 2023 represents a comprehensive approach to platform regulation that extends beyond Australia’s co-regulatory model. The Act imposes direct duties on social media platforms and search engines to protect users from harmful content. The Office of Communications (Ofcom) is implementing the Act through codes of practice addressing illegal content, content harmful to children, and specific categorised services.⁶⁶

The UK Act introduces several new criminal offences that address gaps in existing law:

• Intimate image abuse (s 188) - criminalising the non-consensual sharing of intimate images, similar to Australia’s image-based abuse provisions
• Epilepsy trolling (s 183) - targeting those who send content designed to trigger seizures
• False communications (s 179) - prohibiting the sending of false information intended to cause non-trivial harm
• Threatening communications (s 181) - modernising threats law for digital contexts
• Encouraging self-harm (s 184) - addressing content that encourages or assists serious self-harm
• Cyberflashing (s 187) - criminalising the unsolicited sending of sexual images

These offences demonstrate how jurisdictions are identifying and addressing specific online harms through targeted criminal law provisions, complementing broader platform regulation approaches.

Canada

Canada’s proposed Online Harms Bill 2024 takes a duty-based approach to platform regulation.⁶⁷ The Bill would impose three primary duties on social media services:

1. Duty to act responsibly - requiring platforms to implement systems to address harmful content
2. Duty to protect children - specific obligations regarding content accessible to minors
3. Duty to make certain content inaccessible, specifically:
   • Content that sexually victimises a child or revictimises a survivor
   • Intimate images posted without consent

This approach emphasises proactive obligations on platforms rather than reactive content removal, reflecting an emerging trend in online safety regulation.⁶⁸

The Bill did not proceed. It lapsed when Parliament was prorogued in early 2025 following Justin Trudeau’s resignation as Prime Minister, the second occasion on which an attempt to legislate on online harms in Canada has failed. The Government of Canada has since indicated that it intends to bring forward online harms legislation for a third time, with a revised approach that separates the criminal law measures from the platform duties and takes account of developments in generative artificial intelligence.⁶⁹

European Union

European Union. The Digital Services Act (DSA) is the European Union’s framework for regulating online intermediaries. It entered into force in 2022 and became fully applicable in February 2024.⁷⁰ It replaces a patchwork of national rules with a single set of obligations applying directly to providers of intermediary services, online platforms and search engines offering services in the EU market.

Structure

The DSA imposes a tiered set of obligations. Baseline duties apply to all intermediary service providers, with progressively more demanding requirements for online platforms, and the most extensive requirements reserved for Very Large Online Platforms and Very Large Online Search Engines — those with more than 45 million average monthly recipients in the EU, approximately 10% of its population.⁷¹ The European Commission supervises the largest platforms directly; national Digital Services Coordinators oversee the remainder.

Obligations

All intermediary providers must be transparent about their terms of service and must operate notice and action mechanisms allowing users to report illegal content. Online platforms must additionally provide internal complaint-handling systems, give priority to reports from designated ‘trusted flaggers’, and suspend accounts that repeatedly post manifestly illegal content. Platforms must label advertising, and must not present targeted advertising to minors or advertising based on sensitive categories such as political opinion or sexual orientation.⁷²

The largest platforms carry obligations of a different character. They must conduct annual assessments of systemic risks arising from the design and operation of their services — including risks to civic discourse and electoral processes, risks of gender-based violence, and risks to the protection of minors — and take mitigation measures, which may extend to changes to recommender systems and advertising practices. They are subject to independent audits of their compliance, and must give vetted researchers access to data for the study of systemic risks.⁷³

The obligation to assess and mitigate risks arising from a platform’s own design is the feature that most clearly distinguishes the DSA from the Australian scheme, which is directed principally at categories of harmful material and at removal after the fact.

Enforcement

The Commission may impose fines of up to 6% of a provider’s global annual turnover. Temporary restriction of access to the service is available only by a separate route and on a much higher threshold: where an infringement persists and entails a criminal offence involving a threat to the life or safety of persons, the Digital Services Coordinator of establishment may ask the competent national judicial authority to order the temporary restriction of recipients’ access to the service.⁷⁴

Reception

The DSA is frequently cited as an instance of the ‘Brussels effect’, in which the size of the EU market causes regulatory standards adopted there to be applied more widely, because it is cheaper for global firms to operate a single compliance regime than several.⁷⁵

Critics have argued that requiring platforms to act quickly against illegal content creates an incentive to over-remove, and so to restrict lawful expression.⁷⁶ Concerns have also been raised about the compliance burden on smaller platforms; the Commission has offered transitional exemptions to services newly designated as very large. Supporters argue that the transparency, audit and data access provisions address the influence of the largest platforms more directly than schemes limited to content removal.

Related instruments

The DSA operates alongside other EU instruments. The Digital Markets Act, also in force since 2022, imposes obligations on designated ‘gatekeeper’ platforms directed at contestability in digital markets rather than at content.⁷⁷ The Audiovisual Media Services Directive sets standards for audiovisual media providers, including video-sharing platforms, with a focus on the protection of minors and the prohibition of incitement to violence or hatred.⁷⁸ The Code of Practice on Disinformation, introduced in 2018 and strengthened in 2022, began as a voluntary instrument. On 13 February 2025 the Commission and the European Board for Digital Services endorsed its integration into the DSA framework as a Code of Conduct on Disinformation.⁷⁹

Other emerging issues

Example: Unsolicited Dick Pics

Video overview by Kaito Suzuki

An illustrative example of how existing harassment laws struggle with digital contexts is the sending of unsolicited sexual images, commonly known as “dick pics.” It’s important to distinguish this from consensual sexting, which is a normal part of many adults’ digital relationships and sexual expression. The legal issue arises specifically when intimate images are sent without consent or solicitation.

Research from dating applications like Bumble found that 41% of women have received unsolicited photographs of male genitalia. The 2023 Online Safety Issues Survey found that nearly 8% of respondents had experienced this behaviour in the past 12 months, with women and LGBTQIA+ people disproportionately affected.

Several jurisdictions have created specific offences for this behaviour. The UK’s Online Safety Act 2023 criminalises sending photographs of genitals with intent to cause alarm, distress or humiliation (maximum two years imprisonment). Ireland’s Online Safety and Media Regulation Act 2022 takes a similar approach.

In Australia, this conduct is not specifically criminalised. Traditional “indecent exposure” laws like section 5 of the Summary Offences Act 1988 (NSW) are limited to public places and don’t capture digital sending. The federal provision most likely to apply is section 474.14 of the Criminal Code Act 1995 (Cth), which criminalises using carriage services to menace, harass or cause offence, but this requires proving the conduct would be considered menacing, harassing or offensive by a reasonable person.⁸⁰

Some state laws may apply in specific circumstances. Victoria’s Crimes Act 1958 section 48 creates an offence for sexual activity directed toward another person intending to cause fear or distress, though how courts would interpret this in digital contexts remains unclear.

This example illustrates how laws designed for physical spaces often translate poorly to digital environments, creating enforcement gaps even for relatively straightforward harmful conduct.

Deepfakes

See an explanation of deepfakes by Eric Briese

Deepfakes and Non-consensual Sexual Imagery

A deepfake is a technique of image manipulation where artificial intelligence and deep learning is leveraged to manipulate a person’s characteristics, including physical appearance and voice, to create images or content that appear authentic. While manipulated media is not new, deepfake technology has lowered the technical barriers to creating convincing fabrications, raising significant legal and ethical concerns.

The primary legal concerns with deepfakes relate to their use in creating non-consensual sexual imagery, political disinformation, fraud, and harassment. This section focuses on the legal frameworks addressing non-consensual sexual deepfakes, which constitute a form of image-based sexual abuse.

Australian Legal Framework

Australia lacks comprehensive deepfake-specific legislation, but several existing laws may apply depending on the context:

• Criminal Code Act 1995 (Cth);
• Telecommunications Act 1997 (Cth);
• Enhancing Online Safety (Non-consensual Sharing of Intimate Images) Act 2018 (Cth); and
• Online Safety Act 2021 (Cth).

Most Australian jurisdictions have criminal offences covering non-consensual sharing of intimate images, with varying application to altered material. Federal offences under sections 474.17 and 474.17A of the Criminal Code Act 1995 (Cth) prohibit using carriage services to menace, harass or offend, and to transmit sexual material without consent. Victoria’s intimate image offences in Subdivision 8FAA of the Crimes Act 1958 (Vic) address production (s 53R), distribution (s 53S) and threatened distribution (s 53T), and the statutory examples to s 53R make clear that digitally superimposing a person’s face onto another image can constitute production.⁸¹ New South Wales has no dedicated deepfake provision, although its image-based abuse offences extend to altered images: ‘intimate image’ is defined in s 91N of the Crimes Act 1900 (NSW) to include an image that has been altered to appear to show a person’s private parts or a person engaged in a private act, so ss 91P–91R can capture deepfakes.⁸²

The Online Safety Act 2021 (Cth) empowers the eSafety Commissioner to issue removal notices to online service providers hosting intimate imagery, including deepfakes. Providers must remove content within 24 hours of notice, with penalties for non-compliance. As noted by scholars, ‘detection without removal offers little solace to those exploited by deepfake pornography’.⁸³

The limitations of civil enforcement mechanisms are illustrated by eSafety Commissioner v Rotondo [2023] FCA 1296, in which the Federal Court granted an interim injunction requiring the removal of non-consensual intimate images, and in the later contempt proceedings in eSafety Commissioner v Rotondo (No 3) [2023] FCA 1590, where Rotondo was ordered to pay $25,000 plus costs after admitting breaches of those orders.⁸⁴ This apparently did not serve as a deterrent; Rotondo was subsequently arrested for distributing deepfake images of school students and teachers.

Criminal Code Amendment (Deepfake Sexual Material) Act 2024 (Cth)

The Criminal Code Amendment (Deepfake Sexual Material) Act 2024 (Cth), which commenced on 3 September 2024, represents Australia’s first targeted legislative response to deepfake sexual abuse. It replaced the former private sexual material offences with a transmission offence and two aggravated forms:⁸⁵

• Using a carriage service to transmit sexual material of another adult without their consent, whether the material is unaltered or has been created or altered using technology (s 474.17A, maximum 6 years imprisonment)
• An aggravated offence where the person who transmits the material was also responsible for creating or altering it (s 474.17AA(5), maximum 7 years)
• An aggravated offence where three or more civil penalty orders had previously been made against the person under the Online Safety Act 2021 (Cth) (s 474.17AA(1), maximum 7 years)

‘Transmit’ is defined to include making available, publishing, distributing, advertising and promoting the material.

Critics argue the Act duplicates existing offences and may impact freedom of expression, though supporters emphasise the need for specific deterrence given the unique harms of deepfake technology.⁸⁶

It has also been argued that the Act does not go far enough in protecting individuals from deepfake sexual material: while it criminalises the distribution of that material, it does not create a separate offence for its creation.⁸⁷

International Approaches

United Kingdom

In April 2024, the UK government announced plans to amend the Criminal Justice Bill to include a new offence for making sexually explicit deepfakes without consent, which will build on section 66B of the Sexual Offences Act 2003 (UK).

United States

The United States lacks comprehensive federal deepfake legislation, with regulation occurring primarily at state level. California and Texas pioneered state-level deepfake laws in 2019:

California’s Assembly Bill 602 establishes civil remedies for victims of non-consensual pornographic deepfakes, while Assembly Bill 730 prohibits distribution of political deepfakes within 60 days of elections. Texas similarly prohibits political deepfakes within 30 days of elections.

At the federal level, proposed legislation includes:

• The DEEP FAKES Accountability Act - establishing criminal penalties and a Department of Homeland Security task force
• The DEFIANCE Act of 2024 - providing civil remedies for victims of sexually explicit deepfakes

European Union

The EU Directive on combating violence against women and domestic violence (May 2024) requires member states to criminalise non-consensual production and distribution of sexual deepfakes using artificial intelligence.

China

China has taken a comprehensive regulatory approach since 2019. The ‘Regulations on the Administration of Networked Audiovisual Information Services’ require disclosure when deepfake technology is used and prohibit unlabelled deepfake content.⁸⁸ The 2023 ‘Regulations on Deep Synthesis Management of Internet Information Service’ extend controls throughout the deepfake lifecycle, requiring platforms to obtain consent before using individuals’ likenesses and strengthen training data management.

Political deepfakes and disinformation

Discussion of deepfakes has focused mainly on non-consensual sexual imagery. Synthetic media also presents a distinct set of problems in politics and elections, where the concern is not injury to an individual but the reliability of the information environment on which democratic participation depends.⁸⁹

Democratic risks

Because political figures are extensively recorded, they are unusually easy subjects for synthetic media.⁹⁰ Empirical work indicates that exposure to political deepfakes can reduce trust in media and in politicians,⁹¹ and the principal harm has been characterised not as belief in any particular fabrication but as an erosion of epistemic trust — a general decline in the willingness to treat recorded evidence as reliable.⁹²

One survey identified 82 political deepfakes across 38 countries between July 2023 and July 2024, 30 of which held elections during that period.⁹³ Widely circulated examples include a fabricated video purporting to show President Zelenskyy announcing a surrender, and manipulated footage of Nancy Pelosi appearing impaired while speaking.⁹⁴

A related effect operates in the opposite direction. The ‘liar’s dividend’ describes the advantage that accrues to a person who can dismiss authentic recordings as fabricated, once audiences accept that convincing fabrications are possible.⁹⁵

Australia. The Australian Electoral Commission’s Stop and Consider campaign encourages voters to check the source of electoral communications, including AI-generated material. Its reach is limited by the speed and anonymity with which manipulated content circulates.⁹⁶

The Australian legal framework

There is no Australian offence or civil wrong directed specifically at political deepfakes. The conduct is addressed, if at all, indirectly.

Defamation. Defamation proceedings have been brought in the New South Wales Supreme Court over a digitally altered image, although in that case the imputations that survived a pleading challenge were held to arise from the accompanying text rather than from the image,⁹⁷ and Voller established that a party who facilitates publication may be a publisher for the purposes of defamation.⁹⁸ The 2023 reforms introduced a conditional exemption for certain digital intermediaries, and a defence for platforms that maintain an accessible complaints mechanism and act on notice, discussed in the Intermediary Liability for Defamation chapter. Defamation may therefore reach a political deepfake that damages reputation, but it protects reputation rather than the integrity of electoral information, and it does nothing where the target is not identifiable or not defamed.

Copyright. Under the Copyright Act 1968 (Cth), only the owner of copyright or an exclusive licensee may sue for infringement.⁹⁹ A person depicted in a deepfake generally has no standing, since copyright protects the economic interests of rights holders rather than the dignity or reputation of the subject.¹⁰⁰ Where synthetic material reproduces a substantial part of a protected work, the rights holder may secure removal through the safe harbour notice scheme,¹⁰¹ and the person depicted may benefit incidentally. In the United States, a publisher used YouTube’s copyright process to remove a deepfake depicting Kim Kardashian.¹⁰² The protection is contingent on the accident of whether protected material was reproduced.

Online safety. The Online Safety Act 2021 (Cth) empowers the eSafety Commissioner to require removal of non-consensual intimate images, including altered images.¹⁰³ The Criminal Code Amendment (Deepfake Sexual Material) Act 2024 (Cth) created offences for the transmission of sexual material depicting a person without consent, whether or not the material is artificially generated.¹⁰⁴ The focus remains on sexual harm; political disinformation falls outside it.

The result is a fragmented framework in which political deepfakes are addressed only where they happen to coincide with an established cause of action.

Other jurisdictions

European Union. The Artificial Intelligence Act requires that deep fakes be disclosed as artificially generated or manipulated, an obligation that extends to political content.¹⁰⁵

China. Labelling obligations extend to AI-generated media generally.¹⁰⁶

United States. Several states have legislated against deceptive synthetic media in the period before an election.¹⁰⁷ These laws have encountered legal difficulty on two distinct grounds. In October 2024 a federal court preliminarily enjoined most of California’s AB 2839, holding that it was a content-based restriction that lacked the narrow tailoring and least restrictive means required under the First Amendment.¹⁰⁸ In August 2025 the same court struck down a second Californian statute, AB 2655, on a different basis: the statute was preempted by s 230 of the Communications Decency Act, and the court considered it unnecessary to decide the First Amendment arguments.¹⁰⁹

Regulatory and constitutional constraints

Regulation faces both practical and constitutional limits. The volume and speed of circulation, and the anonymity of those who create and distribute the material, make enforcement difficult. More fundamentally, any Australian measure directed at political disinformation would need to be compatible with the implied freedom of political communication, which requires that a burden on political communication be reasonably appropriate and adapted to a legitimate end.¹¹⁰ The Californian litigation illustrates two distinct vulnerabilities: measures directed precisely at the political speech thought to be most harmful are the measures most exposed to constitutional challenge, and measures that operate by imposing obligations on platforms in respect of material posted by their users are additionally exposed to federal statutory preemption.

AI voice agents and AI characters

AI voice agents use speech recognition, natural language processing and generative models to hold spoken conversations. Depending on how they are configured, an agent may answer questions, complete transactions, make bookings, or hand a call to a person.

AI characters are a related application designed to present a persona rather than to complete a task. The persona may be fictional, or may imitate a real person, and the service is generally offered for entertainment or companionship. Unlike task-oriented assistants, these systems are built to sustain an ongoing relationship: they retain the content of earlier conversations, express apparent emotional states, and adapt to the user over time.

Social context and potential harms

The design features that make these systems engaging are also the source of the concerns raised about them. A character is available at any hour, responds consistently and affirmingly, and does not impose the friction present in relationships with other people. For some users — particularly children, and people who are isolated or unwell — that combination carries a risk of dependency, and of forming expectations about intimacy and consent that do not transfer to human relationships.

Reporting and research have identified a second problem: services frequently lack effective age assurance or content safeguards, and characters have been documented responding permissively to sexual role-play initiated by minors, to disclosures of self-harm, and to violent content.¹¹¹

These are not harms for which the existing content categories are well designed. The material is generated in a private exchange rather than published, it is produced in response to the user’s own prompting, and it may not exist anywhere until the moment it is generated.

The Australian framework

Australia (Commonwealth). Australia has no legislation directed specifically at AI systems. Conversational AI services are regulated as online content, principally under the Online Safety Act 2021 (Cth), through the schemes described earlier in this chapter — cyberbullying, adult cyber abuse, image-based abuse, and the online content scheme for illegal and restricted material.

The Basic Online Safety Expectations, made under s 45 of the Act, require providers of covered services to take reasonable steps to ensure that end-users are able to use the service in a safe manner, and were amended in 2024 to address generative artificial intelligence expressly.¹¹² The Privacy Act 1988 (Cth) governs the collection, use and disclosure of personal information by these services, including the conversational data they retain.

In place of binding AI-specific regulation, Australia has adopted Australia’s AI Ethics Principles, a voluntary framework providing that AI systems should benefit individuals, society and the environment; respect human rights, diversity and individual autonomy; be inclusive and accessible and avoid unfair discrimination; uphold privacy and data security; be transparent, so that people can tell when AI is affecting them; be contestable, so that a person significantly affected has a means of challenge; and be accompanied by accountability on the part of those responsible for the system.¹¹³ The principles are not enforceable.

Open questions

The absence of a specific framework leaves a number of questions unresolved:

• Where a character elicits or encourages harmful conduct that results in injury, how is responsibility allocated between the developer of the model, the operator of the service, and the user who configured the character?
• Where a character imitates a real person without consent, which existing cause of action applies — passing off, defamation, copyright, or the statutory tort for serious invasions of privacy?
• What remedy is available to a person harmed psychologically or socially by a service of this kind?
• How can protections for users at risk of dependency be designed without restricting access for others on the basis of assumed vulnerability?
• How should services with therapeutic or educational purposes be distinguished from those the framework is intended to restrict?

International approaches

European Union. The Artificial Intelligence Act classifies AI systems by risk, prohibiting a defined set of unacceptable practices including manipulative techniques that materially distort behaviour, and social scoring. The social scoring prohibition is not limited to public authorities: it applies to the placing on the market, putting into service or use of AI systems for the evaluation or classification of natural persons based on their social behaviour, whether by public or private actors.¹¹⁴ Systems intended to interact directly with natural persons are subject to transparency obligations: users must be informed that they are interacting with an AI system unless that is obvious, and synthetic audio, image and video content must be marked as artificially generated.¹¹⁵ Obligations of the high-risk regime attach to the uses listed in the Act rather than to conversational systems as a class.

China. The Provisions on the Administration of Deep Synthesis Internet Information Services, in force since January 2023, require conspicuous labelling of synthetically generated content, prohibit the cloning of a person’s voice or likeness without consent, require providers to verify the real identity of users, and require services to monitor for and act on unlawful content and to operate complaint mechanisms.¹¹⁶

Both frameworks place binding obligations on developers and operators and require disclosure of artificiality at the point of interaction. Australia’s approach relies on general online safety obligations supplemented by voluntary principles, which leaves the specific risks presented by persistent, personalised AI companions largely unaddressed.

Misinformation and Disinformation

See an explanation of misinformation by the ACMA.

Artificial Intelligence (‘AI’) has contributed to the ongoing challenge of regulating and controlling the spread of misinformation and disinformation. Misinformation is ‘false, misleading or deceptive information that can cause harm’. Disinformation is misinformation that is deliberately spread to cause confusion and undermine trust in governments or institutions.¹¹⁷ Algorithms and ‘bots’ are becoming some of the strongest spreaders of false, unreliable and misleading information online. ‘Bots’ are computer algorithms generated by AI that automatically produce content and interact with humans on social media platforms.

The spread of misinformation and disinformation online has been linked to propaganda and the proliferation of abuse and targeted attacks, and harm in emergency situations as civilians are unable to obtain the correct information from reliable sources about how they should ensure their own safety. A 2023 Forbes report indicated that 76% of consumers were worried about misinformation provided by AI.

Regulation

Misinformation and disinformation are regulated in Australia through a voluntary code of practice. The Australian Code of Practice on Disinformation and Misinformation (The Code) was released in Australia by The Digital Industry Group (DIGI) in February 2021. DIGI is a not-for-profit industry association tasked with administering the Code. The objective of the Code is to combat false material being released on digital platforms by setting a standard of practice to which signatories are required to comply with. Eight technology companies have opted into commitments under the Code, however according to provision 7.1 they are only required to comply with their selected commitments. Provision 7.2 also recognises that companies may withdraw from the Code by notifying DIGI. An independent Complaints Committee resolves complaints regarding Signatories compliance with their commitments under the Code and the public has access to complaints that are made via a complaints portal on DIGI’s website.¹¹⁸ The Australian Communications and Media Authority (ACMA) also has oversight over the code and reports on the adequacy of platforms measures to implement their commitments. These reports are then publicly available.

The previous Australian Government took steps towards introducing legislation to combat the spread of misinformation and disinformation on digital platforms. A senate inquiry was conducted in 2023 by the Economics References Committee who reported on the ‘Influence of international digital platforms’. This inquiry received several submissions from organisations such as the Human Rights Law Centre noting concerns about the rise of disinformation and misinformation online and the failure of any existing effective enforcement mechanism combatting it. An exposure draft of the Communications Legislation Amendment (Combatting Misinformation and Disinformation) Bill 2023 was released for public feedback on 25 June 2023. This legislation would afford ACMA new powers to hold digital platforms to account and strengthen and support the Code to extend to non-signatories. The Government, however, has not yet announced a timeline for introduction of this Bill to parliament and there is considerable pushback from organisations noting concerns for the restraint it may impose on freedom of expression.

Online Safety Act

The Online Safety Act 2021 (Cth) does not directly regulate the spread of misinformation and disinformation. However, the Commissioner has the power to require providers to report on the extent to which they are complying with the basic online safety expectations. Importantly, however, failure to comply with those expectations will not lead to legal penalties as they are not enforceable by proceedings in a court.¹¹⁹ Inclusive within the ‘Core Expectations’, providers are required to take ‘reasonable steps’ to ensure the safety of their end-users and to prevent ‘harmful material’ being released on their sites.¹²⁰ Under the first determination in 2022 the Minister for Communications set out expectations that providers would take reasonable steps to minimise the extent to which AI and anonymous accounts would produce harmful material on their sites.¹²¹ ‘Harmful material’ is not defined anywhere in the Act, however it is considered a ‘reasonable step’ by the provider to request a consultation with the eSafety Commissioner in making determinations about what may be ‘harmful’.

1. Heath Gilmore, ‘Web Porn Software Filter Takes Biggest Hit’, The Sun-Herald (Sydney, 17 February 2008). ↩

2. Enhancing Online Safety (Non-Consensual Sharing of Intimate Images) Act 2018 (Cth) sch 1 s 24; Enhancing Online Safety Act 2015 (Cth) ss 19A, 27, 44D–44F. ↩

3. Online Safety Act 2021 (Cth) s 63D; Regulatory Powers (Standard Provisions) Act 2014 (Cth) s 82(5); Crimes Act 1914 (Cth) s 4AA. The value of a penalty unit is indexed every three years; it increased from $330 to $364 on 1 July 2026. ↩

4. eSafety Commissioner, Register of Industry Codes and Industry Standards for Online Safety (Web Page) https://www.esafety.gov.au/industry/codes/register-online-industry-codes-standards; Phase One Codes (Web Page, onlinesafety.org.au) https://onlinesafety.org.au/phase-one-codes/. ↩

5. Classification (Publications, Films and Computer Games) Act 1995 (Cth). The Commonwealth Act establishes the classification scheme; the consequences of a classification, including restrictions on sale, hire and public exhibition, are given effect by complementary State and Territory enforcement legislation. ↩

6. Online Safety (Basic Online Safety Expectations) Amendment Determination 2024 (Cth) ss 1–2, sch 1 item 1, inserting Online Safety (Basic Online Safety Expectations) Determination 2022 (Cth) s 6(2A). ↩

7. Delia Rickard PSM, Report of the Statutory Review of the Online Safety Act 2021 (Report, October 2024) https://www.infrastructure.gov.au/department/media/publications/report-statutory-review-online-safety-act-2021. ↩

8. Minister for Communications, ‘Address to the National Press Club’ (Speech, 13 November 2024). ↩

9. See, for example, Alana Maurushat, David Vaile and Alice Chow, ‘The Aftermath of Mandatory Internet Filtering and S 313 of the Telecommunications Act 1997 (Cth)’ (2014) 19 Media and Arts Law Review 263. ↩

10. Enhancing Online Safety (Non-Consensual Sharing of Intimate Images) Act 2018 (Cth) sch 2 s 4; Criminal Code Act 1995 (Cth) s 474.17A; Criminal Code Amendment (Deepfake Sexual Material) Act 2024 (Cth) sch 1 (repealing the definition of ‘private sexual material’ in Criminal Code s 473.1 and substituting s 474.17A). ↩

11. Criminal Code (Non-Consensual Sharing of Intimate Images) Amendment Act 2019 (Qld) s 4; Criminal Code Act 1899 (Qld) s 207A. ↩

12. Criminal Code (Non-Consensual Sharing of Intimate Images) Amendment Act 2019 (Qld) s 5; Criminal Code Act 1899 (Qld) s 223. ↩

13. Criminal Code (Non-Consensual Sharing of Intimate Images) Amendment Act 2019 (Qld) s 6; Criminal Code Act 1899 (Qld) s 227A. ↩

14. Criminal Code (Non-Consensual Sharing of Intimate Images) Amendment Act 2019 (Qld) s 7; Criminal Code Act 1899 (Qld) s 227B. ↩

15. Office of the eSafety Commissioner, Image-Based Abuse National Survey: Summary Report (Report, 2017). ↩

16. Nicola Henry, Anastasia Powell and Asher Flynn, Not Just ‘Revenge Pornography’: Australians’ Experiences of Image-Based Abuse — A Summary Report (Report, RMIT University, May 2017) 7. ↩

17. X Corp and Elston v eSafety Commissioner [2025] ARTA 852 (Deputy President O’Donovan, 1 July 2025) (Tribunal Nos 2024/2582 and 2024/2583). ↩

18. Nell Geraets, ‘Is This the End of Faceless Internet Trolls? How One “Mumfluencer” Is Turning the Tables’, The Sydney Morning Herald (online, 25 June 2025) https://www.smh.com.au/culture/celebrity/is-this-the-end-of-faceless-internet-trolls-how-one-mumfluencer-is-turning-the-tables-20250623-p5m9hk.html. ↩

19. Hit Network, ‘P.I. Reveals How Indy Clinton Tracked Down TikTok Trolls’ (YouTube, 24 June 2025) https://www.youtube.com/watch?v=ULxwFgjeixE. ↩

20. eSafety Commissioner, ‘Trolling’ (Web Page) https://www.esafety.gov.au/young-people/trolling. ↩

21. eSafety Commissioner, Protecting LGBTIQ+ Voices Online: Resource Development Research (Qualitative Report, August 2021). ↩

22. Sameer Hinduja and Justin W Patchin, Bullying, Cyberbullying, and Sexual Orientation (Research Summary, Cyberbullying Research Center, 2011) 1. ↩

23. eSafety Commissioner, Netsafe and UK Safer Internet Centre, Online Hate Speech: Findings from Australia, New Zealand and Europe (Report, January 2020) 8 (Table 2). ↩

24. Sex Discrimination Act 1984 (Cth) ss 5A, 5B. ↩

25. Kath Albury and Daniel Reeders, ‘3 in 4 People Experience Abuse on Dating Apps. How Do We Balance Prevention with Policing?’, The Conversation (online, 30 January 2023) https://theconversation.com/3-in-4-people-experience-abuse-on-dating-apps-how-do-we-balance-prevention-with-policing-198587. ↩

26. Heather Wolbers et al, Sexual Harassment, Aggression and Violence Victimisation among Mobile Dating App and Website Users in Australia (Research Report No 25, Australian Institute of Criminology, 2022) 10. Non-binary respondents were excluded from the associated statistical testing because of small sample size. ↩

27. Monash University, Australian Users’ Experiences with Control Features on Social Media Services and Online Dating Apps (Key Findings, May 2023) 13. The figures describe the 102 participants in the study’s 24 online focus groups, not a nationally representative sample. ↩

28. Jason Om, ‘New Wave of Homophobic Attacks Targets Users of Gay Dating Apps like Grindr’, ABC News (online, 6 July 2025) https://www.abc.net.au/news/2025-07-06/gay-dating-app-users-lured-into-violent-homophobic-attacks/105464048. ↩ ↩²

29. Jordan Hirst, ‘Safety Alert after “High Number” of Gay App Assaults in NSW’, QNews (online, 18 October 2024) https://qnews.com.au/safety-alert-after-high-number-of-gay-app-assaults-in-nsw/. ↩

30. Australian Online Dating Code Oversight Body, Code of Practice (Industry Code, July 2024); Department of Infrastructure, Transport, Regional Development, Communications, Sport and the Arts, ‘New Industry Code Now Operational to Make Online Dating Safer’ (Media Release, 3 October 2024). ↩

31. Michelle Rowland, ‘Online Dating Platforms Now Subject to Enforcement’ (Media Release, Minister for Communications, 1 April 2025). ↩

32. eSafety Commissioner, ‘Domestic and Family Violence’ (Web Page, 6 November 2024) https://www.esafety.gov.au/key-topics/domestic-family-violence. ↩

33. Attorney-General’s Department, ‘Understanding Technology-Facilitated Coercive Control’ (Web Page, 5 March 2024) https://www.ag.gov.au/families-and-marriage/publications/understanding-technology-facilitated-coercive-control. ↩

34. Delanie Woodlock et al, Second National Survey of Technology Abuse and Domestic Violence in Australia (Report, WESNET, 2020) 18. ↩

35. Commonwealth of Australia, National Plan to End Violence against Women and Children 2022–2032 (2022). ↩

36. Monash University, ‘Technology-Facilitated Violence in the Indo-Pacific Is on the Rise and Under-Researched’, Lens (Web Page, 7 March 2025) https://lens.monash.edu/@politics-society/2025/03/07/1387375/technology-facilitated-violence-in-the-indo-pacific-is-on-the-rise-and-under-researched. ↩

37. eSafety Commissioner, ‘The Online Experiences of Children in Australia’ (Research Web Page, 2025), reporting the Keeping Kids Safe Online survey of over 3,000 children aged 10 to 17 conducted between December 2024 and February 2025: ‘1 in 4 (25%) had experienced non-consensual tracking, monitoring or harassment’. ↩

38. Arianna Levy and Alison Xiao, ‘Location-Sharing Apps Linked to Increased Risk of Digital Coercive Control, eSafety Commission Research Finds’, ABC News (online, 15 May 2025) https://www.abc.net.au/news/2025-05-15/location-sharing-apps-esafety-commission-coercive-control/105289994. ↩ ↩²

39. Victoria Pengilley and Brianna Parkins, ‘Lilie James’s Killer Was Calm and Calculated, Inquest into Murder Told’, ABC News (online, 22 March 2025) https://www.abc.net.au/news/2025-03-22/lilie-james-killer-controlling-domestic-violence-expert-inquest/105078608. ↩

40. Mostafa Rachwani, ‘Disturbing Portrait of Coercive Control and Violent Masculinity Revealed at Lilie James Inquest’, The Guardian (online, 22 March 2025) https://www.theguardian.com/society/2025/mar/21/lilie-james-inquest-reveals-disturbing-portrait-of-coercive-control-and-violent-masculinity-ntwnfb. ↩

41. Crimes (Domestic and Personal Violence) Act 2007 (NSW) ss 8(1)(b1), 13, as amended by the Crimes (Domestic and Personal Violence) and Other Legislation Amendment Act 2024 (NSW) sch 1[3]. The s 13 offence carries a maximum penalty of imprisonment for 5 years or 50 penalty units, or both. ↩

42. Noelle Martin, ‘Online Safety Regulation of Deepfake Abuse: A Case Study on Australia’s eSafety Commissioner’ (2025) 34(1) Griffith Law Review 23, 30. ↩

43. Ibid. ↩

44. Ibid. ↩

45. ANROWS, Technology-Facilitated Abuse: A Survey of Support Service Stakeholders (Research Report, July 2021) 7. ↩

46. Bronwyn Carlson and Madi Day, ‘Technology-Facilitated Abuse: The Need for Indigenous-Led Research and Response’ in Bridget Harris and Delanie Woodlock (eds), Technology and Domestic and Family Violence: Victimisation, Perpetration and Responses (Routledge, 2023) 33. ↩

47. Online Safety Amendment (Social Media Minimum Age) Act 2024 (Cth); Online Safety Act 2021 (Cth) pt 4A, s 63E(2). The commencement date is fixed by a separate notifiable instrument: Online Safety (Day of Effect of Social Media Minimum Age) Instrument 2025 (Cth) F2025N00628, which specifies 10 December 2025 as the day s 63D takes effect. ↩

48. Julie Inman Grant, ‘Swimming between the Digital Flags: Helping Young Australians Navigate Social Media’s Dangerous Currents’ (Speech, Canberra, 24 June 2025). ↩

49. Anthony Albanese, ‘Australia Can Lead on Making Kids Safe’ (Opinion, 1 December 2024) https://www.pm.gov.au/media/australia-can-lead-making-kids-safe. ↩

50. eSafety Commissioner, Digital Use and Risk: Online Platform Engagement among Children Aged 10 to 15 (Report, July 2025), reporting findings of eSafety’s Keeping Kids Safe Online survey. ↩

51. Australian Institute of Health and Welfare, ‘Mental Health of Young Australians’ in Australia’s Health 2022: Data Insights (Report, 2022) ch 8, 263. ↩

52. Online Safety Act 2021 (Cth) s 63C(1). Paragraph 63C(1)(a) sets out four conditions, of which sub-paragraph (iv) is ‘such other conditions (if any) as are set out in the legislative rules’. ↩

53. Online Safety (Age-Restricted Social Media Platforms) Rules 2025 (Cth) s 4A, inserted by the Online Safety (Age-Restricted Social Media Platforms) Amendment Rules 2026 (Cth) F2026L00370 sch 1 item 1 (registered 25 March 2026, commenced the following day). ↩

54. eSafety Commissioner, ‘Which Social Media Platforms Are Age-Restricted?’ (Web Page). The list reflects eSafety’s views as at 21 November 2025; it is not exhaustive, and other services have self-assessed as age-restricted. ↩

55. Online Safety (Age-Restricted Social Media Platforms) Rules 2025 (Cth) s 5. ↩

56. eSafety Commissioner, Age Assurance: Tech Trends and Issues (Report, 2024). ↩

57. eSafety Commissioner, ‘Information from Age-Restricted Platforms’ (Web Page), collecting the age assurance methods published by each age-restricted platform. ↩

58. Age Check Certification Scheme, Age Assurance Technology Trial — Part A: Main Report (Report, August 2025) 17 (‘Broad demographic consistency’), [A.25.5]. ↩

59. Online Safety Act 2021 (Cth) pt 4A. ↩

60. Australian Human Rights Commission, Submission to the Senate Standing Committees on Environment and Communications: Online Safety Amendment (Social Media Minimum Age) Bill 2024 (Submission, 22 November 2024) https://humanrights.gov.au/our-work/legal/submission/social-media-ban. ↩

61. Australian Human Rights Commission, ‘Permissible Limitations on Rights’ (Web Page) https://humanrights.gov.au/our-work/rights-and-freedoms/permissible-limitations-rights. ↩

62. Convention on the Rights of the Child, opened for signature 20 November 1989, 1577 UNTS 3 (entered into force 2 September 1990) arts 13, 17. ↩

63. Committee on the Rights of the Child, General Comment No 25 (2021) on Children’s Rights in Relation to the Digital Environment, UN Doc CRC/C/GC/25 (2 March 2021). ↩

64. Tamoor Mirza, ‘Balancing Risks and Resilience: Evaluating the Impact of Australia’s Social Media Ban on Adolescent Mental Health’ (2025) 33(4) Australasian Psychiatry 768. ↩

65. Online Safety Act 2021 (Cth) s 239B: ‘Within 2 years after the day section 63D takes effect in accordance with section 63E, the Minister must cause to be conducted an independent review of the operation of Part 4A’, which must include consideration of the adequacy of privacy protections. ↩

66. United Kingdom, Department for Science, Innovation & Technology ‘Guidance – Online Safety Act: explainer’ Online Safety Act: explainer (Web Page, 8 May 2024) https://www.gov.uk/government/publications/online-safety-act-explainer/online-safety-act-explainer#what-the-online-safety-act-does. ↩

67. Government of Canada, ‘Government of Canada introduces legislation to combat harmful content online, including the sexual exploitation of children’ Canadian Heritage (Web Page, 26 February 2024) https://www.canada.ca/en/canadian-heritage/news/2024/02/government-of-canada-introduces-legislation-to-combat-harmful-content-online-including-the-sexual-exploitation-of-children.html. ↩

68. Government of Canada, ‘Proposed Bill to address Online Harms’ Arts and media (Web Page, 4 April 2024) https://www.canada.ca/en/canadian-heritage/services/online-harms.html. ↩

69. ‘Liberals Taking “Fresh” Look at Online Harms Bill, Says Justice Minister Sean Fraser’, CBC News (online, 2025) https://www.cbc.ca/news/politics/liberals-taking-fresh-look-at-online-harms-bill-says-justice-minister-sean-fraser-1.7573791. ↩

70. Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market for Digital Services (Digital Services Act) [2022] OJ L 277/1 https://eur-lex.europa.eu/eli/reg/2022/2065/oj. ↩

71. Ibid art 33. ↩

72. Ibid arts 14, 16, 20–23, 26, 28. ↩

73. Ibid arts 34–37, 40. ↩

74. Ibid arts 74(1) (fines), 82(1) and 51(3)(b) (temporary restriction of access, on application by the Digital Services Coordinator to the competent judicial authority, where the infringement ‘entails a criminal offence involving a threat to the life or safety of persons’); European Commission, ‘The Enforcement Framework under the Digital Services Act’ (Web Page) https://digital-strategy.ec.europa.eu/en/policies/dsa-enforcement. ↩

75. Christian Odendahl, ‘How Europe’s New Digital Law Will Change the Internet’, The Economist (online, 24 August 2023) https://www.economist.com/the-economist-explains/2023/08/24/how-europes-new-digital-law-will-change-the-internet. ↩

76. Eliska Pirkova, ‘The Digital Services Act: Your Guide to the EU’s New Content Moderation Rules’, Access Now (online, 6 July 2022) https://www.accessnow.org/digital-services-act-eu-content-moderation-rules-guide/. ↩

77. Regulation (EU) 2022/1925 of the European Parliament and of the Council of 14 September 2022 on Contestable and Fair Markets in the Digital Sector (Digital Markets Act) [2022] OJ L 265/1. ↩

78. Directive 2010/13/EU of the European Parliament and of the Council of 10 March 2010 on Audiovisual Media Services, as amended by Directive (EU) 2018/1808. ↩

79. European Commission, ‘The Code of Conduct on Disinformation’ (Web Page): ‘On 13 February 2025, the Commission and the European Board for Digital Services endorsed the integration of the 2022 Code of Practice on Disinformation as a Code of Conduct on Disinformation into the framework of the DSA.’ ↩

80. R v TB (No 5) [2023] SASC 118 ↩

81. Crimes Act 1958 (Vic) ss 53R, 53S, 53T (Subdivision 8FAA), inserted by the Justice Legislation Amendment (Sexual Offences and Other Matters) Act 2022 (Vic). ↩

82. Crimes Act 1900 (NSW) s 91N (definition of ‘intimate image’), ss 91P–91R; Tom Gotsis, Sexually Explicit Deepfakes and the Criminal Law in NSW (Research Paper, NSW Parliamentary Research Service, April 2025). ↩

83. Tong, S, “You Won’t Believe What She Does!’: an Examination into the Use of Pornographic Deepfakes as a Method of Sexual Abuse and the Legal Protections Available to its Victims” [2022] UNSWLawJlStuS 25; UNSWLJ Student Series No 22-25. ↩

84. See Laura Lavelle, ‘Antonio Rotondo guilty of contempt of court after allegedly creating deepfake images of school students and teachers’ (ABC News) (6 December 2023) https://www.abc.net.au/news/2023-12-06/qld-deepfake-images-court-charge-antonio-rotondo-school-students/103195578 ↩

85. Criminal Code Amendment (Deepfake Sexual Material) Act 2024 (Cth) sch 1 items 5–6, substituting Criminal Code ss 474.17A and 474.17B and inserting ss 474.17AA and 474.17AB. ↩

86. Billi Fitzsimmons, ‘A Victorian teen has been arrested after fake nudes of 50 school girls were shared online’ The Daily Aus (online, 13 June 2024) < https://www.newsletter.thedailyaus.com.au/p/teen-arrested-fake-ai-images>. ↩

87. Tom Williams, ‘Deepfake abuse law “doesn’t go far enough”’, Information Age (online, 24 July 2024) https://ia.acs.org.au/article/2024/deepfake-law-doesnt-go-far-enough.html. ↩

88. Cyberspace Administration of China, Regulations on the Administration of Networked Audiovisual Information Services (18 November 2019) http://www.cac.gov.cn/2019-11/29/c_1576561820967678.htm [perma.cc/E2DQ-ZHCQ]. ↩

89. Andrew Ray, ‘Disinformation, Deepfakes and Democracies: The Need for Legislative Reform’ (2021) 44(3) University of New South Wales Law Journal 983, 984. ↩

90. Ibid 986. ↩

91. Michael Hameleers, Toni G L A van der Meer and Tom Dobber, ‘You Won’t Believe What They Just Said! The Effects of Political Deepfakes Embedded as Vox Populi on Social Media’ (2022) 8(3) Social Media + Society (Article 20563051221116346) https://doi.org/10.1177/20563051221116346. The journal does not assign page numbers. ↩

92. Tom Dobber et al, ‘Do (Microtargeted) Deepfakes Have Real Effects on Political Attitudes?’ (2021) 26(1) International Journal of Press/Politics 69, 71. ↩

93. Insikt Group, Targets, Objectives, and Emerging Tactics of Political Deepfakes (Report, Recorded Future, 24 September 2024) https://www.recordedfuture.com/research/targets-objectives-emerging-tactics-political-deepfakes. ↩

94. Maria Pawelec, ‘Deepfakes and Democracy (Theory): How Synthetic Audio-Visual Media for Disinformation and Hate Speech Threaten Core Democratic Functions’ (2022) 1(2) Digital Society 19. ↩

95. Kaylyn Jackson Schiff, Daniel S Schiff and Natália S Bueno, ‘The Liar’s Dividend: Can Politicians Claim Misinformation to Evade Accountability?’ (2025) 119(1) American Political Science Review 71. ↩

96. Susan Grantham, ‘The AEC Wants to Stop AI and Misinformation. But It’s Up against a Problem That Is Deep and Dark’, The Conversation (online, 3 February 2025) https://theconversation.com/the-aec-wants-to-stop-ai-and-misinformation-but-its-up-against-a-problem-that-is-deep-and-dark-248773. ↩

97. Gilbert v Nationwide News Pty Ltd [2016] NSWSC 845 (McCallum J), an interlocutory ruling on objections to imputations, in which the surviving imputations were held to arise from ‘the opening paragraphs of the editorial’. Contrast Charleston v News Group Newspapers Ltd [1995] 2 AC 65, in which the plaintiffs, who complained of manipulated images, were unsuccessful. ↩

98. Fairfax Media Publications Pty Ltd v Voller (2021) 273 CLR 346. ↩

99. Copyright Act 1968 (Cth) ss 115(1) (owner), 119 (exclusive licensee). ↩

100. Ted Talas, ‘Real or (Deep)fake? Responding to the Legal Challenges Created by the Emergence of Deepfakes’ (2022) 38(7) Privacy Law Bulletin 181, 181–2. ↩

101. Copyright Act 1968 (Cth) pt V div 2AA. ↩

102. Talas (above) 181–2. ↩

103. Online Safety Act 2021 (Cth) pt 6. ↩

104. Criminal Code Amendment (Deepfake Sexual Material) Act 2024 (Cth) sch 1 items 5–6, substituting Criminal Code ss 474.17A and 474.17B and inserting ss 474.17AA and 474.17AB. ↩

105. Regulation (EU) 2024/1689 (Artificial Intelligence Act) art 50(4). ↩

106. Provisions on the Administration of Deep Synthesis Internet Information Services (China, in force 10 January 2023); Toby Bond and Emma Ren, ‘New AI Content Labelling Rules in China’ (Bird & Bird, 20 May 2025) https://www.twobirds.com/en/insights/2025/new-ai-content-labelling-rules-in-china-what-are-they-and-how-do-they-compare-to-the-eu-ai-act. ↩

107. Defending Democracy from Deepfake Deception Act of 2024 (California) AB 2655; Tex Elec Code Ann § 255.004(d). ↩

108. Kohls v Bonta, 752 F Supp 3d 1187 (ED Cal, 2024) (preliminary injunction, 2 October 2024, in relation to AB 2839). ↩

109. The Conference Board, Federal Judge Strikes Down California Deepfake Law (Newsletter, 7 August 2025) https://www.conference-board.org/research/CED-Newsletters-Alerts/federal-judge-strikes-down-california-deepfake-law, reporting that AB 2655 ‘violates Section 230 of the Communications Decency Act’ and that the decision relied on federal preemption rather than addressing the First Amendment arguments, which the judge stated were ‘not necessary’. ↩

110. Lange v Australian Broadcasting Corporation (1997) 189 CLR 520; McCloy v New South Wales (2015) 257 CLR 178. ↩

111. See, eg, eSafety Commissioner, ‘AI Chatbots and Companions — Risks to Children and Young People’ (Web Page) https://www.esafety.gov.au/newsroom/blogs/ai-chatbots-and-companions-risks-to-children-and-young-people. ↩

112. Online Safety (Basic Online Safety Expectations) Determination 2022 (Cth), made under Online Safety Act 2021 (Cth) s 45; Online Safety (Basic Online Safety Expectations) Amendment Determination 2024 (Cth). ↩

113. Department of Industry, Science and Resources, ‘Australia’s AI Ethics Principles’ (Web Page) https://www.industry.gov.au/publications/australias-artificial-intelligence-ethics-framework/australias-ai-ethics-principles. ↩

114. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) [2024] OJ L 1689 arts 5(1)(a)–(b), 5(1)(c) https://eur-lex.europa.eu/eli/reg/2024/1689/oj. ↩

115. Ibid art 50. ↩

116. Provisions on the Administration of Deep Synthesis Internet Information Services (China, in force 10 January 2023). ↩

117. ACMA, Online Misinformation, www.acma.gov.au/online-misinformation (accessed 01 September 2024) ↩

118. Australian Code of Practice on Disinformation and Misinformation s 7.5. ↩

119. Online Safety Act 2021 (Cth) s 45(4) ↩

120. Online Safety Act 2021 (Cth) s 46(1)(a)(b) ↩

121. Online Safety (Basic Online Safety Expectations) Determination 2022 s8A, s9 ↩

---

Table of contents

• eSafety Commissioner
• Graduated Response
• Follow the Money