AI has been a frequent topic on this blog recently. Consistent with my twisted sense of humour poking fun at lawyers who don’t check citations and refer to non-existent cases can be amusing. However, what happens when a self represented person uses AI to prepare their case in the Court of Appeal? I might also add at this point that the reason why I know about this case is that I am a part time NCAT member and received an email about being vigilant for AI generated arguments in the Tribunal (the “wild west” frontier of the self represented).
In May v Costaras [2025] NSWCA 178 Chief Justice Bell clearly smelled a rat early on:
“BELL CJ: Ms Costaras, are you, and I don’t mean this critically, I just want to inquire, are you reading from some script or some slides prepared by artificial intelligence?
RESPONDENT: Yes, your Honour, I did get the help of AI.”
That judgment (the AI parts anyway) pretty much speaks for itself….
[4] Some of the respondent’s oral submissions were intelligible and engaged with the matters raised on the appeal. Many of them, however, did not. It is useful to give some examples to highlight the serious shortcomings of the use of Generative AI at least by a person who is not capable of either checking the accuracy or veracity or relevance of what has been generated. Again, I emphasise that in setting out the following passages, I am not being personally critical of the respondent who was doing her best to represent herself. …….
[10] Again, without intending any disrespect to the respondent, she plainly had no understanding of what she was reading out to the Court. Ultimately, I indicated to her that her oral submissions were not very helpful:
“I appreciate you are representing yourself et cetera, but to the extent you’re just relying on something which has been produced by AI, from what you’ve read out, it is not really engaging with the arguments. That doesn’t mean that the Court is against you by any means. But there’s limited utility in just reading out something which has been produced—”
[11] It should also be noted that the respondent’s list of authorities identified under the heading “Cases to be read” the case of Tate v Ragg [2004] NSWCA 306: [66]-[70]. No such case exists. On the assumption that the respondent drew on Generative AI to produce her list of authorities, it is a matter of profound concern that the technology relied upon generated a non-existent case including pinpoint paragraph references to apparently relevant paragraphs. This has been and remains a serious issue: how does Generative AI produce facially credible citations to non-existent cases, still less provide paragraph references to such cases? And if it does so, what reliance can be placed on other legal references or propositions so produced? …..
[15] The problems of unverified use of artificial intelligence in the preparation of submissions are exacerbated where the technology is used by unrepresented litigants who are not subject to the professional and ethical responsibilities of legal practitioners and who, while subject to the Practice Note SC Gen 23, may be unaware of its terms. All litigants are under a duty not to mislead the court or their opponent: Vernon v Bosley (No 2) [1999] QB 18 at 37, 63, cited in Burragubba v State of Queensland (2016) 151 ALD 471; [2016] FCA 984 at [228]; see also, in relation to the obligations of unrepresented litigants, Barton v Wright Hassall LLP [2018] UKSC 12; [2018] 1 WLR 1119 at [18] (Barton)
[16] It is and will remain important for judicial officers to be conscious of the potential use of Generative AI by unrepresented litigants in legal proceedings and it is legitimate to inquire, as the Court did of the respondent in the present case, whether Generative AI has been used in the preparation of materials placed before the Court. Such use may introduce added cost and complexity to the proceedings and, where unverified, add to the burden of other parties and the Court in responding to it.
There is also a very instructive quote from the UK High Court of Justice case of Ayinde v The London Borough of Haringey [2025] EWHC 1383 (Admin) at [5]-[9] (Ayinde) . Clearly this issue is bigger in the UK than here.
The AI generated creative commons photo acknowledgment