ChatGPT has found itself in trouble again, and this time it happened because of its hallucination in a court case. A defense lawyer appealing his client’s murder conviction submitted a brief containing made-up police testimony and witness testimony fabricated by OpenAI’s ChatGPT, as revealed by New Mexico’s highest court. The New Mexico Supreme Court fined the attorney Stephen Aarons and held him in contempt for failing to verify the accuracy of the court filing, which Aarons said he prepared with help from the AI program.
The court said that the file contained, ‘false testimony from wholly fabricated witnesses.’ The panel also said Aarons has, ‘demonstrated a lack of remorse and a lack of concern for his client.’ The justices fined Aarons $5,000 and said they will refer him to the attorney disciplinary board for investigation. Aarons, in a statement to Reuters, said he had used ChatGPT to summarize the trial proceedings when he agreed to take up the defendant’s appeal last year, and did not understand the degree to which AI could ‘hallucinate’ facts.
He said, ‘I am remorseful but hopeful that the disciplinary board takes into account it was an honest mistake. It is a lesson learned for all professionals who rely upon this powerful but sometimes unstable technology.’ That’s not happening for the first time, as dozens of lawyers have been sanctioned for filing briefs where AI made up case citations or misquoted the law.
Aarons’ filing appears to have gone further, containing fabricated witness testimony in a criminal appeal. Aarons, a private attorney based in Santa Fe, was handling the appeal of Oscar Renee Sandoval, who pleaded not guilty to the murder of his children before he was convicted and sentenced last year to life in prison. The appeal is still pending and was assigned on September 2 to Kim Chavez Cook, a New Mexico public defender.
