When New Mexico defense attorney Stephen Aarons fed a murder trial transcript into ChatGPT, he expected a bulletproof summary. Instead, he received a work of fiction that ultimately landed him in contempt of court and carrying a five-thousand-dollar fine. The state Supreme Court did not blink. They penalized Aarons not merely for experimenting with artificial intelligence, but for the profound professional negligence of failing to check his own paperwork before dropping it onto the judicial desk.
This incident is far from an isolated slip-up by a lone practitioner. It represents a dangerous, systemic collision between overworked legal professionals seeking automated shortcuts and probabilistic software designed to invent plausible sounding untruths. Across the country, courts are grappling with briefs containing phantom case citations, nonexistent judicial precedents, and now, entirely fabricated witness testimony in high-stakes criminal appeals.
The Mechanics of a Judicial Phantom
Large language models do not think, reason, or recall facts. They predict the next token based on statistical probabilities derived from vast training corpora. When an attorney feeds thousands of pages of dense trial transcripts into a consumer-grade chatbot and demands a condensed summary, the system encounters context windows that strain its architectural limits.
When the model encounters gaps in its capacity or its internal statistical mapping, it does not stop and raise a hand to confess ignorance. It extrapolates. It fills the void with synthetic prose that mimics the authoritative tone of legal documents.
In Aarons' case, the artificial intelligence generated descriptions of police testimony and witness statements that never occurred during the actual trial. Details about what a shooter wore—fictional statements about dark pants and a white shirt—slid seamlessly into an appellate brief because the software produced text optimized to sound correct rather than text tethered to reality.
For decades, the legal profession relied on associates, paralegals, and manual indexing. These traditional methods had error rates, of course, but human errors usually stemmed from fatigue, misinterpretation, or poor typing. They rarely involved inventing a breathing human being out of thin air to supply imaginary testimony on a murder charge.
The Reality of Professional Oversight
During oral arguments, New Mexico Supreme Court Justice C. Shannon Bacon expressed public disbelief at the defense lawyer's apparent ignorance of technology pitfalls.
"Counsel, do you watch the news? Do you listen to the radio? Do you read anything about what's going on in the world?" Justice Bacon asked from the bench. "Because the problem with lawyers relying on AI hallucinations is an above-the-fold story every single day."
That observation cuts straight to the core of institutional culpability. Legal technology warnings have saturated trade journals, mainstream newspapers, and continuing legal education seminars for years. Sanctions handed down in federal and state courts against attorneys who cited nonexistent federal cases made national headlines long before this New Mexico appeal.
Yet, practitioners under intense billing pressures continue to gamble with tools they fundamentally misunderstand. The appeal in question involved a life sentence for a conviction of murdering the mother of the defendant's children. Treating a capital or murder appeal with the administrative casualness of copy-pasting chatbot output reveals a systemic rot within segments of the defense bar. When attorneys treat caseload volume as an excuse to abdicate primary verification duties, the integrity of the adversarial system begins to fracture.
Why Bans and Warnings Fail
Law firms and court systems have tried various containment strategies. Some jurisdictions now require explicit certifications stating whether generative tools were used to draft filings. Other courts threaten immediate monetary sanctions or mandatory reporting to disciplinary boards for any submission containing unverified citations or facts.
These measures treat the symptom while ignoring the structural disease. The legal market operates on an economic model that punishes thoroughness and rewards output speed. Solo practitioners and small firm attorneys handling court-appointed indigent defense cases face crushing caseloads with minimal resources. For an attorney buried under tens of thousands of pages of discovery and trial logs with zero budget for human transcription analysis or large support staffs, the temptation to offload cognitive labor to a free or low-cost browser tab is immense.
The software vendors themselves offer disclaimers hidden in terms of service agreements, warning users that outputs may be inaccurate. But these warnings function as legal shields for tech companies rather than practical safeguards for courtrooms. The interface of a chat application encourages trust. It speaks in a calm, confident, professional register. It never stutters, hesitates, or adds a footnote admitting it guessed.
The Broader Erosion of Trust
The fallout extends far beyond individual fines or professional reprimands. Every time a court uncovers fabricated testimony or ghost case law generated by algorithms, public trust in the administration of justice takes another hit.
Skeptics of judicial reform already question the efficiency and fairness of modern criminal proceedings. When appellate courts must spend valuable judicial resources playing detective to verify whether a quoted witness actually sat on a witness stand, the entire apparatus grinds down.
Defense attorneys have an ethical duty of competence and diligence under professional responsibility rules. Blindly submitting unvetted algorithmic output satisfies neither standard. As long as the economic pressures driving the legal sector remain unchanged, attorneys will continue searching for shortcuts. Until the profession confronts the reality that artificial intelligence is an unreliable drafting assistant rather than an autonomous junior associate, courtrooms will continue unearthing ghosts hidden inside routine legal briefs.
