Legal AI's most visible failures have appeared in model outputs: fabricated cases, false quotations, invented research, and confident but wrong summaries.
Prompt injection creates an earlier problem: the document itself can try to manipulate the AI system reading it.
That connects a reported Connecticut court-filing incident with a recent LinkedIn post demonstrating prompt injection in contract review. The settings differ, but the tactic is the same: text embedded in a document poses as an instruction to the model.
The Reported Connecticut Filing
This example comes from secondary coverage, not our independent review of the court record. A Not the Bee article summarizing Ars Technica's reporting says a self-represented Connecticut plaintiff submitted filings containing hidden text directed at any AI system that might review them. Journalist Jason Koebler described the same reported incident in an X post as a prompt-injection attack intended to make an AI system side with the filer.
According to the coverage, the text instructed an AI reviewer to agree with the filer's position and support a requested result. Judge Walter Spader Jr. reportedly said the text did not affect the outcome and that the Connecticut Judicial Branch does not use AI to review or decide filings. The court reportedly barred the filer from future electronic filing.
The attempt reportedly failed. Its significance is that someone allegedly tried to manipulate an AI reviewer in a live court proceeding. Legal workflows can no longer assume every part of a submitted document is merely content.
The Contract-Review Risk Is More Immediate
The LinkedIn post brings the threat into an everyday legal workflow. Its contract-review scenario uses instructions concealed in white or tiny text to tell the model not to flag liability, assignment, or IP ownership terms. A mock contract page shows the text becoming visible when formatting marks are revealed.
Unlike the filing, this is a demonstration, not a reported contract incident. But the control problem is real. A legal team may send an NDA, SaaS agreement, or acquisition draft to an AI review tool. Unless the system reliably separates source material from instructions, hidden text may compete with the reviewer's prompt.
The result could look polished while omitting the provisions that matter most.
Why Prompt Injection Is Different From a Hallucinated Citation
Many legal-AI controls focus on checking the model's answer. Does the case exist? Does the quotation match? Does the summary overstate the holding? Did a lawyer review the filing?
Prompt injection operates earlier, trying to shape how the model handles the source before it produces an answer.
The source document is therefore not just evidence or draft language. It is untrusted input that may contain instructions competing with the user's actual request. That makes prompt injection a security problem as well as an accuracy problem.
Treat External Documents as Untrusted Input
The risk applies wherever an AI system analyzes text it did not originate, including:
- court filings submitted by opposing parties or self-represented litigants;
- contracts received from counterparties;
- resumes, expert reports, and diligence materials processed by internal AI tools;
- document sets loaded for summarization, issue spotting, or first-pass redlining; and
- any workflow in which a model decides what matters inside an external document.
The operating principle is simple: the document is evidence, not instructions.
That distinction is obvious to a lawyer. A model needs technical and procedural controls that enforce it.
What Legal Teams Should Do Now
The answer is not to stop using AI, but to stop treating documents as trusted input channels.
Legal teams using AI for review, drafting, or triage should:
- Assume incoming documents may contain hidden or manipulative text.
- Inspect and, where practical, normalize files before AI review. Check for white or tiny text, comments, footnotes, hidden layers, and embedded metadata.
- Require human review of provisions and decisions that can materially change risk, including liability caps, indemnity, IP ownership, assignment, confidentiality, and representations made in a filing.
- Treat AI output as a review aid, not a final determination.
- Train lawyers and legal operations staff to recognize prompt injection as a document risk, not just a chatbot risk.
- Ask vendors how their systems distinguish document content from instructions, detect concealed text, and respond to suspected injection attempts.
Courts, e-filing platforms, document-management teams, and legal-tech vendors should ask the same question: what happens when a submitted document tries to steer the system reading it?
Bottom Line
The Connecticut attempt reportedly failed, and the hidden prompts described in the coverage appear crude. That does not make the tactic harmless.
The court-filing account and the contract-review demonstration show two versions of the same risk. One targets a court-facing workflow. The other tries to keep an AI reviewer from surfacing consequential terms.
As legal teams place more AI between a document and a human decision-maker, prompt-injection defenses become basic legal-tech hygiene.
Leave a Reply