In a decision published last week, Connecticut judge Walter Spader Jr. confirmed that the hidden text had no impact in a case where a man alleged a healthcare provider was improperly withholding access to records. The court weighed his filing on the merits, Spader said, but nevertheless, the attempted attack sets a “dangerous” precedent. This will likely not be the last time US courts see the malicious tactic, as AI tools become more commonplace in court systems.
Shrunk to tiny-point type and colored white on a white background, the text appeared to be an attempt at prompt injection, with the plaintiff, Matthew Elliott, seemingly hoping to shift the court’s favor after earlier arguments he raised were defeated.
The plan didn’t work, but Elliott faced modest sanctions anyway because he continued adding hidden text to filings even after the court warned him that he could face penalties for what was ultimately deemed a “serious litigation abuse.” [...]
In his defense, Elliott claimed that the most concerning prompt that the judge flagged was an attempt to “audit” the court as a public service, out of fears that the court seemed to be letting AI unfairly decide cases.
But Spader suggested that if Elliott was truly concerned that the court was improperly using AI, he was “free to write so in plain, visible words that everyone could see and answer.” The fact that he hid the text is “evidence of its malicious purpose,” Spader said. [...]
Pro se litigants use chatbots wrong
Spader said that it’s “unsurprising” that people would start using prompt injection to attempt to sway court rulings since the attack is so common in other areas, such as in job hunting, where people hide text in resumes primarily reviewed by AI. The tactic is now “everywhere,” he said, and courts should be on the lookout for more litigants sneaking adversarial AI instructions into filings.
To Spader, there is a lesson to be learned from Elliott’s failed prompt injection attacks that he thinks “reaches well beyond this case.”
Elliott seemingly turned to prompt injection after using AI to build his case as a pro se litigant without a legal expert to assist in drafting his arguments. Such use is widespread among pro se litigants these days, Spader acknowledged, but those inexperienced in the courtroom are seemingly using chatbots in a way that hurts their cases, he suggested.
What frequently happens, Spader explained, is that pro se litigants build their argument backward, asking the chatbot to help them advocate only for their position, without ever asking the chatbot for the actual truth or to advance opposing arguments. This is “a genuine hazard of the technology, and one that judges now see often,” Spader said, as chatbot sycophancy then entrenches litigants in their arguments despite any ruling to the contrary. In Elliott’s case, defending his arguments fiercely meant turning to prompt injection to try to force the court to agree with him.
“An argument prompted only to agree with its author is, in the end, dishonest even with its author,” Spader said. “Those using these tools must ask them to test a position as readily as to advance it.”
Image: Liudmila Chernetska | iStock/Getty Images Plus
[ed. See also: Israel Is Paying Millions to Train AI Chatbots How to Talk About Gaza. It's Working (Drop Site):]***
"Since October, former Trump campaign manager Brad Parscale has been quietly overseeing an operation posting hundreds of blog posts on behalf of Israel. One article, titled “The Reality Behind Gaza’s ‘Journalists’: Terror Ties, Propaganda, and the Laws of War,” asserts that a majority of journalists in Gaza were linked to terrorist organizations. Another casts doubt on the killing of Hind Rajab, a five-year-old Palestinian girl killed by the Israeli military in 2024.The key intended audience of these sites is not concerned Americans, it’s not even humans—most of the sites average a few hundred unique visitors each month. Instead, Parscale and his firm, Clock Tower X, created them as part of a $46.5 million contract with the Israeli government to try and influence artificial intelligence-powered chatbots, tools like Claude or ChatGPT.
Parscale has made his goal of influencing artificial intelligence—often referred to as “LLM poisoning”—explicit. In his initial agreement with Israel, Parscale said that he would deploy “websites and content to deliver GPT framing results on GPT conversations” as part of the contract. More recently, his team even told Axios they are “seeing success” at getting popular AI systems to incorporate information from their sites, though they declined to provide data.
And it is working, according to disinformation experts who reviewed a Drop Site analysis of chatbot queries and training data, meaning tens of millions of Americans who use chatbots are increasingly likely to receive answers manipulated by Parscale on behalf of the Israeli government."
[ed. More here (Politico).]
