There have now been dozens of stories of bad lawyers using AI results in filings, most of which show up in the form of fake case citations. That's old hat at this point. But how about prompt injections in court filings? There have been a few people warning that it's coming. A year ago, Louisiana...
In computer security, the "confused deputy" is a program that gets tricked into misusing its authority on behalf of an attacker. Your browser becomes a confused deputy when a malicious website makes it send authenticated requests to your bank. The deputy has legitimate access. The attacker doesn't. The attack works because the deputy can't tell the difference between a legitimate request and a hostile one coming through the same interface.