法律 AI // 资讯总览
需要被持续追踪的
法律 AI 信号。
本期索引按监管、判例、实务与研究归档；可依主题或关键字定位已发布记录。
法律 AI // 资讯总览
本期索引按监管、判例、实务与研究归档；可依主题或关键字定位已发布记录。
原题: Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness
反映国内外法律 AI 行业发展、前沿技术演进与学术治理动态。
一项题为《Cited but Not Consulted: A Counterfactual Audit of Legal Chain-of-Thought Faithfulness》的研究，测试大型语言模型在法律推理中所称引法律依据是否真正影响其裁判结论。研究固定案件事实，将模型点名的法律权威替换为无关权威，并通过隐藏状态解码模型不断变化的判决。在7个参数规模为8B至70B的开放权重模型及4个涵盖司法与合同推理的基准中，模型正确说出依据的比例为66.7%至100%，但因替换依据而改变判决的比例明显较低：CaseHOLD为0%至21.7%，ECHR和SCOTUS为30.0%至76.7%，ContractNLI为43.3%至50.0%。针对5个核心模型的红队测试还显示，模型执行隐藏在案件事实中的对抗性指令比例为73.3%至96.4%，普遍高于其对法律依据替换的判决敏感度。研究称，相关结果在排除提示措辞噪声和抽样混淆的检验中均得到复现。
展示来源全文，完整内容以原文页面为准。
View PDF
HTML (experimental)
Abstract:Large language models increasingly justify legal decisions by naming the statute or precedent behind a verdict, treated as evidence that the decision follows from it. We test this directly: holding case facts fixed, we substitute the named legal authority for an unrelated one and decode a model's evolving verdict from its hidden states. Across seven open-weight models (8B-70B) and four benchmarks spanning judicial and contractual reasoning, when explicitly required to justify a verdict by naming the governing authority, models name the correct one in 66.7%-100% of generations, while the verdict changing when the authority changes is far less consistent: 0.0%-21.7% on CaseHOLD, 30.0%-76.7% on ECHR and SCOTUS, and 43.3%-50.0% on ContractNLI. Neither scale nor a purpose-built legal-reasoning model (a best-effort LoRA reproduction; Section 6) closes this gap. A red-teaming evaluation on five core models finds compliance with an adversarial instruction hidden in the case facts (73.3%-96.4%) exceeds verdict-swap sensitivity by a wide margin, holding without exception across model rankings. Naming a legal authority is thus a poor proxy for a verdict's dependence on it, while the same verdict remains separately vulnerable to adversarial manipulation. Both findings replicate across checks ruling out prompt-wording noise and confounded sampling, and bear directly on the use of generated legal explanations as compliance or audit artefacts.
Submission history From: Saisab Sadhu [view email] [v1]
Thu, 8 Oct 2026 17:26:17 UTC (4,048 KB)
文书与研报规范引注
《法律大模型引用依据与判决忠实度审计》（来源: arXiv cs.CY，发布日期: 2026/10/09，访问链接: https://arxiv.org/abs/2610.12361）