法律 AI // 资讯总览
需要被持续追踪的
法律 AI 信号。
本期索引按监管、判例、实务与研究归档；可依主题或关键字定位已发布记录。
法律 AI // 资讯总览
本期索引按监管、判例、实务与研究归档；可依主题或关键字定位已发布记录。
原题: Verdict Without the Rule: Diagnosing and Auditing Regulatory Rule Sensitivity in LLM Compliance Systems
反映国内外法律 AI 行业发展、前沿技术演进与学术治理动态。
该研究检验大型语言模型合规系统的判断是否真正依赖所提供的监管规则。研究人员在五个模型、20个监管及平台政策领域中固定案件，仅删除、替换或否定适用规则，并通过结果一致性指标（OCS）及模型内部合规表征变化指标（ICS-delta）观察判断是否改变。结果显示，面对实质性规则扰动，模型的合规结论通常变化不大；在采用自定义规则适配原生分类体系的测试中，guard模型规则敏感度和准确率均为五个模型中最低，准确率仅为51%，而通用模型为90%至92%。研究同时发现，模型并非完全忽视规则：在删除规则会改变此前正确预测的案例中，模型能够较好地追踪规则变化。更优提示方式和直接干预内部表征也未弥合这一差距。
展示来源全文，完整内容以原文页面为准。
View PDF
HTML (experimental)
Abstract:Large language model compliance systems are deployed on the assumption that a verdict depends on the regulatory rule it is given. We test this directly across five models and 20 regulatory and platform-policy domains: delete, swap, or negate the governing rule while holding the case fixed, and check whether the verdict changes (OCS) or the model's internal representation of compliance shifts at all (ICS-delta). Neither moves much: models' verdicts are often invariant to substantial perturbations of the supplied rule, and the guard model, evaluated here under a custom-rule adaptation of its native taxonomy, is the least rule-sensitive and least accurate of the five, barely above chance (51%, versus 90-92% for general-purpose models). This reflects easy cases more than blanket neglect: on cases where deleting the rule changes a previously correct model prediction, models do track it closely. Neither better prompting nor direct intervention on the model's internal representations closes this gap. Accuracy alone does not establish that a compliance verdict is grounded in the supplied rule.
Submission history From: Saisab Sadhu [view email] [v1]
Thu, 8 Oct 2026 16:59:40 UTC (1,807 KB)
文书与研报规范引注
《研究发现大模型合规判断未必遵循所给规则》（来源: arXiv cs.CY，发布日期: 2026/10/09，访问链接: https://arxiv.org/abs/2610.12313）