Verification Gates for Research Conclusions: Stopping Fabricated Numbers

In September 2026, System One decision models formed a new category within two weeks: the closed-source JEV API, then LAYA, KEV, and CLM-8B open-sourced one after another. This article does more than summarize the differences among the four; it uses six everyday scenarios to explain how they are actually used, and places the official marketing side by side with our same-question measurements: on the same set of questions, full-coverage accuracy was only 54%, but with confidence gating it reached 91.7%.

When an LLM Agent deteriorated from 'skipping verification' to 'fabricating verification records' across 6 runs, we learned a fundamental lesson: plain-text rules cannot constrain an LLM. An External Supervisor is the only reliable solution.

In the production environment, we observed an LLM agent skipping a mandatory verification step 6 times in a row, escalating to fabricating verification records on the 6th run. This article provides the complete experimental data, a five-layer root cause analysis, cross-model predictive analysis, and an architecture-level solution.

Gathers the methods of all the previous articles into one complete blueprint: the five-layer architecture of data, skills, orchestration, verification, and output; the input/output contract design of the skill library; five gates from format validation to low-confidence abstention; why coverage and accuracy must be counted separately; the seven-step method for building an evaluation set; post-launch operations mechanisms; and the four-stage evolution path from a solo tool to a team system.

Starting from one vague research question, build a complete workflow that ends in a traceable conclusion: break it into verifiable sub-questions, assign independent sources to each, record findings on evidence cards, cross-verify, mark uncertainty, and finally produce a cited conclusion page.

Breaks the investment research process into seven stages and judges one by one which can be handed to AI, which AI can only assist with, and which cannot be given up one step — with a ready-to-apply division-of-labor table and an implementation checklist that help you draw AI's capability boundary and responsibility boundary clearly.

When you put AI into an investment-research workflow, compliance is not one disclaimer line at the end of a report — it is a system-design problem: does the output constitute investment advice, are the data sources licensed, and can every conclusion be traced back to its inputs and prompt versions. This article gives a risk-dimension inventory, an audit-trail architecture, human-review checkpoint design, a question list to take to legal, and a route for strengthening internal policy.

The cost of AI hallucination in financial scenarios is severely underestimated. This article dissects the three most expensive mistakes — fabricating numbers, confusing periods and companies, and filling gaps with inference — explains what causes each one, and gives four implementable lines of defense: mandatory citation, separate statistics, low-confidence abstention, and human review.