Skip to content
TruthVouch Docs

Natural Language Inference (NLI) Scoring

Natural Language Inference (NLI) is the technology that powers hallucination detection. It measures whether one text statement logically follows from another — helping TruthVouch determine if an AI response aligns with your verified facts or contradicts them.

What is NLI?

NLI (also called Textual Entailment) answers: “Does Sentence A logically imply Sentence B?”

Three possible relationships:

Entailment (Implication)

Premise logically implies hypothesis:

Premise: "TruthVouch was founded in 2023"
Hypothesis: "TruthVouch was founded more than one year ago"
Score: ENTAILMENT (1.0)

Neutral (Unrelated)

Premise and hypothesis have no logical connection:

Premise: "TruthVouch monitors AI hallucinations"
Hypothesis: "The sky is blue"
Score: NEUTRAL (0.5)

Contradiction (Negation)

Premise contradicts hypothesis:

Premise: "TruthVouch is a SaaS platform"
Hypothesis: "TruthVouch is installed on-premises only"
Score: CONTRADICTION (0.05)

Scoring Scale

NLI scores range from 0.0 to 1.0:

1.0 ─────────────────────────────────── ENTAILMENT (matches)
0.5 ─────────────────────────────────── NEUTRAL (unclear)
0.0 ─────────────────────────────────── CONTRADICTION (conflicts)

Higher scores indicate stronger alignment between the premise and hypothesis. Thresholds for alert severity are configurable in your dashboard.

How TruthVouch Uses NLI

TruthVouch uses a purpose-built NLI model optimized for business and technical text. The model converts both your verified truth and the AI claim into semantic representations, then compares them to determine their relationship. This process happens in milliseconds, enabling real-time hallucination detection.

Limitations

NLI scoring has inherent limitations:

  • Ambiguous Text: Unclear or vague premises reduce accuracy
  • Subjective Claims: Opinion-based statements are difficult to classify
  • Implicit Context: Some claims require broader context to evaluate properly
  • Domain Specificity: Very specialized domains may have lower accuracy than business/technical domains
  • Complex Logic: Multi-step logical chains are harder to verify than direct claims

Configuring Detection Sensitivity

NLI scores are converted to alert severity in your dashboard. You can choose from preset configurations:

  • Strict: Minimize missed hallucinations (fewer false negatives) — more alerts
  • Standard: Balanced detection and false positive rate — recommended
  • Permissive: Minimize false alerts (fewer false positives) — some hallucinations may not trigger

Configure sensitivity in Dashboard → Settings → Detection Thresholds.

Common Score Patterns

Exact Match

Identical or nearly identical texts:

Premise: "TruthVouch costs $349/month"
Hypothesis: "TruthVouch costs $349/month"
Score: 0.99 (ENTAILMENT)

Paraphrase

Different wording, same meaning:

Premise: "TruthVouch was founded in 2023"
Hypothesis: "TruthVouch's founding year is 2023"
Score: 0.94 (ENTAILMENT)

Partial Match

Subset relationship:

Premise: "TruthVouch monitors 9+ LLM models"
Hypothesis: "TruthVouch monitors ChatGPT"
Score: 0.78 (Weak entailment, not guaranteed)

Negation

Opposite meaning:

Premise: "TruthVouch is cloud-based"
Hypothesis: "TruthVouch is on-premises"
Score: 0.08 (CONTRADICTION)

Temporal Shift

Different time period:

Premise: "Founded in 2023"
Hypothesis: "Founded in 2024"
Score: 0.12 (CONTRADICTION)

Unrelated

No semantic connection:

Premise: "TruthVouch monitors AI"
Hypothesis: "The Earth orbits the Sun"
Score: 0.51 (NEUTRAL)

Best Practices for Better Detection

Create Precise Truth Nuggets: “Starter plan is $349/month” detects better than “affordable pricing.”

Keep Nuggets Updated: Stale truths reduce detection accuracy. Review quarterly.

Use Multiple Query Variants: TruthVouch generates multiple query phrasings to catch different hallucination patterns.

Monitor Low-Confidence Alerts: Review borderline detections to tune your sensitivity settings.

Next Steps