Hallucination
Hallucination is when an AI makes something up and presents it as fact. It might cite a research paper that does not exist, give you a command with the wrong flags, or describe a feature that a product never had. The dangerous part is that hallucinated answers sound just as confident as correct ones. This is why you should always verify AI-generated information against authoritative sources, especially for technical work.
Hallucination in AI refers to model outputs that are fluent and plausible but factually incorrect, fabricated, or ungrounded. It is an inherent property of autoregressive language models: they predict the most likely next token based on statistical patterns, not by reasoning from verified facts.
Types of hallucination:
- Factual fabrication: inventing facts, statistics, dates, or citations that do not exist
- Entity confusion: mixing up attributes of different but related entities
- Unfaithful generation: contradicting the provided context (in RAG scenarios, the model ignores retrieved documents and generates from its priors)
- Logical inconsistency: generating internally contradictory statements within the same response
- Confident uncertainty: presenting uncertain information with high confidence and no hedging
Causes:
- Training data contains errors or conflicting information
- The model has no mechanism for “I do not know”; it always generates something
- Statistical patterns favor plausible-sounding completions over accurate ones
- Knowledge cutoff: the model lacks information about events after its training date
- Ambiguous prompts that do not constrain the output sufficiently
Mitigation strategies:
- RAG: ground responses in retrieved source documents
- Citations: require the model to cite specific sources, making fabrication auditable
- Temperature reduction: lower temperature reduces creative (but potentially wrong) outputs
- Self-consistency: generate multiple responses and cross-check for agreement
- Constrained decoding: limit output to known-valid values (structured output, tool use)
- Human review: always verify critical outputs (code, configurations, security decisions)
Detecting and mitigating hallucination
# Anti-hallucination prompt pattern: constrain to context
system_prompt = """Answer ONLY based on the provided context.
If the context does not contain enough information to answer,
say "I don't have enough information to answer this."
Never invent facts. Cite the specific section of context used."""
# Structured output to reduce hallucination
from pydantic import BaseModel
class NetworkAnswer(BaseModel):
answer: str
confidence: str # "high", "medium", "low"
source: str # Which context section was used
caveats: list[str] # Limitations or uncertainties
# Force the model to output structured data
# If it cannot fill 'source', hallucination becomes visible
# Self-consistency check: ask the same question 3 times
# If answers disagree, flag for human review
responses = [ask_llm(question) for _ in range(3)]
if len(set(responses)) > 1:
print("WARNING: Inconsistent responses detected")
print("Flagging for human review")Hallucination is the primary limitation preventing AI from being fully autonomous in critical tasks. In software development, hallucinated API calls with incorrect parameter names compile but fail at runtime. In cybersecurity, a hallucinated CVE number or remediation step could lead teams in the wrong direction. Legal firms have been sanctioned for submitting court briefs containing AI-hallucinated case citations. The RAG pattern is the most practical mitigation: by providing the model with actual source documents, it generates answers grounded in real data rather than training-time statistics. For IT professionals, the rule is: trust AI for drafting, brainstorming, and acceleration, but verify every factual claim, command, configuration, and code snippet against official documentation before deploying to production.