Hallucination

In detail

Hallucination is the failure mode where a language model generates fluent, confident output that contains invented facts, fake citations, made up function signatures or wrong numbers. The model is not lying in any intentional sense. It is pattern matching to what plausible text in that context would look like, even when it does not have grounded knowledge. Hallucination rates vary by task, model and prompt design and have decreased substantially with newer models, but they have not been eliminated.