Hallucination in Language Models is NEVER Going Away 

what a paper. full of useful info.

A simpler interpretation is that recall is over fitting the data while hallucinations are over generalization, ie it's not a bug but a feature, it's working as intended. Separating recall and generalizations is probably the way to solve the issues, which is what RAG does. Over generalisation is key for creativity and reasoning, because you apply known pattern and extrapolate, ie hallucinate. Categorizing type of hallucinations is probably better, ie context of when to apply it, or literal failure like hallucinations cause by collision of homonymous concept, such as type safety in programming vs in society. They are equivalent to the mangled hands ✋ in generative art. I call these continuity knots.