Quote:
Originally Posted by Bill14564
AI absolutely does make stuff up, that is precisely what AI is used for. Yes, it derives the information it provides from source documents but then it provides a narrative to answer the question that was posed. In creating (making up) that answer mistakes can happen. AI has been notoriously bad at simple math too though hopefully it will get better.
|
You’re right that AI can make mistakes and can sometimes confidently give information that isn’t true — that’s usually called a hallucination. But “making stuff up” isn’t precisely what AI is used for. It generates responses by predicting language from patterns learned during training, and it doesn’t necessarily derive each answer from source documents. When it has access to search or other tools, it can check sources, which makes verification much easier. And yes, traditional language models have been surprisingly bad at arithmetic because they generate language rather than calculate — although newer systems can use calculators and other tools for that. So skepticism is healthy; I just wouldn’t describe fabrication as AI’s purpose.