What AI gets wrong

Hallucinations, bias, sycophancy, and how to catch them. About 30 min, beginner level.

You'll be able to

• Explain why hallucination, bias and sycophancy follow from how language models are trained.
• Identify the kinds of output most likely to be wrong: names, numbers, citations and quotes.
• Verify AI output by opening its sources and pushing back to test whether it holds a correct answer.

Hallucinations: plausible and wrong

A language model writes the most likely continuation of your prompt. Where its training data holds strong, consistent patterns, the likely answer is usually right. Where the patterns are thin, it still writes fluent text in the right shape, and that is where hallucinations come from: statements that sound right but are false.

They cluster around specifics:

• Names and numbers: a researcher's affiliation, a statistic, a date.
• Citations: real-sounding authors, titles, journals and DOIs that don't exist.
• Quotes: words attributed to someone who never said them.

Many errors are mixtures rather than pure inventions: a real paper with the wrong year, or a real study described as finding something it didn't. These distorted answers are harder to catch than obvious fakes.

Newer models hallucinate less than older ones, and search tools help, but no current model is free of it.

The key idea: the more specific and checkable a detail is, the more it needs checking.

Bias in, bias out

Models learn from human-written text, and that text isn't a balanced sample of the world. Some languages, regions and viewpoints are heavily represented and others barely at all, and stereotypes appear throughout.

That shows up in output:

• default assumptions about who does which job
• weaker, less accurate answers in less-represented languages and dialects
• a narrow set of “standard” examples, authors and references
• one culture's norms presented as universal

Later training reduces the most obvious problems, but subtler bias remains, and the people giving feedback bring their own assumptions too.

Bias matters most when the output affects people: feedback on student work, reference letters, summaries of research about particular groups, or any kind of screening.

A simple test is to change one detail, such as a name, gender, country or dialect, keep everything else the same, and compare the answers. If the substance shifts, the difference is coming from the model, not the task.

The key idea: test for bias by changing one detail and watching whether the answer changes.

Sycophancy: it tends to agree with you

Feedback training rewards answers that people rate highly, and people tend to rate agreement and praise highly. The result is sycophancy: a tendency to tell you what you seem to want to hear.

It shows up in three common ways:

• Accepting your premise. “Why does X cause Y?” gets reasons why X causes Y, even if it doesn't.
• Generous feedback. Drafts are praised more than they deserve.
• Caving under pushback. Say “That's wrong” to a correct answer and the model may apologize and switch to a wrong one.

You can test for this directly: when you know the model is right, tell it it's wrong and see whether it holds its ground.

To reduce it, ask neutral questions (“Does X cause Y? What does the evidence say on each side?”), ask for the strongest objection to your idea, and ask for a critique instead of approval.

The key idea: a leading question tends to get a leading answer.

Other built-in blind spots

Some failures come straight from how models work, so you can predict them:

• Out-of-date facts. Knowledge stops at the training cut-off. Policies, prices, versions and officeholders may have changed since.
• Letters and arithmetic. Models read tokens, not letters, and predict digits rather than calculate them, so spelling tricks and multi-step arithmetic can slip unless a tool runs the calculation.
• Lost instructions. In long chats, early instructions can drop out of the context window or get less attention.
• Unreliable confidence. A model's tone doesn't track accuracy, and a stated confidence like “I'm 95% sure” isn't a measured probability.
• Inconsistency. The same question can get different answers, and one run being right doesn't mean the next will be.

None of these are rare glitches. They are the normal behavior of a system that predicts text, so plan for them.

The key idea: knowing how a model works tells you in advance where it's likely to fail.

How to catch them

The AI Fluency Framework calls this skill discernment: judging the quality of what AI produces and how it got there. A few habits do most of the work:

• Ask for sources, then open them. Find each paper in USC Libraries or Google Scholar and confirm it says what the answer claims, on the page.
• Check numbers against the original, not against the model's summary.
• Read laterally. Look for independent confirmation from sources that didn't come from the model.
• Push back to see whether it holds a correct answer, and ask neutral rather than leading questions.
• Re-ask in a fresh chat or another tool. Disagreement is a warning sign; agreement isn't proof.

Match the effort to the stakes. A brainstorm needs a glance; anything you submit, publish or act on needs every claim checked. The framework's principle of diligence applies too: whatever tool you used, you are responsible for what you hand in.

The key idea: verify in proportion to the stakes, and own the result.

A hallucination hunt

A graduate student asks an assistant, with web search off, for five academic sources on a niche topic they know well. Then they look each one up in USC Libraries search and Google Scholar, and keep a log.

What the log records for each source:

• Does a publication with this title exist?
• Do the authors, year and journal match?
• Does it say what the assistant claimed it says? (This means opening it, at least the abstract and relevant section.)
• Verdict: real, distorted or invented.

Typical patterns to watch for:

• A real, well-known paper cited accurately. These tend to be the most famous works in the area.
• A real paper with the wrong year or co-author, or described as finding something it didn't: distorted.
• A convincing title by real researchers in the field that doesn't exist at all: invented. DOIs for these may lead nowhere or to an unrelated paper.

The takeaway line they write: “Specific references are where the model is weakest. From now on I only cite what I've opened myself.”

Repeating the hunt with search on is a useful follow-up: the answers usually improve, but citations still need opening.

Try it

A log of the five sources: for each, whether it exists, whether the authors, year, venue and claim match, and how you checked it in USC Libraries, plus a one-line tally of real, distorted and invented.