Because they were trained on reddit…
They don’t know that they don’t know.
Think about how often you see a question raised and then responded with “I don’t know” in the internet data.
Wikipedia, textbooks, book books, forums, wiki how…
The vast majority of time, the format of what you see is:
<short question>? <long winded answer or how to guide>And thats what the llms are all trained on, that’d the most common pattern, so llms replicate that pattern.
The majority if their training data is off wikis abd textbooks, which pretty much never have “I dunno” anywhere. If an answer isn’t known, instead, the page simply doesn’t exist in the first place.
This just demonstrates a funadmentally flawed understanding of LLMs… they don’t know anything they are generating the text that is stastically likely to follow. They will still generate what is most stastically likely for things they “don’t know”
I loved the confident response I got after a few corrections.
“Thank you for your patience and assistance in correcting my mistakes, I will finally provide you with the correct response to your question…” Continues with a completely incorrect response lol
They’re trained on reams of work emails from people who would lose their livelihoods if they admitted to not knowing something
