A new research paper from OpenAI asks why large language models like GPT-5 and chatbots like ChatGPT still hallucinate, and whether anything can be done to reduce those hallucinations.
In a blog post summarizing the paper, OpenAI defines hallucinations as “plausible but false statements generated by language models,” and it acknowledges that despite improvements, hallucinations “remain a fundamental challenge for all large language models” — one that will never be completely eliminated.
To illustrate the point, researchers say that when they asked “a widely used chatbot” about the title of Adam Tauman Kalai’s Ph.D. dissertation, they got three different answers, all of them wrong. (Kalai is one of the paper’s authors.) They then asked about his birthday and received three different dates. Once again, all of them were wrong.
How can a chatbot be so wrong — and sound so confident in its wrongness? The researchers suggest that hallucinations arise, in part, because of a pretraining process that focuses on getting models to correctly predict the next word, without true or false labels attached to the training statements: “The model sees only positive examples of fluent language and must approximate the overall distribution.”
“Spelling and parentheses follow consistent patterns, so errors there disappear with scale,” they write. “But arbitrary low-frequency facts, like a pet’s birthday, cannot be predicted from patterns alone and hence lead to hallucinations.”
The paper’s proposed solution, however, focuses less on the initial pretraining process and more on how large language models are evaluated. It argues that the current evaluation models don’t cause hallucinations themselves, but they “set the wrong incentives.”
The researchers compare these evaluations to the kind of multiple choice tests random guessing makes sense, because “you might get lucky and be right,” while leaving the answer blank “guarantees a zero.”
Techcrunch event
San Francisco
|
October 27-29, 2025
“In the same way, when models are graded only on accuracy, the percentage of questions they get exactly right, they are encouraged to guess rather than say ‘I don’t know,’” they say.
The proposed solution, then, is similar to tests (like the SAT) that include “negative [scoring] for wrong answers or partial credit for leaving questions blank to discourage blind guessing.” Similarly, OpenAI says model evaluations need to “penalize confident errors more than you penalize uncertainty, and give partial credit for appropriate expressions of uncertainty.”
And the researchers argue that it’s not enough to introduce “a few new uncertainty-aware tests on the side.” Instead, “the widely used, accuracy-based evals need to be updated so that their scoring discourages guessing.”
“If the main scoreboards keep rewarding lucky guesses, models will keep learning to guess,” the researchers say.
Source link
#bad #incentives #blame #hallucinations #TechCrunch
![‘Alice in Borderland: Retry’ Makes Up For the Netflix Series Reheating ‘Squid Game’ Nachos
Not to pick at an irritated scab, but when the eighth and final season of Game of Thrones… happened… I, a fan of the series via the show, naively assumed all author George R.R. Martin had to do was pen a different bookend to the series to recover all the lost goodwill of the series. Little did I know that was a touchy subject among book-readers who’ve been tortured by the author’s perpetual procrastination for years. Condolences. While the same scenario isn’t fully analogous with what’s happened with Netflix’s live-action Alice in Borderland TV show and VIZ Media’s English manga release of the series’ extended epilogue, Alice in Borderland: Retry, the manga more than retroactively makes up for the huge disappointment I felt witnessing the show I loved go full Squid Game in the worst way possible in its final season. [embed]https://www.youtube.com/watch?v=49_44FFKZ1M[/embed] Alice in Borderland is a cutting series in which a man named Arisu gets trapped in a sci-fi death game where a group of sequestered survivors must compete in a litany of challenges in an abandoned city. Think Gantz meets Kaiji. Its rules are cruel, but never completely ass-pull, with players logically outfoxing each other to win games lest a laser from the sky punch them through the top of their heads.
The show was so good I didn’t realize it was based on a manga by Haro Aso (Zom 100: Bucketlist of the Dead) that was also adapted into an anime—a fact that speaks to its overall quality in a sea of lackluster live-action Netflix projects. And for two seasons it came and went, adapting that story prior to Squid Game mania and, subsequently, in the throes of its pop culture takeover. That is, until a third season was released last year, ripping Arisu and his lover Usagi from their peaceful lives post-death games and throwing them back into the fray like a kid with a magnifying glass looming over an ant hill.
[embed]https://www.youtube.com/watch?v=_8p6YkEPVco[/embed] I was rocking with the series for a bit despite the show clearly making stuff up as it went along until it committed the tacky, cardinal sin of copying what Squid Game’s third season did in its final moments, desperately sequel-baiting itself by asking the question no one was curious about in a stinger: What if it took place in America? May that answer never come because of the road it took to tarnish the show’s reputation. As far as I’m concerned, there was no third season. Alice in Borderland Retry is now available in print and digital! Read a free preview: https://t.co/OJcjm9wliH pic.twitter.com/8iqGzwjTIV — VIZ Media (@VIZMedia) July 21, 2026 So imagine my surprise to find that Viz Media’s recently released localization of Aso’s 2020 epilogue manga to the series, set 10 years after its conclusion, would retroactively act as a remedy to the Netflix series’ final season I wish I could wipe from my mind palace. The manga starts out almost exactly the same as the third season of the Netflix series (duh, the manga came out first), with Arisu going from a NEET to a psychologist and Usagi expecting their first child. That’s the last thing similar between the two.
Unlike the Netflix series’ contrived reasoning to get everyone back into the games, Arisu winds up in the games for a pretty simple reason. He and five other participants get hit by a billboard and wind up in the Borderlands. If you’ve watched the show or read the manga, this inciting incident is far from the cataclysmic events that send Arisu to the Borderlands the first time around, but what it lacks in spectacle, it more than makes up for in the challenge that awaits them. In contrast to the more-is-more inventiveness of the Netflix series’ games in its final season, Arisu and company only have to play one game, which is essentially musical chairs but for keeps. Despite the simplicity of the game, the stakes of the 300+ page epilogue far exceed those of the Netflix series, where its collective of players, composed of an insurance employee, a nursery school teacher, an emergency room nurse, a doctor, and an unemployed man, have to debate whose life matters more. [embed]https://www.youtube.com/watch?v=Dq2p_B5MD0c[/embed] It’s rare that manga get a chance to reach a conclusion without getting the axe, and just as rare for series on an indefinite hiatus to start back up regardless of whether a creator is healthy, willing, and able to spin more narrative yarn years after the fact. Which begs the question whether the juice is worth the squeeze to loop back to a series that’s already over. Evidently, I wasn’t the only one in doubt of what, if anything, could be gained from revisiting a series whose manga concluded pretty succinctly. Aso had his doubts too, which he lays bare near the midpoint of the manga, writing the following:
It’s been ten years since I started working on Alice in Borderland. Like Arisu, I didn’t want to ever revisit a horrifying world like Borderland, but I was given the opportunity to write a sequel and I just couldn’t pass it up. What does it mean to live? What do our lives mean? Now, at 40 years old, I want to retry answering these questions. While its game is a deceptively simple one, a fatal road of agony, betrayal, and second-guessing the inherent good in the human spirit left me in awe as I flipped through its pages and found that Aso still had it in him to concoct an intense, not-so-unwinnable death game, and that what he had to say serves as a footnote to a series that’s more than earned its place in the pantheon of death game narratives. Want more io9 news? Check out when to expect the latest Marvel, Star Wars, and Star Trek releases, what’s next for the DC Universe on film and TV, and everything you need to know about the future of Doctor Who. #Alice #Borderland #Retry #Netflix #Series #Reheating #Squid #Game #NachosAlice in Borderland,Manga,Netflix,Viz Media ‘Alice in Borderland: Retry’ Makes Up For the Netflix Series Reheating ‘Squid Game’ Nachos
Not to pick at an irritated scab, but when the eighth and final season of Game of Thrones… happened… I, a fan of the series via the show, naively assumed all author George R.R. Martin had to do was pen a different bookend to the series to recover all the lost goodwill of the series. Little did I know that was a touchy subject among book-readers who’ve been tortured by the author’s perpetual procrastination for years. Condolences. While the same scenario isn’t fully analogous with what’s happened with Netflix’s live-action Alice in Borderland TV show and VIZ Media’s English manga release of the series’ extended epilogue, Alice in Borderland: Retry, the manga more than retroactively makes up for the huge disappointment I felt witnessing the show I loved go full Squid Game in the worst way possible in its final season. [embed]https://www.youtube.com/watch?v=49_44FFKZ1M[/embed] Alice in Borderland is a cutting series in which a man named Arisu gets trapped in a sci-fi death game where a group of sequestered survivors must compete in a litany of challenges in an abandoned city. Think Gantz meets Kaiji. Its rules are cruel, but never completely ass-pull, with players logically outfoxing each other to win games lest a laser from the sky punch them through the top of their heads.
The show was so good I didn’t realize it was based on a manga by Haro Aso (Zom 100: Bucketlist of the Dead) that was also adapted into an anime—a fact that speaks to its overall quality in a sea of lackluster live-action Netflix projects. And for two seasons it came and went, adapting that story prior to Squid Game mania and, subsequently, in the throes of its pop culture takeover. That is, until a third season was released last year, ripping Arisu and his lover Usagi from their peaceful lives post-death games and throwing them back into the fray like a kid with a magnifying glass looming over an ant hill.
[embed]https://www.youtube.com/watch?v=_8p6YkEPVco[/embed] I was rocking with the series for a bit despite the show clearly making stuff up as it went along until it committed the tacky, cardinal sin of copying what Squid Game’s third season did in its final moments, desperately sequel-baiting itself by asking the question no one was curious about in a stinger: What if it took place in America? May that answer never come because of the road it took to tarnish the show’s reputation. As far as I’m concerned, there was no third season. Alice in Borderland Retry is now available in print and digital! Read a free preview: https://t.co/OJcjm9wliH pic.twitter.com/8iqGzwjTIV — VIZ Media (@VIZMedia) July 21, 2026 So imagine my surprise to find that Viz Media’s recently released localization of Aso’s 2020 epilogue manga to the series, set 10 years after its conclusion, would retroactively act as a remedy to the Netflix series’ final season I wish I could wipe from my mind palace. The manga starts out almost exactly the same as the third season of the Netflix series (duh, the manga came out first), with Arisu going from a NEET to a psychologist and Usagi expecting their first child. That’s the last thing similar between the two.
Unlike the Netflix series’ contrived reasoning to get everyone back into the games, Arisu winds up in the games for a pretty simple reason. He and five other participants get hit by a billboard and wind up in the Borderlands. If you’ve watched the show or read the manga, this inciting incident is far from the cataclysmic events that send Arisu to the Borderlands the first time around, but what it lacks in spectacle, it more than makes up for in the challenge that awaits them. In contrast to the more-is-more inventiveness of the Netflix series’ games in its final season, Arisu and company only have to play one game, which is essentially musical chairs but for keeps. Despite the simplicity of the game, the stakes of the 300+ page epilogue far exceed those of the Netflix series, where its collective of players, composed of an insurance employee, a nursery school teacher, an emergency room nurse, a doctor, and an unemployed man, have to debate whose life matters more. [embed]https://www.youtube.com/watch?v=Dq2p_B5MD0c[/embed] It’s rare that manga get a chance to reach a conclusion without getting the axe, and just as rare for series on an indefinite hiatus to start back up regardless of whether a creator is healthy, willing, and able to spin more narrative yarn years after the fact. Which begs the question whether the juice is worth the squeeze to loop back to a series that’s already over. Evidently, I wasn’t the only one in doubt of what, if anything, could be gained from revisiting a series whose manga concluded pretty succinctly. Aso had his doubts too, which he lays bare near the midpoint of the manga, writing the following:
It’s been ten years since I started working on Alice in Borderland. Like Arisu, I didn’t want to ever revisit a horrifying world like Borderland, but I was given the opportunity to write a sequel and I just couldn’t pass it up. What does it mean to live? What do our lives mean? Now, at 40 years old, I want to retry answering these questions. While its game is a deceptively simple one, a fatal road of agony, betrayal, and second-guessing the inherent good in the human spirit left me in awe as I flipped through its pages and found that Aso still had it in him to concoct an intense, not-so-unwinnable death game, and that what he had to say serves as a footnote to a series that’s more than earned its place in the pantheon of death game narratives. Want more io9 news? Check out when to expect the latest Marvel, Star Wars, and Star Trek releases, what’s next for the DC Universe on film and TV, and everything you need to know about the future of Doctor Who. #Alice #Borderland #Retry #Netflix #Series #Reheating #Squid #Game #NachosAlice in Borderland,Manga,Netflix,Viz Media](https://gizmodo.com/app/uploads/2026/07/Alice-in-Borderland-1280x853.jpg)
Post Comment