General Discussion
Related: Editorials & Other Articles, Issue Forums, Alliance Forums, Region ForumsYou've probably heard that AI chatbots can completely fabricate quotes. It happened yesterday in a DU thread.
The thread is in the Science Fiction forum, at https://www.democraticunderground.com/12002305 . I hope you'll read all of it. But see in particular the OP and replies 8, 11, 12, 15, 17, 19, 20 and 21.
DUer raccoon was trying to remember a partucular time travel story. There were a number of replies. One, from Goonch, had both a title and author, and a quote that was apparently the ending of the story:
Elias punched the coordinates for 1924 into the brass console, desperate to see his grandmother one last time. He pulled the lever, expecting the smell of ozone and the sight of her garden; instead, he was met with an absolute, crushing silence .
When the flash faded, there was no garden. There was no air. Through the reinforced glass of his pod, Elias stared at the cold, indifferent glow of distant nebulae [3]. He hadn't accounted for galactic driftwhile he had traveled back a century in time, the Earth had continued its relentless orbit around the Sun, and the Sun had continued its 500,000-mile-per-hour sprint around the Milky Way.
He had reached the right time, but the Earth was billions of miles away . He was a ghost in the vacuum, a man who had forgotten that in the universe, you can never go back to the same place twice.
It looked like a pretty convincing answer. If I'd just run across it, I doubt I'd've thought twice about its accuracy.
But I knew it couldn't be right because I'd posted a reply hours earlier, mentioning the same story title and author, and linking to pages I'd found referring to it. I didn't have a quote from the story, and I've never read it.
But I had run across and linked to pages summarizing the story, about a girl named Marla (not a man named Elias) who went forward - not backward - in time, but also ended up in outer space.
So I asked about the source of the quotation above.
Turned out it had been fabricated by a chatbot. I've read lots of articles and social media posts about chatbots fabricating quotes, but it was the first time I'd seen this happen in a thread I was posting in. I thought it would be a perfect example to post here to show why chatbot responses should never be trusted without careful checking.
I want to thank Goonch for posting the explanation, which included the chatbot admitting the quote was apparently fabricated. Goonch also posted another helpful message with what the chatbot, Google's, had said later about chatbot dangers and the need to verify information, and it's an interesting read.
SheltieLover
(78,613 posts)AZJonnie
(3,294 posts)And it pulled some of it's explanations from ... dun-dun-duh ...
Democratic Underground!
highplainsdem
(60,988 posts)other posts on DU, from me and/or other DUers, about chatbots making things up? Could you mail that response to me here, if you don't want to post it?
AZJonnie
(3,294 posts)Claude's citation for the below text IS the thread that you and I are talking in right now
Verbatim paste:
When short fiction circulates online, people:
Change character names (Elias → Marla, etc.) to personalize or improve it for their own post.
Edit details like dates or the specific time‑travel instructions to fit a different prompt, meme, or context (e.g., Reddit writing prompts, AI demos).
Let AIs regenerate the same idea, which preserves some lines but changes surface details like names and wording.
Because theres no canonical publication to anchor it, each repost or AI regeneration becomes a version, and they drift apart over time.
Is there an original version?
With no traceable first publication (e.g., an early dated story on a known authors site, magazine, or book), there is no reliably verifiable original here. What youre seeing is a meme‑like text: one early version got popular, and later copies, edits, and AI paraphrases created the Elias, Marla, and altered‑directions variants, none of which can be definitively proven to be the first.
In short: both versions are internet variants of the same micro‑fiction idea, not differing print editions of a book passage, and the differences come from casual edits and AI/reader rewrites rather than from an author‑sanctioned original.
highplainsdem
(60,988 posts)are no fan-fiction versions of that story, AFAIK, and none of the searches I did for different sentences in that fake quote written by another chatbot turned up any results.
Claude completely invented a reason for what happened that had nothing to do with the reality I explained here.
The story itself is real. The other chatbot got the title and author right, and mentioned one of the books the story has been published in, but invented a fake ending.
I'd give Claude an F, but there really should be a grade Z for being so incredibly wrong.
AZJonnie
(3,294 posts)In fact, I use Claude all the time, and it is very solid at getting quotes right when the attribution is clear-cut, but not when it finds a forum discussion thread like this. You also don't know the prompt that provoked this response which was very lackadaisically composed on my part.
AI getting this completely right would be outside the scope of a cheap or free subscription. This discussion, over multiple threads, involving multiple people, is simply too complicated for it to parse, given there's a limit to how hard it will work on any given response. All that it 'got completely wrong' is that it didn't search the entire history of the entire internet to see if the blurb was from any publication ever posted to the internet.
If I had a $100/month subscription using a model with more horsepower, and I wasn't using a model that specializes in coding work but rather on interpreting conversations, it would've done a better job. You get what you pay for in the AI world, and you need to use the right tool for the task, and you ABSOLUTELY have to compose good prompts
But your overall point is taken: It's established that AI makes mistakes and should be reviewed.
P.S. I understand how and why "writing" generally, and proper attribution in particular is very near and dear to you and understand why this mistake (in Goonch's post) is particularly irksome
highplainsdem
(60,988 posts)the story and author were real. Claude doing a search with any search engine would also have shown that it's a real story.
Hell, Anthropic illegally used so many books for training that Claude might've been trained on the story.
I don't understand why everyone isn't outraged by the theft of the world's knowledge and intellectual property by AI robber barons. What they stole was worth trillions, and everyone involved in the IP theft belongs in prison.
And what's called the AI pollution of our information ecosystem is a catastrophe. GenAI is destroying people's ability to find out what's real and what isn't.
If all the AI bros spent the rest of their lives in prison, and their companies were sued out of existence, it still wouldn't begin to atone for all the harm they've done in just a few years.
And as for coding - one OP I posted about Matt Shumer the other day quoted an article Gary Marcus had linked to, about how unsafe AI-generated code is. It mentioned a survey showed half of developers trust AI coding so much now that they aren't even checking the code. That's a lot of catastrophes waiting to happen.
And then there's AI wrecking education and harming the natural environment...
GenAI is the most harmful tech ever developed, and the stupidest.
AZJonnie
(3,294 posts)Therefore Claude assumed that the other AI knew what it was talking about, which it did. Because the Goonch's AI said it "appears to be a modern AI-generated "re-telling" or a hallucination of the specific prose", and THAT is indeed true i.e. that IS "what (the offending quote) *appears* to be". Claude then made the one (and only major) mistake of interpreting "appears to be" to mean "is", and riffed on the idea, which is describing a real phenomenon/how this sort of thing happens generally. That isn't that bad IMHO.
The reason the 'canonical' connection was not made is because either the text of the short story is not visible to it, OR it does not significantly match Goonch's quote, which I provided it with and implied the original it may feature a protagonist named Marla or Elias, but is largely matching this text provided.
To test this hypothesis, here's the rundown with what happened in Claude when I tried what Goonch tried, switching it up enough that Claude wouldn't find the DU thread we're on:
I'm trying to remember a story I read (scifi) where the protagonist tried do time travel (forget if it was the past or future) but it didn't work because the universe had grown or something like that and they ended up in space instead. Can you think what that sounds like?
I think they tried to move exactly 1 year and the time traveler was female?
I think the protagonist's name is Marla, does that help?
Key matching details:
The protagonist is a girl named Marla Nixbok who thinks of herself as a futuristic type and becomes obsessed with a forgotten time machine built by a mad scientist. (snip)
Is the full text available to you? If so, can you recall a couple paragraphs around the denouement where she finds herself in space, that exact moment?
I understand and agree with the large majority of your complaints in general, but I'm not coding missile guidance systems over here. Heck a lot of what I do is purely internal for our inhouse systems, which are web-based.
And I have never seen Claude do anything REMOTELY like what happened with Goonch.
ETA: If someone used AI to write a story similar to Same Time, Next Year, gave it a name of Elias to use as protagonist, and that user had the "use my input for training" checkbox turned on, that can be why we saw Goonch's answer, even if "someone" never published what AI wrote for it onto the internet itself.
ProfessorGAC
(76,222 posts)There is mo cosmological theory that says you can't go the same place in 2 different times.
It's the opposite.
You can't occupy 2 places at the same time. (A fatal flaw in movies, Back To The Future for instance).
So the bot even got the explanation wrong by making up a rule that doesn't exist.
dalton99a
(92,986 posts)AZJonnie
(3,294 posts)I think it makes sense if the assumption is that everything is moving at all times due to the expansion of the universe.
Another way of stating it is that the idea of "the same place" is a construct of the human mind. Despite your perception, you are not in the same place that you were .000000001 seconds ago, and will never be at that same place again. If you COULD somehow return to 'the same place you are right now', it could only happen if you returned to the same time it is right now.
IOW, the "place" you are at any given split second is not just a location, it's also a time.
No?
ProfessorGAC
(76,222 posts)I see a big difference between the likelihood of occupying the exact same spot at 2 times & a cosmological prohibition of doing so.
We agree on the extremely low probability, especially given our inability to traverse interstellar distances.
But, there is nothing in astrophysics that say we cannot do it if we had a ST style warp drive.
The theory does, in fact, prohibit the opposite, because it would require the creation of mass (which hasn't happened since shortly after the Big Band), with the same quantum properties. The former is not possible; the latter stupendous improbable. (Heisenberg and all that)
So, we're on the same page that it would be extraordinarily difficult to occupy the same space at more than one time, but there is nothing in the theory that absolutely prevents it.
I'm not a cosmologist, but I've been an avid student/obersver of that field and the math is not terribly different than that used in quantum chemistry, which I do know.
On board?
AZJonnie
(3,294 posts)I do know enough to say that whether or not there's a 'law' in this regard depends upon how you define "place", because in one definition, "place" also includes a time coordinate. In another (and admittedly much more useful in day to day life) one uses a co-moving coordinate system (I think it's called iirc
) that accounts for the universe's expansion (I think is the basic idea?), such that the concept of "place" actually does continue to exist across time. And using the latter (and more useful) definition, yes, you are totally correct in your assertion. Fair?
ProfessorGAC
(76,222 posts)I can't, however, accept that a point in space "includes" time.
Yes, it's a 4th dimension per Einstein, but with exceptions like Feynmann, time is a marker of where that 3 dimensional coordinate was when the 4th axis was at z'. That 3 dimensional point is at a different set of coordinates in 3 dimensions, but it still exists independent of time.
The 4 dimensional coordinates differ too, but the xyz is directly correlated to time, because distance traveled is velocity time. So, as in simple algebra, time cancels out.
I suppose the tricky part is that time us not a constant as gravity causes curvature in space that won't be the same at a point in the future, and since the source of that gravity has moved, the curvature of space is reduced or even non-existent.
Since time slows down in regions of low curvature, time isn't a constant.
So, we're back to "extraordinarily" difficult & unlikely to occupy the same spot in 2 tines, but not impossible.
I have issues of time travel forward too, because it's traveling into nothingness unless, also like Feynmann, we believe the outcomes are determinate. That smacks too much of predestination which I can't accept.
So, as far as I understand it, only time travel into the past is allowable and only to a period before one was born. Otherwise we're back to places at the same time and the "where'd the extra matter come from" question raises its head.
Fun conversation! Thanks
RockRaven
(18,988 posts)There are MANY stories (one is too many, and you'd think people would learn from others' mistakes) of lawyers getting in hot water with a judge for using AI and citing cases which simply don't exist. It is unprofessional to the extreme, yet it keeps happening even in law firms which institute training on the matter for their partners/associates/staff.
Same with science journal articles. Totally made up citations of articles which don't exist.
highplainsdem
(60,988 posts)about chatbots making stuff up.
I posted this on February 23, 2023. I''ve posted hundreds of OPs here about the problems and dangers with AI.
https://www.democraticunderground.com/100217674825
