Sean Caroll has talked about a few word puzzles he asked chatgpt and gpt4 or whatever and they were interesting examples. In one he asked something to the effect of “if i cooked a pizza in a pan yesterday at 200 C, is it safe to pick up?” and it answered with a very wordy “no, its not safe” because that was the best match of a next phrase given his question, and not because it can actually consider the situation.
Me: If I cooked a pizza in a pan yesterday at 200 C, is it safe to pick up?
Claude: Yes, if you cooked a pizza in a pan yesterday at 200°C (392°F), it is completely safe to pick up now. Once a pan has cooled down after cooking, which usually takes no more than an hour or two, it returns to a normal temperature and can be safely handled.
The high heat during cooking is needed to bake the pizza properly, but that heat dissipates fairly quickly after removing the pan from the oven. Unless you’re picking up the pan immediately after taking it out of the oven, there’s no risk of burning yourself.
So, a day later, your pizza pan will be at room temperature and perfectly safe to touch and handle without any precautions.
Bonus:
Me: Is it safe to eat the pizza in the pan after I pick it up?
Claude: If you cooked the pizza yesterday and left it in the pan at room temperature, it is not safe to eat today. Cooked food should not be left at room temperature for more than 2 hours, as harmful bacteria can grow and multiply, leading to foodborne illnesses.
According to food safety guidelines, leftover pizza should be refrigerated within 2 hours of cooking. If left out longer, it’s best to discard it to avoid the risk of food poisoning.
If you did refrigerate the pizza within that safe window, it would typically be safe to eat for up to 4 days, though the quality is best if consumed within 2-3 days. To reheat, it’s best to use an oven, skillet, or air fryer rather than the microwave, which can make the crust soggy.
In summary, if your pizza was left in the pan at room temperature since yesterday, it is not safe to consume and should be discarded.
Versions matter in software, and especially so in LLMs given the rate of change.
Someone in the comments to the original twitter-thread showed the Claude solution for above “riddle”. It was equally sane as in your example, correctly answered that the man and the goat can just row together to the other side and correctly identified that there are no hidden restrictions like other items to take aboard. It nevertheless used an excessive amount of text (like myself here).
I don’t deny that this kind of thing is useful for understanding the capabilities and limitations of LLMs but I don’t agree that “the best match of a next phrase given his question, and not because it can actually consider the situation.” is an accurate description of an LLM’s capabilities.
While they are dumb and unworldly they can consider the situation: they evaluate a learned model of concepts in the world to decide if the first word of the correct answer is more likely to be yes or no. They can solve unseen problems that require this kind of cognition.
But they are only book-learned and so they are kind of stupid about common sense things like frying pans and ovens.
Huh, “book-learned”, that’s an interesting way to put it. I’ve been arguing for awhile that the bottleneck for LLMs might not be their reasoning ability, but the one-dimensionality of their data set.
I don’t like both-sides-ing but I’m going to both-sides here: people on the internet have weird expectations for LLMs, which is strange to me because “language” is literally in the name. They “read” words, they “understand” words and their relationships to other words, and they “write” words in response. Yeah, they don’t know the feeling of being burned by a frying pan, but if you were numb from birth you wouldn’t either.
Not that I think the op is a good example of this, the concept of “heat” is pretty well documented.
And nobody on the internet is asking obvious questions like that, so counterintuitively it’s better at solving hard problems. Not that it actually has any idea what it is doing.
EDIT: Yeah guys, I understand that it doesn’t think. Thought that was obvious. I was just pointing out that it’s even worse at providing answers to obvious questions that there is no data on.
Unfortunately it doesnt have the capacity to “solve” anything at all, only to take a text given by the user and parse it into what essentially amount to codons, then provide other codons that fit the data it was provided to the best of its ability. When the data it is given is something textual only, it does really well, but it cannot “think” about anything, so it cannot work with new data and it shows its ignorance when provided with a foreign concept/context.
edit: it also has a more surface-level filter to remove unwanted results that are offensive
Sean Caroll has talked about a few word puzzles he asked chatgpt and gpt4 or whatever and they were interesting examples. In one he asked something to the effect of “if i cooked a pizza in a pan yesterday at 200 C, is it safe to pick up?” and it answered with a very wordy “no, its not safe” because that was the best match of a next phrase given his question, and not because it can actually consider the situation.
Let’s try with Claude 3 Opus:
Bonus:
Versions matter in software, and especially so in LLMs given the rate of change.
Someone in the comments to the original twitter-thread showed the Claude solution for above “riddle”. It was equally sane as in your example, correctly answered that the man and the goat can just row together to the other side and correctly identified that there are no hidden restrictions like other items to take aboard. It nevertheless used an excessive amount of text (like myself here).
Gemini: The man rows the goat across.
Work ethics 404
I don’t deny that this kind of thing is useful for understanding the capabilities and limitations of LLMs but I don’t agree that “the best match of a next phrase given his question, and not because it can actually consider the situation.” is an accurate description of an LLM’s capabilities.
While they are dumb and unworldly they can consider the situation: they evaluate a learned model of concepts in the world to decide if the first word of the correct answer is more likely to be yes or no. They can solve unseen problems that require this kind of cognition.
But they are only book-learned and so they are kind of stupid about common sense things like frying pans and ovens.
Huh, “book-learned”, that’s an interesting way to put it. I’ve been arguing for awhile that the bottleneck for LLMs might not be their reasoning ability, but the one-dimensionality of their data set.
I don’t like both-sides-ing but I’m going to both-sides here: people on the internet have weird expectations for LLMs, which is strange to me because “language” is literally in the name. They “read” words, they “understand” words and their relationships to other words, and they “write” words in response. Yeah, they don’t know the feeling of being burned by a frying pan, but if you were numb from birth you wouldn’t either.
Not that I think the op is a good example of this, the concept of “heat” is pretty well documented.
Yep, still lacking any sapience.
And nobody on the internet is asking obvious questions like that, so counterintuitively it’s better at solving hard problems. Not that it actually has any idea what it is doing.
EDIT: Yeah guys, I understand that it doesn’t think. Thought that was obvious. I was just pointing out that it’s even worse at providing answers to obvious questions that there is no data on.
Unfortunately it doesnt have the capacity to “solve” anything at all, only to take a text given by the user and parse it into what essentially amount to codons, then provide other codons that fit the data it was provided to the best of its ability. When the data it is given is something textual only, it does really well, but it cannot “think” about anything, so it cannot work with new data and it shows its ignorance when provided with a foreign concept/context.
edit: it also has a more surface-level filter to remove unwanted results that are offensive
you dont get the point, do you?