Feel like we’ve got a lot of tech savvy people here seems like a good place to ask. Basically as a dumb guy that reads the news it seems like everyone that lost their mind (and savings) on crypto just pivoted to AI. In addition to that you’ve got all these people invested in AI companies running around with flashlights under their chins like “bro this is so scary how good we made this thing”. Seems like bullshit.
I’ve seen people generating bits of programming with it which seems useful but idk man. Coming from CNC I don’t think I’d just send it with some chatgpt code. Is it all hype? Is there something actually useful under there?
My perspective is that consciousness isn’t a binary thing, or even a linear scale. It’s an amalgamation of a bunch of different independent processes working together; and how much each matters is entirely dependent on culture and beliefs. We’re artificially creating these independent processes piece by piece in a way that doesn’t line up with traditional ideas of consciousness. Conversation and being able to talk about concepts one hasn’t personally experienced are facets of consciousness and intelligence, ones that the latest and greatest LLMs do have. Of course there others too that they don’t: logic, physical presence, being able to imagine things in their mind’s eye, memory, etc.
It’s reductive to dismiss GPT4 as nothing more than mimicry; saying it’s just a mathematical text prediction model is like saying your brain is just a bunch of neurons. Both statements are true, but it doesn’t change what they can do. If someone could accurately predict the moves a chess master would make, we wouldn’t say they’re just good at statistics, we’d say they’re a chess master. Similarly, regardless of how rich someone’s internal world is, if they’re unable to express the intelligent ideas they have in any intelligible way we wouldn’t consider them intelligent.
So what we have now with AI are a few key parts of intelligence. One important thing to consider is how language can be a path to other types of intelligence; here’s a blog post I stumbled across that really changed my perspective on that: http://www.asanai.net/2023/05/14/just-a-statistical-text-predictor/. Using your example of mathematics, as we know it falls apart doing anything remotely complicated. But when you help it approach the problem step-by-step in the way a human might - breaking it into small pieces and dealing with them one at a time - it actually does really well. Granted, the usefulness of this is limited when calculators exist and it requires as much guidance as a child to get correct answers, but even matching the mathematical intelligence of a ten year old is nothing to sneeze at.
To be clear I don’t think pursuing LLMs endlessly will be the key to a widely accepted ‘general intelligence’; it’ll require a multitude of different processes and approaches working together for that to ever happen, and we’re a long way from that. But it’s also not just getting carried away with the hype to say the past few years have yielded massive steps towards ‘true’ artificial intelligence, and that current LLMs have enough use cases to change a lot of people’s lives in very real ways (good or bad).
Thanks for that article, it was a very interesting read! I think we’re mostly agreeing about things :) This stood out to me from there as an encapsulation of the conversation:
“Statistics” is probably an insufficient term for what these things are doing, but it’s helpful to pull the conversation in that direction when a lay person using one of those things is likely to assume quite the opposite, that this really is a person in a computer with hopes and dreams. But I agree that it takes more than simply consulting a table to find the most likely next word to, to take an earlier example, write a haiku about Danny DeVito. That’s synthesizing two ideas together that (I would guess) the model was trained on individually. That’s very cool and deserving of admiration, and could lead to pretty incredible things. I’d expect that the task of predicting words, on its own, wouldn’t be stringent enough to force a model to develop “true” intelligence, whatever that means, to succeed during training, but I suppose we’ll find out, and probably sooner than we expect.
Well put! I think I kinda misunderstood what you were saying, I guess we sort of reached the same conclusion from different directions. And yeah, it does seem like we’re hitting the limits of what can be achieved from the current underlying word-prediction mechanisms alone, with how diminishing the returns are from dumping more data in. Maybe something big will happen soon, but it looks to me like LLMs will stagnate for a while until they’re taken in a fundamentally new direction.
Either way, what they can do now is pretty incredible, and equally interesting to me is how it’s making us reevaluate our ideas of consciousness and intelligence on a large scale; it’s one thing to theorize about what could happen with an ‘intelligent’ AI, but the reality of these philosophical questions being so thoroughly challenged and dissected in mundane legal and practical matters is wild.