Back

Why is this interesting?

The recent eye-popping advances in AI have come from models that scan huge datasets, either generated by humans (e.g. “all competitive Go games” or “all the text we could find on the web”) or by computer (“the AI plays itself a billion times”), and imitating the patterns in that dataset — with no underlying model of meaning, no experience to check against, no underlying theory formation, just parroting.
3/

These systems produce such striking results, it raises the question of whether our brains are any different. Are we also just fancy pattern mimics?

Would an LLM gain human-like intelligence if only we had more processing power?

This result suggests that no, there’s something else our brains do. The tactic the AI missed would be painfully obvious to a human player. There’s still something our brains do — theorizing, generalizing, reasoning through the unfamiliar — that these AIs don’t.
4/

As usual on Ars, there are actually some good comments.

This person wisely reminds us that AI has been littered with bold predictions that human-like AI is just around the corner — and every time, we realized that we’d failed to understand what the hard part even was. The Marvin Minsky quote here is eye-popping:
https://arstechnica.com/information-technology/2023/02/man-beats-machine-at-go-in-human-victory-over-ai/?comments=1&post=41644794
5/

But does this matter? Sure, the AI did a faceplant on some bizarro strategy that would never fool a competent human. So what? It’s just a board game.

OK, what if it’s a self-driving car?

Have you ever encountered a traffic situation that was just totally bizarre, but had a common-sense solution like “wait” or “just go around?” What would an AI do in that situation?

Think of the reports of self-driving Teslas suddenly swerving or accelerating straight into an obvious crash.
6/

Or as this comment remarks: what if it’s an autonomous killbot? (arguably a superset of the previous item, I know) https://arstechnica.com/information-technology/2023/02/man-beats-machine-at-go-in-human-victory-over-ai/?comments=1&post=41644720

Several comments point out the parallels to the recent story about Marines defeating an AI with Jim-Carrey-style nonsense antics that bore no relationship to the training data: https://arstechnica.com/information-technology/2023/02/man-beats-machine-at-go-in-human-victory-over-ai/?comments=1&post=41644757

The linked article: https://taskandpurpose.com/news/marines-ai-paul-scharre/
7/

This is probably what’s going on with the hilarious ChatGPT faceplants making the rounds on social media.

People try to fool GPT with esoteric questions, but those are easy for it: if anybody anywhere on the web already answered the question, no problem — and making it esoteric just narrows the search space.

But give it a three-digit addition problem, and there’s no single specific example to match. And GPT can’t turn all those examples into a generalized theory of how to do addition.
8/

What’s the moral here?

1. Beware the AI hype.

2. Know your AI history, at least a little.

3. If AI ever does become intelligent (whatever that means), it’s going to be •weird• for a long time first.

4. Beware your own heuristics about what “intelligence” looks like, because •that• is a place where our brains are as easily fooled as the Go AI was. Our brains didn’t evolve around things like GPT, and we’re easily bamboozled by them.
/end

Correction from some replies:

Commenters correctly pointed out that the specific AIs in question (KataGo and Leela Zero) did not use human play as training data, only AI play. Thanks! Edited posts to fix.

Correction of some replies:

None of the variants of AlphaGo are the AI in question.

From @Iguanadelmar: Yes, interesting indeed: https://kolektiva.social/@Iguanadelmar/109892672212437525

The point of my thread was that deep learning has weird blind spots that a human would be unlikely to develop, and that these blinds spots are intrinsic to the approach.

The additional insight I take from the fact that it was computer analysis that found the AI’s weakness is that our human metacognition makes it hard for us to find deep learning’s blind spots. Again, we’re too easily bamboozled.

From @psu_13: This paper gets at the underlying problems of that now-embarrassing Minsky quote. It gets at many of the things I was talking about upthread, and much more beside. Thanks for the recommendation!
https://pgh.social/@psu_13/109892784391376542

From @Globaltom, a very good example of a human handling a novel problem not in the training data:
https://mstdn.ca/@Globaltom/109892880540137200

I tend to think that Roomba had the right idea with respect to AI: build stupid-ish machines that assist humans by handling the easy 80% of dreary tasks, and complement humans instead of trying to mimic humans.

From @mykl: https://infosec.exchange/@mykl/109893061731112567

The mistake in that line of reasoning is imagining that “intelligence” (whatever that means) is a linear continuum. Computers are already far “smarter” than humans in specific ways. (Can you sum a billion numbers in one second?)

Part of my point is that what we call AI won’t progress toward human intelligence like going up a staircase. It will be •weird• the whole way, and will thwart our intuitions.

This is a good clarification from @ekwessel: https://mstdn.party/@ekwessel/109893444487949870

Please don’t read my thread to mean “Machines can’t handle novel situations,” or “Machines can’t generalize,” or “Machines can’t identify underlying patterns.” All those statements are false! There is no such bright line to be found here.

My argument is a fuzzier one: software is surprising; it’s hard to know what the hard part is; we should expect to see ML impress us but then faceplant on the seemingly obvious.

Always this (from @ech@mstdn.social):
https://mstdn.social/@ech/109893906069395708

Aha! Here is the paper on the research behind this story (ht ‪@leftpaddotpy‬):

https://arxiv.org/abs/2211.00241

The researchers’ abstract puts it quite crisply: “Our results demonstrate that even superhuman AI systems may harbor surprising failure modes.”

Many replies take one of these two viewpoints:

“Ha! Humans win after all!”

“Whatever, brains are just neurons and so is this, therefore human-like AI is still forthcoming.”

Both rather miss the point: “AI systems harbor surprising failure modes.” Yes, humans will patch the problem, and AIs will keep winning at Go.

But those failure modes that •would have been easily detectable by humans• indicate intrinsic gaps in the •structure• of these “deep learning” systems, not just their scale.

The lesson here isn’t about who wins at Go. It’s about the danger of extrapolating from narrow domains (like winning at Go) to open-ended problems (like driving a car) or general human-like intelligence.

Software is surprising.

AI is surprising.

“Intelligence” — whatever that means — is surprising.

We keep realizing we’ve failed to understand what the hard part even is. We are fools if we do not expect that from the outset.

@suzannealdrich "Calculating a derivative is like squeezing a tube of toothpaste, calculating an integral is like putting the toothpaste back into the tube"

@Cyb3rVix3n @inthehands I think the Chinese room was trying to make a different philosophical point--that even an artificial system *that could act in a flawlessly human way* would lack subjectivity or intentionality. Which I have doubts about, for reasons that have been argued about for decades as described on that Wikipedia page.

But these systems aren't actually acting completely like humans; they're just getting parts of it uncannily right, and stumbling elsewhere.

@mattmcirvin @Cyb3rVix3n @inthehands we just need a bigger room

@Iguanadelmar @inthehands Exactly. I think a lot of the arguments in the thread don't hold specifically because the human followed a recipe from another AI.

@lana @Iguanadelmar I specifically addressed this interesting point in the postludes.

And no, 80% of the thread doesn't hinge at all on whether it is a human or a computer that identified the weakness.

@eurobubba Touché.

@steve @leftpaddotpy And it’s good work, imo! Expecting to be surprised is a surprisingly hard lesson for people to come to about software.

@steve Exactly so. There are two difficult layers, both at the heart of engineering: (1) prepare for the unexpected; seek it out relentlessly; mitigate after and mitigate before; build in redundancy; and (2) intuitions about human failure modes create blind spots for us when reasoning about AI; no failure mode is too weird, too stupid, or too obvious to consider.

@adredish I would hope that people would have learned not to be surprised when software fails in bizarre-to-humans ways, but as it turns out, that’s a hard lesson to learn!

I reserve my greatest disdain for the engineers who should know better, but keep pumping the GAI hype juice anyway.

@jedbrown @Globaltom Yeah, I’m convinced there is no working approach to cleaning — not automated, not hired, not systemic, not expert, not anything — for people with kids. We live in the chaos. We breathe the chaos.

@inthehands @jedbrown you learn magic, levitate everything off the floor, then sweep/vacumn underneath. Easy.

@SciencesPoulet @Iguanadelmar Adversarial approaches help. The point is that deep learning for critical systems will leave us playing endless whack-a-mole with failures we have trouble anticipating. It’s a tough model on which to build a good engineering practice.

@glenatron @inthehands Engineers that dismiss philosophy as worthless should not be allowed to work on anything related to humans or the environment (i.e. anything). Dangerous people susceptible to the dumbest scams and capable of inhuman horrors without a shred of conscience or second thoughts.

@arclight @inthehands i don't disagree but it is very common. Every couple of years there's a new book from a physicist claiming to have "solved" philosophy and it's always misbegotten from the start because they haven't grasped that the scientific method and logic and the nature of numbers are all philosophy. If you can solve it with physics it's not metaphysics, that's right there in the name! Other fields do the same and as a philosophy grad it is infuriating.

@glenatron @arclight @inthehands
"It's simple" are often the first words said by someone that doesn't understand a subject

@stefanieschulte Same here. The specialized topics are the ones that show the underlying texture of the model best, I think: sometimes repeated an accurate and focused summary, sometimes throwing recognizable fragments together into nonsense, and sometimes just plagiarizing.

@mattmcirvin @inthehands My favourite part is that, once it started acting arrogant and obtuse, it decided that "the date is actually 2022" was the most likely next token. The mistake it made about the movie's release date damaged its ability to output the correct date.

@wizzwizz4 @inthehands It's an interesting problem for this kind of approach--you have certain categories of text in the corpus describing specific events in time, that were appropriate things to say at the time that text was written, but it's now possible to deduce that the text is obsolete. But only if you have an understanding of how time works.