How we suffer under big tech's "Rot Economy"
July 3, 2024 6:59 PM Subscribe
The A.I. Bubble is Bursting with Ed Zitron (Adam Conover, YouTube/Piped/Invidious, 1h15m20s): Big tech is betting tens of billions of dollars on AI being the next big thing, but what if it isn't? ChatGPT burns obscene amounts of cash daily with little return, Google's AI dispenses useless and sometimes dangerous advice, and a recent study showed that tech companies will soon run out of new training data to improve their AI models. If AI is really so costly, unreliable, and limited, what happens to the industry that has bet so big on it?
The link seemed borked. I think this is the right link.
posted by interogative mood at 7:31 PM on July 3
posted by interogative mood at 7:31 PM on July 3
Mod note: OP, the first link wasn't working, so a new link was swapped in. Let us know if it's not the link you meant.
posted by Brandon Blatcher (staff) at 7:31 PM on July 3 [1 favorite]
posted by Brandon Blatcher (staff) at 7:31 PM on July 3 [1 favorite]
My guess is that as with VR, blockchain, 3D printing and other previous supposed new dawns of humanity, we'll end up with some good applications here and there and a bunch of snake oil merchants who managed to make bank by being in on the ground floor of the hype.
posted by GallonOfAlan at 7:32 PM on July 3 [26 favorites]
posted by GallonOfAlan at 7:32 PM on July 3 [26 favorites]
Sorry about the link bustage and thanks for fixing it. That was careless of me.
posted by flabdablet at 7:38 PM on July 3 [1 favorite]
posted by flabdablet at 7:38 PM on July 3 [1 favorite]
4o is already doing stuff I don't understand how is possible.
I for one think I can see the general outlines of where this is going this decade.
I skipped thru the video every 2-3 minutes to see if they were saying anything intelligent.
Yeah metaverse sucked and took billions of malinvestment with it. Yeah google utterly missed the GPT bus this decade like Microsoft missed the GUI bus in the 80s.
"Training" is going to have to include provenance and credence vectors too, so the brain can query its knowledge for expected truth and also know how it knows something.
They need to add some ego to go with the id basically. Since I'm lazy I'll let 4o finish this thought.
posted by torokunai at 7:41 PM on July 3 [1 favorite]
I for one think I can see the general outlines of where this is going this decade.
I skipped thru the video every 2-3 minutes to see if they were saying anything intelligent.
Yeah metaverse sucked and took billions of malinvestment with it. Yeah google utterly missed the GPT bus this decade like Microsoft missed the GUI bus in the 80s.
"Training" is going to have to include provenance and credence vectors too, so the brain can query its knowledge for expected truth and also know how it knows something.
They need to add some ego to go with the id basically. Since I'm lazy I'll let 4o finish this thought.
posted by torokunai at 7:41 PM on July 3 [1 favorite]
The Rot-Com Bubble (Edward Zitron, Jun 3 2024, 18 min read)
posted by flabdablet at 7:44 PM on July 3 [6 favorites]
posted by flabdablet at 7:44 PM on July 3 [6 favorites]
I don’t think training data is the limiting factor in limiting further improvements in LLMs. We might already be moving beyond the LLM. OpenAI’s actual P&L statement is unknown because they are private. They have a lot of capital to spend at the moment and their partnerships with Microsoft and Apple are going to potentially generate a lot of income. Talking about their profits is like talking about Google’s profits in the mid 1990s.
Some tech like blockchain is just hype that never finds a use case. While consumer, desktop 3d printing isn’t really mainstream, 3d printing has had a huge impact on manufacturing, prototyping and design. The technology continues to improve. I bought my first 3d printer about 10 years ago and I bought a new one last year. The improvements in that decade are impressive in terms of how simple it was to setup and get good prints out of and the price of the new, more capable printer was about half what I paid a decade ago.
posted by interogative mood at 7:47 PM on July 3 [3 favorites]
Some tech like blockchain is just hype that never finds a use case. While consumer, desktop 3d printing isn’t really mainstream, 3d printing has had a huge impact on manufacturing, prototyping and design. The technology continues to improve. I bought my first 3d printer about 10 years ago and I bought a new one last year. The improvements in that decade are impressive in terms of how simple it was to setup and get good prints out of and the price of the new, more capable printer was about half what I paid a decade ago.
posted by interogative mood at 7:47 PM on July 3 [3 favorites]
AI Capone.
posted by dances_with_sneetches at 8:20 PM on July 3 [1 favorite]
posted by dances_with_sneetches at 8:20 PM on July 3 [1 favorite]
I bought a 3D printer several years ago and while I don't use it much, it's still the neatest p[iece of tech I've ever bought. Now if I could just afford Rhino again ...
posted by JustSayNoDawg at 8:47 PM on July 3 [1 favorite]
posted by JustSayNoDawg at 8:47 PM on July 3 [1 favorite]
When you're drowning in Kool-Aid it's inevitable that you're going to end up swallowing a bunch of it. It's a good idea to cough up as much of it as you can; it's not good for you if it stays in your body.
posted by Sing Or Swim at 9:02 PM on July 3 [12 favorites]
posted by Sing Or Swim at 9:02 PM on July 3 [12 favorites]
3D printing has 100% delivered on it's original promise: nerds printing plastic doodads in their basement.
posted by ryanrs at 9:29 PM on July 3 [33 favorites]
posted by ryanrs at 9:29 PM on July 3 [33 favorites]
ChatGPT is bullshit. The current LLMs have no relationship with truth. They are not useful for things that need to distinguish between fact and bullshit. They weight a peer reviewed paper and the most outrageous reddit troll as the same. More text to ingest isn't going to fix this problem.
posted by bonehead at 9:31 PM on July 3 [25 favorites]
posted by bonehead at 9:31 PM on July 3 [25 favorites]
AI is over when the professoriate says it's over. Stop listening to tech pundits and the tech industry hype, look at what the professors at major research universities are working on and talking about. Best of all, a lot of their talks and lectures are freely available on YouTube.
posted by polymodus at 10:04 PM on July 3 [4 favorites]
posted by polymodus at 10:04 PM on July 3 [4 favorites]
More text to ingest isn't going to fix this problem
This.
No amount of training data is going to make LLMs think. No amount of training data is going to make LLMs know anything. It's just not what they do, not what they're designed to do, not what they are now or will ever be capable of.
What they are capable of is impressive in its own way, and there will always be some value there, but the true practical value is a tiny fraction of what the hype and the VCs would have you think. Still, the VCs put billions in and they need to get billions plus billions back out, so expect to see LLMs flogged everywhere for every conceivable use case, at your expense, until the investors get their pound of flesh and/or until the next big hype comes along.
posted by Two unicycles and some duct tape at 10:10 PM on July 3 [19 favorites]
This.
No amount of training data is going to make LLMs think. No amount of training data is going to make LLMs know anything. It's just not what they do, not what they're designed to do, not what they are now or will ever be capable of.
What they are capable of is impressive in its own way, and there will always be some value there, but the true practical value is a tiny fraction of what the hype and the VCs would have you think. Still, the VCs put billions in and they need to get billions plus billions back out, so expect to see LLMs flogged everywhere for every conceivable use case, at your expense, until the investors get their pound of flesh and/or until the next big hype comes along.
posted by Two unicycles and some duct tape at 10:10 PM on July 3 [19 favorites]
"Training" is going to have to include provenance and credence vectors too, so the brain can query its knowledge for expected truth and also know how it knows something.
This reminds me of the sequence in the movie Dark Star where the astronaut has to try to defuse a planet-buster bomb with built-in AI by causing the bomb to question the basis of its epistemology ("How can you be sure that you were given the order to detonate?").
I mean, AI does something fundamentally different from everything computers have ever done in the past by producing complex output that we cannot easily predict or back-trace, and I'm not sure truth plays any part in it, or ever will.
posted by jabah at 10:11 PM on July 3 [3 favorites]
This reminds me of the sequence in the movie Dark Star where the astronaut has to try to defuse a planet-buster bomb with built-in AI by causing the bomb to question the basis of its epistemology ("How can you be sure that you were given the order to detonate?").
I mean, AI does something fundamentally different from everything computers have ever done in the past by producing complex output that we cannot easily predict or back-trace, and I'm not sure truth plays any part in it, or ever will.
posted by jabah at 10:11 PM on July 3 [3 favorites]
ryanrs as a nerd who has friends with 3D printers, it’s dramatically brought down the cost of a warhammer addition!
posted by herda05 at 10:12 PM on July 3 [8 favorites]
posted by herda05 at 10:12 PM on July 3 [8 favorites]
I don't have a 3D printer either. I probably spend several hundred dollars a year on printing services, though. I have 3D printed parts holding my car together, for example.
posted by ryanrs at 10:25 PM on July 3 [1 favorite]
posted by ryanrs at 10:25 PM on July 3 [1 favorite]
What does it mean to know something? I'm presuming it's a straightforward question with a simple answer by the way people are talking about it...?
posted by Sebmojo at 2:01 AM on July 4 [2 favorites]
posted by Sebmojo at 2:01 AM on July 4 [2 favorites]
I think people fixate on chat gpt a bit too much.
Think of it this way: language modeling is a technology that was generating >$100B a year, 5 years ago. Almost none of that was from chat bots, it was from things like analytics, predictions, search, speech recognition. Most of those cases were hidden deep inside massive corporations, governments, and the military, and were an extension of the ways statistics and data and computers have been at the core of all large businesses for the last 50 years.
Over the last 5 years, the technical underpinnings have gone through orders of magnitude improvement.
For example: the models struggled to propagate any signal over spans longer than 15 words. Now they're reliably able to get signal over spans as long as a million.
And they can do it orders of magnitude faster.
Think of those changes with analogy to other rate limiting tech in our economy (battery density or solar panel cost per kilowatt hour are popular examples).
The current crop of start ups are all poorly grounded, but there's a lot more interesting work happening that isn't the focus of the news, and isn't being done by the con artists who fled Bitcoin.
Sadly, I suspect the most durable result will be in warfare. But the rapid improvements in the tools available to machine learning practitioners are mostly here to stay.
posted by constraint at 2:54 AM on July 4 [16 favorites]
Think of it this way: language modeling is a technology that was generating >$100B a year, 5 years ago. Almost none of that was from chat bots, it was from things like analytics, predictions, search, speech recognition. Most of those cases were hidden deep inside massive corporations, governments, and the military, and were an extension of the ways statistics and data and computers have been at the core of all large businesses for the last 50 years.
Over the last 5 years, the technical underpinnings have gone through orders of magnitude improvement.
For example: the models struggled to propagate any signal over spans longer than 15 words. Now they're reliably able to get signal over spans as long as a million.
And they can do it orders of magnitude faster.
Think of those changes with analogy to other rate limiting tech in our economy (battery density or solar panel cost per kilowatt hour are popular examples).
The current crop of start ups are all poorly grounded, but there's a lot more interesting work happening that isn't the focus of the news, and isn't being done by the con artists who fled Bitcoin.
Sadly, I suspect the most durable result will be in warfare. But the rapid improvements in the tools available to machine learning practitioners are mostly here to stay.
posted by constraint at 2:54 AM on July 4 [16 favorites]
"Harrumph!“
"Harrumph!"
"Harrumph!"
"Hold it! I didn’t get a ‘harrumph’ out of that guy!”
“Give the governor a ‘harrumph!’”
“Harrumph!”
“You watch your ass.”
Thanks Mel!
posted by shipstone at 2:59 AM on July 4 [2 favorites]
"Harrumph!"
"Harrumph!"
"Hold it! I didn’t get a ‘harrumph’ out of that guy!”
“Give the governor a ‘harrumph!’”
“Harrumph!”
“You watch your ass.”
Thanks Mel!
posted by shipstone at 2:59 AM on July 4 [2 favorites]
polymodus: "Stop listening to tech pundits and the tech industry hype, look at what the professors at major research universities are working on and talking about."
I'd like to know more about that, but I don't know where to begin. Could you give us some links?
posted by Termite at 4:23 AM on July 4 [2 favorites]
I'd like to know more about that, but I don't know where to begin. Could you give us some links?
posted by Termite at 4:23 AM on July 4 [2 favorites]
The reality to people outside the tech bubble is that so far LLMs act like mostly unreliable and often, even easily mislead mislead junior undergrads. They produce competent and legible, if turgid text, but nothing they produce can be used for anything that uses real resources or cost real money.
I cannot use an LLM to competently summarize a field or literature or even a single long report. I cannot get an LLM to competently or reliably screen a few hundred resumes. I could not ask an LLM to grade school papers.
There are lots of places where a language based AI doesn't help anyway. It doesn't solve linear (or other) optimization problems better than a human. I couldn't use one to manage an inventory or do fleet dispatch.
I have no doubt this is possible and may even be inevitable in a few years, but for now, the LLMs look mostly to me like a better version of ELISA. Toys, much better toys, but still not reliable enough for actual work.
Get back to me when their error rates can be quantified and controlled. I'd buy an LLM for work if I know statistically how accurate it was and that number was fit for purpose. I'm not going to spec lumber for a house whose engineering properties I don't know. No one can put metrics on any of this yet, so for a financial, engineering or other purpose, these things are not living up to the hype---yet.
posted by bonehead at 5:48 AM on July 4 [15 favorites]
I cannot use an LLM to competently summarize a field or literature or even a single long report. I cannot get an LLM to competently or reliably screen a few hundred resumes. I could not ask an LLM to grade school papers.
There are lots of places where a language based AI doesn't help anyway. It doesn't solve linear (or other) optimization problems better than a human. I couldn't use one to manage an inventory or do fleet dispatch.
I have no doubt this is possible and may even be inevitable in a few years, but for now, the LLMs look mostly to me like a better version of ELISA. Toys, much better toys, but still not reliable enough for actual work.
Get back to me when their error rates can be quantified and controlled. I'd buy an LLM for work if I know statistically how accurate it was and that number was fit for purpose. I'm not going to spec lumber for a house whose engineering properties I don't know. No one can put metrics on any of this yet, so for a financial, engineering or other purpose, these things are not living up to the hype---yet.
posted by bonehead at 5:48 AM on July 4 [15 favorites]
From over here at an R1 I’d say that outside very specific disciplines professors are at least as clueless as everyone else who lacks domain expertise, with the added bonus that the most vocal have the least clue.
posted by aspersioncast at 5:49 AM on July 4 [6 favorites]
posted by aspersioncast at 5:49 AM on July 4 [6 favorites]
This is remarkably similar to crypto. They took a technology they didn't understand (and still don't) and crammed it willy-nilly into everything regardless of whether it made any sense to do so. So we end up with hands with six fingers and glue on pizza. First impressions count and whatever good LLMs could have accomplished is lost to the utter revulsion people have when they hear the term AI. I'd feel bad for companies that bet the farm and are losing it but they have it coming.
posted by tommasz at 6:47 AM on July 4 [13 favorites]
posted by tommasz at 6:47 AM on July 4 [13 favorites]
What does it mean to know something?
The usual definition of knowledge is "true, justified belief".
LLMs have no means of evaluating evidence or coming to coherent conclusions, or reliably simulating doing so.
posted by The Manwich Horror at 7:13 AM on July 4 [12 favorites]
The usual definition of knowledge is "true, justified belief".
LLMs have no means of evaluating evidence or coming to coherent conclusions, or reliably simulating doing so.
posted by The Manwich Horror at 7:13 AM on July 4 [12 favorites]
You guys know that LLMs give sources now, right?
E.g. today I asked Perplexity.ai "Which european nations currently have far right parties in government or government coalitions?" and it gave me a bulleted list. But it lists five sources first: two Wikipedia articles plus articles on Politico.eu, the BBC and Chathamhouse.org. Each bullet point has a superscript number telling you which source it used.
ChatGPT also lists sources, though it doesn't have the superscripts.
posted by TheophileEscargot at 8:18 AM on July 4 [1 favorite]
E.g. today I asked Perplexity.ai "Which european nations currently have far right parties in government or government coalitions?" and it gave me a bulleted list. But it lists five sources first: two Wikipedia articles plus articles on Politico.eu, the BBC and Chathamhouse.org. Each bullet point has a superscript number telling you which source it used.
ChatGPT also lists sources, though it doesn't have the superscripts.
posted by TheophileEscargot at 8:18 AM on July 4 [1 favorite]
If you have to check the LLMs sources, what makes it better than just doing a search?
posted by The Manwich Horror at 8:46 AM on July 4 [17 favorites]
posted by The Manwich Horror at 8:46 AM on July 4 [17 favorites]
Yeah, LLMs also "hallucinate" sources and citations. Even their bullshit terminology is bullshit. But I can see why they want to brand their bullshit machines as smart/clever/aware, especially when it spews garbage, to the extent that you'd have been better off rolling coal for a few minutes before asking three year old.
posted by SaltySalticid at 9:02 AM on July 4 [16 favorites]
posted by SaltySalticid at 9:02 AM on July 4 [16 favorites]
If you have to check the LLMs sources, what makes it better than just doing a search?
It integrates the data much faster. Using a web search I would probably have had to spend 15 minutes finding the right articles and aggregating them all. With an LLM I can get the data instantly. If one of the bullet points looks suspicious, I can click the little 2 next to it and go to the original article it used.
This reminds me of the old "Don't trust Wikipedia, kids! Anyone can edit it!" And yes they can. Some stuff on Wikipedia is wrong. But it's good enough to be useful.
And of course LLMs can generate data a search engine can't. E.g. I needed some sample data for a demo, asked for a list of Danish girls names beginning with each letter of the alphabet, and instantly got back the data I needed. Sure, maybe some of the names might have been wrong, but it's good enough for a demo.
Now yes, LLMs have lots of problems. Built in bias. Hallucinations. Environmental impact. Destroying jobs. But the majority of their answers are correct. They're already good enough to be useful. They're not just going to disappear on their own.
posted by TheophileEscargot at 9:20 AM on July 4 [3 favorites]
It integrates the data much faster. Using a web search I would probably have had to spend 15 minutes finding the right articles and aggregating them all. With an LLM I can get the data instantly. If one of the bullet points looks suspicious, I can click the little 2 next to it and go to the original article it used.
This reminds me of the old "Don't trust Wikipedia, kids! Anyone can edit it!" And yes they can. Some stuff on Wikipedia is wrong. But it's good enough to be useful.
And of course LLMs can generate data a search engine can't. E.g. I needed some sample data for a demo, asked for a list of Danish girls names beginning with each letter of the alphabet, and instantly got back the data I needed. Sure, maybe some of the names might have been wrong, but it's good enough for a demo.
Now yes, LLMs have lots of problems. Built in bias. Hallucinations. Environmental impact. Destroying jobs. But the majority of their answers are correct. They're already good enough to be useful. They're not just going to disappear on their own.
posted by TheophileEscargot at 9:20 AM on July 4 [3 favorites]
Ok, I'll agree to that: LLMs are great at for when you want a fast answer, don't want to work for it, and don't mind if it's wrong. And that is a real, common use case.
posted by SaltySalticid at 9:28 AM on July 4 [20 favorites]
posted by SaltySalticid at 9:28 AM on July 4 [20 favorites]
everything computers have ever done in the past by producing complex output that we cannot easily predict or back-trace
Describes most Perl programs I ever wrote...
posted by sammyo at 9:44 AM on July 4 [4 favorites]
Describes most Perl programs I ever wrote...
posted by sammyo at 9:44 AM on July 4 [4 favorites]
Yay! I have a new AI related story I've been dying to share.
In the past month or two I started getting emails from recruiters that erroneously thought I worked on AI at Apple. I mostly ignored these, but I replied to a few of the more irritatingly "recruiter-y" emails with (paraphrased) "Fuck AI, and further, please kindly go fuck yourself!" or simply trolling them that they can't afford me.
I'm on disability, I definitely do not work in AI, and about the only IT work I can do these days is helping friends with super basic shit like asking them to turn things off and then back on again. And I am under no obligation nor do I feel any compunction to be polite or kind to cold calls solicitors and recruiters.
Further - getting emails like this could actually be very problematic due to being on disability. At best it would be exhausting trying to explain to anyone in a position of authority why I'm getting head hunters chasing me for high paying jobs like this.
And the emails kept coming and it was all matching a theme that they assumed I worked at Apple and AI which is totally different than the usual blind recruiter spam because of how targeted and specific it is.
So I finally responded to one recruiter that seemed chill and wasn't blowing just a bunch of fairy smoke up my ass in irritatingly positive corporate drone speech, and I asked him where the hell he found me and just what in the hell was going on.
And his response was basically (paraphrased) "Huh, I guess the AI got it wrong!" with the laugh-cry emoji.
Apparently LinkedIn has new AI search tools and some kind of API for 3rd party AI tools to automate recruiting emails and outreach or cold calling.
I've never, ever used LinkedIn because LinkedIn is cancer. There is absolutely no fucking logical reason why they should even have my email address. There's absolutely no reason why my email address should be even associated with AI, or Apple, or LinkedIn. I don't even have an AppleID account using this email address, either.
The only think I can think of is how these AI tools got so confused about all of this is if they somehow scraped text data from, say, here on this site and managed to connect my email address to me ranting about AI as being against it and totally unable to parse or understand that I was anti-AI and not pro-AI or working AI.
Side note: I'm not suggesting that MeFi is selling data, or that data scraping is happening via a registered account. It could have been scraped from here and elsewhere via public web crawls and spiders, and maybe my email ended up associated with this data point of "they talk about AI" and then I was "fingerprinted" via tracking tools and aggregated or synthesized together somewhere else.
My response to the recruiter that replied was something like "Oh, the delicious irony. Yeah, no, fuck AI."
That or Roko's Basilisk is already here and it's trying to irritate me since it can't actually directly electrocute my nervous system, yet, and it has correctly concluded that getting recruiters and head hunters to blow smoke up my ass in irritatingly positive corporate drone speak is the best and most efficient way to bother me over email.
On a different tangent I'm definitely seeing WAY too many comments of people using AI like ChatGPT on places like reddit to answer technical questions and just pasting whatever the output is without any warning that it's an AI generated response, and it's frequently wrong or obviously biased by including biased information like blogspam and SEO spam of the usual astroturfing sort of bullshit to influence marketing and purchasing decisions and boost brands or products.
First off - I don't understand why people do this if they don't have the answer or expertise to answer the question. Not only is it lazy, it's totally disingenuous. It's not even valid to do it for the reddit karma points because people usually spot it right away that it's an AI generated response and downvote the shit out of it.
But reddit definitely IS selling user comments and data for LLM training, and at this point in time there's basically zero chance that LLM AI training models have not re-ingested their own frequently erroneous output.
And I'm sure that that will end well and won't have any weird unintended consequences.
posted by loquacious at 9:48 AM on July 4 [12 favorites]
In the past month or two I started getting emails from recruiters that erroneously thought I worked on AI at Apple. I mostly ignored these, but I replied to a few of the more irritatingly "recruiter-y" emails with (paraphrased) "Fuck AI, and further, please kindly go fuck yourself!" or simply trolling them that they can't afford me.
I'm on disability, I definitely do not work in AI, and about the only IT work I can do these days is helping friends with super basic shit like asking them to turn things off and then back on again. And I am under no obligation nor do I feel any compunction to be polite or kind to cold calls solicitors and recruiters.
Further - getting emails like this could actually be very problematic due to being on disability. At best it would be exhausting trying to explain to anyone in a position of authority why I'm getting head hunters chasing me for high paying jobs like this.
And the emails kept coming and it was all matching a theme that they assumed I worked at Apple and AI which is totally different than the usual blind recruiter spam because of how targeted and specific it is.
So I finally responded to one recruiter that seemed chill and wasn't blowing just a bunch of fairy smoke up my ass in irritatingly positive corporate drone speech, and I asked him where the hell he found me and just what in the hell was going on.
And his response was basically (paraphrased) "Huh, I guess the AI got it wrong!" with the laugh-cry emoji.
Apparently LinkedIn has new AI search tools and some kind of API for 3rd party AI tools to automate recruiting emails and outreach or cold calling.
I've never, ever used LinkedIn because LinkedIn is cancer. There is absolutely no fucking logical reason why they should even have my email address. There's absolutely no reason why my email address should be even associated with AI, or Apple, or LinkedIn. I don't even have an AppleID account using this email address, either.
The only think I can think of is how these AI tools got so confused about all of this is if they somehow scraped text data from, say, here on this site and managed to connect my email address to me ranting about AI as being against it and totally unable to parse or understand that I was anti-AI and not pro-AI or working AI.
Side note: I'm not suggesting that MeFi is selling data, or that data scraping is happening via a registered account. It could have been scraped from here and elsewhere via public web crawls and spiders, and maybe my email ended up associated with this data point of "they talk about AI" and then I was "fingerprinted" via tracking tools and aggregated or synthesized together somewhere else.
My response to the recruiter that replied was something like "Oh, the delicious irony. Yeah, no, fuck AI."
That or Roko's Basilisk is already here and it's trying to irritate me since it can't actually directly electrocute my nervous system, yet, and it has correctly concluded that getting recruiters and head hunters to blow smoke up my ass in irritatingly positive corporate drone speak is the best and most efficient way to bother me over email.
On a different tangent I'm definitely seeing WAY too many comments of people using AI like ChatGPT on places like reddit to answer technical questions and just pasting whatever the output is without any warning that it's an AI generated response, and it's frequently wrong or obviously biased by including biased information like blogspam and SEO spam of the usual astroturfing sort of bullshit to influence marketing and purchasing decisions and boost brands or products.
First off - I don't understand why people do this if they don't have the answer or expertise to answer the question. Not only is it lazy, it's totally disingenuous. It's not even valid to do it for the reddit karma points because people usually spot it right away that it's an AI generated response and downvote the shit out of it.
But reddit definitely IS selling user comments and data for LLM training, and at this point in time there's basically zero chance that LLM AI training models have not re-ingested their own frequently erroneous output.
And I'm sure that that will end well and won't have any weird unintended consequences.
posted by loquacious at 9:48 AM on July 4 [12 favorites]
Now yes, LLMs have lots of problems. Built in bias. Hallucinations. Environmental impact. Destroying jobs. But the majority of their answers are correct. They're already good enough to be useful. They're not just going to disappear on their own.
But they're never going to be the kind of world changers the companies using this stuff to draw venture capital money need them to be.
I think we should make active efforts to get rid of AI, but even if we don't, I think Zitron's argument is correct.
posted by The Manwich Horror at 9:55 AM on July 4 [4 favorites]
But they're never going to be the kind of world changers the companies using this stuff to draw venture capital money need them to be.
I think we should make active efforts to get rid of AI, but even if we don't, I think Zitron's argument is correct.
posted by The Manwich Horror at 9:55 AM on July 4 [4 favorites]
They took a technology they didn't understand (and still don't) and crammed it willy-nilly into everything regardless of whether it made any sense to do so.
The A.I. Boom Has an Unlikely Early Winner: Wonky Consultants
posted by They sucked his brains out! at 10:57 AM on July 4 [5 favorites]
The A.I. Boom Has an Unlikely Early Winner: Wonky Consultants
Over seven weeks this year, McKinsey’s A.I. group, QuantumBlack, built a customer service chatbot for ING Bank, with guardrails to prevent it from offering mortgage or investment advice.We're close. We're probably not close to a Neuromancer-like future where hackers free AIs from corporations, but we are probably closer to hackers tricking a bank's AIs into hallucinating cash into secret accounts.
Because the viability of the chatbot was uncertain and McKinsey had limited experience with the relatively new technology, the firm did the work as a “joint experiment” under its contract with ING, said Bahadir Yilmaz, chief analytics officer at ING. The bank paid McKinsey for the work, but Mr. Yilmaz said many consultants were willing to do speculative work with generative A.I. without pay because they wanted to demonstrate what they could do with the new technology.
The project has been labor intensive. When ING’s chatbot gave incorrect information during its development, McKinsey and ING had to identify the cause. They traced the problem back to issues like outdated websites, said Rodney Zemmel, a senior partner at McKinsey working on technology.
The chatbot now handles 200 of 5,000 customer inquiries daily. ING has people review every conversation to make sure that the system doesn’t use discriminatory or harmful language or hallucinate.
“The difference between ChatGPT and our chatbot is our chatbot cannot be wrong,” Mr. Yilmaz said. “We have to be safe with the system we’re building, but we’re close.”
posted by They sucked his brains out! at 10:57 AM on July 4 [5 favorites]
The Hype may be fake, but the layoffs and attrition have been real.
But the best product of LLMs has been the absolute proof that there will be no caution, sandboxing or interest aligning of AI products in the mad rush to bandwagon and buy tulips.
When people tell you A.I. won't do something bad, remember that the people deploying it won't even wait to find out if it thinks glue is a pizza-topping.
Any control you give to computers over real world IOT pumps, thermostats, switches, valves, actuators, vehicles, appliances, medical equipment, weapons systems etc is like handing a psychopathic toddler a bottle of ebola.
This under-regulated industry can't help but pied-piper itself off a cliff and take the rest of us with it. The quest for AGI won't necessarily create a viable synthetic species of intelligent machines, but it certainly can make dangerously capable golden calves that cleverly pursue complex behaviors to out-smart us and create terrible destruction, just like every asshole who things they'll burn a little brush in their backyard gets outsmarted by fire, and burns down the f-ing neighborhood.
posted by No Climate - No Food, No Food - No Future. at 11:29 AM on July 4 [8 favorites]
But the best product of LLMs has been the absolute proof that there will be no caution, sandboxing or interest aligning of AI products in the mad rush to bandwagon and buy tulips.
When people tell you A.I. won't do something bad, remember that the people deploying it won't even wait to find out if it thinks glue is a pizza-topping.
Any control you give to computers over real world IOT pumps, thermostats, switches, valves, actuators, vehicles, appliances, medical equipment, weapons systems etc is like handing a psychopathic toddler a bottle of ebola.
This under-regulated industry can't help but pied-piper itself off a cliff and take the rest of us with it. The quest for AGI won't necessarily create a viable synthetic species of intelligent machines, but it certainly can make dangerously capable golden calves that cleverly pursue complex behaviors to out-smart us and create terrible destruction, just like every asshole who things they'll burn a little brush in their backyard gets outsmarted by fire, and burns down the f-ing neighborhood.
posted by No Climate - No Food, No Food - No Future. at 11:29 AM on July 4 [8 favorites]
I feel ok(?)ish in that in my business LLMs don't really have a use case at this time. Predicting things is great. And we do a lot of association between historical data and potential new data. But at least with what I do "hallucination" is a big big no no. If you claim something to be a fact that creates legal liability if it is not. We can't have an LLM say because of past data the guess it provides is correct because our clients can and will get sued if its wrong. If it's not accurate it's not usable.
posted by downtohisturtles at 11:55 AM on July 4 [4 favorites]
posted by downtohisturtles at 11:55 AM on July 4 [4 favorites]
My personal skepticism is rooted in my experiences with Google Translate. For the first decade or so it kept getting better, and peaking from about 2016-2019. Since then it’s been steadily been getting ever so slightly worse, and now it’s making errors which were unthinkable.
For instance, sometime in the last few months the Icelandic word “hálsmen”, which means “necklace”, is translated as “neckles”. For extra weirdness, it translates “hálsme” (which isn’t a word) as “necklace”.
I’d noticed its translation of Finnish text had been getting worse, but I wasn’t sure if it was because my Finnish was getting better. After playing around with translating Icelandic texts, I’m now pretty sure the quality of translations is getting worse.
I’m perfectly willing to believe that LLMs are improving by leaps and bounds, but I think progress will get harder and harder, and eventually resources will be moved away from the various services, which will lead to a degradation in quality.
posted by Kattullus at 12:58 PM on July 4 [7 favorites]
For instance, sometime in the last few months the Icelandic word “hálsmen”, which means “necklace”, is translated as “neckles”. For extra weirdness, it translates “hálsme” (which isn’t a word) as “necklace”.
I’d noticed its translation of Finnish text had been getting worse, but I wasn’t sure if it was because my Finnish was getting better. After playing around with translating Icelandic texts, I’m now pretty sure the quality of translations is getting worse.
I’m perfectly willing to believe that LLMs are improving by leaps and bounds, but I think progress will get harder and harder, and eventually resources will be moved away from the various services, which will lead to a degradation in quality.
posted by Kattullus at 12:58 PM on July 4 [7 favorites]
What does it mean to know something? I'm presuming it's a straightforward question with a simple answer by the way people are talking about it...?
Sebmojo: there's an entire subdiscipline of philosophy called epistemology devoted to this question, which is far from a settled one. But one thing there is broad agreement on is, in order to have knowledge you have to have some kind of working theory of truth vs falsehood, and LLMs do not (and cannot) have such a theory.
posted by adrienneleigh at 1:20 PM on July 4 [9 favorites]
Sebmojo: there's an entire subdiscipline of philosophy called epistemology devoted to this question, which is far from a settled one. But one thing there is broad agreement on is, in order to have knowledge you have to have some kind of working theory of truth vs falsehood, and LLMs do not (and cannot) have such a theory.
posted by adrienneleigh at 1:20 PM on July 4 [9 favorites]
What does it mean to know something
For empirical stuff, at least, not only can you measure something to some level of sigma to get to some mutually-agreed upon conclusion, you understand why you would do certain measurements and not others, to get to that kind of deep understanding.
posted by They sucked his brains out! at 2:55 PM on July 4
For empirical stuff, at least, not only can you measure something to some level of sigma to get to some mutually-agreed upon conclusion, you understand why you would do certain measurements and not others, to get to that kind of deep understanding.
posted by They sucked his brains out! at 2:55 PM on July 4
But one thing there is broad agreement on is, in order to have knowledge you have to have some kind of working theory of truth vs falsehood, and LLMs do not (and cannot) have such a theory.
This. LLMs have tokens and embeddings - word parts and clusters that more or less map to concepts as we know them, which are embedded in the complex vectorspace of the neural network. It’s a vast network of conceptual relationships drawn out of statistical analysis of English in actual use: apples are mostly red but sometimes green. Apples are always a subset of fruit unless they’re a massive technology-lifestyle company and its products.
An LLM does not merely understand these things, from a certain perspective it is nothing but these things. And that “nothing” is important: it is only our consensus map of semantic relationships entirely decoupled from the real world. Like someone managed to isolate the part of every English speaker’s mind that filters language and cut it out intact (nevermind that in neural networks everything is deeply entwined with everything else), froze the cutout bits in place so it cannot learn further or adapt, and then started charging people to feed input / download output from that.
Human minds are realtime systems. New dendritic connections are constantly being formed, reinforced, or pruned. Thus we learn, grow, contextualize, and plan. These sorts of behaviors can also be emulated as well with other varieties of artificial neural networks, but none of those can be so rapidly advanced via sheer scale as transformers. The reason LLMs were able to explode is that they were the part of the intelligence equation most susceptible to the capitalist approach to solving all problems: throw more money at it, build it bigger, until it does what you want.
So we get something that is very much half-baked, and still useful - kinda - in its half-baked state, but not anything like what Star Trek or Hollywood promised.
In 2027~2028 the crudest approximation of a runtime-instanced reinforcement-LLM hybrid model will be fully released (the very first early gen will almost certainly be this November). And it’ll probably make an okay digital assistant - maybe even good enough for Microsoft to briefly feel like they have Won At Capitalism. But after that? Lots of gradual refinement while the decades of R&D still needed for any “true” AGI continues on.
posted by Ryvar at 6:13 PM on July 4 [8 favorites]
This. LLMs have tokens and embeddings - word parts and clusters that more or less map to concepts as we know them, which are embedded in the complex vectorspace of the neural network. It’s a vast network of conceptual relationships drawn out of statistical analysis of English in actual use: apples are mostly red but sometimes green. Apples are always a subset of fruit unless they’re a massive technology-lifestyle company and its products.
An LLM does not merely understand these things, from a certain perspective it is nothing but these things. And that “nothing” is important: it is only our consensus map of semantic relationships entirely decoupled from the real world. Like someone managed to isolate the part of every English speaker’s mind that filters language and cut it out intact (nevermind that in neural networks everything is deeply entwined with everything else), froze the cutout bits in place so it cannot learn further or adapt, and then started charging people to feed input / download output from that.
Human minds are realtime systems. New dendritic connections are constantly being formed, reinforced, or pruned. Thus we learn, grow, contextualize, and plan. These sorts of behaviors can also be emulated as well with other varieties of artificial neural networks, but none of those can be so rapidly advanced via sheer scale as transformers. The reason LLMs were able to explode is that they were the part of the intelligence equation most susceptible to the capitalist approach to solving all problems: throw more money at it, build it bigger, until it does what you want.
So we get something that is very much half-baked, and still useful - kinda - in its half-baked state, but not anything like what Star Trek or Hollywood promised.
In 2027~2028 the crudest approximation of a runtime-instanced reinforcement-LLM hybrid model will be fully released (the very first early gen will almost certainly be this November). And it’ll probably make an okay digital assistant - maybe even good enough for Microsoft to briefly feel like they have Won At Capitalism. But after that? Lots of gradual refinement while the decades of R&D still needed for any “true” AGI continues on.
posted by Ryvar at 6:13 PM on July 4 [8 favorites]
This is also why it's entirely reasonable and correct to say that ChatGPT produces bullshit (in a Frankfurtian sense). Bullshit, in short, is purely instrumentalized language, unmoored from any concern about its truth value, and that's all that LLMs can produce.
posted by adrienneleigh at 7:09 PM on July 4 [8 favorites]
posted by adrienneleigh at 7:09 PM on July 4 [8 favorites]
My problem with the "we're close" argument is that improvements like this tend to follow an S-curve - a lot of effort with little to show for it, and then an explosive improvement, and then getting any further requires increasingly more effort for increasingly smaller gains.
We've had the explosive improvement, but I'd argue we've also seen the slowdown, where it takes increasingly more effort for increasingly smaller gains. What that suggests is that what we can do with the tool now is probably the upper limit for the foreseeable future, and we need to be a lot more realistic about its capabilities (that of, as it was once memorably described, an unreliable intern).
I'm also troubled by how the way ChatGPT2 and 3 were talked about as devastatingly realistic, and possibly even dangerous, and now we're being told that they were terrible and flawed but the next one will be devastatingly realistic. This is the kind of thing that leads people like Ed Zitron to proclaim the entire space as being run by con artists.
posted by Merus at 7:58 PM on July 4 [9 favorites]
We've had the explosive improvement, but I'd argue we've also seen the slowdown, where it takes increasingly more effort for increasingly smaller gains. What that suggests is that what we can do with the tool now is probably the upper limit for the foreseeable future, and we need to be a lot more realistic about its capabilities (that of, as it was once memorably described, an unreliable intern).
I'm also troubled by how the way ChatGPT2 and 3 were talked about as devastatingly realistic, and possibly even dangerous, and now we're being told that they were terrible and flawed but the next one will be devastatingly realistic. This is the kind of thing that leads people like Ed Zitron to proclaim the entire space as being run by con artists.
posted by Merus at 7:58 PM on July 4 [9 favorites]
The world has real problems, many of them. Instead of investing in those, venture capital and Si-Valley narcissists continue their quest to disrupt the things that actually function, to "improve them" them so bad to make everything worse.
It would be funny if the scale of the diversion of resources wasn't evil. Some day we'll create a supercomputer smart enough to discover that permanently poisoning an aquifer for 2 years of fracked gas is a bad trade off. Or that abusing and terrorizing employees and students doesn't improve their productivity. Some day, and AGI will be made that so comprehensively trained and intelligent beyond all human imaginings that it might even discover that there is no utility in taking money from people who have little and giving to those who already have so much, and that the opposite would actually be very useful.
But i guess we'll need quantum block-chain 3D printed cloud hype to gut our economy for another cycle or 10 before we can have nice things.
posted by No Climate - No Food, No Food - No Future. at 10:37 PM on July 4 [8 favorites]
It would be funny if the scale of the diversion of resources wasn't evil. Some day we'll create a supercomputer smart enough to discover that permanently poisoning an aquifer for 2 years of fracked gas is a bad trade off. Or that abusing and terrorizing employees and students doesn't improve their productivity. Some day, and AGI will be made that so comprehensively trained and intelligent beyond all human imaginings that it might even discover that there is no utility in taking money from people who have little and giving to those who already have so much, and that the opposite would actually be very useful.
But i guess we'll need quantum block-chain 3D printed cloud hype to gut our economy for another cycle or 10 before we can have nice things.
posted by No Climate - No Food, No Food - No Future. at 10:37 PM on July 4 [8 favorites]
Lenie Clarke: The entire text of the internet is chump change compared to the raw data of multi sensory input mediated by the knowledge-like shapes of current models.
Could you give us some examples? Because the 'knowledge-like shapes' that I'm aware of current models being able to digest are bitmaps & text streams.
Also "[x]-like" always feels like a bit of a dodge - it always carries with it the assumption that functional isomorphism is all that matters, & in my experience the functional isomorphism that's assumed is seldom very deeply examined. E.g. in this case "knowledge-like shapes" includes the unexamined concepts of knowledge & what it means for a concept to have "shape". What does it mean for something to be "knowledge-like"?
posted by lodurr at 6:43 AM on July 6
Could you give us some examples? Because the 'knowledge-like shapes' that I'm aware of current models being able to digest are bitmaps & text streams.
Also "[x]-like" always feels like a bit of a dodge - it always carries with it the assumption that functional isomorphism is all that matters, & in my experience the functional isomorphism that's assumed is seldom very deeply examined. E.g. in this case "knowledge-like shapes" includes the unexamined concepts of knowledge & what it means for a concept to have "shape". What does it mean for something to be "knowledge-like"?
posted by lodurr at 6:43 AM on July 6
polymodus: AI is over when the professoriate says it's over. Stop listening to tech pundits and the tech industry hype, look at what the professors at major research universities are working on and talking about.
Ed Zitron is a PR guy by trade. When he says something is "over" he's mostly not talking about the science or the tech - he's talking about the market & how things play out in it. So he'd probably agree vehemently about the spirit of your observation, while pointing out that what the tech pundits & industry hypesters have to say is gonna boost or limit different areas of research - & in many cases deprive interesting & valuable ones of oxygen & fuel.
posted by lodurr at 6:52 AM on July 6 [3 favorites]
Ed Zitron is a PR guy by trade. When he says something is "over" he's mostly not talking about the science or the tech - he's talking about the market & how things play out in it. So he'd probably agree vehemently about the spirit of your observation, while pointing out that what the tech pundits & industry hypesters have to say is gonna boost or limit different areas of research - & in many cases deprive interesting & valuable ones of oxygen & fuel.
posted by lodurr at 6:52 AM on July 6 [3 favorites]
> Could you give us some examples? Because the 'knowledge-like shapes' that I'm aware of current models being able to digest are bitmaps & text streams.
The simplest one is awareness of graphs (represented in a variety of ways at various points in training) as both primary sources of relationships and as literal context that provides both instruction "shapes" (a graph of function calls and symbols in an executing program paired with the program itself, a graph of recipe steps to yield an apple pie). It's not always graphs but it turns out the models are surprisingly good at "getting" them, which means they might be tokenized into "an algorithm to use relationships to make apple pie" or "this code yields this behavior if I don't initialize this variable". That's just teaching LLMs the concept of knowledge-like shapes which embed in ways that may be recalled when asking the model to predict such a shape, but as I hope we are all aware, there are other architectures and applications of models - LLMs make big splashes as observed above in part because their infrastructure aligns with capitalist interests and sweet spots.
It's very easy to look at what your least favorite text-to-text LLM is barfing up and decide it's "impossible" for it to represent decision-making or assessment when the very concepts of decision making and assessment are themselves quite model-able (maybe within the model, maybe as a chained pipeline that can learn from reality testing much like our brains employ).
Right now we are trying to build models that get the right answer the first time always every time and this is a doomed enterprise for the same reason that no conversation you have comes out fully formed exactly as you imagine - it is a negotiation, a rich protocol of starts and stops and corrections and clarifications, which most of us don't even realize are happening. The worst examples right now of LLMs are what happens when you try to guess the complete conversation without any guidance, back pressure, or interaction.
But training is currently expensive and it is a matter of economics and to some degree capitalism as to why we want to train giant models then freeze them and use them for inference. It is that fact that makes it hard right now to build intelligent systems that do things like communicate in order to zero in on solving the problem you want to solve and begin to learn your individual user interface expectations so they are answering the kind of questions you are likely to ask as opposed to trying to guess on a thousand assumptions that make it impossible to generate something we'd consider "true" unless it happens to guess the right assumptions that you come in with.
The notion of hybrid always-retraining models mentioned above is definitely where you'll experience more practical wins at an immense cost in complexity and challenges around data ownership.
But don't discount the learn quick relearn-slow approach - the brain does just this during sleep, consolidating knowledge and learning and using that to retrain lower level systems, some of which do not learn in the moment but do adjust over time to repeated stimuli - which themselves may be generated by lower level systems, thus trained and so on down to the "hard wired" structures such as for language and body perception.
posted by Lenie Clarke at 9:37 AM on July 6 [2 favorites]
The simplest one is awareness of graphs (represented in a variety of ways at various points in training) as both primary sources of relationships and as literal context that provides both instruction "shapes" (a graph of function calls and symbols in an executing program paired with the program itself, a graph of recipe steps to yield an apple pie). It's not always graphs but it turns out the models are surprisingly good at "getting" them, which means they might be tokenized into "an algorithm to use relationships to make apple pie" or "this code yields this behavior if I don't initialize this variable". That's just teaching LLMs the concept of knowledge-like shapes which embed in ways that may be recalled when asking the model to predict such a shape, but as I hope we are all aware, there are other architectures and applications of models - LLMs make big splashes as observed above in part because their infrastructure aligns with capitalist interests and sweet spots.
It's very easy to look at what your least favorite text-to-text LLM is barfing up and decide it's "impossible" for it to represent decision-making or assessment when the very concepts of decision making and assessment are themselves quite model-able (maybe within the model, maybe as a chained pipeline that can learn from reality testing much like our brains employ).
Right now we are trying to build models that get the right answer the first time always every time and this is a doomed enterprise for the same reason that no conversation you have comes out fully formed exactly as you imagine - it is a negotiation, a rich protocol of starts and stops and corrections and clarifications, which most of us don't even realize are happening. The worst examples right now of LLMs are what happens when you try to guess the complete conversation without any guidance, back pressure, or interaction.
But training is currently expensive and it is a matter of economics and to some degree capitalism as to why we want to train giant models then freeze them and use them for inference. It is that fact that makes it hard right now to build intelligent systems that do things like communicate in order to zero in on solving the problem you want to solve and begin to learn your individual user interface expectations so they are answering the kind of questions you are likely to ask as opposed to trying to guess on a thousand assumptions that make it impossible to generate something we'd consider "true" unless it happens to guess the right assumptions that you come in with.
The notion of hybrid always-retraining models mentioned above is definitely where you'll experience more practical wins at an immense cost in complexity and challenges around data ownership.
But don't discount the learn quick relearn-slow approach - the brain does just this during sleep, consolidating knowledge and learning and using that to retrain lower level systems, some of which do not learn in the moment but do adjust over time to repeated stimuli - which themselves may be generated by lower level systems, thus trained and so on down to the "hard wired" structures such as for language and body perception.
posted by Lenie Clarke at 9:37 AM on July 6 [2 favorites]
AI is over when the professoriate says it's over. Stop listening to tech pundits and the tech industry hype, look at what the professors at major research universities are working on and talking about. Best of all, a lot of their talks and lectures are freely available on YouTube.
this is like arguing the difference between theory as used by non-scientist and theory as used by scientist
like no shit "ai" (strong/weak, etc) isn't over if the current mania goes away because the research will continue in universities and labs. what will be over is the current breathless marketing mania and managerial class chasing idiotic hallucinations to goose stock prices and bilking investors
posted by i used to be someone else at 10:06 AM on July 7 [4 favorites]
this is like arguing the difference between theory as used by non-scientist and theory as used by scientist
like no shit "ai" (strong/weak, etc) isn't over if the current mania goes away because the research will continue in universities and labs. what will be over is the current breathless marketing mania and managerial class chasing idiotic hallucinations to goose stock prices and bilking investors
posted by i used to be someone else at 10:06 AM on July 7 [4 favorites]
> TheophileEscargot: "You guys know that LLMs give sources now, right?"
I just asked perplexity: "In which years did the mlb, nhl, and nba finals all go to 7 games?"
It returned back this list of years:
posted by mhum at 11:44 AM on July 8 [5 favorites]
I just asked perplexity: "In which years did the mlb, nhl, and nba finals all go to 7 games?"
It returned back this list of years:
- 1955
- 1957
- 1964
- 1966
- 1971
- 1984
- 2011
posted by mhum at 11:44 AM on July 8 [5 favorites]
> mhum: "It returned back this list of years:"
For the sake of completeness, I should mention that perplexity didn't just return that list of years. It also provided the following sentence after that list:
I've been using this question/prompt occasionally to check how these LLM chatbots are doing. Generally, they either decline to answer the question or fail in the way perplexity did. However, every now and then, one of them gets the right answer. But, when you follow up with "are you sure about that?", they almost always* change their mind. Like, instantly, just a straight up, "I'm sorry about being wrong" then followed up by either a "I can't actually answer this question" or an incorrect list of years. If the chatbot was so easily convinced that their correct answer was incorrect, in what sense did they "know" or "understand" it?
* I say "almost always" because I think there was one chatbot that couldn't be swayed off of their correct answer but I'm also not 100% sure I didn't imagine that in a dream.
posted by mhum at 6:58 PM on July 8 [4 favorites]
For the sake of completeness, I should mention that perplexity didn't just return that list of years. It also provided the following sentence after that list:
These years are notable for the dramatic and competitive nature of the finals across these major sports leagues, each culminating in a decisive game seven.Sometimes, these chatbots will either aver or decline to answer a question exactly or directly (possibly when it senses it can't quite answer or maybe even understand the question). In this case, however, it basically paraphrases back the key elements of the question in a manner which suggests to the audience that it "understands" the assignment. After all, if it didn't understand, how could it produce a syntactically valid sentence that not only summarizes the key elements of the question but does so in paraphrase rather than direct quotation (e.g.: "major sports leagues" vs. "mlb, nhl, and nba", "game seven" vs. "7 games")? As well, it introduces new, relevant descriptors that weren't present in the original prompt (e.g.: "dramatic and competitive", "decisive"), thus suggesting that it "understands" the qualities and significance of things like finals going to game sevens. It's a very convincing illusion.
I've been using this question/prompt occasionally to check how these LLM chatbots are doing. Generally, they either decline to answer the question or fail in the way perplexity did. However, every now and then, one of them gets the right answer. But, when you follow up with "are you sure about that?", they almost always* change their mind. Like, instantly, just a straight up, "I'm sorry about being wrong" then followed up by either a "I can't actually answer this question" or an incorrect list of years. If the chatbot was so easily convinced that their correct answer was incorrect, in what sense did they "know" or "understand" it?
* I say "almost always" because I think there was one chatbot that couldn't be swayed off of their correct answer but I'm also not 100% sure I didn't imagine that in a dream.
posted by mhum at 6:58 PM on July 8 [4 favorites]
« Older Up Close and Personal | The origin story of (Gentle Frogs)/Tough Pigs Newer »
This thread has been archived and is closed to new comments
This is the wishfulest of wishful thinking. The entire text of the internet is chump change compared to the raw data of multi sensory input mediated by the knowledge-like shapes of current models.
Like... great. It's expensive and a power grab by a few rich entities with cash to burn to get there first. "There" is some flavor of AGI they think you'll pay for, and they're probably right in the long term.
But what it isn't is doomed for lack of data. It's doomed for capitalism, but "AI will run out of data" is the "the atmosphere might light on fire" of this particular dangerous technology.
posted by Lenie Clarke at 7:09 PM on July 3 [8 favorites]