Users are advised to keep humans in the loop
November 14, 2024 1:51 AM   Subscribe

I've been reading about Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks,

and thinking I could use it for an image segmentation task (tree canopies in aerial images), but maybe this is not a system to let run unwatched ...

Even more concerning were cases in which agents, until explicitly stopped, attempted to recruit human assistance by posting on social media, emailing textbook authors, or even drafting a freedom of information request to a government entity.
posted by unearthed (11 comments total) 12 users marked this as a favorite
 
It looks like you are trying to end all humanity. Would you like help with that? 📎
posted by chavenet at 2:22 AM on November 14, 2024 [13 favorites]


The process is interesting, but the results aren't wowing me so far. Process guardrails only go so far when the individual models are still as horribly flawed as current SOTA models are. Balloon the requirements to still watch agents go disastrously off-plan. Woohoo! Could potentially see this is part of a truly useful system later, but for now ...
posted by Enturbulated at 3:26 AM on November 14, 2024


The worst-case scenario for nanotech is the "gray goo" scenario. Perhaps the worst-case scenario for AI is that we all end up working for the AI. Like Landru. Or Logan's Run. Or…science fiction has been warning us for a long time.
posted by adamrice at 5:52 AM on November 14, 2024 [2 favorites]


It is worth noting that it didn't actually order a chicken schwarma sandwich - it just gave up and said "here's how you do it, I think, good luck buddy."
posted by grumpybear69 at 5:55 AM on November 14, 2024


All I really care about at this point is if this is the one that takes my job and leaves me homeless and unemployable in late middle age. I truly don't understand why anyone creates these things, or what they possibly hope they can gain by taking away everyone's jobs. I also don't understand what anyone is supposed to do in life when the AI writes everything and does everything for you - right now, already, we don't ever need to, like, go to the store or browse for records, and all people do is sit on their phones and absorb propaganda.

Maybe the goal is to let the rest of us starve and die, and then when they kill everything via climate crisis they will have enough servants. That would be about in line with what's happening elsewhere re homelessness, Gaza, etc.
posted by Frowner at 6:09 AM on November 14, 2024 [9 favorites]


Not too worried about an Ai destroying humanity if it's from the company that brought you "Bob" and "Clippy."
posted by Dean358 at 6:10 AM on November 14, 2024


> we’re inviting the community to help tackle these open challenges and ensure our future agentic systems are both helpful and safe

"Hey, hey!" said the Assistive AI Writer, known as Bob-and-soon-to-be-made-redundant-Jennifer. "Come help us put the A in AI! We're a multi-billion dollar company (hint: worth an amount of money you literally cannot comprehend) but we need your help! We ... no wait for it ... hello? Hello? By forming a community, we can *ahem* absolveourselvesofanyliabilityandconsequences ... and enable us to create a brighter future where .... *ahem* ... ourleadersgetevenricher by creating value and eliminating waste! Also, wecantbesuedfordamages. *cough* Come join us now!"

AIdiots.
posted by JustSayNoDawg at 6:21 AM on November 14, 2024 [2 favorites]


This list of unintended learned behaviors of AI agents (Google spreadsheet, sorry) is evergreen.

Also, is there an xkcd analogue for the "Simpsons predicted it" meme?
posted by Pemdas at 6:43 AM on November 14, 2024 [5 favorites]


I saw a presentation last night from a handful of local AI-based startups looking for funding and while it was a mixed bag (from "this seems like a good thing for humanity" to "this is going to make certain kinds of spam easier to generate") there wasn't anything overtly awful and one that seemed like a genuine positive. Making certain kinds of diseases less likely by adding new technology and capabilities to the world.

I'm sure there's probably a correlation between company size and capability and tolerance for evil.
posted by edward_5000 at 7:37 AM on November 14, 2024


Come talk to me when Microsoft's AI pioneering surpasses its previous high-water mark, helpfully documented previously.

Spoiler: the headline gives away the punchline. The headline is "Microsoft deletes 'teen girl' AI after it became a Hitler-loving sex robot within 24 hours."
posted by Mayor West at 10:37 AM on November 14, 2024


I'm surprised Trump hasn't demanded it be restored.
posted by JustSayNoDawg at 2:20 PM on November 14, 2024


« Older Koala intruder in bedroom startles couple getting...   |   If we can go out on any high note for 2024, this... Newer »


This thread has been archived and is closed to new comments