Did you see the thing where it was [advising people to drink pee for 24 hours in order to pass kidney-stones](https://www.youtube.com/watch?v=WfaOzxlliUA)?

Unrelated to Google's AI, but related to AI in general, [the results of this AI-written cake recipe were quite funny](https://www.youtube.com/watch?v=nUqPOsgu0uo). :tongue:

And they just keep coming, how about [some spicy gasoline spaghetti](https://mastodon.social/@ThePlant/112496961845510096) lol

Quoting: PenglingDid you see the thing where it was [advising people to drink pee for 24 hours in order to pass kidney-stones](http://www.youtube.com/watch?v=WfaOzxlliUA)?

Quoting: Liam Dawe*This counter will never need to be incremented or reset.

I am very much enjoying seeing all these utterly ridiculous AI things from Google. Arse Technica did a [nice overview](https://arstechnica.com/information-technology/2024/05/googles-ai-overview-can-give-false-misleading-and-dangerous-answers/).

The overview was pretty good until the end they put some platitudes about how it is "improving all the time", diminishing these serious concerns in a way that can dangerously mislead the general public.

Most people without a somewhat technical understanding of machine learning and LLMs see these mistakes as "bugs", as "growing pains" while the tech is being perfected... when they are in fact fundamental limitations of this approach. It isn't a few implementation errors you can debug and fix; to fix the general problem, all they can do is research and hope someone discovers a totally new technique, which might not even exist to be discovered.

What those scammy AI companies do is add "filters" to deal with those particular edge cases (which is why sometimes you can't replicate the problems people reported on social media), but they do nothing for all the other bullshit it will spew in the future. We keep seeing people "tricking" LLMs into assuming personas and talking about stuff it was not supposed to talk about, or using hypotheticals and double negatives. Sometimes we will even see these filters overcompensate and insist on a pre-programmed answer to one of those detected flaws, even when that answer isn't really relevant (but the question is similar enough). It all showcases how this approach is never foolproof: you need filters to protect your filters, you need your heuristics to cover every possibility. Instead of programming all correct answers into a program, they have to instead program all wrong answers so that their bullshit generator doesn't make up stuff that looks too bad.

Last edited by eldaking on 24 May 2024 at 5:36 pm UTC