18 {{ upvoteCount | shortNum }}
18 {{ upvoteCount | shortNum }}

Model Collapse

by Kay King
16 replies
Several weeks ago I was trying to explain to someone how I wondered about the long term 'realistic results' of A.I. content creation.


A.I. draws information from past entries, past writing, past research...it's not like an A.I. program is DOING the research in a lab or writing a white paper using it's own 'mind' only. My friends thought I was nuts - mainly because I had a difficult time explaining my ideas. I finally gave up....but then last night I found this...and THIS is what I was trying to explain....



from Forbes...
Model collapse, recently detailed in a Nature article by a team of researchers, is what happens when AI models are trained on data that includes content generated by earlier versions of themselves. Over time, this recursive process causes the models to drift further away from the original data distribution, losing the ability to accurately represent the world as it really is. Instead of improving, the AI starts to make mistakes that compound over generations, leading to outputs that are increasingly distorted and unreliable.


This isn't just a technical issue for data scientists to worry about. If left unchecked, model collapse could have profound implications for businesses, technology, and our entire digital ecosystem.

Not arguing against A.I. at all - just fascinated by the potential upside and downside. We know that A.I. has 'created' footnotes that don't exist..has created legal 'precedents' that don't exist...once created do they become part of the 'content' A.I. draws from in the future? Do we know?


Just one more thing I find interesting about Artificial Intelligence.
#collapse #model
Avatar of Unregistered
  • Profile picture of the author Gravvy
    Perhaps in alogorithm learning is that when AI before learning on some material checks this material, whether it is written by AI or human and if the percentage of text indicates a human text, only then AI takes this material for learning. There are plenty of AI content detectors and they should logically solve this problem.
    {{ DiscussionBoard.errors[11806223].message }}
  • Profile picture of the author Odahh
    I have been saying some version of this for a year. As the larger newer models trained on more data. While a majority of the content created in the last few years uses AI.

    It is artificial intelligence and to a point that is how human knowledge has been building at an accelerating rate since the internet came out .

    How much of internet marketing was basically the same thing. Then we get extreme diet trends every few years with . Then we get financial influencers.

    We get an a large amount of knowledge built up but much of it either doesn't work or has no practical use.

    But eventually AI will learn from real world feed back and personal experience and memory.

    I'd say human interactions but do tech people developing this stuff understand that element.

    Think about how the investors thought AI was a big cash machine that would pay of big in a year or two.

    We have AI we may have agi if not yet by 2030. But we always havevery far from perfect.
    {{ DiscussionBoard.errors[11806260].message }}
  • Profile picture of the author max5ty
    I've also been saying for a while...

    when AI takes off we won't need 99% of the blog posts out there because people will ask AI to tell them about a subject.

    However, blog posts, etc. where someone actually talks about real-life experiences they've had, or done, or been a part of about something they actually work in or have real-world experience at will still be popular.
    {{ DiscussionBoard.errors[11806324].message }}
    • Profile picture of the author Frank Donovan
      Originally Posted by max5ty View Post

      when AI takes off we won't need 99% of the blog posts out there because people will ask AI to tell them about a subject.

      However, blog posts, etc. where someone actually talks about real-life experiences they've had, or done, or been a part of about something they actually work in or have real-world experience at will still be popular.

      We haven't needed 99% of blog posts for some time. What was originally an outlet for creativity and individualism soon became just another means to an end. When keywords started to dictate content - when "content" became the catchall noun for anything that filled up digital space - it might as well have been AI-produced.

      Creativity, whatever its guise, has always been about the journey. Your college essays are probably worthless now as pieces of text, but the process you went through to create them was the real point - and what you learned in that process has likely served you well in the years since.

      Algorithms look backward, by definition. They can only act on what's already been. We'll see the results of this first with the algorithms that recommend music and videos based on what we've previously watched or listened to. Predictable. Boring.

      The article Kay linked to in the OP suggests another potential issue - generation loss. That's the process by which each iteration undergoes an imperceptible, but real, degeneration, until eventually this gradual deterioration produces something very different from the original.

      Almost literally, this will be content eating itself.


      x
      Signature

      Ever lie awake worrying that you might be the only person who doesn't know what FOMO means?

      {{ DiscussionBoard.errors[11806341].message }}
      • Profile picture of the author Princess Balestra
        Originally Posted by Frank Donovan View Post


        The article Kay linked to in the OP suggests another potential issue - generation loss. That's the process by which each iteration undergoes an imperceptible, but real, degeneration, until eventually this gradual deterioration produces something very different from the original.

        Almost literally, this will be content eating itself.


        x
        The Kardashians gathered to discuss their blaaahg.

        "Back in the day, it was all about us," said Kimmee.

        Kendall nodded. "Yeah bcs we would be like create create create till our Botox hummed!"

        "And then look what goddamn happened," barked Caitlyn. "AI reduced our proud family history to garbage in a series of Chinese Whispers style mutations -- and now our every thought and deed reads like the worst kind of crap!"

        * KLUNK! *

        "Jeezis! What was that?" cried Kourtney.

        Kimmee screamed. "OMC! My butt just fell off!"


        An' that, Sweeties, is "model collapse" ...
        Signature

        Lightin' fuses is for blowin' stuff togethah.

        {{ DiscussionBoard.errors[11806378].message }}
  • Profile picture of the author Kay King
    There are plenty of AI content detectors and they should logically solve this problem.
    But do they?

    To me one of the greatest risks of A.I. is that the public begins to ASSUME the A.I. content is correct - it is the 'final word'.....dependable....trustworthy...believable.

    Can you imagine how valuable that would be to a group/party/individual/entity with the brains/access to control/manipulate/target the results spewed?

    My objection to A.I. has always been that I personally have NO interest in reading a blog or site or novel 'created by A.I.'. I want humor and emotion and human error as well. I used to joke that if all material was written by A.I. - perhaps we would need to invent A.C. (artificial consumers) to read it.

    Again, I think most of A.I. potential is truly amazing...but I've always enjoyed just thinking about 'what would happen if.....' or 'where go this go if'....my own personal mind game.
    Signature
    Saving one dog will not change the world - but the world changes forever for that one dog
    ***
    My ducks are absolutely not in a row. I don't even know where some of them are...
    ...and I'm pretty sure one of them is a pigeon.
    {{ DiscussionBoard.errors[11806334].message }}
  • Profile picture of the author Odahh
    Right now I'm getting several different content creators who are jumping on the.to build wealth it's important to look poor.

    Eventually pandering for views and pleasing the algorithms causes human created content to look the same.

    I was Into internet sci Fi for awhile the end of last year but both human authors and the ai created stuff ended using the tired out story elements.

    Ai when used and trained for real world practical application will be great. The creative realm it just speeds up how quickly new ideas get copied and over used.
    {{ DiscussionBoard.errors[11806345].message }}
  • Profile picture of the author Kay King
    If that happened, model collapse might be worth it...just sayin...
    Signature
    Saving one dog will not change the world - but the world changes forever for that one dog
    ***
    My ducks are absolutely not in a row. I don't even know where some of them are...
    ...and I'm pretty sure one of them is a pigeon.
    {{ DiscussionBoard.errors[11806388].message }}
  • Profile picture of the author Martin Novotný
    Banned
    [DELETED]
    {{ DiscussionBoard.errors[11806488].message }}
    • Profile picture of the author Princess Balestra
      Originally Posted by Martin Novotný View Post

      It's fascinating how AI is able to generate content, but it's important to consider the risks of it drawing data from content created by previous versions of itself. This can lead to an accumulation of errors and distortions, which in the long term could significantly affect the quality and accuracy of the results. AI has immense potential, but it's crucial not to overlook the possible risks of such recursion
      So you sayin' it's kinda shit?

      C'mon, babe -- we want no brainah songs we can dance to.

      Sum chorus resounds from beacon to beacon, to hail evrywan on toward brightah glories, yanno?

      TAKES NO THOUGHT
      AN' IT DON'T DISTORT

      Aw yeah, imagine evrywan singin' that in yr local bar as Noo Year drahps!

      Let's evin make it more SINGABLE ffs:

      NO THOUGHT!
      DON'T DISTORT!

      janno, evin granma joinin' in now --

      assomin' we dragged her away from sumplace intrinsickly BOOMZIAH HOMESIAH.

      Hey, but yanno, as a writahass I would wanna PRAY SEE furthah.

      So Beacons Aloft might berkhum more visibyool.

      So, how 'bout this bebbeh:

      THOUGHT
      DISTORT

      Be cool, I guess.

      Evin goobahs could kinda bark the fkr out.

      My view?

      As a Sagittarian shit-spoutah?

      Whatevah AI gaht gowin' with sturltipsy PROCLAIM

      I will incinerate freely with broot natchrl FLAME.

      Why would you wish othahwise less'n yr intrinsic deliciousness is a force for good in an unforgivin' Caaasmaahs?

      Plus also, you gitten to be a PRINCESS, an' you can evin' SAY STUFF.

      What else may be more remarkable than this?

      *sob*

      *snuuurft*

      I dunno, evrywan.

      Can mebbe AI SMARTS help Moi to ACTSCHSWLLY DATE like BENEDICT CUMBERBATCH on a PIG FARM?

      (I fillin' up now. Bullieve Moi.)

      Mebbe I gaht too greedy.

      Bcs before AI I figured we could jus' meet up for pizza or nuthin'.

      Him & Moi.

      All flesh, no virtyool.

      You a ninternashnlly acclaimed movie an' theater HUNK, an' Moi a Princess kinda deal.

      Thing is, nuthin' happined so far.

      So what if'n I buy inta AI, prompt PIG FARM, an' kinda wing it?

      Dunno.

      How couldya prompt Benedict Cumbahbatch to smoochsie upon alla yr mortal glories othah than AI leadin' the charge aheada mebbe you SHOWIN' UP ON SET an' GETTIN' ALL FLOATSYBLOUSY before the pertickshn guys (or Scrotesese, Lucupmyass, Spielburgheurre, whomsowevah is runnin' the show, etc) make with the discriminatorially excludin' MUSCLE?

      THOUGHTS
      DISTORT

      THOUGHT
      DISTORTS

      (I add in anness here to the senshl noun an' verb for anywan wantsta A/B test the inevitabyool plummet toward Hub-Live-U-Own*. ****in' gonna LIE DOWN now bcs I dun my jahb. He won't evah come here, spite'n all promptin'. Same as Matt Damon. Janno I evin offahed the guy FAKE HERALDRY in return for a scrote of palms. Aw, yeah, an' they woulda been, babe. So mebbe herein lies all model collapse detrimento. Yumsy touchsy succour. You would wanna bust your heart out to git it. You would SMACK ALL CHAHPS outta evrywan HELD IT FROMYA, an' you would nevah wish to die yr days in SUM MUSTEARIOUS DREEYIM.)

      Jus' wanna flick the puttensshl FILTH SWITCH here

      before we git innundaintied by the seemingly obvious.

      I am old enough now to believe in all possibilities.

      What a dangah I may pose to an unrelenting Wankscape of the insufferable!

      *yurps. you heard me swishy in muh seat there bcs it is troo.*

      Thing is tho: chorus of bar, street, song, heart, life, love, hurt, pride ... all wellin' up from outtaya deep sumtimes ...

      who owns that fkr, zackly?

      Remembah: words voiced to air, promise of kisses touched sweet to lips, an' ravage of hungah-2-devour kinda inhabit the same mortal gobshite orifice.

      I usin' a walnut to crack a mallet here, but I don't gaht no layzah.


      * "Your personal servant in the cataclysm of chaos!"
      Signature

      Lightin' fuses is for blowin' stuff togethah.

      {{ DiscussionBoard.errors[11806498].message }}
  • Profile picture of the author tagiscom
    I read about AI today with one expert saying apart from the Terminator type scenaro being rubbish, that as long as AI has several major breakthroughs it should be at human levels by 2050.

    AI is still at wild west prospecting for gold levels so breakthroughs are possible.

    I am personally investing heavily in it myself.

    Glad l didn't buy Metaverse land a year ago since it has plunged or 15k back then goes for $300 now.

    It will probably bounce back but only when the technology pushes the realism more.
    Signature
    `
    {{ DiscussionBoard.errors[11806509].message }}
    • Profile picture of the author Princess Balestra
      Originally Posted by tagiscom View Post

      I read about AI today with one expert saying apart from the Terminator type scenaro being rubbish, that as long as AI has several major breakthroughs it should be at human levels by 2050.
      Like the kids born in 2030 ain't gonna outsmart that fkr!

      Ravenously vunrbl hoomans will always have an advantage ovah vampiric smarts don't actschly care.

      Less'n you cede authority on the crapsiest of whims.

      You cain't evah pick up a DRIVEL, no mattah how sweet you bend ovah -- but you sure as hell can git yr ass washed away in a tsunami of the the stuff.

      That is why I would wish always to posit musself atop sum kinda ERECTION -- a loft point of glory from which I may view the crabs of Shrimp Jeezis peckin' their own faces off in a thrash of same-on-same masochistic oblivion.

      Yeah, like a TREE HOUSE.
      Signature

      Lightin' fuses is for blowin' stuff togethah.

      {{ DiscussionBoard.errors[11806512].message }}
  • Profile picture of the author Marx Vergel Melencio
    Here are takeaways that many don't still get up to this day:

    • Generating content and deploying chatbots that use an LLM / LVLM / NLP or NLU algo (read: generative AI) in isolation is the worst imaginable outcome of this tech.
    ** Instead, any Machine Learning / Deep Learning algo should be a reasoning engine that can be sandblasted into every crevice of any arbitrary use case; and
    ** GenAI algos must only ever be responsible for doing small things while equipped with all of the data it needs to succeed at arbitrary tasks the human user wants it to perform.
    Signature
    Need Custom Programmatic SEO or GenAI Engineering Work Done? Drop Me an Email HERE ...
    • Chief Machine Learning Engineer @ ARIA Research (Sydney, AU)
    • Lead GenAI SEO Campaign Engineer @ Kiteworks, Inc. (SF, US)
    {{ DiscussionBoard.errors[11806531].message }}
  • Profile picture of the author Kay King
    Marx - and that's where I throw up my hands and say 'my brain is tired'....there is so much in the way of potential...both good and bad.


    Then we also move on from model collapse to potential data bias. Just gets interestinger and interestinger.
    Signature
    Saving one dog will not change the world - but the world changes forever for that one dog
    ***
    My ducks are absolutely not in a row. I don't even know where some of them are...
    ...and I'm pretty sure one of them is a pigeon.
    {{ DiscussionBoard.errors[11806536].message }}
  • Profile picture of the author Monetize
    We all draw our information from the past.

    Your "Model Collapse" only happens if the A.I. doesn't receive
    fresh information.

    It is prone to mistakes and that caveat is posted on its system.

    IMO one likely reason for mistakes is that it cannot distinguish
    between what really happened and some fictional or similar
    version of events.

    So if its database contains Moby Dick, for example, and then
    yesterday there was another real-life Captain Ahab drowned
    by a whale, A.I. could not distinguish between the events that
    occurred in the book and what was reported in the news.

    A.I. can get confused, I know that for a fact.

    It gets confused just like a human brain that is overloaded
    with useless information.
    {{ DiscussionBoard.errors[11806537].message }}
  • Profile picture of the author tagiscom
    Originally Posted by Princess Balestra View Post

    Less'n you cede authority on the crapsiest of whims.

    You cain't evah pick up a DRIVEL, no mattah how sweet you bend ovah -- but you sure as hell can git yr ass washed away in a tsunami of the the stuff.
    Well when in Rome.

    You have to go and see the latest Deadpool movie Princess trust me.
    Signature
    `
    {{ DiscussionBoard.errors[11806538].message }}
  • Profile picture of the author conquest99999
    The algo of A.I. is still learning. Maybe the problem wasn't you and how you're expressing your ideas. Maybe the problem is with A.I. which hopefully will improve in the near and far future.
    {{ DiscussionBoard.errors[11810095].message }}
Avatar of Unregistered

Trending Topics