• Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
Wednesday, November 18, 2026
No Result
View All Result
Subscribe Now
  • Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Trump Administration Revokes Pay Increase for Head Start Educators

    Trump Administration Revokes Pay Increase for Head Start Educators

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

    Beloved Former Los Angeles Times Columnist Steve Harvey Passes Away at 80

    Beloved Former Los Angeles Times Columnist Steve Harvey Passes Away at 80

  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
The Insight Post
  • Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Trump Administration Revokes Pay Increase for Head Start Educators

    Trump Administration Revokes Pay Increase for Head Start Educators

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

    Beloved Former Los Angeles Times Columnist Steve Harvey Passes Away at 80

    Beloved Former Los Angeles Times Columnist Steve Harvey Passes Away at 80

  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
No Result
View All Result
No Result
View All Result
Home Science

Could AI Eat Itself to Death? Synthetic Data Could Lead To “Model Collapse”

by Theinsightpost
August 14, 2024
in Science
0 0
0
Could AI Eat Itself to Death? Synthetic Data Could Lead To “Model Collapse”

Generative AI’s reliance on extensive data has led to the use of synthetic data, which Rice University research shows can cause a feedback loop that degrades model quality over time. This process, called ‘Model Autophagy Disorder’, results in models that produce increasingly distorted outputs, highlighting the necessity for fresh data to maintain AI quality and diversity. Credit: SciTechDaily

Rice University’s findings reveal that repetitive synthetic data training can lead to ‘Model Autophagy Disorder’, deteriorating the quality of generative AI models. Continuous reliance on synthetic data without fresh inputs can doom future AI models to inefficiency and reduced diversity.

Generative Progressive Artifact Amplification

Generative artificial intelligence (AI) models trained on synthetic data generate outputs that are progressively marred by artifacts. In this example, the researchers trained a succession of StyleGAN-2 generative models using fully synthetic data. Each of the six image columns displays a couple of examples generated by the first, third, fifth, and ninth generation model, respectively. With each iteration of the loop, the cross-hatched artifacts become progressively amplified. Credit: Digital Signal Processing Group/Rice University

The Risks of Autophagous Training

“The problems arise when this synthetic data training is, inevitably, repeated, forming a kind of a feedback loop ⎯ what we call an autophagous or ‘self-consuming’ loop,” said Richard Baraniuk, Rice’s C. Sidney Burrus Professor of Electrical and Computer Engineering. “Our group has worked extensively on such feedback loops, and the bad news is that even after a few generations of such training, the new models can become irreparably corrupted. This has been termed ‘model collapse’ by some ⎯ most recently by colleagues in the field in the context of large language models (LLMs). We, however, find the term ‘Model Autophagy Disorder’ (MAD) more apt, by analogy to mad cow disease.”

Training Loops Schematic
Richard Baraniuk and his team at Rice University studied three variations of self-consuming training loops designed to provide a realistic representation of how real and synthetic data are combined into training datasets for generative models. Schematic illustrates the three training scenarios, i.e. a fully synthetic loop, a synthetic augmentation loop (synthetic + fixed set of real data), and a fresh data loop (synthetic + new set of real data). Credit: Digital Signal Processing Group/Rice University

Mad cow disease is a fatal neurodegenerative illness that affects cows and has a human equivalent caused by consuming infected meat. A major outbreak in the 1980-’90s brought attention to the fact that mad cow disease proliferated as a result of the practice of feeding cows the processed leftovers of their slaughtered peers ⎯ hence the term “autophagy,” from the Greek auto-, which means “self,”’ and phagy ⎯ “to eat.”

“We captured our findings on MADness in a paper presented in May at the International Conference on Learning Representations (ICLR),” Baraniuk said.

The study, titled “Self-Consuming Generative Models Go MAD,” is the first peer-reviewed work on AI autophagy and focuses on generative image models like the popular DALL·E 3, Midjourney, and Stable Diffusion.

Impact of Training Loops on AI Models

“We chose to work on visual AI models to better highlight the drawbacks of autophagous training, but the same mad cow corruption issues occur with LLMs, as other groups have pointed out,” Baraniuk said.

The internet is usually the source of generative AI models’ training datasets, so as synthetic data proliferates online, self-consuming loops are likely to emerge with each new generation of a model. To get insight into different scenarios of how this might play out, Baraniuk and his team studied three variations of self-consuming training loops designed to provide a realistic representation of how real and synthetic data are combined into training datasets for generative models:

  • fully synthetic loop ⎯ Successive generations of a generative model were fed a fully synthetic data diet sampled from prior generations’ output.
  • synthetic augmentation loop ⎯ The training dataset for each generation of the model included a combination of synthetic data sampled from prior generations and a fixed set of real training data.
  • fresh data loop ⎯ Each generation of the model is trained on a mix of synthetic data from prior generations and a fresh set of real training data.
AI Generated Dataset Without Sampling Bias
Progressive transformation of a dataset consisting of numerals 1 through 9 across 20 model iterations of a fully synthetic loop without sampling bias (top panel), and corresponding visual representation of data mode dynamics for real (red) and synthetic (green) data (bottom panel). In the absence of sampling bias, synthetic data modes separate from real data modes and merge. This translates into a rapid deterioration of model outputs: If all numerals are fully legible in generation 1 (leftmost column, top panel), by generation 20 all images have become illegible (rightmost column, top panel). Credit: Digital Signal Processing Group/Rice University

Progressive iterations of the loops revealed that, over time and in the absence of sufficient fresh real data, the models would generate increasingly warped outputs lacking either quality, diversity, or both. In other words, the more fresh data, the healthier the AI.

Consequences and Future of Generative AI

Side-by-side comparisons of image datasets resulting from successive generations of a model paint an eerie picture of potential AI futures. Datasets consisting of human faces become increasingly streaked with gridlike scars ⎯ what the authors call “generative artifacts” ⎯ or look more and more like the same person. Datasets consisting of numbers morph into indecipherable scribbles.

“Our theoretical and empirical analyses have enabled us to extrapolate what might happen as generative models become ubiquitous and train future models in self-consuming loops,” Baraniuk said. “Some ramifications are clear: without enough fresh real data, future generative models are doomed to MADness.”

AI Generated Dataset With Sampling Bias
Progressive transformation of a dataset consisting of numerals 1 through 9 across 20 model iterations of a fully synthetic loop with sampling bias (top panel), and corresponding visual representation of data mode dynamics for real (red) and synthetic (green) data (bottom panel). With sampling bias, synthetic data modes still separate from real data modes, but, rather than merging, they collapse around individual, high-quality images. This translates into a prolonged preservation of higher quality data across iterations: All but a couple of the numerals are still legible by generation 20 (rightmost column, top panel). While sampling bias preserves data quality longer, this comes at the expense of data diversity. Credit: Digital Signal Processing Group/Rice University

To make these simulations even more realistic, the researchers introduced a sampling bias parameter to account for “cherry picking” ⎯ the tendency of users to favor data quality over diversity, i.e. to trade off variety in the types of images and texts in a dataset for images or texts that look or sound good. The incentive for cherry-picking is that data quality is preserved over a greater number of model iterations, but this comes at the expense of an even steeper decline in diversity.

“One doomsday scenario is that if left uncontrolled for many generations, MAD could poison the data quality and diversity of the entire internet,” Baraniuk said. “Short of this, it seems inevitable that as-to-now-unseen unintended consequences will arise from AI autophagy even in the near term.”

AI Sampling With Bias
The incentive for cherry picking ⎯ the tendency of users to favor data quality over diversity ⎯ is that data quality is preserved over a greater number of model iterations, but this comes at the expense of an even steeper decline in diversity. Pictured are sample image outputs from a first, third, and fifth generation model of fully synthetic loop with sampling bias parameter. With each iteration, the dataset becomes increasingly homogeneous. Credit: Digital Signal Processing Group/Rice University

Reference: “Self-Consuming Generative Models Go MAD” by Sina Alemohammad, Josue Casco-Rodriguez, Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaei, Daniel LeJeune, Ali Siahkoohi and Richard Baraniuk, 8 May 2024, International Conference on Learning Representations (ICLR), 2024.

In addition to Baraniuk, study authors include Rice Ph.D. students Sina Alemohammad; Josue Casco-Rodriguez; Ahmed Imtiaz Humayun; Hossein Babaei; Rice Ph.D. alumnus Lorenzo Luzi; Rice Ph.D. alumnus and current Stanford postdoctoral student Daniel LeJeune; and Simons Postdoctoral Fellow Ali Siahkoohi.

The research was supported by the National Science Foundation, the Office of Naval Research, the Air Force Office of Scientific Research, and the Department of Energy.

ShareTweetSend
Previous Post

Rassie Erasmus puts Springboks’ No 1 world ranking on the line

Next Post

Russia’s Belgorod declares state of emergency after Ukraine shelling

Related News

Groundbreaking Electric Comb Promises Hair Regrowth in Mice Within Four Weeks, Paving the Way for Future Hair-Loss Solutions
Science

Groundbreaking Electric Comb Promises Hair Regrowth in Mice Within Four Weeks, Paving the Way for Future Hair-Loss Solutions

November 18, 2026
Miniature LSD Doses Administered in Rapid Succession Show Antidepressant Potential in Mice: Exploring the Impact of Timing vs. Tolerance
Science

Miniature LSD Doses Administered in Rapid Succession Show Antidepressant Potential in Mice: Exploring the Impact of Timing vs. Tolerance

November 14, 2026
Investigation Underway: Experts Seek Answers Behind Sudden Brooklyn Building Collapse
Science

Investigation Underway: Experts Seek Answers Behind Sudden Brooklyn Building Collapse

November 12, 2026
Preserve the Unseen: The Essential Role of Parasites in Ecosystems
Science

Preserve the Unseen: The Essential Role of Parasites in Ecosystems

November 11, 2026
Next Post
Russia’s Belgorod declares state of emergency after Ukraine shelling

Russia's Belgorod declares state of emergency after Ukraine shelling

Discussion about this post

Subscribe To Our Newsletters

    Customer Support


    1251 Wilcrest Drive
    Houston, Texas
    77042 USA
    Call-832.795.1420
    e-mail – news@theinsightpost.com

    Subscribe To Our Newsletters

      Categories

      • Africa
      • Africa-East
      • African Sports
      • American Sports
      • Arts
      • Asia
      • Australia
      • Business
      • Business Asia
      • Business- Africa
      • Canada
      • Defense
      • Education
      • Egypt
      • Energy
      • Entertainment
      • Europe
      • European Soccer
      • Finance
      • Germany
      • Ghana
      • Health
      • Insight
      • International
      • Investing
      • Japan
      • Latest Headlines
      • Life & Living
      • Markets
      • Mobile
      • Movies
      • New Zealand
      • Nigeria
      • Politics
      • Scholarships
      • Science
      • South Africa
      • South America
      • Sports
      • Tech
      • Travel
      • Travel-Africa
      • UK
      • USA
      • Weather
      • World
      No Result
      View All Result

      Recent News

      Expert Challenges Obama-Era Housing First Initiative Following LAHSA Budget Reductions

      Expert Challenges Obama-Era Housing First Initiative Following LAHSA Budget Reductions

      November 18, 2026
      Alphabet’s Valuation Estimated at 1 Through Discounted Cash Flow Analysis

      Alphabet’s Valuation Estimated at $291 Through Discounted Cash Flow Analysis

      November 18, 2026
      Crimea Bridge Targeted in Renewed Assault, Reports Russian Governor

      Crimea Bridge Targeted in Renewed Assault, Reports Russian Governor

      November 18, 2026
      FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

      FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

      November 18, 2026
      • Home
      • Advertise With Us
      • About Us
      • Corporate
      • Consumer Rewards
      • Forum
      • Privacy Policy
      • Social Trends

      Theinsightpost ©2026 | All Rights Reserved. Theinsightpost is an Elnegy LLC company, registered in Texas, USA

      Welcome Back!

      Login to your account below

      Forgotten Password?

      Retrieve your password

      Please enter your username or email address to reset your password.

      Log In

      Add New Playlist

      We are using cookies to give you the best experience on our website.

      You can find out more about which cookies we are using or switch them off in .

      No Result
      View All Result
      • Home
      • Insight
      • Blog
      • Business
      • Entertainment
      • Health
      • Politics
      • Shop
        • Gift Shop
        • Value Shop
        • Store
        • Bargain Shop
        • Discount
      • Sports
      • Tech
      • Travel
      • USA
      • Video
      • World
        • Asia
        • Africa
        • South America
        • North America
        • Europe
        • Oceania

      Theinsightpost ©2026 | All Rights Reserved. Theinsightpost is an Elnegy LLC company, registered in Texas, USA

      The Insight Post
      Powered by  GDPR Cookie Compliance
      Privacy Overview

      This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.

      Strictly Necessary Cookies

      Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.

      Cookie Policy

      More information about our Cookie Policy