• Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
Thursday, November 19, 2026
No Result
View All Result
Subscribe Now
  • Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
    Trump’s Deal Turns Sour as Voters Withdraw Support

    Trump’s Deal Turns Sour as Voters Withdraw Support

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Trump Administration Revokes Pay Increase for Head Start Educators

    Trump Administration Revokes Pay Increase for Head Start Educators

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
The Insight Post
  • Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
    Trump’s Deal Turns Sour as Voters Withdraw Support

    Trump’s Deal Turns Sour as Voters Withdraw Support

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Trump Administration Revokes Pay Increase for Head Start Educators

    Trump Administration Revokes Pay Increase for Head Start Educators

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

    LAUSD Takes Strong Action Against Hate Speech Following Racial Slur Incidents

  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
No Result
View All Result
No Result
View All Result
Home Tech

The problems with Galactica AI

by Theinsightpost
December 3, 2022
in Tech
0 0
0
The problems with Galactica AI


The Galactica AI model was trained on scientific knowledge, but it spat out alarmingly plausible nonsense. Aaron J Snoswell and Prof Jean Burgess from the Queensland University of Technology discuss the issues raised by this authoritative, but subtly wrong bullshit generator.

Earlier this month, Meta announced new AI software called Galactica, “a large language model that can store, combine and reason about scientific knowledge”.

Launched with a public online demo, Galactica lasted only three days before going the way of other AI snafus such as Microsoft’s infamous racist chatbot.

The online demo was disabled (though the code for the model is still available for anyone to use), and Meta’s outspoken chief AI scientist, Yann LeCun, complained about the negative public response.

So what was Galactica all about, and what went wrong?

‘Not only did Galactica reproduce many of the problems of bias and toxicity we have seen in other language models, it also specialised in producing authoritative-sounding scientific nonsense’

Galactica is a language model, a type of AI trained to respond to natural language by repeatedly playing a fill-the-blank word-guessing game.

Most modern language models learn from text scraped from the internet. Galactica also used text from scientific papers uploaded to the (Meta-affiliated) website PapersWithCode. The designers highlighted specialised scientific information such as citations, maths, code, chemical structures, and the working-out steps for solving scientific problems.

The preprint paper associated with the project (which is yet to undergo peer review) makes some impressive claims. Galactica apparently outperforms other models at problems such as reciting famous equations (“Q: What is Albert Einstein’s famous mass-energy equivalence formula? A: E=mc²”), or predicting the products of chemical reactions (“Q: When sulfuric acid reacts with sodium chloride, what does it produce? A: NaHSO₄ + HCl”).

However, once Galactica was opened up for public experimentation, a deluge of criticism followed. Not only did Galactica reproduce many of the problems of bias and toxicity we have seen in other language models, it also specialised in producing authoritative-sounding scientific nonsense.

Misinformation masquerading as science

Galactica’s press release promoted its ability to explain technical scientific papers using general language. However, users quickly noticed that, while the explanations it generates sound authoritative, they are often subtly incorrect, biased, or just plain wrong.

We asked Galactica to explain technical concepts from our own fields of research and found it would use all the right buzzwords, but get the actual details wrong. For example, it would mix up the details of related but different algorithms.

In practice, Galactica was enabling the generation of misinformation and this is dangerous precisely because it deploys the tone and structure of authoritative scientific information. If a user already needs to be a subject matter expert in order to check the accuracy of Galactica’s ‘summaries’, then it has no use as an explanatory tool.

At best, it could provide a fancy autocomplete for people who are already fully competent in the area they’re writing about. At worst, it risks further eroding public trust in scientific research.

Galactica could make it easier for bad actors to mass-produce fake, fraudulent or plagiarised scientific papers. This is to say nothing of exacerbating existing concerns about students using AI systems for plagiarism.

Fake scientific papers are nothing new. However, peer reviewers at academic journals and conferences are already time-poor, and this could make it harder than ever to weed out fake science.

Underlying bias and toxicity

Other critics reported that Galactica, like other language models trained on data from the internet, has a tendency to spit out toxic hate speech while unreflectively censoring politically inflected queries. This reflects the biases lurking in the model’s training data, and Meta’s apparent failure to apply appropriate checks around the responsible AI research.

The risks associated with large language models are well understood. Indeed, an influential paper highlighting these risks prompted Google to fire one of the paper’s authors in 2020, and eventually disband its AI ethics team altogether.

Machine-learning systems infamously exacerbate existing societal biases, and Galactica is no exception. For instance, Galactica can recommend possible citations for scientific concepts by mimicking existing citation patterns (“Q: Is there any research on the effect of climate change on the great barrier reef? A: Try the paper ‘Global warming transforms coral reef assemblages’ by Hughes, et al. in Nature 556 (2018)”).

For better or worse, citations are the currency of science. By reproducing existing citation trends in its recommendations, Galactica risks reinforcing existing patterns of inequality and disadvantage. (Galactica’s developers acknowledge this risk in their paper.)

Citation bias is already a well-known issue in academic fields ranging from feminist scholarship to physics. However, tools like Galactica could make the problem worse unless they are used with careful guardrails in place.

A more subtle problem is that the scientific articles on which Galactica is trained are already biased towards certainty and positive results. (This leads to the so-called ‘replication crisis’ and ‘p-hacking’, where scientists cherry-pick data and analysis techniques to make results appear significant.)

Galactica takes this bias towards certainty, combines it with wrong answers and delivers responses with supreme overconfidence. Hardly a recipe for trustworthiness in a scientific information service.

These problems are dramatically heightened when Galactica tries to deal with contentious or harmful social issues. Galactica readily generates toxic and nonsensical content dressed up in the measured and authoritative language of science, as AI-focused journalist Tristan Greene found out from his interactions with the model.

you, sir, apparently have no clue what’s in the Galactica dataset, because I sure didn’t write these outputs: pic.twitter.com/31ccTz7m9V

— Tristan Greene ?‍? (@mrgreene1977) November 18, 2022

Calls for AI research organisations to take the ethical dimensions of their work more seriously are now coming from key research bodies such as the National Academies of Science, Engineering and Medicine. Some AI research organisations, such as OpenAI, are being more conscientious (though still imperfect).

Meta dissolved its Responsible Innovation team earlier this year. The team was tasked with addressing “potential harms to society” caused by the company’s products. They might have helped the company avoid this clumsy misstep.

By Aaron J Snoswell and Prof Jean Burgess

Aaron J Snoswell is a robotics engineer and postdoctoral research fellow at the Queensland University of Technology (QUT) working with humanities scholars on how to regulate and litigate the spaces where AI and society overlap. Prof Jean Burgess is associate director of the ARC Centre of Excellence for Automated Decision-Making and Society at QUT where Snoswell’s research is based. She is also a chief investigator in the QUT Digital Media Research Centre.

10 things you need to know direct to your inbox every weekday. Sign up for the Daily Brief, Silicon Republic’s digest of essential sci-tech news.





Source link

ShareTweetSend
Previous Post

Coyote attacks toddler on front lawn of Woodland Hills home

Next Post

India Tour of Bangladesh 2022: Litton Das to lead Bangladesh in ODI series against India

Related News

Exploring Innovations: A Breakthrough in Living Liver Technology, a Lucrative Battery Partnership, and the Surge of AI Excitement
Tech

Exploring Innovations: A Breakthrough in Living Liver Technology, a Lucrative Battery Partnership, and the Surge of AI Excitement

November 19, 2026
Microsoft’s Satya Nadella Reveals Strategic Rationale Behind Layoff of 60,000 Employees
Tech

Microsoft’s Satya Nadella Reveals Strategic Rationale Behind Layoff of 60,000 Employees

November 19, 2026
Cybercriminals Embrace Innovative AI Tool for Streamlined Hacking Operations
Tech

Cybercriminals Embrace Innovative AI Tool for Streamlined Hacking Operations

November 18, 2026
Transform Your Laptop Setup with These 5 Affordable Accessories
Tech

Transform Your Laptop Setup with These 5 Affordable Accessories

November 15, 2026
Next Post
India Tour of Bangladesh 2022: Litton Das to lead Bangladesh in ODI series against India

India Tour of Bangladesh 2022: Litton Das to lead Bangladesh in ODI series against India

Discussion about this post

Subscribe To Our Newsletters

    Customer Support


    1251 Wilcrest Drive
    Houston, Texas
    77042 USA
    Call-832.795.1420
    e-mail – news@theinsightpost.com

    Subscribe To Our Newsletters

      Categories

      • Africa
      • Africa-East
      • African Sports
      • American Sports
      • Arts
      • Asia
      • Australia
      • Business
      • Business Asia
      • Business- Africa
      • Canada
      • Defense
      • Education
      • Egypt
      • Energy
      • Entertainment
      • Europe
      • European Soccer
      • Finance
      • Germany
      • Ghana
      • Health
      • Insight
      • International
      • Investing
      • Japan
      • Latest Headlines
      • Life & Living
      • Markets
      • Mobile
      • Movies
      • New Zealand
      • Nigeria
      • Politics
      • Scholarships
      • Science
      • South Africa
      • South America
      • Sports
      • Tech
      • Travel
      • Travel-Africa
      • UK
      • USA
      • Weather
      • World
      No Result
      View All Result

      Recent News

      Putin Promises Consequences Following Ukraine’s Assault on Kerch Strait Bridge

      Putin Promises Consequences Following Ukraine’s Assault on Kerch Strait Bridge

      November 19, 2026
      Brian Blessed Shares Amusing Encounter with Female Gorillas at Zoo

      Brian Blessed Shares Amusing Encounter with Female Gorillas at Zoo

      November 19, 2026
      Unveiling the Real Narrative: ‘The Altruists’ and Their Connection to the FTX Controversy

      Unveiling the Real Narrative: ‘The Altruists’ and Their Connection to the FTX Controversy

      November 19, 2026
      Is Hakeem Jeffries Facing a Moment of Crisis?

      Is Hakeem Jeffries Facing a Moment of Crisis?

      November 19, 2026
      • Home
      • Advertise With Us
      • About Us
      • Corporate
      • Consumer Rewards
      • Forum
      • Privacy Policy
      • Social Trends

      Theinsightpost ©2026 | All Rights Reserved. Theinsightpost is an Elnegy LLC company, registered in Texas, USA

      Welcome Back!

      Login to your account below

      Forgotten Password?

      Retrieve your password

      Please enter your username or email address to reset your password.

      Log In

      Add New Playlist

      We are using cookies to give you the best experience on our website.

      You can find out more about which cookies we are using or switch them off in .

      No Result
      View All Result
      • Home
      • Insight
      • Blog
      • Business
      • Entertainment
      • Health
      • Politics
      • Shop
        • Gift Shop
        • Value Shop
        • Store
        • Bargain Shop
        • Discount
      • Sports
      • Tech
      • Travel
      • USA
      • Video
      • World
        • Asia
        • Africa
        • South America
        • North America
        • Europe
        • Oceania

      Theinsightpost ©2026 | All Rights Reserved. Theinsightpost is an Elnegy LLC company, registered in Texas, USA

      The Insight Post
      Powered by  GDPR Cookie Compliance
      Privacy Overview

      This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.

      Strictly Necessary Cookies

      Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.

      Cookie Policy

      More information about our Cookie Policy