• Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
Friday, November 20, 2026
No Result
View All Result
Subscribe Now
  • Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
    Texas Execution Prolonged As Staff Encounter Difficulties Finding Vein, Inmate Reinaldo Dennes Reports Pain

    Texas Execution Prolonged As Staff Encounter Difficulties Finding Vein, Inmate Reinaldo Dennes Reports Pain

    Trump’s Deal Turns Sour as Voters Withdraw Support

    Trump’s Deal Turns Sour as Voters Withdraw Support

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Trump Administration Revokes Pay Increase for Head Start Educators

    Trump Administration Revokes Pay Increase for Head Start Educators

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
The Insight Post
  • Home
  • Insight
  • Blog
  • Business
  • Entertainment
  • Health
  • Politics
  • Shop
    • Gift Shop
    • Value Shop
    • Store
    • Bargain Shop
    • Discount
  • Sports
  • Tech
  • Travel
  • USA
    Texas Execution Prolonged As Staff Encounter Difficulties Finding Vein, Inmate Reinaldo Dennes Reports Pain

    Texas Execution Prolonged As Staff Encounter Difficulties Finding Vein, Inmate Reinaldo Dennes Reports Pain

    Trump’s Deal Turns Sour as Voters Withdraw Support

    Trump’s Deal Turns Sour as Voters Withdraw Support

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    FAA Administrator Takes Responsibility as Mullin Faces Tough Questions Over Nationwide Ground Stop

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Mary Peltola Emerges Victorious in Alaska Senate Race, Making History for Democrats

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Stranded in Conflict: Palestinians Navigate the Tenuous Balance of War and Peace Amid Ceasefire Efforts

    Trump Administration Revokes Pay Increase for Head Start Educators

    Trump Administration Revokes Pay Increase for Head Start Educators

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Unexpected Triumph: Gay Candidates Achieve Unforeseen Victories in Challenging Races

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Microsoft Outlook Experiences Prolonged Outage Affecting Users Nationwide

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

    Gunfire Erupts in D.C.: Authorities Search for Suspect Amid Rising Concerns Over Public Safety

  • Video
  • World
    • Asia
    • Africa
    • South America
    • North America
    • Europe
    • Oceania
No Result
View All Result
No Result
View All Result
Home Tech

You can now run a GPT-3 level AI model on your laptop, phone, and Raspberry Pi

by Theinsightpost
March 14, 2023
in Tech
0 0
0
You can now run a GPT-3 level AI model on your laptop, phone, and Raspberry Pi


Ars Technica

Things are moving at lightning speed in AI Land. On Friday, a software developer named Georgi Gerganov created a tool called “llama.cpp” that can run Meta’s new GPT-3-class AI large language model, LLaMA, locally on a Mac laptop. Soon thereafter, people worked out how to run LLaMA on Windows as well. Then someone showed it running on a Pixel 6 phone, and next came a Raspberry Pi (albeit running very slowly).

If this keeps up, we may be looking at a pocket-sized ChatGPT competitor before we know it.

But let’s back up a minute, because we’re not quite there yet. (At least not today—as in literally today, March 13, 2023.) But what will arrive next week, no one knows.

Since ChatGPT launched, some people have been frustrated by the AI model’s built-in limits that prevent it from discussing topics that OpenAI has deemed sensitive. Thus began the dream—in some quarters—of an open source large language model (LLM) that anyone could run locally without censorship and without paying API fees to OpenAI.

Open source solutions do exist (such as GPT-J), but they require a lot of GPU RAM and storage space. Other open source alternatives could not boast GPT-3-level performance on readily available consumer-level hardware.

Enter LLaMA, an LLM available in parameter sizes ranging from 7B to 65B (that’s “B” as in “billion parameters,” which are floating point numbers stored in matrices that represent what the model “knows”). LLaMA made a heady claim: that its smaller-sized models could match OpenAI’s GPT-3, the foundational model that powers ChatGPT, in the quality and speed of its output. There was just one problem—Meta released the LLaMA code open source, but it held back the “weights” (the trained “knowledge” stored in a neural network) for qualified researchers only.

Advertisement

Flying at the speed of LLaMA

Meta’s restrictions on LLaMA didn’t last long, because on March 2, someone leaked the LLaMA weights on BitTorrent. Since then, there’s been an explosion of development surrounding LLaMA. Independent AI researcher Simon Willison has compared this situation to the release of Stable Diffusion, an open source image synthesis model that launched last August. Here’s what he wrote in a post on his blog:

It feels to me like that Stable Diffusion moment back in August kick-started the entire new wave of interest in generative AI—which was then pushed into over-drive by the release of ChatGPT at the end of November.

That Stable Diffusion moment is happening again right now, for large language models—the technology behind ChatGPT itself. This morning I ran a GPT-3 class language model on my own personal laptop for the first time!

AI stuff was weird already. It’s about to get a whole lot weirder.

Typically, running GPT-3 requires several datacenter-class A100 GPUs (also, the weights for GPT-3 are not public), but LLaMA made waves because it could run on a single beefy consumer GPU. And now, with optimizations that reduce the model size using a technique called quantization, LLaMA can run on an M1 Mac or a lesser Nvidia consumer GPU.

Things are moving so quickly that it’s sometimes difficult to keep up with the latest developments. (Regarding AI’s rate of progress, a fellow AI reporter told Ars, “It’s like those videos of dogs where you upend a crate of tennis balls on them. [They] don’t know where to chase first and get lost in the confusion.”)

For example, here’s a list of notable LLaMA-related events based on a timeline Willison laid out in a Hacker News comment:

  • February 24, 2023: Meta AI announces LLaMA.
  • March 2, 2023: Someone leaks the LLaMA models via BitTorrent.
  • March 10, 2023: Georgi Gerganov creates llama.cpp, which can run on an M1 Mac.
  • March 11, 2023: Artem Andreenko runs LLaMA 7B (slowly) on a Raspberry Pi 4, 4GB RAM, 10 sec/token.
  • March 12, 2023: LLaMA 7B running on NPX, a node.js execution tool.
  • March 13, 2023: Someone gets llama.cpp running on a Pixel 6 phone, also very slowly.
  • March 13, 2023, 2023: Stanford releases Alpaca 7B, an instruction-tuned version of LLaMA 7B that “behaves similarly to OpenAI’s “text-davinci-003” but runs on much less powerful hardware.
Advertisement

After obtaining the LLaMA weights ourselves, we followed Willison’s instructions and got the 7B parameter version running on an M1 Macbook Air, and it runs at a reasonable rate of speed. You call it as a script on the command line with a prompt, and LLaMA does its best to complete it in a reasonable way.

A screenshot of LLaMA 7B in action on a MacBook Air running llama.cpp.
Enlarge / A screenshot of LLaMA 7B in action on a MacBook Air running llama.cpp.

Benj Edwards / Ars Technica

There’s still the question of how much the quantization affects the quality of the output. In our tests, LLaMA 7B trimmed down to 4-bit quantization was very impressive for running on a MacBook Air—but still not on par with what you might expect from ChatGPT. It’s entirely possible that better prompting techniques might generate better results.

Also, optimizations and fine-tunings come quickly when everyone has their hands on the code and the weights—even though LLaMA is still saddled with some fairly restrictive terms of use. The release of Alpaca today by Stanford proves that fine tuning (additional training with a specific goal in mind) can improve performance, and it’s still early days after LLaMA’s release.

As of this writing, running LLaMA on a Mac remains a fairly technical exercise. You have to install Python and Xcode and be familiar with working on the command line. Willison has good step-by-step instructions for anyone who would like to attempt it. But that may soon change as developers continue to code away.

As for the implications of having this tech out in the wild—no one knows yet. While some worry about AI’s impact as a tool for spam and misinformation, Willison says, “It’s not going to be un-invented, so I think our priority should be figuring out the most constructive possible ways to use it.”

Right now, our only guarantee is that things will change rapidly.





Source link

ShareTweetSend
Previous Post

China’s Xi to meet Putin in Moscow, speak to Zelenskyy: reports – POLITICO

Next Post

Chelsea appoint Nigerian manager as new Head Coach

Related News

Tech Turmoil: Bitcoin and AI Stocks Tumble Amid Intensifying Market Selloff
Tech

Tech Turmoil: Bitcoin and AI Stocks Tumble Amid Intensifying Market Selloff

November 20, 2026
Uber Launches Pilot Program for Safe Children’s Rides with Trusted Drivers
Tech

Uber Launches Pilot Program for Safe Children’s Rides with Trusted Drivers

November 20, 2026
Exploring Innovations: A Breakthrough in Living Liver Technology, a Lucrative Battery Partnership, and the Surge of AI Excitement
Tech

Exploring Innovations: A Breakthrough in Living Liver Technology, a Lucrative Battery Partnership, and the Surge of AI Excitement

November 19, 2026
Microsoft’s Satya Nadella Reveals Strategic Rationale Behind Layoff of 60,000 Employees
Tech

Microsoft’s Satya Nadella Reveals Strategic Rationale Behind Layoff of 60,000 Employees

November 19, 2026
Next Post
Chelsea appoint Nigerian manager as new Head Coach

Chelsea appoint Nigerian manager as new Head Coach

Discussion about this post

Subscribe To Our Newsletters

    Customer Support


    1251 Wilcrest Drive
    Houston, Texas
    77042 USA
    Call-832.795.1420
    e-mail – news@theinsightpost.com

    Subscribe To Our Newsletters

      Categories

      • Africa
      • Africa-East
      • African Sports
      • American Sports
      • Arts
      • Asia
      • Australia
      • Business
      • Business Asia
      • Business- Africa
      • Canada
      • Defense
      • Education
      • Egypt
      • Energy
      • Entertainment
      • Europe
      • European Soccer
      • Finance
      • Germany
      • Ghana
      • Health
      • Insight
      • International
      • Investing
      • Japan
      • Latest Headlines
      • Life & Living
      • Markets
      • Mobile
      • Movies
      • New Zealand
      • Nigeria
      • Politics
      • Scholarships
      • Science
      • South Africa
      • South America
      • Sports
      • Tech
      • Travel
      • Travel-Africa
      • UK
      • USA
      • Weather
      • World
      No Result
      View All Result

      Recent News

      Ablakwa Appointed Chair of Nordic-African Foreign Ministers for 2027

      Ablakwa Appointed Chair of Nordic-African Foreign Ministers for 2027

      November 20, 2026
      Rams’ Star Receiver Puka Nacua to Miss Remainder of Season Following Knee Surgery, Coach Sean McVay Confirms

      Rams’ Star Receiver Puka Nacua to Miss Remainder of Season Following Knee Surgery, Coach Sean McVay Confirms

      November 20, 2026
      Tech Turmoil: Bitcoin and AI Stocks Tumble Amid Intensifying Market Selloff

      Tech Turmoil: Bitcoin and AI Stocks Tumble Amid Intensifying Market Selloff

      November 20, 2026
      Clash in Gqeberha: Ngobese-Zuma Pledges to Challenge Law Enforcement Following Arrests

      Clash in Gqeberha: Ngobese-Zuma Pledges to Challenge Law Enforcement Following Arrests

      November 20, 2026
      • Home
      • Advertise With Us
      • About Us
      • Corporate
      • Consumer Rewards
      • Forum
      • Privacy Policy
      • Social Trends

      Theinsightpost ©2026 | All Rights Reserved. Theinsightpost is an Elnegy LLC company, registered in Texas, USA

      Welcome Back!

      Login to your account below

      Forgotten Password?

      Retrieve your password

      Please enter your username or email address to reset your password.

      Log In

      Add New Playlist

      We are using cookies to give you the best experience on our website.

      You can find out more about which cookies we are using or switch them off in .

      No Result
      View All Result
      • Home
      • Insight
      • Blog
      • Business
      • Entertainment
      • Health
      • Politics
      • Shop
        • Gift Shop
        • Value Shop
        • Store
        • Bargain Shop
        • Discount
      • Sports
      • Tech
      • Travel
      • USA
      • Video
      • World
        • Asia
        • Africa
        • South America
        • North America
        • Europe
        • Oceania

      Theinsightpost ©2026 | All Rights Reserved. Theinsightpost is an Elnegy LLC company, registered in Texas, USA

      The Insight Post
      Powered by  GDPR Cookie Compliance
      Privacy Overview

      This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.

      Strictly Necessary Cookies

      Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.

      Cookie Policy

      More information about our Cookie Policy