Choosing Wisely: Lessons for Leaders in AI Integration

The path to their collective goal is equally succinct: continuous research, customer feedback and growth.

And grow, they have. Since we spoke with Malkin when he became CEO seven years ago, Carnegie has added 500 new employees, 500 part-time tutors, four new adaptive AI products, countless research projects and a new Canadian headquarters.

Today, after seven years of sitting in on classrooms, talking with school leaders, a pandemic and the nationwide leap into AI-mania, Malkin has unique insights to share about how schools can choose AI tools wisely, how Carnegie fits into the AI landscape and how AI can be a source of positive reinforcement for learners.

EdSurge: What makes Carnegie Learning’s approach to AI different from other edtech offerings?

Malkin: Our origin story is one of an AI-driven product initially launched by Carnegie Mellon University through Carnegie Learning. Twenty-five years ago, Carnegie Mellon University created the first adaptive AI-driven tutor for teaching middle school mathematics, MATHia, which is still one of our flagship products.

Carnegie Learning was way ahead of its time with that early version of AI. Of course, the technology wasn’t as advanced as today, but it was still an artificial intelligence-driven, adaptive learning-driven product. When the generative AI revolution hit, we were well-positioned to step up as a leader, take that AI knowledge and apply it in ways that directly support students and teachers.

We have the people who understand the technology behind AI. We have the researchers who have studied it for 25 years. AI was already part of Carnegie Learning’s DNA, and now we’re moving fast and furious to integrate it into products in new and impactful ways.

Drawing on such deep experience, what do you think school leaders and teachers can look at when choosing effective AI tools?

Many technology solutions already exist in education, but only a limited few demonstrate actual value, and AI is no different. Anybody can build a technology product centered around education, but not everybody can create a technology product influenced by learning science and research that truly makes a difference.

Everyone has access to a large language model (LLM) that will allow you to build an education application that will ostensibly look the same across many companies in the sector. Only those companies that understand cognitive models and learning science and have data to add something substantive to that LLM will provide a meaningfully differentiated product.

At Carnegie Learning, for example, we’re thoughtful and purposeful about integrating generative AI into our products as an enhancement to the work that we already do, and that’s critical.

We are integrating it into our curriculum. It’s not yet another tool that educators and students access. It’s a tool built into the Carnegie Learning ecosystem that gives educators and students one more arrow in their quiver to help solve a challenge or motivate them to learn more.

Schools should consider those things as they adopt AI tools. What is the tool’s actual value to students and teachers? Is it thoughtful, purposeful and — most critical — based on research?

Research is obviously vital at Carnegie. What does that look like in practice?

We have a large research team that constantly researches the efficacy of our products and undertakes a system of continuous improvement to make our solutions better all the time.

Their ethos is relentless questioning and exploring.

Our research team constantly challenges itself: Let’s find out if there is bias in this math problem, test it in different communities and find out which language resonates positively to improve outcomes. How can we better understand student misconceptions? Are there patterns in the types of mistakes students are making? This is critical.

They are never satisfied with the status quo and are very intentional about introducing enhancements to products and content that will make a difference.

Carnegie’s research team is also involved in large-scale studies like the Gold Standard RAND Study funded by the U.S. Department of Education and smaller studies with districts and schools. Collecting data and independent validation, and I stress independent valuation, which is unique, is essential to the credibility of our products. All of that informs our processes to make our products more efficacious. That’s a key part of our goal.

How do you see AI supporting that goal of giving all students access to equitable, personalized learning?

If we use this technology in the right way, it can give students more of what they wish for (what I remember wishing for as a student): more engagement, empowerment and context. Students today deserve that.

While having students understand theory is great, they must also understand the practical applications of their learning. “What can I do with this knowledge? How can I take it beyond the classroom?”

We spend a lot of time sitting in classrooms and observing; there’s no reason curriculum and technology can’t be inspirational for all students. Technology like adaptive AI can help in a major way through personalization. And the more we can do that, the more we will succeed in igniting the passion inside every learner.

I struggled as a middle school math student, and I remember sitting in the classroom as the class moved forward while I was still wrangling with concepts that prevented me from going farther faster. It’s a tough place to be and an awful feeling. I’m motivated by helping all students, but especially students who need additional support. Carnegie Learning’s products, especially our AI-driven solutions, are well positioned to help students be on grade level all the time.

If we can help students achieve greater and greater results, we’ll have accomplished something amazing. That’s really what we’re doing here.