Your next Windows computer upgrade will most likely have a new type of processor. It is called an NPU and is the most important requirement for Microsoft’s next generation of Windows computers—Copilot+ PCs.
While the new Snapdragon processors and Microsoft’s requirements brought NPUs into the spotlight, they’ve been around longer than you think.
However, these new processors mean you no longer have to rely on cloud servers to run generative processes.
So what are NPUs? How do they work? What are their benefits and drawbacks? I’ll explain everything in this article.
What Is an NPU?
NPUs (short for neural processing units) are dedicated chips that execute complex AI operations. They process multiple data points in parallel and simulate brain neurons. This computing method is the ideal way to run neural networks and tackle functions like natural language processing, image recognition, and generative tasks.
hey are faster than GPUs and CPUs for these and more AI use cases as a result.
As mentioned, NPUs are not as new as their current hype. Before the Microsoft Build event, they’ve resided in CPUs, GPUs, smartphones, and servers. They’ve also gone by different names, such as tensor cores, neural engines, and DSPs (digital signal processors).
For example:
- Intel and AMD built NPUs into their flagship Meteor Lake and Phoenix CPUs.
- NVIDIA RTX series GPUs have built-in NPUs (tensor cores) for AI features like DLSS 3 and ray tracing.
- Qualcomm has used its hexagon NPU since its Snapdragon 855 smartphone chipset in 2018.
- Apple introduced its first neutral engine in the A11 Bionic in 2017.
How NPUs have evolved
NPUs have also evolved from what they used to be. This evolution was primarily driven by the ever-expanding list of AI use cases, especially generative AI.
Earlier versions focused on less complicated object identification use cases, which relied on simpler neural networks known as CNNs (convolutional neural networks). But AI video and photography increased the stakes. NPUs that could handle more advanced neural network models were developed as a result.
Generative AI, which involves using AI to generate new content, brought even higher power and performance demands.
Creating new text, image, video, and audio output requires more memory and data transfer, which could lay waste to limited bandwidth and spike energy consumption. Today’s NPUs solve these problems for smaller devices as they are integrated into large systems on chips (SoC).
What are SoCs, and why are they important?
What most people call CPUs today are systems on chips or systems-on-a-chip. They are integrated circuits that contain different processing units, such as CPUs, GPUs, and TPMs. This chip design significantly saves space, improves power efficiency, and boosts performance.
The CPU has to hand off tasks to the GPU, TPU, or NPU when users initiate specific tasks. If these processors are separate, the CPU must transfer data over relatively long distances on the same board, which would consume more bandwidth and drive up energy usage. It means laptops, tablets, and phones will easily overheat and slow down because they lack space.
Also read: Ultimate Guide: How to Stop a Laptop from Overheating
That is why most gaming laptops with separate GPUs consume more power and run loud fans to dissipate excessive heat.
Integrating these processors on a single chip resolves these issues and allows smaller form factors to perform faster and consume less energy.
NPUs vs. Traditional Processors
We’ve established that NPUs are better at handling complex AI tasks than other types of processors. But that doesn’t mean they can replace them. CPUs and GPUs still have important computing roles to play. With the emergence of AI, NPUs should be seen as complementary processors.
NPUs ensure AI features will not bog down your GPU and CPU. They will allow them to handle regular and demanding functions.
On the flip side, NPUs are not versatile and cannot handle most of the processes CPUs and GPUs are designed to execute.
-
NPUs vs. CPUs
CPUs (central processing units) are commonly considered every computer’s brain. They are general-purpose processors that deal with data manipulation and transfer, logical operations, input and output tasks, some graphics processing, and emulation. Heavy tasks like simulation and development rely mostly on CPUs. They also determine whether to send tasks to the GPU or NPU.
While they can do most things, it’s better to outsource specialized tasks to other processors. They are especially not optimized for parallel processing like NPUs. You’ll notice significant slowdowns if you force machine learning and AI operations to run on the CPU.
-
NPUs vs. GPUs
GPUs are designed to handle demanding graphical computations. They’re more valuable to gamers, video editors, and 3D modelers. That said, NVIDIA has made the argument that GPUs are great for AI tasks because they have numerous integrated cores and excel at parallel processing.
But their AI capabilities are mostly targeted at graphics-related tasks and features, such as DLSS and live streaming HDR. These tasks also mostly run on AI-optimized cores known as tensor cores.
Dedicated NPUs do a better job at other use cases, such as system-wide information processing and other generative applications.
-
Windows and NPUs
Microsoft’s Copilot+ PC is mostly responsible for the current spotlight on NPUs. The company is boldly betting on AI to revolutionize Windows and transform how people interact with their computers.
Its new Copilot+ PCs offer exciting new integrated and generative AI capabilities powered by dedicated NPUs. That way, GPUs and CPUs are not burdened by these new classes of features.
Copilot+ PC is Microsoft’s new class of Windows computers. Devices with the Copilot+ brand ship with AI features that will not run on other computers. They include:
- Cocreator in Paint, which helps users refine their sketches,
- Restyle Image for advanced image editing,
- Live Captions for real-time audio transcription and translation,
- Recall, which helps people retrieve forgotten information in seconds, and
- Windows Studio Effects for enhancing the video calling experience.
Do you need a Copilot+ PC? Our rundown on these next generation computers will tell you all you need to know.
We’ve seen teasers in the latest Windows 11 build, such as the preview version of the Copilot app and Image Creator in Paint.
However, Microsoft requires NPUs that can pull off 40 trillion operations per second (TOPS) to run the main Copilot+ AI features. Currently, only the new Snapdragon X processors can manage that capability. The new chipsets are behind Microsoft’s claim that Copilot+ PCs are the “fastest and most intelligent” Windows devices ever.
We analyze Microsoft’s declaration in detail and provide more insight in our comparison between Copilot+ PCs and traditional Windows devices.
The new PCs have also had their share of PR disasters due to the controversies surrounding Recall, which could discourage some users from making the switch.
To be fair to Microsoft, the newly released Copilot+ PCs offer incredible performance and battery life. They also perform better when running on batteries than other Windows computers.
The role of NPUs in Copilot+ PCs
NPUs primarily allow AI processes to run locally without the need to use cloud servers. It’s one of the main reasons Microsoft insists on processors that can execute at least 40 trillion operations per second.
For example, whenever you press the Copilot button, launch Recall, activate Live Captions, or turn on Cocreator, Windows will automatically route those processes to the NPU. You can also see the NPU’s real-time graph in the Task Manager tick up to show that it’s busy.
NPUs also handle other integrated AI functions, such as graphics upscaling (automatically improving video and image equality, especially in games).
Additionally, they free up computing resources for GPUs and CPUs by handling tasks related to AI that these processors may have had to handle in the past. These tasks include things like graphics upscaling and object recognition. That way, you’ll notice significant performance improvements.
Other Types of Devices Using NPUs
As mentioned, NPUs are not exclusive to computers. They’ve also been integral to other smart devices for years.
Mobile chip makers such as Qualcomm, Samsung, Apple, and Huawei have used NPUs for years to power features like voice assistants, facial recognition, predictive typing, translation, and task automation.
NPUs are also part of IoT devices, such as Smart TVs that automatically upscale image quality and gadgets like speakers with integrated AI assistants.
Are NPUs the Only Type of AI Processors?
No. NPUs are just among the many types of processors built for handling complex AI tasks. Players like NVIDIA, Google, Amazon, Qualcomm, IBM, AMD, and Apple also have different types of processes dedicated to AI.
\Most of this hardware targets enterprise-scale processes and is built for things like robotics, driverless vehicles, security and surveillance, and cloud computing.
Outlook for AI Chips
According to Statista, the AI chips market was worth $53.7 billion in 2023 and is primed to surpass $70 billion in 2024 and $90 billion in 2025.
We’re currently seeing mass AI adoption across different sectors, from healthcare to law and business.
As for PCs, Microsoft’s new direction is wrapped up in artificial intelligence, and its NPU requirement means every computer will have an AI chip going forward.
NPUs: Looking Forward
NPUs are now making it increasingly possible to truly achieve incredible machine learning and AI ambitions. Users can now run generative AI tasks on their devices without subscribing to cloud solutions. The requirement for Copilot+ PCs and future AI capabilities also means you now have to look out for NPU-powered computers on your next purchase.
However, there are no guarantees that these features will come as bundled packages with Windows or if Microsoft will introduce a Windows-as-a-service subscription model.
Still, there’s a lot to anticipate, as Intel and AMD’s drive to ship Copilot+ PCs means users will get even more powerful and efficient computers. Windows 12 speculations are also not fading away, as some experts believe Copilot+ PC is a primer for the new operating system.
Let us know what you think about NPUs and if the technology has bought you over.
