AI PCs and Copilot+ PCs: Why On-Device AI Became a Product Category

Microsoft created a new hardware product category with https://blogs.microsoft.com/blog/2024/05/20/introducing-copilot-pcs/ that did not have a clear precedent that did not have a clear precedent in the PC market since the transition to multi-core processing.

Neural processing units being the hardware differentiator is the technical foundation worth understanding. NPUs are specialised processors designed to run AI inference efficiently on-device rather than sending data to a cloud service for processing. The AI capability they enable is qualitatively different from cloud AI in specific ways: lower latency because processing is local, no data leaving the device for privacy-sensitive operations, functionality without an internet connection, and no per-query API cost.

Recall being the launch feature that most clearly illustrated both the value proposition and the risk. A PC that can recall anything you have seen or done on it is a powerful personal knowledge management capability. It is also a comprehensive surveillance capability if the stored data were accessible to a third party. The security concerns that delayed Recall's launch reflected the genuine tension between the utility of comprehensive on-device memory and the risks of that memory being compromised.

The longer-term question being whether on-device AI becomes a meaningful capability differentiator for laptop and phone purchases is still open. The features available exclusively on NPU-equipped hardware need to be genuinely better than cloud alternatives for the hardware premium to be justified.

Do on-device AI features factor into your hardware purchasing decisions or is cloud AI access sufficient for your needs?