Tested: How badly Windows on Arm compares to the new Mac M1s
After Apple released its impressive M1 Arm chip on its new Macs, and Microsoft followed with its long-awaited 64-bit X86 emulator, we had just one question: How does Windows on Arm compare to MacOS on Arm? The answer: badly. Very, very badly.
Running Windows apps on Arm processors has a few wrinkles. For one, there are only two chips currently powering Windows on Arm machines: Qualcomm’s own processors, such as the Snapdragon 8cx and Snapdragon 8cx Gen 2, as well as the derivative SQ1 and SQ2 processors Microsoft co-designed with Qualcomm. The latter two processors both appear in Microsoft’s Surface Pro X tablet.
Until last week, WOA devices have only been able to run apps coded natively for the Snapdragon Arm architecture, or run 32-bit apps coded for X86 processors natively. Last week, after an awkward delay, Microsoft finally published its long-awaited 64-bit X86 emulator, allowing Windows on Arm PCs to run 64-bit X86 apps via emulation. The vast majority of apps today are optimized for 64-bit processors and the larger amount of memory they can address. Because the apps are being emulated and not running natively, they will run more slowly than native code. Apple, too, has shipped Macs running on its own 64-bit Arm chip, the M1, and shipped a finalized 64-bit emulator alongside it.
Given the glowing reviews by our sister site, Macworld, we know how well the new MacBook Air (M1) and other M1-based hardware performed. Now that Microsoft has shipped its own 64-bit emulator, we can more directly compare how well Windows on Arm compares to Mac OS on Arm.
Jason Cross/IDGApple and its M1-powered MacBook Air have accomplished what Microsoft hasn’t: delivering a viable new Arm ecosystem of hardware and software.
How we tested
Our testbed was Microsoft’s Surface Pro X, running on a first-generation SQ1 chip, a more powerful version of Qualcomm’s Snapdragon 8cx. (We did not have an SQ2-powered Surface Pro X to test.) We downloaded and installed Windows Insider Build 21277 and the additional code, such as Adreno GPU drivers, to allow 64-bit X86 apps to run. (Microsoft warned that not every app would work, even with its emulator.) We used Apple’s MacBook Air (M1) as a comparison.
We already had a good idea of how slow Microsoft’s Surface Pro X is—that was evident from our original review. But these benchmarks provide insight into just how slowly the Surface Pro X and its SQ1 chip run with the new 64-bit X86 instruction emulator layered on top. We hewed closely to the test suite from Macworld’s MacBook Air review, including GeekBench 5, Cinebench R23, HandBrake, and a representative game, Rise of the Tomb Raider. We added a third Windows laptop for reference: the HP Pavilion x360 Convertible 14, a decidedly average $700 laptop with a fairly pedestrian Core i5-1035G1 inside.
To be fair, Microsoft’s emulator is in preview, and Microsoft promises performance will improve over time. Also, we’re comparing the first-gen SQ1 chip, which maxes out at 3GHz, and not the current SQ2—though the SQ2 offers a teensy upgrade to a 3.1GHz boost clock. We tried testing with the Windows performance slider set to maximum, and the results were unchanged. Windows on Arm lags so far behind the MacBook on M1 that it’s hard to believe further improvements will bring it significantly closer.
Enough preamble—let’s look at how soundly Apple’s MacBook with the M1 chip trounces Windows on Arm’s best.
How Microsoft’s SQ1 compares to Apple’s M1
Geekbench offers both a CPU-specific test, and a “compute” benchmark that ropes in the GPU. The current version of Geekbench 5 couldn’t accommodate the SQ1 in the latter test, so we show only the CPU test in single-core and multi-core. We can see that the SQ1 pales in comparison to a Core chip and the Apple MacBook M1.
Mark Hachman / IDGAlthough the Microsoft SQ1 chip inside the Surface Pro X can hang with the Core i5-1035G1 inside our budget HP Pavilion, there’s no way it can compete with the Apple MacBook M1.
Maxon’s Cinebench paints a rendered two-dimensional image. Macworld jumped to the most recent R23 benchmark, which uses a more complex image than the R15 version PCW has used. The new R23 release supports Apple’s silicon, with no specific optimizations for the SQ1 or Qualcomm’s Snapdragon chips. Nevertheless, Windows on Arm running on the Surface Pro X isn’t even in the same league as the Apple Macbook M1.
Mark Hachman / IDGIn pure CPU performance as measured by Cinebench, Apple’s M1 holds more than a fourfold performance advantage over the Surface Pro X and Microsoft’s SQ1.
HandBrake is an open-source video transcoding tool, and a popular benchmark. Its latest version, version 1.4, is written specifically for MacOS, to accommodate the new M1 processors. We used the last public version, 1.33, for our Windows test. But the version is not the real difference here. The SQ1 chugged along at about a frame per second, taking about two hours to transcode a 12-minute 4K video, Tears of Steel, into a 1080p H.265 format. Apple’s MacBook M1 simply blows away the Surface Pro X.
Mark Hachman / IDGAlthough we’re using different versions of HandBrake in our comparison, the tool is not the difference here. The MacBook Air M1 is more than six times faster than the Surface Pro X in video transcoding.
We’d like to say that we were able to run Rise of the Tomb Raider, part of our test suite for gaming PCs, but the Surface Pro X simply wouldn’t. Instead, we ran the HP Pavilion on the benchmark at 1280×800 resolution (Medium settings) to compare against the MacBook M1.
Mark Hachman / IDGMicrosoft’s SQ1 versus other Windows laptops
We took the opportunity once again to see how the Surface Pro X and its SQ1 chip, with the beta X86 emulator, compares against a suite of Windows laptops. Here, too, we ran into compatibility issues. A 3DMark Sky Diver test that we usually use to test 3D performance wouldn’t run on the Adreno GPU. PCMark 8 Creative failed to run, too—though it did when we originally reviewed the Surface Pro X. We weren’t able to run Cinebench R15, a standard workload for our laptop testing. We were, at least, able to run an older version of HandBrake.
HandBrake exposes how weakly the SQ1 performs against modern Windows laptops running X86 processors from Intel and AMD. Even a budget laptop from three years ago, the Chuwi Hi13, topped the Surface Pro X.
Mark Hachman / IDGOnce again, the Surface Pro X finishes roughly four times slower than our 2020 budget laptop, the HP Pavilion x360 Convertible 14.
Conclusion: Windows on Arm needs a miracle
Two years ago, the future of Windows on Arm looked bright. With what we hoped was a 64-bit emulator waiting in the wings, the Snapdragon’s “good-enough” performance could hold its own, especially with the perks of all-day battery life and LTE connectivity. Today, Project Athena/Evo laptops from Intel’s partners have caught up in all these area. Qualcomm hasn’t launched a significant Windows on Arm chip in about two years, and during its recent Snapdragon Tech Summit the company had basically nothing to say about its future PC plans.
Microsoft’s 64-bit X86 emulator is still in beta, so we can’t make definitive statements about its success. But it’s hard to believe that further development will bridge the vast gulf of performance between Windows on Arm and Apple’s M1-based Macs. In six months, Microsoft may be able to boast that its emulation performance has improved by a significant amount. But without the combined miracle of a much better CPU from Qualcomm or another Arm chipmaker and continued improvements from Microsoft, the future of Windows on Arm looks grim.