Will Windows AI Copilot be the ultimate PC accessibility tool?

At Build 2023, Microsoft made quite a big deal about its headlining announcement, the Windows Copilot. The idea is simple: build AI right into the operating system. In the same way that the Edge browser (and soon the entire stack of Office apps) have their own AI copilots, now Windows will have one too. But in talking to the leader of Microsoft’s Windows (and Surface) group, Panos Panay, I wanted to dig into a different possible use case for generative AI as a tool for accessibility. It wasn’t presented this way, in particular, but having access to a virtual assistant that’s actually smart built right into the operating system of computer certainly seems like it would have some serious value in terms of accessibility. When I asked Panay about it, he was enthusiastic about the idea. “I have so much passion in this space. When Copilot starts to present itself, you start realizing, regardless of your mode of input, that works for you.” Microsoft is not new to the world of accessibility — in fact, the company’s been ramping up its offering in all sorts of ways recently, whether it’s features in Windows like Live Captions or its modular Adaptive Accessories. But with Windows Copilot, there’s a new opportunity to bring everything together. Now, there’s always been a basic search function in Windows, but making it hyperintelligent does a lot to transform it into the ultimate accessibility tool. In particular, the Windows Copilot might help those who struggle to fully operate their computer in one way or another. “When you think about accessibility, whether it’s a dexterity problem or if want to use voice or only write or only type — whatever the mode of input is — the Copilot will take it,” said Panay. Being able to control your operating system entirely from the Windows Copilot via voice or text seems like an extremely beneficial feature for some. But the even broader benefit Windows Copilot has is as a discovery function. “Imagine what you can do to enable every single accessibility feature in Windows,” Panay said. “There are things right in front of you for accessibility that we do, but there’s so much in there that’s unique to each person. Imagine if you’re able to just call for it.” Panay mentioned the wealth of accessibility options that are already built right into Windows, whether that’s Magnifier, Color Filters, Narrator, Live Captions, Translation, or even Sticky Keys. But the problem is that many people who could likely benefit from any or some of these features might not know they exist. In theory, you could request that the Copilot help you with something, and it would automatically point you in the right direction. You wouldn’t have to go digging through settings to see if such a thing even exists. We’ll have to wait until we’ve used Windows Copilot ourselves to see just how good it is at these things, but for now, it’s certainly promising.