Microsoft wants you to talk to Windows 11 PCs again — Copilot gets 'conversational' input to complement your mouse and keyboard

Microsoft is pushing to make every Windows 11 PC an AI PC. In new updates, the company is focusing on further integrating AI into Windows 11 apps and experiences (not just chatbots), and making voice as much of an input option as the keyboard and mouse.

The idea, Microsoft says, is that AI on a PC should be easy and natural to interact with in either text or voice; that it should be able to see your screen and information and help; and that it should be able, with your permission, to "take action on your behalf."

"Voice will become the third input mechanism to using your PC," Microsoft executive vice president and consumer chief marketing officer Yusf Mehdi said on a call with reporters ahead of the announcement. He mentioned that the idea is for talking to be additive, not replacing keyboards and mice altogether. Mehdi authored a blog post on the new features.

You will be able to talk to Microsoft's Copilot AI by invoking a "Hey, Copilot" wake word, or use the established Win+C shortcut or Copilot button. With your permission, Copilot will be able to see your screen and make suggestions or provide help. (To enable the feature, you'll have to opt in in settings.)

In example videos, we saw someone talk to Copilot in a music player app, and the AI walked the user through steps to get the best sound quality. In another, someone watched a YouTube video and asked Copilot to identify the microphone the host was using, and then the AI pointed out that Best Buy had it available.

This feature, among others that Microsoft is announcing today, won't require a Copilot+ PC (with one exception), but will instead work on any system running Windows 11.

When asked whether people will want to speak to their computers, particularly in offices with other people, Mehdi said that people already have all sorts of talk in these spaces, and that, between office chit chat and using headphones, workers will find ways to learn how to deal with it.

Microsoft is also making Copilot Vision available anywhere in the world where Copilot is offered. Windows Insiders will soon be able to interact with Copilot Vision using text (previously, you could only talk to it).

Windows Insider Experiences

Microsoft is bringing a handful of new Copilot features to Windows Insider before they go wide, with plenty of opt-in security controls..These include a new feature to build Copilot into the taskbar, called "Ask Copilot," that takes the place of the traditional search bar. Microsoft says this will let you use just one click to jump into chat, search for files, or use Copilot Voice or Vision. This, like many of Microsoft's Copilot features, is opt-in.There's also a new mode called Copilot Actions, which will let Copilot take action on your behalf. An experimental feature will go beyond the web and even let the AI use your local files, including sorting photos and finding information in documents.

Copilot Actions will let you observe every step it takes, and you can take over at any point. It will launch in Copilot Labs with a narrow set of use cases, and Navjot Virk, corporate vice president of Windows Experiences, said that these will include more complicated tasks as the model is trained. Actions will be turned off by default, and may request additional consent before doing anything deemed important.

Windows users will also be able to connect Copilot to services, including OneDrive and Outlook, as well as Google services like Gmail, Google Drive, and Google Calendar. From here, Copilot will be able to see your calendar and other personal data to help you find personalized information like appointments, contact details, and documents.The Copilot Connections will also let you export text from Copilot to Microsoft 365, letting you put results into homework or presentations.

Lastly, Microsoft is integrating Manus, a general AI agent, into File Explorer. This AI, which is also available through a native app on Windows 11, can use your documents to complete tasks like creating a website, without you needing to code.

Copilot Plus update

There's only one AI announcement for Copilot+ computers. The Click-to-Do feature will now be able to add actions from Zoom. For example, if you hover over an email address, there will be an option to invite that person to a Zoom meeting.

In other Windows news, earlier this week, Microsoft dropped support for Windows 10 and is encouraging customers to update to Windows 11 with new and improved devices. Extended support for Windows 10 is available to some customers as well as businesses.

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.