Windows’ built-in personal assistant Cortana has long been one of its consumer selling points. But as competition from Google and Amazon has increased, it has been seen as lagging behind, having to work with Amazon’s Alexa to add support for a wider range of services, for example.
Microsoft certainly was slower at expanding Cortana’s reach to third-party applications, but by building on its own Bot Framework in the Cortana Skills Kit it has provided developers with a multi-channel approach to building and delivering voice-powered services. Using the Bot Framework allows the same code that supports Cortana to work on the web or through Skype.
That same Bot Framework is part of Microsoft’s enterprise collaboration tooling, working with its own Teams service, as well as Slack and Facebook Messenger. So it’s not surprising to see Microsoft taking its Cortana tools and giving them an enterprise spin, to add voice-based natural user interfaces to your enterprise applications.
While the Cortana Skills Kit for Enterprise is still in private beta, the development team has released details of what you’ll need to use to build your own, private, Cortana skills for use with the iOS and Android Cortana mobile apps, with PCs, and with Cortana-enabled smart speakers.
Building on bots
Microsoft’s Bot Framework has been available for some time now. Using Azure to host bot applications running in node.js and in .NET, it’s a way of quickly adding code around Azure’s existing cognitive services. You can use them to capture context from speech, and with tooling like the Language Understanding framework LUIS, you’re able to extract information that can be used to automatically fill forms, attaching speech recognition to existing APIs and to existing services.
SEE: IT leader’s guide to the future of artificial intelligence (Tech Pro Research)
Using the Cognitive Services APIs simplifies much of the machine learning needed to build natural user interfaces. Microsoft has been developing a range of different speech APIs that can be used in conjunction with other services to deliver what would have been complex applications. Using its speech tooling is no different to handling freeform text input, and the resulting bots can be connected to new or existing applications and services.
Azure AD for security
Where the enterprise skill tooling differs from the consumer service is in adding Azure Active Directory support. By using Azure AD, the Cortana Skills Kit for Enterprise can lock down access to only inside your organization, unlike the wide public reach of consumer Cortana skills. You can go even further, and provide specific skills and specific functionality to different groups inside a business, so that marketing-focused skills can’t be accessed by accounting, or by more than the development group that’s building a skill.
Azure AD is at the heart of Microsoft 365, either working in hybrid mode with existing on-premises Active Directory or as a standalone service. By taking a dependency on AAD, Cortana’s enterprise skills can also take advantage of its built-in security features, ensuring that skills are only accessed by authorised users on authorised devices. With skills possibly accessing regulated information, having effective controls is essential — both during deployment and at runtime.
Once your skill is ready for deployment, you can hand it over to your IT admins. They’re able to use a new portal to first test and review code, before adding it to an Azure tenant, ready for use. It’s important to think carefully about how you want corporate Cortana skills to operate. They’re going to need to work within your existing privacy and security policies, and will also need to comply with any appropriate regulations.
That’s going to mean avoiding using Cortana skills with personally identifiable information, as well as protecting health records and financial transactions. You probably also don’t want to make it too easy to trigger a new business process, or update and change databases and other records.
What does an enterprise Cortana do?
While choosing what apps to voice enable requires some thought, that doesn’t stop you being imaginative. A voice query doesn’t always need a spoken response, and you can use tools like Adaptive Cards to deliver cross-platform output in your preferred applications. Voice can trigger applications, opening meetings in Teams, or booking rooms in Outlook. You can perhaps quickly dictate emails, or take advantage of the Microsoft Graph to manage files or contacts.
SEE: How we learned to talk to computers, and how they learned to answer back (cover story PDF)
There’s even the prospect of working with newer technologies, like the Project Rome APIs that power Windows 10’s connected experiences. You might be in a meeting room, giving a presentation through a Surface Hub or similar, opening a document from your PC on a wall display without having to break the flow of your presentation. That may be the key driver for Cortana in the enterprise — as a tool to keep you from losing your concentration on a task, letting you use voice to either go to a new application, or to handle an interaction in the background, without having to switch applications or lift your hands from the keyboard.
Perhaps you’re writing some code, and realise that you need to talk to someone on another team about an interaction with their module. Asking “Hey Cortana, start a Teams chat with Tim about his interfaces” is a lot easier than opening an email client, creating a meeting request, and sending it off. You’re able to carry on coding until the meeting starts.
Avoiding risky business
Bringing Cortana into the enterprise is risky. Voice and other natural user interfaces aren’t commonplace outside the home, and they’re hard to explain. You’ll probably want to start with a simple pilot where you can demonstrate a quick win, before investing in significant development. It’ll probably also be easier to get buy-in if you’re already using tools like the Bot Framework in collaboration and workflow environments.
By building on existing trusted tooling, Microsoft hopes to overcome misgivings early, adding Cortana as a new input to existing multi-channel applications. If users are already using bots in Slack or Skype, taking the extra step to driving them through Cortana isn’t going to be a huge leap.
Cortana Skills Kit for Enterprise is currently in private preview, and you can request an invitation to try out the service.