Microsoft announces Azure AI trio at Inspire 2023

Microsoft announced a slew of new capabilities at its Inspire 2023 conference today. Among them are three new AI capabilities that merit deeper exploration.

These capabilities follow a substantial number of Azure-based AI releases this year from the Redmond-based corporation.

Also: The best AI chatbots

John Montgomery, Microsoft corporate vice president of AI Platform told ZDNET, “Increasingly, companies are looking for tools to customize their AI builds and fit their organization's needs and budget. Our goal is to make it easy for any organization to innovate with AI for things like building chatbots using organizational data and transforming search and document intelligence to get valuable insights and make better business decisions.”

Let's dive in.

Azure AI Vector Search preview

As we've come to know the capabilities of large language models (LLM) like ChatGPT, one of the most amazing (and controversial) features is their ability to find and describe data in response to a query or a prompt. Performing analysis of enormous datasets requires very powerful search capabilities.

Most folks are familiar with the traditional keyword search, where a word or phrase is used as a lookup key for resulting data. But AI engineers have been working with a different kind of search, called vector search.

Also: 7 advanced ChatGPT prompt-writing tips you need to know

Vector search not only looks at text, but it can also search media types like images, video, and audio. This chart gives a quick overview of how keyword search differs from vector search:

Rather than using keywords, vector search creates a numerical representation of the data (called vector embeddings).

Vector search is part of Azure's Cognitive Search offering, which has a number of search mechanisms. Search applications can call on a hybrid set of these search options to perform large data searches, providing a deep bench of search capabilities.

Today, at Microsoft's Inspire 2023, Microsoft announced that vector search is now in public preview and developers can start incorporating this capability into their projects.

Azure Al Document Intelligence preview

AI chatbots like ChatGPT do provide considerable productivity benefits. But they rapidly run into walls once there's any attempt to use them in the corporate environment. 

Also: ChatGPT productivity hacks: Five ways to use chatbots to make your life easier

While LLMs offer tremendous knowledge analysis (and a little bit of randomly generated hallucinatory filler), they lack the ability to process corporate information.

This lack of capability is driven by three key factors:

1. Data in public LLMs (ChatGPT in particular) have temporal bounds. In other words, ChatGPT doesn't know about anything after 2021.
2. Public LLMs definitely don't know about your corporate information, whether it's the contents of user manuals and repair manuals, or financial and strategy planning documents.
3. It's fortunate that there's no way to insert deep corporate information into a public LLM's dataset, because you definitely don't want all that proprietary information out there in public.

But there's enormous potential in combining the capabilities of generative AI chatbots with deep corporate data and documents. Imagine interacting with your proprietary user manuals, or being able to ask questions based on meeting notes going back years. The discovery potential, as well as the cross-departmental knowledge integration potential, is enormous.

That's where Document Generative AI, which is a collaboration between Azure AI Document Intelligence (formerly Azure Form Recognizer) and Azure OpenAI Service, comes in. This capability, also just announced at Inspire 2023, provides support for multiple document types, OCR (with AI-driven error correction), and information extraction.

On top of all that is all the security that Microsoft and Azure bring to all their cloud-based offerings, so you can upload your proprietary data and be assured that it will remain locked down and under your control.

Also: Generative AI is coming for your job. Here are 4 reasons to be excited

Beyond that, the real key to the Azure-based solution is scalability. Individual documents can be small all the way up to huge, and the library of documents can be enterprise-scale as well. Microsoft today announced that you can try out the service via this GitHub repo.

Whisper Model preview

The third main AI Azure AI announcement that caught our attention is the service's new Whisper Model, which handles audio transcription at enterprise scale. This, too, is a collaboration with OpenAI, providing the OpenAI services on top of Azure.

Whisper Model has the ability to understand 57 languages. It supports what Microsoft calls “enhanced readability,” which essentially means the AI can understand what's being said in context and can create transcriptions with more colloquial wording, as appropriate.

Also: ChatGPT vs Bing Chat vs Google Bard: Which is the best AI chatbot?

A big feature here is scalability. Whisper Mode on Azure can scale to transcribe hundreds or even thousands of documents. It can, for example, transcribe thousands of customer service conversations, and index them using some of the tools we've discussed previously.

This also relates to the customization and Azure integration aspects of the service. As we've seen, Azure AI has a wide range of offerings and the new Whisper Model integrates into those offerings, allowing users to build out very customized applications.

It should be mentioned that IBM offers Watson speech-to-text, Google has an AI-driven speech-to-text solution, as does Amazon. Most offer similar capabilities to Whisper Mode in Azure.

The real differentiating factor with the Azure/OpenAI offering is how tightly it integrates into other Azure solutions and includes Azure security capabilities, making it a very natural component to add along with other aspects of custom applications.

Also: How does ChatPT actually work?

Whisper Model was announced at Inspire 2023. Once it is available in preview, users will be able to apply for access to the Azure OpenAI Service, which will then open the door for Whisper testing.

More AI from Inspire 2023

Microsoft's Montgomery said, “From our supercomputing infrastructure and trusted cloud capabilities to cutting-edge AI models and robust responsible AI tooling, we're focused on continuing to make Azure the best place for training, deploying and scaling AI models — both frontier and open.”

In this article, we spotlighted three AI technologies announced at Inspire 2023. Microsoft clearly has put a lot of time and investment in providing AI tools for its Azure clients. Keep reading ZDNET for our ongoing coverage of Inspire 2023, as well as other AI initiatives Microsoft is announcing.

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.