Microsoft is stepping up its efforts to help developers build more secure and trustworthy generative AI applications with the announcement of five new tools in Azure AI. These tools are designed to address the challenges of prompt injection attacks, hallucinations, and content risks, which can undermine the quality and reliability of AI systems.
Prompt Shields, one of the new tools, is designed to detect and block both direct (jailbreak) and indirect prompt injection attacks. Jailbreak attacks involve users manipulating prompts to trick the AI system into inappropriate content generation or ignoring system-imposed restrictions. Indirect attacks, on the other hand, involve hackers manipulating input data, such as websites or emails, to make the AI system perform unauthorized actions. Prompt Shields proactively safeguard the integrity of large language model (LLM) systems and user interactions.
Another tool, Groundedness detection, identifies text-based hallucinations, which occur when a model confidently generates outputs that misalign with common sense or lack grounding data. This feature detects 'ungrounded material' in text to support the quality of LLM outputs.
Microsoft is also introducing safety system message templates to help developers build effective system messages that guide the optimal use of grounding data and overall behavior. These templates, developed by Microsoft Research, can help mitigate harmful content generation and misuse.
To help organizations assess and improve their generative AI applications before deployment, Azure AI Studio now provides automated evaluations for new risk and safety metrics. These safety evaluations measure an application's susceptibility to jailbreak attempts and to producing harmful content. They also provide natural language explanations for evaluation results to help inform appropriate mitigations.
Finally, risk and safety monitoring in Azure OpenAI Service allows developers to visualize the volume, severity, and category of user inputs and model outputs that were blocked by their content filters and blocklists over time. This feature also introduces reporting for potential abuse at the user level, providing organizations with greater visibility into trends where end-users continuously send risky or harmful requests to an AI model.
With these new tools, Azure AI hopes equip customers with innovative technologies to safeguard their applications across the generative AI lifecycle. As generative AI becomes increasingly prevalent, it is crucial for organizations to prioritize security and responsibility in their AI development efforts. Microsoft's commitment to responsible AI principles is evident in these new tools, which empower developers to create game-changing solutions while mitigating risks and promoting trust.