AI Safety - Maginative

Microsoft AI Safety

Microsoft Partners with StopNCII to Combat Intimate Image Abuse

Chris McKay• September 5, 2024 • 1 min read

The pilot program has already shown promising results. By the end of August, the company had taken action on 268,899 images flagged through the StopNCII database, preventing them from appearing in Bing image search results.

AI Policy AI Safety

California AI Safety Bill Clears Assembly, Awaits Final Approval

Chris McKay• August 28, 2024 • 2 min read

The Safe and Secure Innovation for Frontier Artificial Intelligence Models Act squeaked through the Assembly with a 41-9 vote, the minimum number required for passage.

OpenAI AI Safety

OpenAI Has Demonstrated "Strawberry" AI Capabilities to U.S. National Security Officials

Chris McKay• August 27, 2024 • 2 min read

Strawberry, previously known as Q*, is reportedly significant technical leap forward in AI capabilities, particularly in complex problem-solving and reasoning.

OpenAI AI Ethics AI Safety

OpenAI Disrupts Iranian Influence Operation That Used ChatGPT

Chris McKay• August 16, 2024 • 2 min read

The operation was using the AI assistant to produce articles and social media posts on subjects ranging from U.S. politics to global events, which were then distributed across multiple platforms.

Anthropic AI Safety

Anthropic will Pay You Up To $15,000 to Jailbreak its Next-Gen AI Safety System

Chris McKay• August 9, 2024 • 1 min read

The program aims to uncover vulnerabilities that could consistently bypass AI safety guardrails across a wide range of high-risk domains, including chemical, biological, radiological, nuclear, and cybersecurity areas.

OpenAI AI Safety

OpenAI Publishes GPT-4o Model Card Detailing Extensive Safety and Risk Mitigation Measures

Chris McKay• August 9, 2024 • 3 min read

It offers an in-depth look at the extensive safety protocols and risk mitigations that were implemented during the model’s development and deployment.

OpenAI AI Safety

OpenAI Adds AI Safety Expert Zico Kolter to Board

Chris McKay• August 8, 2024 • 2 min read

Kolter currently leads the Machine Learning Department at Carnegie Mellon and brings over a decade of experience in AI research, particularly in developing methods to assess and enhance the safety of AI systems.

Google AI Safety

Google Research Highlights Generative AI Misuse Tactics

Chris McKay• August 2, 2024 • 2 min read

The report highlights two main types of misuse: exploitation of AI capabilities and compromise of AI systems.

OpenAI AI Safety

OpenAI to Provide Government with Early Access to Next Foundation Model

Chris McKay• August 1, 2024 • 3 min read

These government collaborations are becoming increasingly common in the AI industry as companies navigate the complex landscape of rapid advancement and public accountability.

Microsoft AI Safety Responsible AI

Microsoft Urges Congress to Enact Federal Deepfake Fraud Statute

Richard Banfield• July 30, 2024 • 2 min read

President Brad Smith is urging lawmakers to pass a comprehensive 'deepfake fraud statute' that would give law enforcement the necessary legal framework to prosecute deepfake-related crimes.

OpenAI AI Safety

OpenAI Introduces Rule-Based Rewards, an AI-Powered Alternative to RLHF

Chris McKay• July 24, 2024 • 2 min read

RBRs offer a promising approach to using AI to enhance AI safety and efficiency—streamlining the training process while ensuring models remain aligned with desired behaviors.

Responsible AI Cybersecurity AI Safety

Tech Industry Launches Coalition for Secure AI

Richard Banfield• July 18, 2024 • 2 min read

CoSAI will focus on the entire lifecycle of AI systems, from building and integration to deployment and operation. The coalition aims to mitigate risks like model theft, data poisoning, prompt injection, and inference attacks.

OpenAI AI Safety

OpenAI Partners with Los Alamos National Laboratory for Bioscience Safety Research

Chris McKay• July 10, 2024 • 2 min read

The project will assess GPT-4o's ability to assist both expert scientists and novices with complex biological tasks using its visual and voice capabilities.

Microsoft AI Safety Cybersecurity

Microsoft Reveals 'Skeleton Key': A Powerful New AI Jailbreak Technique

Chris McKay• June 28, 2024 • 3 min read

The jailbreak allowed top models to comply fully with requests across various risk categories, including explosives, bioweapons, political content, self-harm, racism, drugs, graphic sex, and violence.

SSI Venture Capital Startups AI Safety

OpenAI Co-Founder Ilya Sutskever Launches Safe Superintelligence Inc.

Chris McKay• June 19, 2024 • 2 min read

SSI will focus solely on building a safe and powerful AI system, and has no near-term plans of selling AI products or services.

AI Safety Responsible AI

AI Whistleblowers Call for Increased Safety Oversight and Employee Protections

Chris McKay• June 4, 2024 • 2 min read

The letter makes a case for stronger whistleblower protections, arguing that current measures are insufficient to address the unique challenges posed by AI.

OpenAI AI Safety

OpenAI Releases Report on Disrupting AI-Powered Influence Operations

Chris McKay• May 30, 2024 • 2 min read

The report provides valuable insights into the evolving landscape of deceptive influence operations and the measures needed to address them.

OpenAI AI Safety

OpenAI Forms Safety and Security Committee

Chris McKay• May 28, 2024 • 2 min read

This move comes amid scrutiny and high-profile concerns about the company's commitment to AI safety, especially following the recent departures of key personnel involved in AI safety and "superalignment" work.

Google AI Safety

Google Introduces Frontier Safety Framework to Identify and Mitigate Future AI Risks

Chris McKay• May 17, 2024 • 2 min read

The Frontier Safety Framework focuses on severe risks posed by advanced AI models, such as those with exceptional autonomy or sophisticated cyber capabilities.

OpenAI AI Safety

OpenAI Introduces Tools to Help Identify AI-Generated Content

Chris McKay• May 7, 2024 • 3 min read

Deepfakes, or AI-generated fake content, have already impacted political campaigning and voting in various countries, including Slovakia, Taiwan, and India.

AI Safety

NIST Launches GenAI Program to Evaluate and Measure Generative AI Technologies

Richard Banfield• April 29, 2024 • 1 min read

NIST GenAI will release benchmarks and evaluate generative AI technologies through a series of challenge problems. One of its key goals is to develop systems that can distinguish between human-created and AI-generated content, starting with text.

AI Safety

DHS Establishes AI Safety and Security Board with Industry Leaders

Chris McKay• April 26, 2024 • 2 min read

The Board will develop recommendations to help critical infrastructure stakeholders, such as transportation service providers, pipeline and power grid operators, and internet service providers, more responsibly leverage AI technologies.

AI Literacy AI Ethics AI Safety

AI as a "New Digital Species": Thoughts on Microsoft AI Chief's TED Talk

Chris McKay• April 23, 2024 • 4 min read

By proposing the "digital species" metaphor, he offers a new lens through which to view AI's potential and pitfalls. It invites us to consider the long-term implications of this technology and our responsibility in shaping its evolution.

Meta AI Safety

Meta to Require "Made with AI" Labels for AI Generated Content

Chris McKay• April 5, 2024 • 2 min read

The new labels will cover a broader range of content than the company's current manipulated media policy

Anthropic AI Safety Research

Anthropic Shares Research on Technique to Exploit Long Context Windows to Jailbreak Large Language Models

Chris McKay• April 2, 2024 • 3 min read

Many-shot jailbreaking works by prompting the model with a large number of fictitious question-answer pairs that depict the AI assistant providing harmful or dangerous responses.