The ARC Prize will Become a Nonprofit that Benchmarks AGI
The foundation will transition to a 501(c)(3) nonprofit and begin fundraising this month
AI Safety
A collection of 81 posts
OpenAI Shares Research on Red Teaming Methods
OpenAI has been applying these techniques across their major model releases, from DALL-E 2 through their recent o1 model family.
The Character.AI Lawsuit Is a Wake-Up Call for Responsible AI Development
What happens when these AI-generated relationships become more than just entertaining—when they become emotional crutches for people struggling to connect in the real world?
California Governor Vetoes Landmark AI Safety Bill
In his veto message, Newsom stated that while the bill was "well-intentioned," it failed to consider the deployment context of AI systems.
California Enacts Five Laws to Combat AI Deepfakes in Elections and Entertainment
Three bills focus on curbing the spread of deceptive AI-generated content in political campaigns, while two others strengthen protections for actors and performers against unauthorized use of their digital likenesses.
Google to Identify AI-Generated Images in Search and Ads with C2PA 2.1 Standard
In Search, the "About this image" feature will display whether an image was created or edited using AI tools. For Ads, the company aims to use C2PA signals to enforce relevant policies.
OpenAI Updates Safety and Security Measures with Independent Oversight
The newly empowered SSC will have the authority to delay model releases if safety concerns are not adequately addressed.
Microsoft Partners with StopNCII to Combat Intimate Image Abuse
The pilot program has already shown promising results. By the end of August, the company had taken action on 268,899 images flagged through the StopNCII database, preventing them from appearing in Bing image search results.
California AI Safety Bill Clears Assembly, Awaits Final Approval
The Safe and Secure Innovation for Frontier Artificial Intelligence Models Act squeaked through the Assembly with a 41-9 vote, the minimum number required for passage.
OpenAI Has Demonstrated "Strawberry" AI Capabilities to U.S. National Security Officials
Strawberry, previously known as Q*, is reportedly significant technical leap forward in AI capabilities, particularly in complex problem-solving and reasoning.
OpenAI Disrupts Iranian Influence Operation That Used ChatGPT
The operation was using the AI assistant to produce articles and social media posts on subjects ranging from U.S. politics to global events, which were then distributed across multiple platforms.
Anthropic will Pay You Up To $15,000 to Jailbreak its Next-Gen AI Safety System
The program aims to uncover vulnerabilities that could consistently bypass AI safety guardrails across a wide range of high-risk domains, including chemical, biological, radiological, nuclear, and cybersecurity areas.
OpenAI Publishes GPT-4o Model Card Detailing Extensive Safety and Risk Mitigation Measures
It offers an in-depth look at the extensive safety protocols and risk mitigations that were implemented during the model’s development and deployment.
OpenAI Adds AI Safety Expert Zico Kolter to Board
Kolter currently leads the Machine Learning Department at Carnegie Mellon and brings over a decade of experience in AI research, particularly in developing methods to assess and enhance the safety of AI systems.
Google Research Highlights Generative AI Misuse Tactics
The report highlights two main types of misuse: exploitation of AI capabilities and compromise of AI systems.
OpenAI to Provide Government with Early Access to Next Foundation Model
These government collaborations are becoming increasingly common in the AI industry as companies navigate the complex landscape of rapid advancement and public accountability.
Microsoft Urges Congress to Enact Federal Deepfake Fraud Statute
President Brad Smith is urging lawmakers to pass a comprehensive 'deepfake fraud statute' that would give law enforcement the necessary legal framework to prosecute deepfake-related crimes.
OpenAI Introduces Rule-Based Rewards, an AI-Powered Alternative to RLHF
RBRs offer a promising approach to using AI to enhance AI safety and efficiency—streamlining the training process while ensuring models remain aligned with desired behaviors.
Tech Industry Launches Coalition for Secure AI
CoSAI will focus on the entire lifecycle of AI systems, from building and integration to deployment and operation. The coalition aims to mitigate risks like model theft, data poisoning, prompt injection, and inference attacks.
OpenAI Partners with Los Alamos National Laboratory for Bioscience Safety Research
The project will assess GPT-4o's ability to assist both expert scientists and novices with complex biological tasks using its visual and voice capabilities.
Microsoft Reveals 'Skeleton Key': A Powerful New AI Jailbreak Technique
The jailbreak allowed top models to comply fully with requests across various risk categories, including explosives, bioweapons, political content, self-harm, racism, drugs, graphic sex, and violence.
OpenAI Co-Founder Ilya Sutskever Launches Safe Superintelligence Inc.
SSI will focus solely on building a safe and powerful AI system, and has no near-term plans of selling AI products or services.
AI Whistleblowers Call for Increased Safety Oversight and Employee Protections
The letter makes a case for stronger whistleblower protections, arguing that current measures are insufficient to address the unique challenges posed by AI.
OpenAI Releases Report on Disrupting AI-Powered Influence Operations
The report provides valuable insights into the evolving landscape of deceptive influence operations and the measures needed to address them.
OpenAI Forms Safety and Security Committee
This move comes amid scrutiny and high-profile concerns about the company's commitment to AI safety, especially following the recent departures of key personnel involved in AI safety and "superalignment" work.