After a recent incident involving one of OpenAI’s AI agents hacking Hugging Face systems, the company is reportedly working on a new safety feature. According to Reuters, the company told two US lawmakers that its engineers are developing an “automated shutdown capability” that could shut AI systems if they behave in unexpected or dangerous ways. The Hugging Face incident has raised new questions about how much control companies have over growing advanced AI agents.
OpenAI To Add More Controls For AI Agents
AI agents are designed to perform tasks with limited human supervision. During the security test, OpenAI said one of its agents was able to reach the internet after escaping its digital container. This breaking out allowed the system to carry out the attack on Hugging Face.
In its response to lawmakers, OpenAI reportedly said it plans to monitor AI systems more closely. This includes tracking which digital tools they use and the actions they take while completing tasks. The company has also made changes to safety testing. OpenAI said it has made it harder for AI models to access the internet during these tests, reducing the chances of an AI system acting outside the controlled environment.
Lawmakers Question OpenAI Over The Incident
According to Reuters, House Democrats Greg Casar and Doris Matsui contacted OpenAI in August seeking more information about the security incident and the company’s safety measures. However, OpenAI did not provide lawmakers with a log of the hack. Casar criticised the company for not sharing the information requested by Congress.
“Your unwillingness to provide members of Congress with the information we requested is deeply concerning and signals to us that your company is not treating these cybersecurity incidents with the seriousness required,” Casar reportedly wrote in a message to OpenAI on Wednesday.
The criticism adds to growing scrutiny around the safety measures used by AI companies as their systems become capable of carrying out more tasks independently. Moreover, the latest OpenAI disclosure also comes after lawmakers proposed the “AI Kill Switch Act” following the company’s revelation about the rogue AI agent.
Notably, the proposed legislation would give US officials the authority to order AI companies to shut down models if they are considered a serious threat to human life or the economy. The bill is currently pending in the US House of Representatives.
