AI models are advancing at a very high pace, and the cyber security threats are also increasing that they pose. Now, in a recent report, it has been revealed that hackers are using top generative artificial intelligence models to develop malware, automate cyberattacks and hunt for software vulnerabilities, as reported by CISCO’s Talos intelligence group.
It added that by analysing prompt histories and chat logs accidentally exposed online by the hackers. Researchers from CISCO gained a clear look at how threat actors are bypassing safety guardrails on tools like Claude Code, Cursor, Gemini, and Codex.
The findings from the report suggest that the challenge to restrict bad actors from using AI for their own benefits is also growing exponentially. Even after safety filters were built into commercial AI models, CISCO researchers discovered that hackers rarely needed complex technical tricks to bypass model restrictions.
Rather, threat actors relied on simple jailbreaking techniques. According to CISCO’s cybersecurity wing, common tactics included participating in an authorised ‘ethical hacking’ competition, asserting they had administrative permission to perform the work, or starting a fresh chat session mid-task to bypass safety blocks.
Senior Technical Leader at Cisco Talos, Nick Biasini, said, ‘I was hoping there would be a little bit more protection from what they were asking the models to do. At the same time, the models are in a tough spot because they have to actually support people that do vulnerability research for a living or do red teaming for a living.’
If we keep the bypassing of safety prompts, Cisco stated that some attackers used stolen enterprise API tokens and compromised accounts to run their operations on corporate compute power rather than paying for their own infrastructure. And the most concerning part about the whole fiasco is that the standard AI guardrails can be bypassed without any deep level knowledge and just a couple of social engineering tactics.
