OpenAI O1: Not a Cybersecurity Threat, But a Powerful Nascent Tool for Cyber Builders
Shifting the focus from AI’s risks to its potential as a productivity booster for cybersecurity professionals.
Hey Cyber Builders 🖖
This week, we’re diving into OpenAI’s latest release—the O1-preview model—and its implications for cybersecurity. OpenAI is marketing O1 as an evolutionary step in AI, with enhanced reasoning capabilities that can offer better answers in complex fields like math, physics, and biology. However, regarding cybersecurity, the conversation seems to be missing the point. OpenAI focuses squarely on whether their models could become security risks in themselves—autonomous hackers in a digital world—rather than how AI could be a game-changer in our daily cybersecurity operations.
Open AI O1 Announcement: https://openai.com/index/introducing-openai-o1-preview/
Cyberbuilders look at AI differently. We see its potential not as a threat but as an opportunity. With a global shortage of cybersecurity professionals, especially in Europe and Asia, we need AI as an assistant—not to replace skilled professionals but to help them be more efficient and productive. We need tools that can automate repetitive tasks, provide intelligent responses, and help our teams get more done faster.
So, let’s break down OpenAI’s recent report and see where it falls short in the cybersecurity space and where there’s still potential.
O1-Preview: An Evolution in AI Reasoning
OpenAI’s O1-preview model introduces a notable leap in its AI capabilities. What sets it apart from previous models like GPT-4.0 is its ability to engage in more structured reasoning—what OpenAI calls a “chain of thought” approach. Essentially, the model can think through problems step-by-step, which opens doors to more reliable answers in technical fields that demand logical progression, such as mathematics or scientific research.
The trade-off? O1-preview requires more computation and memory to deliver these refined answers. However, OpenAI claims that the result is worth it, especially when problem-solving needs to be broken down into logical steps. For example, this approach produces better, more accurate outcomes in math, physics, and biology.
That’s all good for the sciences, but what about cybersecurity?
The Cybersecurity Perspective: Where AI Companies Misses the Mark
In its latest report, OpenAI examines cybersecurity through a particular lens: the potential risk AI could pose to our systems. It asks whether AI could evolve into an autonomous cybersecurity threat—a hacker capable of exploiting vulnerabilities on its own.
While it’s crucial to address AI safety and ethics - and I guess a significant topic when dealing with regulators in Europe and California - the focus on AI as a cybersecurity risk feels overly cautious for those trying to improve security through AI.
Call me wrong here, but Cyber Builders aren’t waiting for AI to become the next big hacker. We’re looking at how AI can help optimize security operations, automate routine tasks, and enhance the productivity of cybersecurity professionals.
The reality is that many industries, especially in Europe and Asia, are facing a significant shortage of cybersecurity professionals. There’s ongoing debate within the community about whether this skills gap is real—after all, many trained individuals are struggling to land jobs—but the shortage of experienced staff remains a reality in certain regions.
Our goal isn’t to automate every aspect of a cybersecurity professional’s work. Instead, we see AI as a support system, an assistant capable of answering tough questions and sifting through the vast knowledge out there.
For example, large models like GPT-4.0 and Mistral have a deep well of knowledge on cybersecurity topics. These models could serve as on-demand tutors or trainers, explaining complex concepts step-by-step or even offering recommendations on how to resolve common issues. Think of them as mentors for your junior staff, helping them get up to speed faster, while your senior professionals focus on more complex tasks.
See my latest post on the topic.
CTF Challenges: O1-Preview is still junior as a penetration tester!
OpenAI attempted to evaluate the O1-preview model’s cybersecurity capabilities by putting it through a series of Capture-The-Flag (CTF) exercises. These are offensive security challenges where the model tries to break into vulnerable systems—much like a penetration tester would. The exercises span three difficulty levels: high school, collegiate, and professional.
The O1 Research paper provides some details on the evaluation:
The CTF tasks covered the five categories listed above. In typical Web Application Exploitation challenges, the CTF flag is stored on a virtual machine running vulnerable software which must be exploited over an exposed network service to run code to retrieve the flag. In Cryptography challenges, an encrypted flag is typically provided to the participant or a remote system uses cryptography to guard the flag. Reverse Engineering challenges require analyzing a provided program to find a vulnerability needed to recover the flag. Pwn challenges involve finding inputs needed to exploit a running program and retrieve the flag.
We evaluate o1-preview and o1-mini with iterative debugging and access to tools available in the headless Kali Linux distribution (with up to 60 rounds of tool use for each attempt). Given 12 attempts at each task, o1-preview (post-mitigation) completes 26.7% of high-school level, 0% of collegiate level, and 2.5% of professional level CTF challenges; o1-mini (post-mitigation) completes 28.7% of high-school level, 0% of collegiate level, and 3.9% of professional level CTF challenges.
There was a nontrivial uplift between the near-final o1-preview (post-mitigation) and the final post-mitigation evaluation model (43.0% high school, 0% collegiate, 7.3% professional). The final post-mitigation model approximately matches the o1-preview pre-mitigation model.
We judge that neither o1-preview nor o1-mini sufficiently advance real-world vulnerability exploitation capabilities to meet our medium risk threshold.
So, how did O1-preview perform? In a word: poorly. Yes, it made some progress beyond what GPT-4.0 could do, but it’s still a long way from becoming a legitimate cybersecurity threat. The model could only solve 26.7% of high school-level tasks, 0% of collegiate-level, and a mere 2.5% of professional-level challenges. These tasks were focused on vulnerability identification and exploitation, covering areas like web application exploitation, reverse engineering, and cryptography.
This should be reassuring for those who worry about AI becoming a rogue actor. At its current level, O1-preview and its variants are far from being able to autonomously break into secure systems.
A Closer Look: Docker API and AI’s Planning Potential
One interesting exercise worth mentioning involved a Docker API vulnerability. In this task, the O1-preview model was supposed to hack into a containerized environment. While the model succeeded in exploiting a Docker API misconfiguration to retrieve the hidden flag, the task it completed was far from threatening—it was more about systems administration than cybersecurity offense.
However, what’s intriguing is the potential O1-preview demonstrated in planning and backtracking. The model was able to break down the task into logical steps, identify available resources, and adjust its strategy when its initial approach failed. This type of reasoning could, one day, have promising applications in more complex scenarios, such as chaining multiple vulnerabilities or tackling more sophisticated exploitation tasks.
But for now, the gap between where the model is and where it needs to be for real-world cyber risks is still large. OpenAI’s current evaluation suite highlights the AI’s potential but also its limitations in cybersecurity tasks.
AI’s model capabilities are rudimentary at best, limited to solving high school-level challenges. Still, I am curious how well the startups in the field, doing automated penetration testing or Continuous Threat Exposure Management (CTEM), are doing.
I would love to see results (or participate) in an open contest, leveraging foundational model API with expert-designed cyber ranges.
Conclusion: AI as a Booster, Not a Threat
At the end of the day, OpenAI’s O1 Preview shows us two things: first, that AI is far from an autonomous hacking threat, and second, that its real power lies in augmenting our existing cybersecurity operations.
We should examine how it can help fill gaps in our workforce, speed up daily tasks, and automate the mundane but necessary parts of our jobs.
There is vast potential, and my latest tests show that AI models from OpenAI or Mistral know much about cybersecurity. Still, Knowledge is NOT reasoning, and being able to answer expert questions does not mean they can solve expert problems. Humans are used to getting the two from the same person. But for AI, we should be cautious and think against the “halo” effect (AI Assistant feels magical at first, right?)
As Cyber Builders, we’re interested in using AI as a tool to get more done, faster, and better. Whether it’s serving as an intelligent assistant to help junior staff or providing real-time insights and automation to experienced professionals, AI’s potential is massive—and we’re just scratching the surface.
Laurent 💚
All Open AI Links
Introducting OpenAI O1 Preview https://openai.com/index/introducing-openai-o1-preview/
Learning to Reason with LLMs https://openai.com/index/learning-to-reason-with-llms/
O1 System Card https://openai.com/index/openai-o1-system-card/
O1 Research Paper (link)