Osterman Research

Cobalt published its sixth annual report on pentesting last month (May 2024). As a company that offers pentesting as a service, Cobalt is well-positioned to leverage its aggregated data set to report on trends and findings year-on-year. The report complements Cobalt’s internal data with a large survey of cybersecurity professionals in the United States and United Kingdom.

Key findings from the report that were of interest here:

• Cobalt conducted 4,068 manual pentesting engagement during 2023, up 31% from the 3,100 it conducted in 2022. With 400 specialist pentesters on call, this averages out at 10 per pentester per year.
• Cobalt listed several reasons why pentest numbers increased: new regulatory compliance requirements, broadening of the attack surface, AI-generated code, the ongoing skills gaps at organizations, and budget reductions.
• AI is one of the major trends covered in the report. There are several concerning conclusions based on Cobalt’s observations. First, tools that increase the speed of software development (including AI features) lead to an increase in the number of security vulnerabilities found, NOT to better quality software. Second, in the rush to embrace “all things AI,” security measures are often overlooked during implementation and during the subsequent changes as models learn. Third, 70% of respondents indicated they had seen evidence of external threat actors using AI to increase the quality and severity of cyberattacks.
• The number of CVEs identified and catalogued in 2023 increased by 15% over 2022. The number of security findings discovered per Cobalt pentest engagement increased 21% in 2023 versus 2022. Some of this will be due to the increased number of CVEs, but not all of it. Cobalt’s pentesters appear to have higher efficacy at finding additional vulnerabilities, possibly due to reduced software quality via AI, better tooling from Cobalt, or more experience versus 2022.
• Large language models (LLMs) need to be tested. Cobalt offers this is a newish service. The three most commonly found vulnerabilities during LLM pentesting engagements in 2023 were prompt injection, model denial of service attacks, and prompt leaking where sensitive information is inappropriately disclosed.
• Organizations are taking longer to fix identified vulnerabilities and are fixing fewer of them, too. This net-nets to unaddressed vulnerabilities creating opportunities for compromise, breach, and other types of attack for a longer period of time – which is good for no one except threat actors.
• Layoffs and budget cuts have a devastating impact on software quality and vulnerability mitigation, along with the physical health and mental wellbeing of remaining staff (with C-level respondents indicating an even higher set of negative outcomes).

For more, get your copy of Cobalt’s report.