Hoxhunt published a report last month called AI-Powered Phishing Outperforms Elite Red Teams in 2025. It was released in full as a blog post, so read away. No download required, no registration, just click and you’re in. The core assertion in the report is that over the past two years, AI-powered phishing has become more effective at getting a user to click a phishing link than human-written phishing messages do. Here’s the chart:

Look at the “effectiveness rate” in the first two data rows – AI (AI-generated phishing messages) goes from 2.9% in March 2023 to 2.1% in November 2024 to 2.78% in March 2025, and human (phishing messages written by an elite red team) goes from 4.2% to 2.3% to 2.25% over the same three time periods. Data row three calculates the relative differences … from AI being 31% less effective, to 10% less effective, to 23.8% more effective … for a 55% improvement in effectiveness over the three time periods.

Hoxhunt says:

• The absolute failure rate metrics are less informative than the relative performance between the two.
• As its AI models improved, the attacks became more sophisticated and harder to detect.
• It’s only a matter of time until AI agents disrupt the phishing landscape, elevating the current effectiveness rate of AI-powered mass phishing to AI-powered spear phishing.
• Organizations should cease-and-desist on compliance-based security awareness training and embrace adaptive phishing training. Hoxhunt offers the latter.

We say:

• Neat research project. We love the emphasis on pushing the boundaries of how AI impacts phishing in a longitudinal study.
• The absolute failure rates above are actually interesting to us – in addition to the relative change. In terms of absolute failure rates, for human-written phishing messages, we read data row two as saying that people have become almost twice as good (failure rate almost halved from 4.2% to 2.25%) at detecting human-written phishing messages from March 2023 to March 2025. Given the data is drawn exclusively from people trained by the Hoxhunt security awareness training platform, that’s interesting.
• For the trend line in the AI phishing data row, it dropped significantly then jumped again – to almost but not quite as high as the March 2023 rate. So … the March 2023 rate set the high water mark, but people have become better at detecting AI-written messages over the three time periods. If Hoxhunt does another comparative study in 6 months, that data point will be the most interesting one to us. Do AI-generated phishing messages increase in effectiveness against people (e.g., a rate of >2.78%) or do people get better at detecting AI messages (e.g., a rate of <2.78%). This study tested how threat actors could use AI agents to write better phishing messages, but in parallel, non-threat actors are also using AI to write better emails in general. This should lift the quality of communication for all and sundry, so does the change in both smooth out the differences making detection more difficult, or does the increased prevalence of using AI to create near-perfect emails throw off signals that AI was involved.
• It would be even more interesting to have done the same study with another cohort – those trained using what Hoxhunt calls “compliance-based security awareness training” programs.
• In describing the methodology, Hoxhunt says “The experiment involved a large set of users (2.5M) selected from Hoxhunt’s platform, which has millions of enterprise users, providing a substantial sample size for the study” and “the AI was instructed to create phishing attacks based on the context of the user” (e.g., role, country). This is why data breaches are such a menace to the current and future phishing landscape – where threat actors aggregate data breach records to create profiles of potential targets and use AI agents to craft profile-specific phishing attacks.

What do you think?

Here’s some notes on our briefing with Paul Mander (GM, Optery for Business) of Optery. The briefing was organized by the Optery team. 

Key takeaways from the briefing and some additional research:

• Optery has been in business for four years. There’s a consumer play and a business play. Paul is the GM of the business side.
• The key idea behind Optery is the removal of unwanted, unwarranted, or excessive personal data held across the internet, most of it due to surreptitious data sharing agreements between a site where an individual has an account or presence and a host of data aggregators and brokers. This sharing of data results in the creation of personal and sensitive data sets on individuals – name, address, contact details, etc. – being more widely available than what the individual has directly authorized. Optery says they find an average of more than 100 profiles per person.
• On the consumer side, individuals can sign up to Optery’s service for free. Optery periodically scans for personal data held by data broker companies (at over 330 sites at the time of the briefing) – and can scan on demand for a consumer, too. If data that the individual wants to control is located on any sites they haven’t authorized, Optery will send removal requests on behalf of the individual. While there is no fee for periodic scanning, there is a fee for instantiating the data removal process. Pricing for removal is graduated by the number of sites and the approach taken by Optery. See Optery’s site for consumer pricing and details, including the list of data broker sites covered by removal requests (depending on pricing tier).
• On the business side, Optery thinks about employee PII as an “attack surface” – that the ability for someone to gain knowledge about employees based on data they / their business hasn’t authorized for release and aggregation, creates opportunities for assessing weaknesses for cyberattacks (e.g., phishing, BEC, voice phishing) and physical attacks (it’s an increasingly dangerous world). Within the growing realm of data privacy regulations and compliance mandates that cover an increasing set of companies, being proactive about data removal from unauthorized sites reduces this attack surface. 
• We really liked the idea of proactively reducing the amount of unauthorized data available on individual and employees – data that can be used to build profiles for cyberattacks (such as phishing) or physical threats (if they know where you live, they can harass / stalk / harm you … or your family for use as leverage against you).

For more, see Optery.

Key findings per the report:

• On receiving the phishing simulation message, 10.4% of all recipients clicked the link. This opened a web page that masqueraded as a valid site and asked for username and password details. Of those who had clicked, 65% entered their details and lost their credentials. Here’s one of the diagrams from the report.

• Aaarrgghhh.
• Per Fortra, “Phishing links don’t click themselves – human beings, however well-intentioned, do.“
• Click rates varied by industry – education was worst (16.7% vs. 10.4% average), finance was best (6.3% vs. 10.4% average). There’s a full breakdown in the report.
• The percentage of recipients-who-clicked-the-link who then submitted their password also varies by industry. Education takes worst place again – 72.8% of those who clicked lost their credentials. Finance is third from best, at 45.2%. Agriculture and food were in first place / best place – at 29.1%.
• A decade ago, the Verizon 2013 Data Breach Investigation Report said this about the mathematics of phishing: sending 10 phishing messages almost guarantees a click. Put another way, 10%. Page 38 of the VDBIR 2013 has this box:

• A decade later, click rates remain the same or are slightly worse.
• Yes, users need to be trained – especially as threats become more sophisticated due to AI, phishing toolkits, MFA bypass as routine, etc. Don’t stop doing that. But … revisit / reassess / recheck the efficacy of whatever technical protections you are using and keep those phishing and BEC emails as far away from a user’s inbox as possible.
• On that note, you should read our report on the role of AI in email security.

Organizations have insufficient cyber insurance coverage

CYE, a cyber risk quantification firm, calculated that the average organization is only covered for 25% of the financial risk associated with a cyber breach. The data comes from external (not CYE) and internal (CYE) sources. CYE said that 80% of organizations lack sufficient coverage, and the maximum insurance gap is 3000% – which would mean that the organization is only covering 0.03% of its total cost due to a data breach. 

The associated report (it’s not in the press release) says that the gap could be “in the ability or willingness of cyber insurers to provide adequate insurance coverage,” which is the major trend line we’ve seen the most due to the spate of high-cost ransomware incidents that decimated insurance company profits. We agree that it could also be due to an inability to quantify risk internally, but would see the insurance market dynamics as the major influence. Regardless of the impact on insurance policy issuance, the value of risk quantification is that it shows what an organization is not covered against from an insurance POV, thus highlighting to senior leadership the urgent work that needs to be done to actually improve security posture – not merely shift the cost of a breach to a third-party. Press release Report

MDRs are failing to help enough; a better approach is needed

A study by Radiant Security found significant shortcomings in MDR (managed detection and response) services, such as a lack of context about their environment (34%), more escalations than a SOC team can handle (32%), and a long time frame for remediating incidents (44% take more than 4 weeks per incident). Also, 70% of respondents said they are saving only 25% or less of their time after outsourcing to an MDR service – which doesn’t line up with the outsourcing value proposition. Radiant says it’s time for a new approach, and it has some ideas on what organizations should be doing instead. Press release

Microsoft and Google continue to top the list of most impersonated brands

Microsoft and Google continue to be in first and second places respectively as the most impersonated brands used in phishing attacks, followed by LinkedIn, Apple, and DHL in 1Q 2024. Amazon brand impersonation has dropped from 4Q 2023 to 1Q 2024, and Airbnb has made a first time appearance in 1Q 2024. Be careful what you click on peeps! InfoSecurity Magazine

Date: July 22, 2021

Focus of the Webinar

Watch this hour-long webinar where Michael Sampson, Senior Analyst at Osterman research reviews the results of their recent survey of 130 cybersecurity professionals, How to Reduce the Risk of Phishing and Ransomware.

 You’ll learn how organizations view their security posture, including:

• Organizational effectiveness against various threats
• Most popular security incidents
• The concerns keeping security teams up at night
• The most-pressing ransomware concerns
• The capabilities to handle different threats

Register to view the recording

Date: June 30, 2021

Osterman Research conducted a brand new and independent study on the rise of phishing and ransomware attacks. 130 cybersecurity professionals were interviewed.

With new strains of ransomware and malware threats on the rise, your organization and data is continually at risk. Watch this webinar to learn how you can reduce your exposure to these threats, including:

• The most effective mitigations against phishing and ransomware attacks.
• Five pillars of phishing and ransomware prevention.
• Best practices to reduce the risk of phishing and ransomware.
• Recommendations for improving security and steps to take today.

Register to watch the recording

This webinar will present insights from a new, in-depth survey of IT and security professionals at large enterprises that use Microsoft 365 for their corporate email. Covering organizations in both the US and UK, the webinar and survey report (a copy of which will be provided to all webinar participants) will give IT and security professionals, from CISOs to email admins, the opportunity to benchmark their own Microsoft 365 security posture and planning with answers from their peers.

Michael Osterman will cover views expressed on issues like security performance, the most prevalent models for SOCs, SIEM integration, how the migration to Microsoft 365 was managed, types of additional security they’ve deployed, time spent remediating breaches, security budget growth, and how users report suspicious emails, along with expectations on new defensive technologies and services.

Register to watch the recording

Sponsored by Agari, Carbon Black, GoSecure, KnowBe4, MDaemon, Proofpoint, SlashNext, Trend Micro, Trustwave and Zix

Executive Summary

The “network perimeter” today is almost non-existent. Almost all organizations operate a large and growing number of cloud services for mission-critical and non-mission-critical purposes, sometimes just at a departmental level (one source estimates that there are nearly 1,200 cloud services in use in the typical large enterprise and that the vast majority of these are not “enterprise-ready”). Mobile devices – many employee-owned – are regularly used to access corporate data resources and sensitive data assets. These devices typically contain a large number of apps, many of which can be exploited to steal login credentials and other sensitive information. IoT devices are now commonplace and the number of these devices in the workplace is skyrocketing, employees continue to use conventional endpoint devices like desktop and laptop computers, and the “Bring Your Own” trend has expanded from personally-owned and managed devices (BYOD) to personally-owned and managed cloud, mobile and desktop/laptop applications of many types.

In short, the network in most organizations has a dramatically expanded attack surface. There is no longer a defensible perimeter that can fully protect corporate data, and so new approaches, technologies and practices are needed to protect corporate data and finances.

Request a Copy

This webinar will present insights from a new, in-depth survey of IT and security professionals at large enterprises that use Microsoft 365 for their corporate email. Covering organizations in both the US and UK, the webinar and survey report (a copy of which will be provided to all webinar participants) will give IT and security professionals, from CISOs to email admins, the opportunity to benchmark their own Office 365 security posture and planning with answers from their peers.

Michael Osterman will cover views expressed on issues like security performance, the most prevalent models for SOCs, SIEM integration, how the migration to Microsoft 365 was managed, types of additional security they’ve deployed, time spent remediating breaches, security budget growth, and how users report suspicious emails, along with expectations on new defensive technologies and services.

Register to watch the recording

Microsoft 365 is a capable and robust platform with a wide collection of features and functions. Like any large platform with diverse capabilities and a diverse user base, customers must analyze if it provides the depth of capability or specialized functionality they require in areas like security, archiving, authentication, eDiscovery, encryption, and file sharing.

To help IT and security managers identify such features or performance gaps, Osterman Research has just published “Why Your Company Needs Third-Party Solutions for Microsoft 365.” 

Join Michael Osterman, Principal Analyst, Osterman Research will review the results contained in the report of detailed research into Microsoft 365’s functionality across several areas. 

Takeaways will include:

• How prevalent it is for IT organizations using Microsoft 365 today to contract additional security, archiving or other capabilities
• Limitations in the embedded security included in standard Microsoft 365 packages 
• Considerations in evaluating Microsoft’s Advanced Threat Protection module
• What is needed to make Microsoft 365’s eDiscovery capabilities really work for you
• What particular issues must be understood by companies with hybrid environments

Register to watch the recording