The Evolution of Cybersecurity Testing: Pen Testing, BAS, BAS+, & AEV

No one questions the need for organizations to have robust methodologies to assess their cybersecurity defenses. This is a known fact. The broader question is, what is the most effective way to test your cyber defenses? Traditional pen testing, while useful, has significant limitations in measuring the effectiveness of an organization's cyber defenses, fueling innovation of more advanced tools/methods like Breach and Attack Simulation (BAS), Breach and Attack Emulation (BAS+), and Adversarial Exposure Validation (AEV). This paper compares each methodology to highlight its unique characteristics, benefits, and limitations.

Additionally, this article compares traditional cybersecurity methodologies with newer advanced threat-focused techniques and their fit within Gartner’s Continuous Threat Exposure Management (CTEM).

Overview of Cybersecurity Testing

Each approach and technology has a distinct approach and purpose in cybersecurity testing:

1. Penetration Testing: A manual and time-bound process where security professionals attempt to exploit vulnerabilities within a system to assess security weaknesses.
2. Breach and Attack Simulation (BAS): An automated testing approach that simulates cyberattacks using a sequence of simple, predefined atomic tests to assess whether security controls can detect or block specific actions.
3. Breach and Attack Emulation (BAS+): A more advanced form of BAS that emulates real-world adversarial techniques in a more dynamic, contextually aware manner. BAS+ emulates ‘real-world’ adversarial attacks by adapting behavior based on the environment and context, providing a more accurate understanding of security posture.
4. Adversarial Exposure Validation (AEV): A proactive, continuous approach that leverages adversarial emulation to actively test and validate an organization’s defenses against complex, evolving threats. AEV provides critical, ongoing insights into an organization’s resilience to thwart cyber attacks (i.e., phishing, malware, ransomware, or insider threat) to measure preparedness and improve security posture.

The following table offers a detailed comparison between these approaches:

Comparative Analysis

1. Pen Testing: Snapshot of Vulnerabilities

Penetration testing, though widely used, provides a static snapshot of an organization’s vulnerabilities. Pen testers use manual and automated methods to exploit weaknesses, giving companies insights into specific vulnerabilities at a given point in time. However, pen testing lacks the ability to simulate the full lifecycle of an attack, and because it is time-bound, it does not account for new threats or evolving attack techniques that may emerge soon after the test. Additionally, pen testing typically ignores the deployed security tooling in place to manage threats.

While pen testing remains essential for certain regulatory compliance and identifying traditional vulnerabilities, it does not provide an ongoing measure of an organization's resilience to adaptive adversaries.

2. BAS: Automated, but Limited in Scope and Realism

Breach and Attack Simulation tools are designed to automate attack sequences against an organization’s security controls, providing a baseline understanding of detection capabilities. However, BAS often uses atomic, serialized tests that lack the complexity and contextual adaptability found in real-world attack scenarios. For example, BAS tools may perform predefined tests like simulating malware execution or command-line abuse but will not dynamically adapt based on observed defenses.

While useful for continuous validation of security controls, BAS tools do not fully simulate an adversary’s tactics, techniques, and procedures (TTPs) in a way that reflects advanced threats, limiting its effectiveness in measuring security posture.

3. BAS+: Moving Towards Realism with Dynamic Emulation

Breach and Attack Emulation (BAS+) improves upon BAS by incorporating real-world threat emulation that is more adaptable to the environment. BAS+ offers a sequence of threat actions that can dynamically adjust based on the security posture of the environment. This creates a more accurate simulation of an adversary’s behavior, making it a better representation of the threats organizations face.

4. Adversarial Exposure Validation (AEV): The Gold Standard for Cyber Fitness

Adversarial Exposure Validation (AEV) takes security assessment a step further by emulating adversarial behavior across the entire attack lifecycle with continuous adaptation to real-world intelligence. AEV actively challenges defenses based on the latest threat landscape, simulating sophisticated TTPs and providing ongoing insights into an organization’s resilience. By using frameworks like MITRE ATT&CK, AEV identifies weaknesses across Initial Access, Execution, Persistence, and other phases, providing actionable insights for continuous improvement.

AEV’s ability to adapt to evolving threat intelligence and mimic sophisticated attackers makes it the ideal solution for organizations looking to achieve high cyber fitness. The methodology allows security teams to track improvements over time, prioritizing risks based on their potential business impact, and enabling them to proactively address gaps in security posture. Expect augmented AI and LLM enhancements to drive automation in the near to mid-term.

Conclusion

The evolution from traditional pen testing to AEV reflects the growing need for proactive, continuous, and contextually aware cybersecurity assessments. While pen testing and BAS provide foundational insights, they lack the sophistication needed to assess resilience against modern, adaptive adversaries. BAS+ offers an improvement but still falls short of comprehensive threat validation.

Adversarial Exposure Validation (AEV) represents the next step in cybersecurity testing, providing dynamic insights into an organization’s ability to withstand real-world threats and adaptive adversaries. By continuously challenging security controls and adapting to the latest adversary behaviors, AEV equips organizations with the necessary insights to strengthen their defenses, enhance resilience, and achieve a more robust security posture. As the threat landscape continues to evolve, AEV will be an essential tool for organizations committed to proactive defense and a high level of cyber fitness.