.png)
Everyone knows there are threats out there hell-bent on destroying our organizations. Innovative businesses everywhere are taking a risk-based approach to prevent mission compromise. This approach involves leveraging a risk assessment framework as part of the cybersecurity program.
Risk assessments help you balance your risk exposure with your organizations’ risk appetite. The return on investment of a risk assessment is only as good as the quality of work you put into it. A third-party perspective designed to challenge internal assumptions is critical to creating an objective result.
Penetration testing and red teaming are tools used to inform the risk assessment. These tools provide a unique perspective through pseudo-exploitation. The entire point of pen testing is to identify gaps in cyber-defenses, much like an actual attacker will.
One common pitfall can completely rob you of the ROI you otherwise receive with quality penetration testing and red teaming. I am speaking to the Achilles heel of pen testing engagements: Feedback loops.
Feedback Loop Defined:
What is a feedback loop? A feedback loop is a cyclic process where testing uses the same input and scope repeatedly, getting pretty much the same results. The perceived attack surface and the actual attack surface may be different, leaving considerable gaps in vulnerability detection. The fact is you can't defend what you don't know about, and your adversaries don't follow the rules. 
"That's out of scope!" - Said no attacker. Ever.'
- Drew Porter
'Never underestimate the creativity of a desperate adversary.'
- William Suthers
You cannot rely on processes that do not align with your actual threat landscape realities.
Don't get me wrong. Penetration testing and red teaming can be effective tools of risk management. However, before defining your scope, conduct an objective attack surface analysis. Your organization is an ever-evolving machine driven and managed by a mix of humans and technology, which must evolve to remain competitive and ahead of the evolving threat landscape.
Things change, so shouldn't the testing scope evolve as well? The answer is yes. Risk assessments, the scope of penetration testing and red team engagements should continually develop based upon an informed third-party perspective discovery process. When we don't constantly evolve, we find ourselves trapped in a feedback loop, questioning why we've lost the value we once obtained from cybersecurity pen-testing operations.
Feedback Loop Perpetuation
- An organization requires or desires security testing to inform the risk assessment
- A testing team resource is identified to conduct security testing
- The engagement scope is based upon what the client knows or thinks they know about their attack surface, which may or may not initially reflect the actual attack surface, but over time will not
- Most trusted third-party assessment firms or teams don't conduct intelligence-driven operations as part of offensive security operations, leading to gaps of risk exposure
- Testing does not reflect real-world attacks
- Any risk exposure identified is managed
- Potential unmanaged attack surface remains
- The risk assessment is misinformed
- The cycle then repeats where one or both parties perpetuate the process by asking the assessed entity for testing scope, which naturally is a repeat from the last assessment of the target environment
- A feedback loop perpetuates
Is The Industry Failing You?
Here is a pattern Truvantis often sees while intaking new clients.
- Truvantis intakes a new client looking for an alternative perspective
- The client purportedly has a mature program with regular clean security assessments and sees no security incidents or is breached repeatedly
- The client provides external penetration testing scope of seven targets
- Intelligence operations uncover a potential 5x larger attack surface
- The client is surprised at the expanded scope
- After some discussion and approval, the modified testing scope leads to the identification of critical issues often leveraged by real-world attackers
- The client mitigates newly found vulnerabilities benefitting overall business resiliency
Protecting Your Organization from Feedback Loops
If your organization is starting to experience a pattern of repeated behavior with a limited value from security testing efforts, you might be a victim of pen testing feedback loops. Your process could likely benefit from a new surface attack analysis and threat landscape gap assessment specific to your business environment.
Our Approach to The Problem
When you select Truvantis as a trusted third-party security partner, you get intelligence-driven operations designed to uncover vulnerabilities associated with real-world risk exposure. Truvantis cybersecurity engagements include Attack Surface Analysis and comprehensive, full-spectrum testing:
- Pen testing engagements use creative blended attacks like real-world adversaries
- Red team engagements, designed to measure security control effectiveness, escalate until the red team gets caught.
The Truvantis approach gives insights into issues that impact operational security objectives. Key decision-makers are then empowered to make well-informed decisions. Teams are empowered to focus on mission objectives. 
Truvantis can help you build a program that prevents feedback loops. We'll work with you to create a program customized to your unique business. Contact Truvantis today for a consultation.
See my blog, "The 0 Day in the Room Nobody is talking about Scope," on scoping pen tests for more discussion on this topic.
William Harvy Suthers III
CISSP, OSCP, OSWP, CPT, CEH
William Suthers is the Director of Technical Services at Truvantis and an open-source security tool developer, security researcher, and security conference speaker, including HushCon and DEF CON.
William has over fifteen years of security assessment and consulting experience with various industries, including security, SCADA, healthcare, government, military, small and large businesses, fiduciary and non-profit. He is certified by numerous security bodies, including Offensive Security, International Information System Security Consortium (ISC)², the Information Assurance Certification Review Board, and the EC-Council. His certifications include CISSP, OSCP, OSWP, CPT, and CEH.
