Thinking like an attacker
I'm interested in the gap between how a system is supposed to work and what its parts allow it to do.
Useful to others as-is. Tested ideas, working code, real findings.
“Think like an attacker” comes up a lot in security advice, but I find it more useful when I can turn it into a question: what does this system allow me to do that its designers weren’t expecting?
The interesting part is often how features interact. A component can behave as intended on its own and still cause trouble when another component interprets its output differently.
For example, a parser might produce text that a renderer treats as executable content. Two permissions that seem limited separately might combine into a way to gain more access. An AI system might treat instructions found in a document as though they came from the person using it.
Those examples need different technical knowledge to investigate, but they share a habit of looking at the relationship between parts. That’s why I connect this approach with systems thinking, including Donella Meadows’s work on how structure influences behaviour.
How I try to use it
I start by working out how the system is intended to operate. What can each user do, which components are meant to trust each other, and what assumptions make the design seem safe?
Then I compare that picture with the code, configuration, and behaviour I can observe. If the two pictures differ, I have something specific to investigate. The difference might be harmless, but at least I can explain why I’m testing it.
CTFs give me a place to practise this because the challenge often depends on spotting an assumption that doesn’t hold. Fuzzing can help explore unexpected inputs, although finding a crash or unusual response still leaves the work of understanding what it means.
Threat-modeling this site asks similar questions from the defensive side. I can look at what each component trusts and consider what would happen if that trust were misplaced.
I’m also interested in how far the habit transfers. Organisations, games, and markets all have interactions that aren’t obvious from looking at their parts separately. That’s one reason I’ve been trying out connections between security research and probability, while keeping in mind that a useful analogy still needs to be tested.
Paths that lead here
- Feeling behind in cybersecurity · Comparing how a system is meant to behave with what it actually does gives me somewhere to start, although I still need technical knowledge to make sense of what I find.
- How I think about this site's security · This is a practical example of examining how the parts of a system fit together.
- How much time should I give a bug hunt? · That's something I want to notice in how I investigate a system, whatever framework I use to plan my time.
- Making HackTheBox easier to start · It connects with looking closely at where a process breaks down.
- Understanding post-quantum cryptography · That's what connects this to thinking about an attacker's options.
- Understanding trading edge and risk · That's one of the reasons it interests me as a system with competing participants, rather than a puzzle with a permanent solution waiting to be found.
- Why I love hero shooters · That's one of the connections I make with security and systems thinking.
Where this note points
- How I think about this site's security · What I want to protect on this site, the choices I've made, and the risks that still need attention.
- How much time should I give a bug hunt? · I'm wondering whether the Kelly criterion can help me think about research time, and where that comparison breaks down.
More from these beds
- Learning to explain things better · Someone told me I wasn't a good explainer, and it made me think about how often I talk past people.
- Why searching online feels tiring · It takes effort to find a useful answer when so many pages sound helpful without telling you very much.
- Why the internet sometimes feels fake · Bots and repetitive content can make the web feel strange, but I don't want suspicion to become my default response to everyone.