Threat Hunting Team Maturity Model
After talking about scaling a hunt team with the concept of "Pack Hunting" in my last post, now I want to approach a topic that has been weighing on my mind as the year comes to a close.
How do I build a kickass hunt program? And how would I know if I did?
Admittedly, this is on my mind because next year my program will have been around long enough to be audited, which means I need to be able to measure my program in an audit-able way. I am not talking about the outcome of the program here, the hunting itself, but rather the processes and procedures that lead to the outcomes. The outcomes are currently measured using KPIs, but I haven't stress tested those enough to share yet.
Developing a Threat Hunting & Research Team Maturity Model
Why a maturity model?
As I looked into how to approach this question I came across the idea of using a maturity model. According to the Institute of Internal Auditors (IIA), a maturity model describes process components that are believed to lead to better outputs and better outcomes. Maturity models are perfect for highlighting continuous process improvement, which makes them well suited for assessing a program that is currently still under development. It also provides an easy starting place for auditors of a new and developing field like threat hunting.
How do you design a maturity model?
To learn more about the necessary components of a maturity model, I read "What makes a useful maturity model? A framework of general design principles for maturity models and its demonstration in business process management" by Jens Pöppelbuß and Maximilian Röglinger.
The authors describe a framework of three design principles that build upon one another based on the purpose of the maturity model.
Relying on their framework, I've decided to try my maturity model at the basic level for now and seek to advance it in the future to descriptive and later prescriptive. I am trying to make sure that the final product can indeed hold up to an audit, so I am purposefully limiting this first attempt.
What goes into a threat hunting maturity model?
I started with the attributes first. What was needed for a world class hunt program? The answer came pretty close to the cliche - "People, Technology, Process"
Talent - I am going to need to be able to recruit, retain and train hunters and huntresses.
Data - Not quite technology, but close. The idea here, I don't want my team tied to specific technology, but we will need specific data to accomplish our mission.
Methodology - Our process and procedures used to complete hunts.
Metrics - Still have to measure outcome of a program consistently and accurately, so can't leave this one out.
Then I rummaged the internet for some good names for my levels, which for me had to start with 0. For labels like these, I suggest just finding words you believe will resonate with your leadership.
Level 0 - Initial
Level 1 - Reactive
Level 2 - Defined
Level 3 - Repeatable
Level 4 - Integrated
Level 5 - Optimized
I purposefully tried to make my model from 0-5. I felt it would help make the leap from one level to the next more manageable, but if you can't come up with enough in your model to fill it in then just reduce the levels.
Next came the hard part, filling in the model. Full disclosure - I spent days staring at an empty table then switching tasks when I would come up empty handed, this part was a struggle for me.
To move forward, I started out adding characteristics of the attributes I thought were indicators of a kickass program.
Talent
For talent, I decided on a few areas I thought showed robust talent management.
Recruting
Training
Career Development Paths
Roles and Responsibilities
Succession Planning
Data
Data needed completely different characteristics.
Visibility
Quality
Automation
Tools
Data Science Integration
I want to briefly explain what I mean by visibility and quality as for my team they have specific meanings.
Visibility is a combination of coverage, retention, and reliability of the data sources.
Quality includes the usability, normalization and standardization of the data.
Methodology
This one was a little easier to poach ideas from pre-existing process maturity models.
Consistency
Documentation
Communication
Relevancy
Metrics
Plenty of models incorporate metrics, so find what includes the characteristics your management and auditors care about and tailor it.
Consistent reporting
Accuracy
Building your own Threat Hunting & Research Team Maturity Model
Chances are this model isn't going to be perfect for your team. So I have uploaded the slide for you to easily edit it.
I also created a worksheet of the questions I used when creating the example above. - 6 Questions to Guide your Maturity Model Development.
One last note, if you are considering building your own model to assist in future audits be sure to document your model development process, your references, and resources. This documentation will be instrumental in showing auditors the thought process behind your model in the absence of an industry recognized one.
Happy Hunting,
K