+/- 40% accuracy and we think it's good?

I was caught by surprise when I was reading Matthew Rosenquist post on the IT@Intel blog by this information about the OCTAVE methodology:

"I have observed the accuracy to be +/- 40% in complex organizations.  I believe this is largely due to multiple tiers of qualitative-to-quantitative analysis and the bias introduced at each level.  Credible sources have expressed a better +/- 20% accuracy for smaller implementations."

Even if Matthew is defending the use of the methodology, these are very strong numbers for me. I cannot see how a methodology with this level of accuracy can be much better than some quick and dirty threat and impact assessment, at least for getting support information for a security strategy definition.

I was always a very big fan of risk based methodologies and frameworks like ISO27002. However, they all seem to suffer from a "first steps syndrome", they are extremely hard to be put in motion and it takes a long time before they start to be effective. Eventually, after a couple of years, you'll start to get some good results. But until you get there you're probably exposed and have some serious gaps on your security posture.

This is not just the case of fixing the urgent gaps first and then starting the everything "in the right way". The gap fixing will become a neverending firefighting and will suck time and resources needed for the big stuff. What we need now is a way to reach a desirable end state by a series of actions that will solve immediate issues while staying in the path for that. And how is that possible?

I'm still not sure, but I'm trying to put something together in that way. That would include:

• More prescriptive directions (like PCI-DSS)
• Quick and dirty, facts based threat assessment
• Actions prioritization based on immediate outcome, reach (threats and assets related) and increasing value over time
• Outcome based metrics

NMAP 5 released

It's kind of stupid to post it in yet another blog, but this will be just a quick note to mention the new NMAP version and also point to a very good post on the SecuriTeam blog about what's new in the new version. A very good summary.

Dunbar's number and security

I've just finished Malcolm Gladwell's book The Tipping Point. As usual, Gladwell's books always bring food for thought on security for me. Security is deeply related to human behaviour, the main subject of his books. The most interesting thing from TP for security is the Dunbar's number. Honestly, when I read about it I thought I've found something like the famous 42, but it was, in fact, some serious and important stuff for our field.

The basic concept on Dunbar's number is that people has a limit for the number of people with whom they can maintain stable social relationships. The actual number, 150, was found in several independent studies, including some new ones about social networking websites like Facebook. The implications of this "hard-coded limit" goes beyond the number of "friends" you can have, as it also relates to the number of people that you can interact while maintaining a personal context, the maximum number of people you can put together as a cohesive group, the list of implications is huge.

It's easy to extrapolate it to security. I can clearly see how it would impact Security Awareness initiatives. It's common to see those initiatives trying to use people as champions for their work groups and departments. The Dunbar's number can be used as rule to define how many champions are necessary and for what groups. It can also be used to define processes around access verification and entitlement review, as we can probably expect that a manager won't be able to effectively answer for "need to know" characteristics of a group bigger than 150 people.

Of course, all these theories need to be tested. However, we must always remember that systems are not only systems to be secured, they have a purpose and they need to perform properly. People are not just "users" they are also human beings. Information is not only data to be protected, it has an infinite range of meanings and context. All research and findings about the Dunbar's number and its applicability into Information Security is just another example of why is so important to security professionals to constantly go through other fields looking for useful information.