CAG, BSIMM and field-assessed security
One of the best blog posts I read from last week was the "Consensus Audit Guidelines are still controls" from Richard Bejtlich. I really like that he is looking at some suggestions (in this case, the CAG) and pointing that's just controls, there is nothing about measuring the outputs. That goes directly to the heart of the metrics issue, it's still very hard to measure success in information security. For instance, each control from the CAG should also have a related metric to be produced to determine how effective it is.That was based on CAG. Today I also found about the "Building Security in Maturity Model", from Garry McGraw, Brian Chess and Sammy Migues. That's another very nice set of controls (ok, "activities", but we can still see them as controls, i.e., something that needs to be deployed in order to mitigate a risk), produced through a very nice approach (putting together the practices from organizations that are doing well). When I was reading the model description the post from Bejtlich immediately came to my mind. There was another nice set of controls, lacking a good set of measurements to ensure it is actually producing the expected result.Whenever a set of controls is written, be it "guidelines", "standards", or a "model", it should point to which problem it tries to solve and how one should check if it's actually happening. That would helps us to have a clear understanding of what works and what does not work on information security. Besides the fact that most of those frameworks/control sets are developed according to a different scope, there's really no way to measure which ones are more effective.Honestly, except for compliance requirements, how would you answer an auditor if he asks "why did you choose this specific control framework to develop your security program?"?