Security generalists (and QSAs...)

This post is not supposed to be a rant about PCI DSS and the quite common low-qualified QSAs that make hell of the life of those pursuing compliance validation. Although it evolved from that, it’s now just an understanding from my part about the role of generalists in Information Security.

They are the glue. But more about that later.

I’ve been working through a PCI validation assessment and during a discussion of findings with the QSA I realized that, in a room full of people (and more than one QSA), no one was really understanding the requirements that were being discussed, their intent and what would be the alternatives that could be acceptable as compensating controls. It was all around custom applications development, so requirements 6.3 to 6.6.

PCI DSS includes a bunch of requirements for secure development of custom applications. There are items for adding security considerations in the early phases of development, doing code review, security functionality testing and vulnerability scanning (not mentioning secure coding itself). My personal point of view is that it’s too prescriptive (a recurring criticism about PCI DSS), where maybe the best thing to have would be some outcome based requirements. After all, what we want are secure applications. Or a better description, applications that can’t be exploited for unauthorized access to cardholder data.

An issue with all the prescriptive requirements is that they force people involved to understand a SDLC. They need to understand exactly what is a code review, functionality testing and vulnerability scanning. Without that you’ll see discussions where those definitions are used interchangeably and just make the assessment messy. If the QSA is one of those who can’t understand the differences, it gets VERY messy. Is that because he is a bad QSA? Yes, from a blunt point of view, as the QSA should be able to understand what he needs to check, but I think we are not being entirely fair with those professionals.

What’s the required background for a QSA? If it’s a guy who used to work with Network Security, then went through the QSA training and passed the exam, is he ready for any assessment? Unless he is one of those curious and ever-learning minds, it’s not a shock if we find he (and other auditors and security professionals in general) is completely ignorant in big pieces of the body of knowledge (BOK) required by his function. How can that happen?

One of  the key answers is how security professionals obtain their credentials. Different than engineers, lawyers and  doctors, we are not required to get a degree in Infosec and sit for a board/college exam. It’s no different than many IT related jobs, but there’s a catch. We are simultaneously asking people to have a minimum level of knowledge in a number of disciplines and not requiring them to prove that they achieved that.

But what about the certifications? CISSP? The QSA test?

All of them will (at least in theory) cover everything, but will gladly allow someone to pass without a clue about pieces of the BOK. There’s a minimum pass mark, but in almost all those credentials exams there is no minimum mark per knowledge domain. So, you can ace the network security piece and go blank the secure development part, and it’s still ok. The obtained credential, however, still implies that you have that minimum skill level in that domain you couldn’t answer a single question.

I’ve seen that multiple times. CISSPs that couldn’t even understand firewall rules or don’t know what an application vulnerability looks like. It is the same thing in the QSA training, so you’ll end up with someone that needs to assess if an organization is doing security functionality testing  but doesn’t even understand how that is different from code reviews.

Civil Engineers, for example, can’t become engineers if they can’t achieve a pass mark in Solid Mechanics. Having to sit through (and pass with a minimum mark on) individual courses that compose the Engineering BOK ensures that no critical gap will exist in an engineer formation. It’s not perfect, of course, but it’s far better that the unrealistic assumptions of minimum skills we currently have in Infosec.

That’s where the Infosec generalist comes to the stage. There are several roles in our field that must be filled with people with minimum skills in each piece of our BOK. QSAs are just one example. If we want to get rid of those “how can he ask something so stupid” moments (ok, reduce…there’s no patch for stupid), we must start forcing people in (or trying to get in) those roles to reach minimum levels on all BOK domains. Let’s change the CISSP credential (or create a new one), for example, forcing the candidate to reach a minimum score on all domains. Same thing for QSAs, CISAs, etc. I’m not sure if I want to advocate the creation of a new certification, but I’m starting to think that it could be useful too. Reducing the pressure for early specialization is also something that we could do to increase the number of good generalists out there.

There are many roles out there that would benefit from good quality generalists. Security organizations within big enterprises normally have consultants or advisors lined with the different LOBs or departments, with attributions that go from access control responsibilities to providing security requirements to new applications and business processes. I’ve met lots of people in those roles, but only a few had the necessary skills set for that.

The interesting aspect of those roles is that they share a common thread: they are often a liaison role, bringing together different groups with their specialists. Without a generalist the dialogue with one or more of those groups is undermined, with that person usually lining up with the group more aligned to his skill set and being seen as “one of them” by the others. Think about it. Developers x Infrastructure, Policy x Technology, Business x Technology, Servers x Networks, Blue Team x Red Team. If there’s someone capable of speaking the language of all those groups he’ll be able to reduce conflict, acting as “the glue” between them.  

There is value in having security generalists. Keep  that in mind when hiring people for those roles, or when considering your career options. Even if your plan is to eventually manage a team of security professionals, being a generalist puts you in an advantage position for that (but don’t forget that “Manager” is also a role that has its own set of minimum skills).