It is widely known that there are problems associated with trying to develop a rating system for vulnerability assessments. Numerical schemes based on statistical models fail to take into account that a single exploitable vulnerability on one machine can lead to a compromise. Subjective rating systems make it difficult to determine whether the overall security posture of a network is improving over time.
In any rating system, however, there has to be some value that represents the highest attainable rating for that system. Is a network that receives this highest rating completely safe? If the highest rating doesn't indicate that the network is perfect, is the rating giving a false sense of security?
In a sample rating system, networks can be rated on a five-point scale from Poor to Excellent. So if a network is rated as Excellent, what does it really mean? In this rating system, it would mean that there is no vulnerability that is exploitable on any of your systems, and that the devices are not providing any information that can be used to gain knowledge about your network or systems. In essence, it's perfect from a security point of view.
So, if we rate a network as Excellent, or even Good, does that give a false sense of security to our customers? If our customers don't fully understand that security is a never-ending process and that they have to remain vigilant to keep that rating, there could indeed be such a misperception. All it takes is one critical vulnerability to be identified on one system, and the overall rating can plummet to Poor overnight.
Some users, trying to glean trending information about their network health, have been confused by shifting ratings: From month to month, a rating may move from Poor to Good, then back to Poor. How can they determine if their network posture is getting better or worse? When asked this question, I advise the user to look at what's causing the changes. Is there suddenly a spate of new vulnerabilities being identified by key software vendors? Are they performing system upgrades to their environment that introduce new vulnerabilities into the environment? Maybe new employee hires with bad security practices are introducing new problems it could be time to offer another round of security training.