Stuck on Defense: Don’t Use a Security Rating Score Alone to Manage Vendor Risk

Written by: Bob Maley

If it rained yesterday, should you wear a raincoat today? It’s not a simple answer. You need more information to make that decision because the sole occurrence of rain yesterday doesn’t necessarily predict today’s weather events.

A Security Rating Score (SRS) is much like a recap of yesterday’s weather. The score provides a retroactive look at a vendor’s cybersecurity posture based on past events. This letter or number grade is meant to serve as an objective assessment of how well that vendor manages its security risks. But, in reality, a SRS score leaves organizations on the defense, evaluating and preparing for risks based on what has already happened instead of what’s likely to happen in the future.

 With something as crucial as third-party risk management (TPRM), this simply isn’t enough. Third-party breaches are a major security concern and more common than you may realize. To ensure vendors don’t invite unnecessary risk into the supply chain, organizations must pursue more proactive TPRM strategies.

A Brief History of Security Rating Scores

In the 2010s, security service organizations (SSOs) started using new technology to gather and analyze massive amounts of data about organizations’ security performance. These companies implemented proprietary scoring methodologies to turn this data analysis into a score representing an overall evaluation of a given organization’s cybersecurity program. 

Security rating scores became a common and accepted measure for security teams to report to stakeholders upon evaluating third-party vendors.

4 Major Issues with SRS Scores

While they might have served a useful purpose a decade ago, SRS scores alone fall short in a modern TPRM program for various reasons:

1. Proprietary (and Secretive) Scoring Methods

SRS companies use proprietary scoring methodologies to determine letter or number grades for each vendor. This means that each SRS company considers different factors and weighs those factors differently to determine an overall score. When companies use multiple rating services to vouch for the security of their service (which is common), they may receive drastically different scores. These conflicting scores can confuse security leaders who want a straightforward evaluation of a potential vendor’s cybersecurity posture.

Moreover, rating services provide little insight into their proprietary scoring methodologies and data collection and curation processes. These undisclosed scoring techniques invite questions about the accuracy and value of the scores provided, plus the nuances between grades. With all these unknowns in play, the letter- or number-grade security rating can only provide a general idea of an organization’s cyber hygiene.

2. Lack of Context to the Impact on Your Business

In addition, letter and number grades don’t contextualize risk in relation to a vendor’s specific services. The same grade can mean vastly different things based on the service an organization provides. For example, while a C grade may be an acceptable level of risk for an organization that manages a marketing website, it wouldn’t be an acceptable level of risk for a vendor with access to high-value data like your customers’ personally identifiable information (PII).

3. False Positives that Send You on a Wild Goose Chase

Unfortunately, there are also countless instances of false positives in an SRS score. These could be the result of multiple factors, including:

• Outdated Vulnerability Information: An SRS score may flag an organization for known vulnerabilities, though the organization may have already patched or mitigated the issues. 
• Shared IP Addresses: Security rating services may flag organizations using shared hosting services or cloud providers for issues actually caused by other entities sharing the IP address. These shared IP addresses may negatively skew a security rating by reflecting problems that aren’t pertinent to the organization​ in question.
• False Malware Detections: It’s also fairly common for security scanners to incorrectly identify malware, leading to unwarranted security concerns and a lower security rating​.
• Misconfigured Security Features: In many cases, traditional security scanners also flag misconfigurations or missing security features as critical issues when they don’t actually impact an organization’s overall security​. 
• External Perception vs. Internal Reality: An SRS score only relies on external scans and publicly available data. Without additional information about an organization’s internal security controls and practices, the final score can be misleading. 

Learn more about why false positives and false negatives happen and the importance of high-quality data.

4. Emphasis on Past Events that Have Little to No Bearing on Future Events

Most concerning is the fact that past events significantly influence an SRS score (e.g., a breach will cause a score to decrease). This puts organizations into a reactionary state, evaluating vendors based on what has happened in the past instead of what is likely to happen in the future. Beyond being reactive versus proactive, this look-back at past events doesn’t evaluate the context and consequences of each event. For example, a low grade based on a past event does not take into account any remediation efforts made to close that vulnerability. The company may currently be more secure than ever in that area. At the same time, a passing grade based on a snapshot of last month’s cybersecurity posture does not fully represent a vendor’s current position. That vendor could have implemented environmental changes within the past few days or weeks that introduced new vulnerabilities or risks. A point-in-time snapshot can’t account for these new changes or help organizations proactively predict and prepare for the future.

Take a Proactive Approach to TPRM with Black Kite

To operate effectively in today’s evolving threat landscape, organizations must take a more proactive approach to TPRM, centered on identifying, analyzing and mitigating risk in their cyber ecosystem in real time. This includes using continuous risk intelligence to determine susceptibility to future threats like ransomware attacks, and proactively quantifying the impact those threats could have on your business.

At Black Kite, we call this proactive approach Cyber Risk Management and believe it represents the next evolution of best practices for TPRM. Dive into the benefits of this approach and learn the practical steps to build a proactive risk management program at any organization. Explore the online interactive eBook, “Stay Secure by Staying Ahead: How to Shift to Proactive Cyber Risk Management.” (No download required.)