How Does Black Kite Calculate Cybersecurity Ratings?

In our 2023 Third-Party Data Breach report, we found that the magnitude of impact from  third-party breaches is increasing. Dramatically. We also found that the number of companies affected by third-party breaches doubled from 2021 to 2022.

In a threat landscape where the threats are growing more sophisticated and more destructive, cybersecurity ratings are more important than ever. Why? Each third-party vendor your company works with broadens your organization’s attack surface, and cybersecurity ratings help you choose the vendors with the best security posture (i.e., most resistant to attacks) and monitor your existing vendors to ensure you’re up to date on any potential threats. In the world of cybersecurity ratings, security experts and executives rely on Black Kite to help them evaluate third-party risk. Black Kite is known for comprehensive, accurate ratings and real-time results packaged in easy-to-read reports. But how do we calculate them? Luckily we’re all about transparency, too.

What are cybersecurity ratings?

According to the National Institute of Standards and Technology (NIST), risk scoring is a type of quantitative analysis that assesses and reports on an organization’s IT assets. Technical ratings (aka cybersecurity ratings) are a form of risk scoring where the assessment tells you about a third-party vendor’s cybersecurity posture, empowering organizations to make confident decisions about the third-party vendors they’re working with.

What does this look like in the real world? Imagine you’re considering working with a third-party vendor to develop an app for your organization. Having information on the app vendor’s patch management and application security vulnerabilities can help you decide if it’s simply too risky to work with them. After all, if the computers and devices on their network are vulnerable, your proprietary information on the vendor’s network could be at risk by that vulnerability.

What goes into Black Kite’s cybersecurity ratings?

At Black Kite, our cybersecurity ratings (distinct from our risk quantification and compliance correlation solutions) analyze data across risk categories from millions of servers. The ratings, composed of controls and categories, enable us to calculate final grades.

Black Kite’s Controls

We perform passive, non-intrusive scans and use an asset discovery engine to collect information on an organization. From IP ranges to external library vulnerabilities and leaked credentials, these data points reflect a company’s vulnerabilities, weaknesses, misconfigurations, and issues. We organize these data points into 298 controls based on MITRE recommendations. MITRE, a non-profit organization whose globally accessible knowledge base includes ATT&CK®, provides a baseline for threat models and methodologies in the private sector, government, and cybersecurity product and service community.

Black Kite’s categories

We organize each of the 298 controls into 20 risk categories. The categories tell you about the type of vulnerabilities represented in the controls. Picture a giant bowl of M&M’s. The categories are colors, and each color/category contains a set of controls or individual M&M’s.

• Digital Footprint – The digital collection of internet-facing assets (IP addresses, domains, subdomains, etc.) that can be traced back to an organization. 
• Patch Management – Does the organization keep network computers and software up to date and capable of resisting low-level cyber attacks? 
• Application Security – Evaluation of security measures at the application level that an organization uses to keep data or code from being hijacked or stolen. 
• Credential Management – Reports on what information an organization’s website unintentionally reveals to users. 
• Hacktivist Shares – Determines if an organization’s name or assets are active on hacker forums, telegram channels, and other hacker platforms.
• SSL/TLS Strength – The strength of an organization’s internet traffic encryption protocols.
• DNS Health – Evaluates vulnerabilities in an organization’s domain name system (DNS).
• Email Security – Checks whether or not an organization’s email system contains misconfigurations. 
• Network Security – Analysis of an organization’s network-level problems (critical ports, unprotected network devices, etc.). 
• DDOS Resiliency – Assess an organization’s ability to rebound from distributed denial-of-service (DDOS) attacks. 
• IP/Domain Reputation – Determines whether an organization’s IP addresses send unwanted requests, if they’re a botnet member, or if they have a poor reputation.
• Website Security – Determines whether there are code or server-level vulnerabilities on an organization’s main website.
• Attack Surface – A look into an organization’s environment or system elements that make it vulnerable to attack. 
• Information Disclosure – Determines whether or not an organization’s website unintentionally reveals sensitive information to users. 
• Social Network – Evaluation of an organization’s social media presence and attack surface on public platforms. 
• Fraudulent Domains – Identifies fraudulent or pirated domains related to an organization.  Fraudulent Applications – Identifies fraudulent or pirated mobile/desktop applications related to or hosted by an organization. 
• Web Ranking – Tracking an organization’s websites and ranking each according to popularity, backlinks, and references. 
• Brand Monitoring – Monitoring and analysis of an organization’s various channels for relevant information concerning the company/brand. 
• CDN Security – Detecting vulnerabilities in an organization’s content delivery networks (CDNs).

How we sort controls into categories

Wondering what our process of sorting controls into categories looks like? Here are two examples. Remember, we group each set of controls into a category that tells us what aspect of the security posture we’re evaluating.

• Example 1: CDN Security (brown M&M’s) – We know that the CDN Security category detects vulnerabilities in an organization’s content delivery networks (CDNs). Remember, each one of these controls is used to give Black Kite a clearer picture of an organization’s posture in the category. CDN controls include:
  • Cloud security 
  • External library vulnerabilities
  • External resource reputation 
  • Unpatched/outdated systems
  • IP regulations
• Example 2: Digital Footprint (blue M&M’s) – The controls collected for this category help Black Kite analyze the digital collection of data traced back to an organization. Digital Footprint controls include:
  • Assets registered/used by the organization
  • Domain names registered
  • Domain aliases 
  • And an additional 20+ M&M’s/controls that fall into this category

We know what you’re thinking.

Why bother putting controls into categories?

We can also call this section, “Why not just grab a handful of M&M’s instead of sorting them?” Organizing our 290+ controls into 20 distinct categories helps in three ways:

1. Calculating a final, weighted grade

We use the Common Weakness Scoring System (CWSS) and Common Weakness Risk Analysis Framework (CWRAF) developed by MITRE, and seven other parameters based on our remediation recommendations, to evaluate and score each control. From there, we calculate a final category score and grade. Here’s how we do it:

• Step 1: We average the control scores into a category score and assign a grade for each category. 
• Step 2: The 20 categories are weighted according to importance (i.e., some M&M’s are bigger than others). For example, the Application Security and Patch Management categories carry more weight in overall security than Brand Monitoring. 
• Step 3:  We calculate a weighted arithmetic mean based on the average grades assigned to all 20 risk categories. Then, we assign a final overall grade based on the arithmetic mean.

2. Providing you with high-level context

At Black Kite, we work hard to make risk assessment easy. This means you get the high-level context in each of your reports. Your report includes a final cybersecurity rating and a breakdown of what each category tracks. Instead of drilling down into individual data points or controls, like lack of HTTP Strict Transport Security (HSTS) support, we provide a letter grade for the SSL/TLS Strength category, which deals with internet traffic encryption, and an overall letter grade for the organization. Based on this information alone, you can quickly make an accurate decision on a potential third-party partner’s risk.

Not annoying you with a report on 290+ data points

But if you want to be annoyed with the in-depth data, we have a technical category report with your name on it!

So there you have it. An in-depth look at the process and formula behind our cybersecurity ratings. At this point, we could recommend eating the (imaginary) M&M’s, but we’ve handled, sorted, and graded them. Just grab a fresh pack.