False positives: A big problem for secret scanners

False positives are a significant but often unnoticed cybersecurity challenge. When a security alert signals a potential threat that turns out to be benign, security and engineering teams scramble to investigate a non-issue. While this may appear to be a minor distraction, repeated false positives can exhaust resources, overwhelm security teams, and greatly increase the risk of missed threats. They can also affect the delivery of other organizational priorities and increase the potential for actual threats to go unnoticed.

»False positives in secret scanning

False positives occur when a secret scanning solution flags legitimate content as suspicious. The primary causes of false positives are:

• Overly sensitive tools
• Solutions without appropriate contextual data
• Secret scanners that lack appropriate functionality

»Sensitivity

Secret scanning solutions configured to be overly sensitive flag any deviation from the norm. Whether these configurations are inadvertent or native functionality, too-strict thresholds mean normal activity triggers alerts that distract security analysts. Secret scanning solutions should offer the ability to configure custom rules so users can adjust the sensitivity of the solution based on their organization’s needs.

»Contextual data

Scanning solutions that have incomplete contextual data or references may not be able to correctly interpret findings. Such tools typically rely on a binary threat detection matrix, either declaring a finding as a threat or not. Additionally, these tools can’t assign severity ratings, so all findings get the same priority. This means that teams must investigate and adjudicate all findings, legitimate or otherwise.

Contextual analysis considers a broader set of criteria in which an event may occur. For example, a secret scanning solution that leverages contextual analysis compares a potentially leaked or unmanaged secret to the secrets management system used by the organization.

»Costs of false positives

False positives attributed to secret scanning solutions increase the risk of breach because they can distract security teams from actual threats, but they can also impact costs as well. These costs include:

• Missed threats: In the aftermath of the high-profile Target data breach of 2013, investigators discovered that the retail company’s security solution actually sent out multiple automated alerts warning of a potential intrusion, but those alerts were ignored. The result? One of the biggest data breaches in history. The Target data breach compromised personal and payment card data of about 40 million customers. Many of those customers fell victim to identity theft. As for Target, the data breach cost the company more than $200 million, including costs for forensic investigations, security enhancements, legal settlements, and customer credit monitoring, among others.
• Wasted time and resources: Each false positive requires an investigation by the security team. Security analysts spend valuable time digging into alerts that pose no actual threat, diverting them from more critical issues.
• Increased labor costs: As the number of false positives increases, so does the workload. Organizations may need to hire additional security analysts to keep up, inflating personnel costs. During a recent survey we conducted, 67 percent of respondents indicated that it takes their organization more than 5 hours to identify a false positive. According to Glassdoor, the average salary for a security engineer in the United States is $165,000. That means that those organizations are incurring at least $413 in increased labor costs for each false positive. For an organization that experiences just 1,000 false positives a year results in more than $400,000 in increases or wasted labor costs.
• Security tool maintenance: Constant tweaking of security tools to reduce false positives can become time-consuming and expensive. Fine-tuning detection rules and thresholds takes significant effort, especially as organizations scale their infrastructure.
• Alert fatigue: Over time, security teams faced with constant false positives may start ignoring alerts altogether, increasing the likelihood that a real threat will slip through the cracks. This takes us back to the example costs of “missed threats” mentioned earlier.
• Burnout and turnover: The repetitive nature of investigating false positives can lead to job dissatisfaction and burnout, pushing skilled analysts out of organizations and leaving companies with talent gaps.
• Decreased trust in security systems: If security tools consistently generate inaccurate alerts, organizations may lose faith in their cybersecurity systems, making them hesitant to rely on automated responses or trust system-generated insights.

»Reduce false positives with HCP Vault Radar

HashiCorp understands the impact of false positives. That’s why we built HCP Vault Radar with an intense focus on reducing false positives when scanning repositories, collaboration tools, or data storage. This includes automatically assigning severity to findings that prioritize the most important items.

Version history: When HCP Vault Radar discovers a finding in the latest version of a file, it assigns a higher priority because it is likely that this finding has not been previously evaluated and is therefore more likely to be a secret.

High entropy: HCP Vault Radar evaluates the entropy (randomness) of content using entropy algorithms. These algorithms are highly effective at identifying random or complex strings that indicate the content may be an exposed secret. Vault Radar also evaluates string literals in code for entropy, which helps identify potentially suspicious strings in any format.

Activeness checks: Active credentials represent the most significant threat. When HCP Vault Radar finds a credential, it will call out to the associated application to check if the secret is still active. Active credentials are marked as critical risks within the prioritization portal. Currently, Vault Radar can test for:

• Google Cloud API keys
• Amazon Web Services (AWS) access keys
• Personal access tokens for GitHub
• JSON web tokens (JWT)

Vault correlation: To further support prioritization, HCP Vault Radar can correlate if a leaked secret is stored in a Vault secrets manager. Most credentials in Vault are used in critical production environments, so HCP Vault Radar gives exposed secrets a higher severity score when they are also found in Vault’s key-value stores. This provides the contextual data that secret scanners have been sorely lacking.

Ignore rules: HCP Vault Radar ‘ignore’ rules allow you to ignore certain events based on a set of rules unique to your organization. There are several types of ignore rules including:

• Path: Path-based ignore rules allow you to ignore entire paths, such as directories used for documentation, or specific files within a resource.
• Resource: Resource ignore rules allow you to create ignore rules that are scoped to a specific resource. You could configure ignore rules on honeypot repositories or documentation repositories that may generate a high level of unimportant alerts
• Secret type: Secret type ignore rules allow you to ignore specific types of secrets. These rules can be configured by data source, meaning you could set a rule to ignore API tokens for a GitHub data source, but Confluence API tokens would still trigger an event. Types of secrets include key value pairs, API keys, passwords, certificates, tokens, credentials, and database credentials.
• Secret: Secret ignore rules allow you to ignore specific secret values that may be used in a data source, such as an example password used in documentation or as an example within the application.

»Remediate unmanaged secrets

In addition to secret scanning, HCP Vault Radar supports a robust set of remediation workflows via ticketing and alerting solutions, including:

• Slack (integration demo)
• PagerDuty
• Splunk
• JIRA
• ServiceNow

These integrations leverage common tools in DevOps, platform engineering, and security teams’ workflows supporting incident response processes. HCP Vault Radar transmits all of the information necessary to remediate its findings, including:

• Author
• Location
• Activeness
• If the secret is in the current version of a document or history
• Whether the secret is publicly accessible

The secret is never exposed in the HCP console. Instead, the user is provided a link to the location where the secret can be found, investigated, and remediated.

»Next steps

HCP Vault Radar is an exciting new addition to HashiCorp Vault’s secret lifecycle management capabilities that helps enterprises reduce the risk associated with credential exposure. Discovery of unmanaged secrets and subsequent remediation workflows further differentiate Vault’s secrets lifecycle management offering by helping organizations take a proactive approach to remediation before a data breach occurs.

»Additional resources

• What is HCP Vault Radar?
• HCP Vault Radar demo
• What is secret scanning
• Solutions to secret sprawl
• Secret remediation best practices
• HCP Vault Radar tutorials