Cybersecurity product management is a nuanced field that requires careful consideration of metrics and their context. While tracking and optimizing metrics is an important responsibility of a product leader, it’s essential to understand that not all metrics are created equal. In this article, we’ll explore two metrics that cybersecurity product leaders are often tempted to track and report on: detection accuracy and conversion rate.
Detection accuracy is a metric that applies to security tooling that triggers alerts notifying users that a specific behavior has been detected. While not all cybersecurity products are designed to generate detections, many do. False positives and false negatives are two types of metrics that are useful to track in the context of detection accuracy.
False positives occur when the tool triggers a detection on normal behavior, resulting in a false alarm. False negatives occur when the tool misidentifies an attack as normal behavior and does not trigger a detection, resulting in a missed attack. Security vendors face an impossible-to-win challenge of reducing the number of false positives and false negatives and bringing them as close to zero as possible.
Every customer’s environment is unique, and applying generic detection logic across all organizations will inevitably lead to gaps in security coverage. Product leaders need to keep in mind that false positives make it more likely that a real, critical detection will be missed, while false negatives mean that the product is not doing the job the tool was bought to do.
Conversion rate is a metric that tracks the percentage of all users or visitors who take a desired action. It’s one of the most important metrics that companies and subsequently, product teams obsess about. Who owns conversions in the organization will depend upon who can influence the outcome.
For example, if the product is fully sales-led and whether the deal gets closed is in the hands of sales, then conversion is owned by sales. If the product is fully product-led and whether a free user becomes a paying customer is in the hands of product, then conversion is owned by marketing and product teams (marketing owns the sign-up on the website, product owns in-app conversion).
In the context of cybersecurity product management, conversion rate can be a tricky metric to track. Cybersecurity products are often complex and require a significant investment of time and resources to implement. As a result, conversion rates may not accurately reflect the value that the product provides.
Product leaders need to be mindful of the fact that cybersecurity products are often sold on the basis of their ability to prevent or mitigate cyber attacks. As such, the value that these products provide may not be immediately apparent to users. Conversion rate may not be the best metric to track in this context.
In conclusion, cybersecurity product management is a nuanced field that requires careful consideration of metrics and their context. While detection accuracy and conversion rate are two metrics that product leaders are often tempted to track and report on, it’s essential to understand that not all metrics are created equal.
Product leaders need to keep in mind that false positives make it more likely that a real, critical detection will be missed, while false negatives mean that the product is not doing the job the tool was bought to do. Additionally, conversion rate may not accurately reflect the value that cybersecurity products provide. By understanding the nuances of these metrics, product leaders can make more informed decisions and drive better outcomes for their organizations.