How to Tell if AI Is an Organizational Paint Job

AI Washing

Too many security startups treat AI as a coat of paint. The useful test is not the word "AI," it is whether the people and the process behind the product actually changed.

At conferences like RSA and Black Hat, this is readily apparent as long-standing products suddenly become "AI-native," rules engines get rebranded as machine learning, and a thin wrapper around a public LLM API is sold as proprietary intelligence. This is AI washing. It is especially common in cybersecurity because the category already lives on fear and novelty: vendors promise automated threat detection, autonomous response, and agentic SOC operations even when the underlying system is still signature matching, static scoring, or human analysts doing the real work.

And the VC market incentivizes this behavior, even if unintended, as seen by an average 250% valuation markup on AI model development. Buyers who take the marketing at face value later discover they purchased glorified automation that still requires heavy tuning, produces the same false-positive burden, and leaves the original vulnerabilities intact.

The damage is not just wasted budget, and it is part of why I wrote the previous blog on how to incorporate AI into your security maturity model. The summary: build mature security process first. That process is what gives you the baseline performance to measure AI capability and innovation against.

The share of VC-backed AI companies scoring low on AI relevance has grown from 16% in 2016 to 42% in 2026. Source: solutionsreview.com

The Cost Is Not Just Wasted Budget

Overstated AI claims create a false sense of coverage, add to your problems in data quality and operational workflows, and can even increase the risk of compromise when companies allocate to and prioritize the wrong areas.

The practitioner read on this year's show floor.

A Better Test Than the Word "AI"

The useful test is not whether a startup uses the term "AI," but whether it can show trained models on relevant data, measurable reductions in analyst workload or dwell time, and an explanation of what happens when the model is wrong.

We started our AI journey at SCYTHE in 2019 when we wrote an internal white paper on offensive game theory. We began incorporating AI elements initially focused on assistive operator capabilities, then expanded to our current architecture, which ships with an MCP server that orchestrates and analyzes across all defensive tools integrated into the platform, Threat Intelligence ingest for attack sequences, and proposes TTP, defensive, and test recommendations.

Confidence is cheap when there are no repercussions for faking it.

The Tell Is People and Process

But how we got there is where the interesting insight comes from. It was not the technology in the product, it was the change in company processes and culture required to take full advantage of AI. This is where you can easily determine if a company is "AI-native" versus slapping AI paint on its rusty origins: can they answer how the people and process sides have changed?

For example, software development with AI can operate at such a speed that the product and quality assurance processes will be overtaken by core development to the point that everything breaks. Agile development methodologies do not naturally address this challenge. The company must find a way to bring those three interdependent processes together.