How to evaluate a FinOps tool without getting sold to

It is 9 a.m. and you have fifteen tabs open. Every FinOps tool promises to cut your cloud bill by 20%.

You open the first three and they say the same thing. Different logos, different dashboards, the same 20%.

The FinOps marketplace is overflowing with tools, all with different offerings, yet promising a similar level of savings.

Something I often see happen is a company evaluates only the handful of tools they’re familiar with. And I don't blame them. Who has the time to compare all of these?

However, making the wrong decision has costs. What if the platform is too complicated and doesn’t get used? What happens to your billing data once it's connected? What happens if you want to leave?

And it doesn't get much easier once you get on a sales call….

How do you cut through the noise?

I’ve been thinking about this problem for a while, so I sat down with Josh from CloudXRay to compare notes. Josh works with companies to improve their FinOps practice and maintains a directory of almost 300 FinOps vendors.

Together we broke down where to start, how to narrow down the tools, and what to ask when you get on a sales call.

I know it’s tempting to start googling FinOps platforms. But please, wait!

There’s something important you need to do first.

Before you ask AI for the “top FinOps platforms” or, heaven forbid, get on a discovery call, you need to get clear on your requirements.

And yes, this means talking to people outside of engineering too.

For Josh, the number one mistake he sees companies making is “jumping in” to a tool, before doing this first step. And let’s not forget about future needs:

“Let's say you're in Azure today, but you know AWS is on the roadmap. You don't want to pick something Azure-specific, you want a solution that handles both.”

Maybe I spend waaaay too much time looking at FinOps tools (this could be true actually), but the “20% savings” line is starting to get old. Or is it 30% now? In any case I’ve stopped noticing it. Because it tells you exactly… nothing.

All it means is that they once saved a customer that much and they’re confident they can save you that too. But they don’t know your situation.

So when a vendor quotes a number, ask:

Sometimes the goal is savings, and that’s great. But maybe you want to go beyond that. Josh argues that “We need to get out of the mindset of what savings a tool will get you, and focus on showing the value of the technology back to the business.”

For example, if your cloud bill increased because a product launch doubled revenue, that’s a success, and a tool that only knows how to flag the increase won't tell you that.

Perhaps the biggest fear of anyone signing a multi-year contract with a SaaS platform is that nobody will use it. No one wants to be that person.

How can you be sure, in advance, that this won’t happen?

Josh’s advice: “You have to make sure there's buy-in. Are finance and product aware of what's going on and why you're doing this?”

Again, this means going outside IT or engineering. Talk to anyone who needs to act on the reports or alerts.

If there's no time, interest, or urgency, then it's worth rethinking the purchase before you sign anything. Because a tool that nobody has the capacity or the reason to open will not change how your organization manages cloud spend, no matter how good its features look in a demo.

And finally, it’s time for the vendor call.

But how to avoid getting “sold” to?

No one wants to sit on a call, listening to a sales pitch about all the shiny features while thinking “Sure, this looks nice, but how does it apply to me?”

Josh says to use your requirements to drive the conversation. “Ask the vendor whether they can meet each need, then have them show you how.”

E.g. ask the vendor to walk through a real scenario from your own environment, like allocating a shared cluster's cost across three teams, instead of clicking through a generic demo dashboard.

Any good FinOps vendor should be able to show you exactly how they would tackle your problem.

Aside from that, you need to know how much effort the tool needs before it delivers anything useful. It can can look great in a demo and still need weeks of engineering time to set up. So ask the vendor to spell it out:

Engineering time has a cost too. If the tool needs a month of work from people who are already busy, that needs to be taken into account alongside the cost of the tool.

Write down your requirements, talk to the people who will actually use the tool, and make every vendor show you your use case. Then ask what it will take from your team to get there.

If a vendor can't do that, you have your answer.