How to Choose Between Different Vendors

Problem

We had to choose a vendor technology partner for our core technology that would be licensed and used. My team started by researching the space and selecting the initial pool of about forty-five potential partners, and eventually narrowed the applicants down to three vendors through a structured process. Only after going through their demos were we to pick the right partner. But, are demos -- and more precisely, what kind of demos -- enough to make the right decision when choosing between different vendors?

Actions taken

At the second stage of our selection process, we asked three vendors to build a demo for us. Not just to do a simple demo but to create a small, one or two weeks time frame with a subset of requirements. We wanted them to provide us with a tailored demo by building some core capabilities that we were looking for; and we also asked for a session where they could walk us through the demo and allow us to run it with their guidance.

The challenge is that it’s easy to be misled by features showcased in a standard demo because it is easy to misunderstand how they may work in your situation, especially if your needs have more nuances that are not covered in the standard demo. Furthermore, vendors often promise things not doablee or replicable in that specific situation. In this particular case, we identified some of the critical gaps only after we were able to review the tailored demo.

One example of this was that to demonstrate a set of actions, the vendor rep switched tabs in a browser to demonstrate how the next set of actions could be done. They were doing it because there was no automatic way to go from one step to another from within the application. I don’t think the vendor was trying to mislead us intentionally -- they just didn’t think it was a critical gap because they were focused on demonstrating the individual features that we had requested. We didn’t notice it during the initial demo done by the reps because the switchover happened quite quickly. It was only when we got the opportunity to run the demo ourselves that we realized how the vendor product worked. Further discussions with the vendor team uncovered the details and we realized it was a significant gap for us.

Lessons learned

It’s important to approach the vendor selection process with a structured process of analysis, but to make the final determination it’s often useful to do a hand-on evaluation of the vendor technology you plan to acquire. This can be effective even when done in a short time frame, by carefully selecting core features you want to evaluate, and having the vendor partner walk you through the demo as you run it yourself, instead of just watching the demo by them.