Proof Of Concept: Tips For Successful Testing

Have you ever skipped the proof-of-concept (POC) stage when buying software and regretted it? The POC is one of the most critical aspects of any IT purchase, and while it can sometimes be challenging to thoroughly test hardware, with software there’s no excuse for failing to do so. The process is usually fairly easy, quick and non-disruptive, making it particularly difficult to later explain away a costly product shortcoming that even a basic POC would have unearthed.

The most important part of the POC is to know what you’re looking for from the outset. With this in mind, it’s very important to work with all stakeholders to first identify what it is you’re trying to accomplish or what problems you need to solve with the software. Additionally, POCs are a great way to compare and contrast multiple vendors at the same time. In doing so, you can often quickly figure out if what their salespeople say is really true, or, if they're just providing a “marketing checkbox” of hyped features that serve no real purpose or merely meet obligatory criteria.

The first test in any POC is the vendor’s reaction. If the vendor tries to talk you out of conducting it or uses delay tactics, that generally means the reps know their product won’t do well. On the other hand, if the vendor is actually pushing for a POC, that could mean the reps are confident in their technology’s ability to perform and have a high rate of success in these situations. Also be wary if the vendor asks you to pay for a POC. Having to pay can depend on the complexity of the software being evaluated. However, if you’re looking at other products that offer a true “try before you buy” approach, a vendor requiring money could indicate issues ranging from possible financial strains and an inability to compete to a lack of ongoing support..

POC criteria should always be developed by a team of business stakeholders, not the vendors. Vendor-supplied criteria for a POC is designed to make the vendor look good, but may not meet your business requirements. That doesn’t mean you can’t change criteria if one vendor has a feature you find useful; it just means you shouldn’t let any one vendor determine the boundaries of the POC-playing field.

Next, be prepared! Most vendors will give you a list of criteria (requirements) for a lab environment to ensure proper testing. If the vendor has agreed to come onsite and help set up the POC, it can be a long day if the lab is not arranged properly. For this reason, ask each vendor for its prerequisites and get things set beforehand. While not always possible, it’s also a good idea to have a separate lab environment for each vendor if you’re doing multiple POCs. For cloud-based applications, this step is actually easier whereas most vendors will provide you with a “sandbox” area to test the software. Still, be ready to provide any data or testing criteria that you specifically want to evaluate.

During the POC, stick to timeframes. Ask each vendor how long the POC should take and then make sure that everyone sticks to the allotted time. Endless POCs don’t really do anyone any good, and if the vendor can’t get things working in the environment you provided - especially with agreed-upon criteria - only allow so many chances to make it work. After all, if it doesn’t function properly in the lab, do you think it will work in production? Moreover, how long installation and configuration takes in the lab also is a good indication of how long it will take in production.

Finally, make sure that you allot enough time for the POC. Vendors understand you have a job to do, but if there’s not sufficient POC time scheduled on your calendar, you’ll never get it done or you may cut corners. Remember, timeframes are also important in keeping salespeople off your back: If you tell them it will take a week, they will call you in a week, and will continue to do so until you answer. The better everyone does in setting proper expectations, the happier all involved will be, and the more likely it is that you’ll get the proof-positive results you’re seeking.