Testing application management applications prior to purchase is tough for many organizations. When many organizations look to purchase software tools to manage their application environment, they typically like to kick the tires of three to four vendors in their environment.
This process typically exposes several deficiencies in organizations' testing environment that makes it difficult to truly evaluate vendors. For those organizations that do have a testing environment, many do not have the types of applications such as Peoplesoft, SAP, custom J2EE and .NET applications that are the most critical to the datacenter installed.
If you are lucky enough to have them installed, they will not have any users, or load in the test environment. So, when you are examining fault and performance tools, they have nothing to monitor. Where there are several applications that will simulate network traffic that helps plan for changes to the network, we have not found similar products in the application testing market that will allow broad testing across a number of discrete applications.
To help organizations evaluate application performance tools, we kicked off a Rolling Review of Applications performance management tools in August. We found that to develop an adequate test environment, we needed to test each application in a production environment to accurately see how well they could detect issues in our environment. This type of testing is nearly impossible, and not recommended, for many businesses as the potential for downtime in your production environment is increased through testing. We caused two minor outages thus far from unknown issues resulting from installing these applications.
