Functional Testing Tools - Mixed Vendor Implementations

The plethora of functional testing tools previously mentioned (Mercury Interactive WinRunner, QuickTest Professional, Segue SilkTest, Compuware TestPartner, IBM Rational Robot, IBM Rational XDE/Functional Tester, and AccordSQA SmarteScript), combined with the fact that some tools work better than others in certain environments or against particular target applications, often creates a need for users to switch tools or at a minimum bring in a new tool to use in parallel with their current tool.



Perhaps a company used one vendor's functional testing tool to test Version 1.0 of their target application, and that tool worked fine with the technologies in 1.0, but now Version 2.0 uses different technologies and that testing tool doesn't support some particular aspect of 2.0. Maybe their previous vendor of choice hasn't released an updated version that works with today's technologies, yet works fine for continuing to support 1.0 revisions.



Perhaps an organization is outsourcing its testing and finding that one of the ramifications of outsourcing is a need to employ a new tool that allows greater reuse of testing assets. That might mean moving from a programmer-centric script-based testing tool to a simpler, keyword-driven testing tool. This may be further driven by a need to shorten the learning curve so that many new testers can learn a particular tool, ramp up quickly, and shorten the gap between hiring on and becoming a productive QA team member.



Change is sometimes necessary. What isn't necessary is completely abandoning an old tool, especially one that works fine for previous (but still maintained and supported) releases and one which has been invested in significantly in terms of man-hours spent creating test assets that work.



Ring-Zero Software sells Test Execution Adapters for IBM Rational TestManager. These adapters allow functional testing tools from a variety of tool vendors to be tied to Rational TestManager. Scripts can be executed and results captured within TestManager, and test results can be tied to requirements and defects. In the big picture, these adapters allow non-Rational functional testing tools to be integrated into the environments of companies that use predominantly Rational tools.



IBM Rational includes integrations out of the box that allow TestManager to work with:


  • IBM Rational Robot

  • IBM Rational Functional Tester

  • IBM Rational Performance Tester

  • IBM Rational Visual Test

  • Command-Line driven tests

  • Java-based tests

  • VB-based tests




At the time of this post, Ring-Zero supports the following tools:




Most importantly, the adapters allow testing tools to coexist within a QA environment. In the event that two particular tools cannot coexist on the same machine, the IBM Rational Test Agent can be used to execute scripts on a remote machine. This means that QA shops can now retain Tool X as they fully or partially implement Tool Y, and that neither tool ends up being orphaned from the rest of the software development process. Of course, the catch is that these adapters require the use of IBM Rational TestManager for test planning. This catch really isn't so bad, however, because the tool also provides a bridge to requirements and defects.



More information on Test Script Execution Adapters is available from Ring-Zero Software.

Functional Testing Tools - traceability to requirements

There are quite a few functional testing tools out there: Mercury WinRunner, QuickTest Professional, SilkTest, TestPartner, IBM Rational Robot, IBM Rational XDE/Functional Tester, AccordSqa SmarteScript, etc. The list doesn't really go on and on; a few of the tools listed (Mercury and IBM Rational tools) own the lion's share of the functional testing tools market.



These functional testing tools do a pretty admirable job of enabling software testers to automate functional testing. Depending on the environment and target application being tested, some tools work better than others. When used in an isolated sense, however, the tools are missing a big piece of the picture: testing what needs to be tested in terms of requirements specifications. Without linking tests back to requirements, generating reports that show what's being tested (or not) and communicate becomes an error-prone manual process. Without an automated link back to requirements, it's hard to know how test results relate to the overall quality of software. In a nutshell, traceability from test cases and test results back to requirements provides context necessary for the QA process to work.



Ring-Zero Software makes integrations that provide this context. Because the company's founders came from IBM / Rational Software, they chose to build integrations that center around Rational's test planning tool, TestManager. Out of the box, Rational provides integrations that let testers tie their tests back to requirements kept in Rational's own requirements tool, IBM Rational RequisitePro, as well as IBM Rational Rose and Microsoft Excel. TestManager has an open architecture and published API that makes it possible to extend Test Manager's ability to tie to other requirements tools and functional testing tools.



Ring-Zero has added to Rational's list of integrations by building bridges to several popular requirements tools made by Rational competitors:



Ring-Zero Test Input Adapter for Telelogic DOORS
Allows Rational TestManager users to work against the market-leader in requirements management tools, DOORS. Ring-Zero's integration leverages DOORS requirements by supporting "suspicion", which basically means that if you link test cases to requirements and the requirements change, the test cases are marked as "suspect" indicating that they may need review since the underlying requirements may have changed in ways that could affect testing. Ring-Zero also leverages DOORS requirements and letting them be used to populate Rational's ManualTest test scripts.



Ring-Zero Test Input Adapter for Borland CaliberRM
TestManager users can work against Borland's entry into requirements management tools. This tool used to be made by StarBase before Borland bought them. This integration is very similar to their DOORS integration, but with CaliberRM instead of DOORS.



Ring-Zero Test Input Adapter for IBM Rational ClearQuest
This is an interesting integration. Usually one would think of submitting defects from test results as a one-way process. Rational has a pretty tight integration with their testing software that lets you do this, but Ring-Zero has gone a step further. Often testers need a way to tie tests back to defects that have been submitted, perhaps to generate reports that demonstrate that all defects for a particular release are being tested. Ring-Zero's integration lets test planners treat defects like requirements in this sense, and if tests linked to defects fail, they can submit more defects.



More information about Ring-Zero Test Input Adapters for TestManager is available from Ring-Zero Software.