1. Test Editor - This feature may be provided either by porting the captured data to a word processor or by a test editor. Without some kind of editing facility, a capture/replay system is useless.The editor is obviously used to correct executed tests, but more important, to create variations on tests without having to return the tests.
2. Smart Comparator - Direct, character-by-character comparators are almost useless because in most realistic testing situations, there are many fields that can be ignored or whose valuse cannot be predicted(e.g., time of day), but whose outcomes do not effect test validity. Smart comparison means the ability to specify the verification criteria including such things as ignoring fields, tolerances on fields, and different timing and sequences.
3. Internal/External - Capture/replay systems may be limited to only working on object code within the same PC or may serve as a terminal to be used over a communication link with another computer whose software is under test; both kinds are available, and both are useful. The internal tool is useful only as a unit test tool, whereas the external variant is useful for higher-level testing, such as system testing.
4. Singel/Multi-thread - The simplest tool can run only one line at a time, which makes it a limited tool for many system testing applications. More elaborate capture/replay system can capture and replay entire scenarios along several lines and do the fancy kinds of timing and sequence comparisons that may be needed.
5. Exception Report - Another useful property. Typically, once regression suites have been built using these tools, test runs may be very long (hours or days) and it is essential that the tool reports comparison failures by exception. Tools that do not do this are useful only in the capture phase but seriously limited as a driver.
|
|
Simulators also allow us to examine system performance. In general, performance tools help us to determine what the software and system performance capabilities are. In practice, it is sometimes hard to find the line that distinguishes a simulator from a performance tool.
Finally there are tools available for automated multi-user client/server load testing and performance measurement. These tools make it possible to create, control, and analyze the performance testing of client/server application - before these applications go on line.
|
|
Experienced programmers know that 20% of the code will cause 80% of the problems, and complexity analysis helps to find that all-important 20%. The McCabe Complexity Metrix were originally published in 1982 .
Complexity metrics is based on the number of decisions in a program. It is important to testers because it provides an indication of the amount of testing (including inspection) necessary to practically avoid defects. In other words, areas of code identified as more complex are candidates for inspections and additional tests. There are other types of complexity metrics which in general are an indicator of program testing cost/schedule and number of defects in the program.
Code comprehension tools help us understand unfamiliar code.They help us to understand dependencies, trace program logic, view graphical representations of the program, and identify dead code. They can be successfully used to identify areas that should receive special attention, such as areas to inspect.
There is a considerable amount of time spent in preparing for a code inspections meeting. This requires extensive analysis, comprehension, and reverse engineering, all of which are made easier by code comprehension tools.
Coverage analysis tools are the oldest known test tools. They have been in use at IBM, among others, for more then tow decades.
Coverage analysis provide a quantitative measure of the quality of tests. In other words, they are a way to find out if the software is being thoroughly tested. If , for instance, we shell try to test branch coverage under an interpreter it will be very difficult, or even meaningless.
This tool, essential to all software test organizations, tells us which parts of the product under test have in fact been executed (covered) by our current tests. They will tell us specifically what parts of the software product are not being covered, and therefore require more tests.
Some companies argue that it is not necessary to achieve full statement coverage, that is, to execute all the statements within the product prior to release. They seem to think it is all right to expect customers to be the first to execute their code fort them. Maybe these companies belong in another line of work. If we aren’t measuring coverage, we do not have a handle on the job we are doing as testers.
Almost all structural tools run the source code into a preprocessor so that it can keep trace of the coverage information. The problem is we now have a new source that’s bigger then the old one so our object module is going to grow in size. The other possible problem is that performance may be impacted because we now have a different program then we did before. However, the final version of the software as delivered doses not include the above preprocessing step, and therefore does not suffer this size and performance penalty.
There are many varieties of coverage, including statement, decision,
condition ,decision/condition, multiple condition, and path. As a minimum,
the place to start is to make sure each statement in the program has been
tested, and that each decision has taken on all possible outcomes at least
once.
| panorama site |
The purpose of test planning is to define the scope, approach, resources (including tools), and schedule of testing activities. The test plan provides the foundation for the entire testing process. Tools don’t eliminate the need to think. As useful as capture/playback tools are they do not replace the need for sound test planning and design.
Perhaps the biggest help here comes from standards. the IEEE/ANSI standard for software test documentation describes the purpose, outline, and content of the test plan, and the appendix of the standard includes examples taken from commercial data processing. Although a few tools have incorporated templates for test plan into them , many companies have found it useful to simply have someone enter the outline for test plans found in the IEEE/ANSI standard into an accessible edit file.
There are useful commercial tools that determine actual project staffing and schedule needs for adequate product testing. Often people rightly complain that schedules are predetermined upper management, and the use of such a tool can provide an objective view of the realities of the project.
The types of tools required for test planning are:
The need for a test case management tool can creep up on us. We begin using a capture/playback tool to build an d automate our tests. Then one day we wake up and find we have thousands of disorganized tests that need to be managed. The best test case managers:
The bad news is - we’re not there yet. The good news is that several of the leading tool vendors claim that they are working hard to accomplish this, and within the next year or so we should see viable offerings.
An ideal error simulation tool should be :
automated without interactive operation while the program is running. smart to determine where the error simulation should be done and only do once for each location .
general with providing the method and sample code for users to easily simulate their own functions rather than system functions only.
"honest" with providing a log file to report where the error simulation
is done and when it was done.
| testdir site |