Software testing

Plus de détails

ELED3111

par Michael Putman

Algebraic Reasoning, Graphing and Geometric Connections

par Rachael Hernandez

Game Design/Production

par HAJI SUHAIMI HAJI SUHAIMI

testing map sharing with audio

par James Holland

Software testing

traceability matrix for specification-based testing

Evaluation-based

Oracle-based testing

In high-volume automated testing, the oracle is probably another program that generates results or checks the sw under test’s results.

an oracle is an evaluation tool that will tell you whether the program has passed or failed a test.

Consistency is an important criterion

Inconsistency may be a reason to report a bug or may reflect intentional design variation

with user’s expectations

with claims (Function behavior is consistent with what people say it’s supposed to be)

with comparable products

with our image (an image the organization wants to project)

with history (past behavior)

Activity-based

performance testing

A significant change in performance from a previous release can indicate the effect of a coding error.

to determine how quickly the program runs (to decide whether optimization is needed)

long sequence testing

called duration testing, reliability testing, or endurance testing

goal:to discover errors that short sequence tests will miss. (memory leaks, thread leak, stack overflow)

testing is done overnight or for days or weeks

load testing

the pattern of events leading to the failure will point to vulnerabilities in the software.

the system under test is attacked by many demands for resources.

installation testing

install the sw in the various ways and on the various types of systems that it can be installed. What happens when you uninstall?

scenario testing

would be classified as coverage-based tests

(use case flow tests) tests derived from use cases

guerilla testing

done by an experienced exploratory tester. (a senior tester might spend a day testing an area that will otherwise be ignored)

A form of exploratory testing

exploratory testing

tester learns about the product, its market, its risks, and the ways in which it has failed previous tests

smoke testing

Smoke test things you expect to work, and if they don’t, you’ll suspect that the program was built with the wrong file or that something basic is broken.

(type of side-effect regression testing) the goal is to prove that a new build is not worth testing.

scripted testing

Manual testing done by a junior tester who follows a step-by-step procedure written by a senior tester.

regression testing

3 kinds

Side-effect regression (stability regression): retesting of substantial parts of the product to prove that the change has caused something that used to work to now be broken.

Old bugs regression: prove a change has caused an old bug fix to become unfixed

Bug fix regression: prove the fix is no good.

Reuse of the same old tests after change

Problems-based

computation constraints

inputs are fine but the processing failed

output constraints

inputs were legal, but output caused a failure (save file)

input constraints

a limit on what the program can handle (IP address)

risk-based testing

the greater the probability of an expensive failure, the more important it is to test that feature as early and as carefully as possible

risk analysis to determine what things to test

Coverage-based

combination testing

Testing two or more variables or functions in combination with each other in order to check bad interactions (e.g. CFU, OCS)

requirements-based testing

Proving that it satisfies every requirement in a requirements document

specification-based testing

often includes every claim made in the manual, in marketing documents or advertisements

Verifying every claim that is made about the product in the specification.

configuration coverage

If you have to test compatibility with 100 printers, and you have tested with 10, you have achieved 10% printer coverage.

statement and branch coverage (Coverage-based testing)

100 % statement and branch coverage if you execute every statement and every branch from one statement to another

100 % statement coverage if your tests execute every statement (or line of code)

path testing

e.g. interface menu, CFNA vs. CFBL.

A (logical) path includes all of the steps that you took that the program passed through in order to get to your current state

state-based testing

A program moves from state to state. In a given state, some inputs are valid, and others are ignored or rejected

domain testing

Change the value of a field in several ways. (import data into the field, type it in, copy paste and so on)

For each var, partition its set of possible values into equivalence classes and pick a small number of representatives from each class

Identify the functions and the variables

menu tour

Walk through all of the menus and dialogs in a GUI product, taking every available choice

feature or function integration testing

Test several functions together, to see how they work together

black box function testing

focuses on things the user can do or select (commands and features)

white box function testing

concentrates on the functions as you see them in the code

called unit testing

function testing

It’s wise to do function testing before doing more complex tests that involve several functions

Test the function thoroughly, to the extent that you can say with confidence that it works.

People-based (testers)

eat your own food

company relies on prerelease versions of its own software, waiting until the software is reliable enough for real use (Beta) before selling it

paired testing

Two testers work together to find bugs (sharing one computer)

subject-matter expert testing

expert: may or may not be someone you would expect to use the product

Give the product to an expert on some issues addressed by the sw and request feedback

bug bashes

A typical bug-bash lasts a half-day and is done when the software is close to being ready to release.

In-house testing using secretaries, programmers, marketers, and anyone who is available

Beta testing

product under test is very close to completion

Testers aren’t part of your organization and are members of your product’s target market

Alpha testing

In-house testing by the test team or other insiders

user testing