Terminology

• We need to check the program for bugs(testing)
• To reduce the chances of bugs, we should understand program requirements - we can design our program and take into account the different scenarios our program might encounter
• Tests do not guarantee program correctness
• Black-Box Testing
  • tests where tester has no/minimal knowledge of implementation, tests are based off of specifications
  • common when developer is different from tester/tests are written before implementation
  • often tests various input classes, e.g. for numbers, positive, negative, zero, extreme values (edge cases)
  • Corner case - test multiple extremes simultaneously
• White-Box Testing
  • Writing tests with full implementation knowledge - common when person writing tests is the one who implemented the code
• Unit Testing
  • Specific components of programs are tested in isolations
  • Component-level tests where tests are written to check functionality in a given module
• Regression Testing
  • After making any changes to code, test should be rerun - modification of code base might introduce new defects
  • Involves re-running existing tests to confirm code has not regressed/gotten worse
• Errors
  • e.g. compiler error, runtime error, logical error, memory error
  • Runtime errors, logical errors, and memory errors indicate the presence of a bug, compiler errors do not
  • Compiler error - program does not follow rules of programming language
    • Always look at the error message!
  • Runtime error - occur during execution of program, indicate execution of illegal operation (e.g. divide by 0)
  • Logical Error - bug, occurs when program has unexpected behaviour (e.g. logical mistake)
  • Memory error - program uses memory in unexpected/illegal away, e.g. access memory outside of memory provided for program to use

Assertion-based Testing

• C assert - allows for testing a condition - stops program and displays a message
• We can also use assert to check correctness of program’s state at a given point by stating it as boolean expressions
• We can add logic to disable tests in production

I/O Testing

• Involves running the program to be tested by giving it different test inputs - output produced by program is then compared with expected, test failed if output differs form expected output
• Test harness - program that is used to test specific functionality of a program/its subset
  • e.g. for C, separate main function that expects user to provide input in custom format to invoke specific functionality
  • Goal s to test functionality of other code, not test harness itself - focus should be on testing implementation, so avoid testing the test harness
• We can use diff to compare two files - can be used in local environment for testing
• We could have also written the test harness with a loop, choose text input, etc.
• Good to write short tests for specific functionalities - failures of such tests quickly tells us what is wrong