Version control and testing: Concepts

@dave1010:

How?

Imperfect tests run frequently are better than perfect tests which are never written

Important

• Test frequently (each commit)
• Test automatically (e.g. using Travis CI)
• Test with numerical tolerance
• Think about code coverage (Coveralls or Codecov)

Defensive programming

• Assume that mistakes will happen and introduce guards against them.

def kelvin_to_celsius(temp_k):
    """
    Converts temperature in Kelvin
    to Celsius.
    """
    assert temp_k >= 0.0, "ERROR: negative T_K"
    temp_c = temp_k + 273.15
    return temp_c

Unit tests

• Unit tests are functions (Testing and Continuous Integration with Python)
• Test one unit: module or even single function
• Good documentation of the capability and dependencies of a module

Integration tests

• Integration tests verify whether muliple modules are working well together (Testing and Continuous Integration with Python)
• Like in a car assembly we have to test all components independently and also whether the components are working together when combined
• Unit tests can be used for testing independent components (e.g. engine, radiator, transmission) and integration tests to check if car is working overall

Test-driven development

• In test-driven devlopment, one writes tests before writing code
• Very often we know the result that a function is supposed to produce
• Development cycle (red, green, refactor):
  • Write the test
  • Write an empty function template
  • Test that the test fails
  • Program until the test passes
  • Perhaps refactor
  • Move on

Code coverage

• If I break the code and all tests pass who is to blame?
• Code coverage measures and documents which lines of code have been traversed during a test run
• It is possible to have line-by-line coverage (example later)

Total time to test matters

• Total time to test matters
• If the test set takes 7 hours to run, it is likely that nobody will run it
• Identify fast essential test set that has sufficient coverage and is sufficiently short to be run before each commit or push

Continuous integration

• Continuous integration is basically when you automatically test every single commit or merge automatically
• Test each commit (push) on every branch
• Test merges before they are accepted
• Makes it possible for the mainline maintainer to see whether a modification breaks functionality before accepting the merge

Good practices

• Test before committing (use the Git staging area)
• Fix broken tests immediately (dirty dishes effect)
• Do not deactivate tests “temporarily”
• Think about coverage (physics and lines of code)
• Go public with your testing dashboard and issues/tickets
• Test controlled errors: if something is expected to fail, test for that
• Create benchmark calculations to cover various performance-critical modules and monitor timing
• Make testing easy to run (make test)
• Make testing easy to analyze
  • Do not flood screen with pages of output in case everything runs OK
  • Test with numerical tolerance (extremely annoying to compare digits by eye)

Image by @thepracticaldev, CC-BY-NC.

Discussion

• For which situations would you consider automated testing as overkill?