Version control and testing: Tools

Unit test frameworks

A test framework makes it easy to run tests across large amounts of code automatically. They provide more control than one single script which does some tests.

• Python
  • pytest
  • nose
  • doctest
  • unittest
• R
  • testthat
• C(++)
  • Google Test
  • Catch2
  • cppunit
  • Boost.Test
  • UnitTest++
• Fortran
  • pFUnit
  • FRUIT
  • Ftunit

pytest

Very easy to set up: Anaconda, Miniconda, virtualenv, or pipenv.

Very easy to use: Prefix a function with “test_” and the test runner will execute it. No need to subclass anything.

def get_word_lengths(s):
    """
    Returns a list of integers representing
    the word lengths in string s.
    """
    return [len(w) for w in s.split()]

def test_get_word_lengths():
    text = "Three tomatoes are walking down the street"
    assert get_word_lengths(text) == [5, 8, 3, 7, 4, 3, 6]

• Example output
• Example project

testthat

• Easily installed from CRAN with install.packages("testthat"), or from GitHub with devtools::install_github("r-lib/testthat")
• Use in package development with usethis::use_testthat()

• Add a new test file with usethis::use_test("test-name"), e.g.:

  # tests/testthat/test_example.R
  # file added by running `usethis::use_test("example")`
  
  context("Arithmetics")
  library("mypackage")
  
  test_that("square root function works", {
    expect_equal(my_sqrt(4), 2)
    expect_warning(my_sqrt(-4))
  })
  

  Tests consist of one or more expectations, and multiple tests can be grouped together in one test file. Test files are put in the directory tests/testthat/, and their file names are prefixed with test_.

• Run all tests in package with devtools::test() (if you use RStudio, press Ctrl+Shift+T):

  > devtools::test()
  Loading mypackage
  Testing mypackage
  ✔ |  OK F W S | Context
  ✔ |   2       | Arithmetics
  
  ══ Results ═════════════════════════════════════════════════════════════════════
  OK:       2
  Failed:   0
  Warnings: 0
  Skipped:  0
  

More information in the Testing chapter of the book R Packages by Hadley Wickham.

Google Test

• Widely used
• Very rich in functionality

#include "gtest/gtest.h"
#include "example.h"

TEST(example, add)
{
    double res;
    res = add_numbers(1.0, 2.0);
    ASSERT_NEAR(res, 3.0, 1.0e-11);
}

• Example output
• Example project

pFUnit

• Very rich in functionality
• Requires modern Fortran compilers (uses F2003 standard)

@test
subroutine test_add_numbers()

   use hello
   use pfunit_mod

   implicit none

   real(8) :: res

   call add_numbers(1.0d0, 2.0d0, res)
   @assertEqual(res, 3.0d0)

end subroutine

• Example output
• Example project

Services to deploy testing and coverage

• Travis CI
• GitLab CI
• Coveralls
• Codecov