Tests

Tests

  • Verify software functionality and prevent regressions
    • catching issues before they impact users
    • regression testing to ensure new changes don’t break existing functionality
  • Document expected system behavior
  • Reduce debugging time and maintenance costs
  • Support continuous integration/deployment
    • Automated testing is essential for CI/CD pipelines

Types of tests

  1. Unit Testing
    • Validate the smallest units of code (e.g., functions or classes) to ensure they perform as expected.
    • Isolated testing of individual components without external dependencies.
  2. Integration Testing
    • Ensure that different components or modules work together as intended.
    • Interactions between units, databases, APIs, or external services.
  3. System Testing
    • Verify that the software behaves according to specified requirements.

Testing in C++

assert

assert is a macro that checks if a condition is true. If the condition is false, it will terminate the program and print an error message.

assert(condition);

For example:

int add(int a, int b) return a + b;

assert(add(1, 2) == 3);  // pass
assert(add(1, -2) == -1); // pass
assert(add(1, -2) == 3); // fail

To include tests in your CMake project, you can create a test subdirectory and put your test code under it.

├── CMakeLists.txt
├── add
│   ├── CMakeLists.txt
│   ├── add.cpp
│   └── add.h
├── main.cpp
└── test
    ├── CMakeLists.txt
    └── test.cpp

In test/CMakeLists.txt, you can write:

add_executable(test test.cpp)
target_link_libraries(test PRIVATE add)

After building the project, a test executable file will be generated in the build/test directory, it won’t affect the main executable.

Vector class example

Recall we implemented the Vector class in midterm. Now we modify this class to a class template and add some test cases.

The assert version have several issues/limitations:

  • Inconvenient to catch exceptions
  • Hard to test template classes/functions
  • All the test cases are mixed in the main function
    • Hard to organize
    • Hard to run specific test cases
    • One test failure will terminate the program

Catch2

Introduction

Catch2 is a modern, C++-native, header-only, test framework for C++11, C++14, C++17, and above.

To use Catch2 in your CMake project, you can add the following lines to your CMakeLists.txt: (A new feature in CMake 3.14)

include(FetchContent)

FetchContent_Declare(
  Catch2
  GIT_REPOSITORY https://github.com/catchorg/Catch2.git
  GIT_TAG        v3.4.0 # or a later release
)

FetchContent_MakeAvailable(Catch2)

Simple test example

#include <catch2/catch_test_macros.h>

TEST_CASE( "Factorials are computed", "[factorial]" ) {
    REQUIRE( factorial( 1) == 1 );
    REQUIRE( factorial( 2) == 2 );
    REQUIRE( factorial( 3) == 6 );
    REQUIRE( factorial(10) == 3'628'800 );
}
  • TEST_CASE(<name>, <tag>) defines a test case.
  • REQUIRE(<condition>) checks if the condition is true.

Run test cases

After building the project, you can run the test cases in several ways:

  • Run all test cases

    ./build/test/test

  • Run test cases with tag [factorial]

    ./build/test/test "[factorial]"

  • Run test cases with name Factorials are computed

    ./build/test/test "Factorials are computed"

Template test cases

Catch2 supports template test cases, you can define test cases with different types. Use our Vector class as an example.

TEMPLATE_TEST_CASE("Vector: Constructor and Default Initialization", "[template][constructor]", int, double) {
    Vector<TestType> vec(3);
    REQUIRE(vec.get_size() == 3);
}
  • int and double are the types to be tested.
  • TestType is the type parameter, it will be replaced by int and double in the test cases.
  • The other part is the same as the simple test example.

Google Test

Introduction

Google Test is another popular testing framework for C++. Google Test has a more complex structure and more features, it’s widely used in the industry.

To use Google Test in your CMake project, similarly, you can add

include(FetchContent)

include(FetchContent)
FetchContent_Declare(
  googletest
  URL https://github.com/google/googletest/archive/03597a01ee50ed33e9dfd640b249b4be3799d395.zip
)

FetchContent_MakeAvailable(googletest)

Simple test example

#include "gtest/gtest.h"

int Factorial(int n);  // Returns the factorial of n

// Tests factorial of 0.
TEST(FactorialTest, HandlesZeroInput) {
  EXPECT_EQ(Factorial(0), 1);
}

// Tests factorial of positive numbers.
TEST(FactorialTest, HandlesPositiveInput) {
  EXPECT_EQ(Factorial(1), 1);
  EXPECT_EQ(Factorial(2), 2);
  EXPECT_EQ(Factorial(3), 6);
  EXPECT_EQ(Factorial(8), 40320);
}
  • EXPECT_EQ(<expected>, <actual>) checks if the expected value is equal to the actual value.
  • Alternatively, you can use ASSERT_EQ(<expected>, <actual>), it will terminate the program if the condition is false.

Type-parameterized test cases

First we take a look at the Test Fixture in Google Test, it’s inherited from ::testing::Test.

template <typename T>
class VectorTest : public ::testing::Test {
protected:
    void SetUp() override {
        vec = Vector<T>(3);
    }

    void TearDown() override {
        // nothing to do
    }
};
  • Setup() will be called before each test case.
  • TearDown() will be called after each test case.

Next we define the test types.

using TestTypes = ::testing::Types<int, double>;

Finally, we register the test cases in the typed test suite, and define the test logic.

TYPED_TEST_SUITE(VectorTest, TestTypes);

TYPED_TEST(VectorTest, ConstructorAndDefaultInitialization) {
    Vector<T> vec(3);
    EXPECT_EQ(vec.get_size(), 3);
}