A test is a small program that checks one thing about your code and then finishes. The important part is not the framework that you use. It is that the test says what the code should do in a way that a person can read, and that it fails for the wrong reason when the code is wrong.
The previous lesson in this module covered "How Do You Debug C++ With a Debugger Instead of print?", and the next lesson in this module is about "How Do CMake and Folder Structure Scale a Project?".
A test is a small program
You do not need a framework to write a test. A test is a program that makes an assertion and then reports whether it held.
#include <cassert>
#include <iostream>
int total_quantity(const int quantities[], int count) {
int total = 0;
for (int i = 0; i < count; ++i) {
total += quantities[i];
}
return total;
}
int main() {
int passed = 0;
{
int values[3] = {5, 3, 2};
assert(total_quantity(values, 3) == 10);
++passed;
}
{
int values[1] = {7};
assert(total_quantity(values, 1) == 7);
++passed;
}
{
int values[1] = {0};
assert(total_quantity(values, 1) == 0);
++passed;
}
std::cout << passed << " tests passed\n";
return 0;
}And the output shows all three of the tests passing.
3 tests passedWhat a good test does
There are four properties that make a test worth having.
the property what it means
------------------------ ----------------------------------------------
it is automatic it runs with one command and says pass
or fail
it is repeatable it gives the same answer every time
it is independent it does not need the other tests to pass
it fails clearly the message says what was expected and what
happened
Test the behaviour and not the implementation
A test that reaches inside a class and checks a private field breaks every time you refactor. A test that calls the public method and checks the answer survives every refactor that keeps the promise.
#include <cassert>
#include <iostream>
class Basket {
public:
void add(const std::string& sku, int quantity) {
if (quantity <= 0) {
return;
}
lines_.push_back(sku + " x" + std::to_string(quantity));
}
std::size_t line_count() const { return lines_.size(); }
private:
std::vector<std::string> lines_;
};
int main() {
Basket basket;
basket.add("A-100", 2);
basket.add("B-200", 0);
// The public promise: only the valid line was added.
assert(basket.line_count() == 1);
std::cout << "the promise holds\n";
return 0;
}And the output confirms that the promise holds.
the promise holdsThe shape of a test suite
And a suite is a set of the tests with a small runner that counts them and reports the failures. The runner is usually part of the framework, and when you do not have a framework you write it yourself in about twenty lines.
#include <functional>
#include <iostream>
#include <string>
#include <vector>
struct TestCase {
std::string name;
std::function<bool()> run;
};
int main() {
std::vector<TestCase> tests;
tests.push_back({"the basket starts empty", []() {
Basket basket;
return basket.line_count() == 0;
}});
tests.push_back({"a valid line is counted", []() {
Basket basket;
basket.add("A-100", 2);
return basket.line_count() == 1;
}});
tests.push_back({"a zero quantity is ignored", []() {
Basket basket;
basket.add("A-100", 0);
return basket.line_count() == 0;
}});
int failed = 0;
for (const TestCase& test : tests) {
if (test.run()) {
std::cout << "pass " << test.name << '\n';
} else {
std::cout << "FAIL " << test.name << '\n';
++failed;
}
}
std::cout << (tests.size() - failed) << " of " << tests.size() << passed\n";
return failed == 0 ? 0 : 1;
}Using a framework
And the frameworks are worth using once the project grows. They give you the runner, the reporting, the filtering and the integration with the build. The most common ones in the C++ world are GoogleTest and Catch2.
#include <gtest/gtest.h>
TEST(Basket, StartsEmpty) {
Basket basket;
EXPECT_EQ(basket.line_count(), 0u);
}
TEST(Basket, CountsAValidLine) {
Basket basket;
basket.add("A-100", 2);
EXPECT_EQ(basket.line_count(), 1u);
}
TEST(Basket, IgnoresAZeroQuantity) {
Basket basket;
basket.add("A-100", 0);
EXPECT_EQ(basket.line_count(), 0u);
}And the output is the same shape whatever framework you use.
[==========] Running 3 tests from 1 test suite.
[----------] 3 tests from Basket
[ RUN ] Basket.StartsEmpty
[ OK ] Basket.StartsEmpty
[ RUN ] Basket.CountsAValidLine
[ OK ] Basket.CountsAValidLine
[ RUN ] Basket.IgnoresAZeroQuantity
[ OK ] Basket.IgnoresAZeroQuantity
[==========] 3 tests from 1 test suite ran.
[ PASSED ] 3 tests.Quick reference
- A test is a small program that checks one thing and then finishes.
- You write the assertion so that the failure message says what was expected.
- You test the public promise and not the private internals.
- The tests must be independent, so you can run them in any order.
- You add a failing test before you fix a bug, and you keep it forever.
- You use the plain assertions in a small project and a framework in a large one.
Try it yourself: write three tests for a small function that you already have, using the plain assertions. Then run them and confirm that each one fails when you break the function in a different way. Finally, move the tests into their own file, link them against the code separately, and check that the build now has two targets.