Automated tests tell you that a change didn't break something that used to work. Different kinds of tests trade speed, cost and confidence in different ways. The test pyramid is a simple model for how many of each kind to write. It helps you get a test suite that is fast to run and still catches real bugs.
The three layers#
The pyramid has three layers. Each layer up tests a bigger piece of the system, and you need fewer tests at each step up.
| Layer | What it tests | Speed | Typical count |
|---|---|---|---|
| Unit | One function or class, with no I/O | Milliseconds | Hundreds to thousands |
| Integration | Your code plus one real dependency (database, queue, HTTP API) | Seconds | Tens to hundreds |
| End-to-end (E2E) | The full system, as a user or client sees it | Seconds to minutes | A handful |
The names aren't strict. Some teams say "small, medium, large" instead, based on what a test is allowed to touch. The idea stays the same: push each check down to the lowest layer that can catch the bug.
Cost, speed and confidence#
Why fewer tests as you go up? Each layer buys you something different. Higher tests give more realistic confidence, but they cost more to write, run and debug.
| Unit | Integration | E2E | |
|---|---|---|---|
| Run time | Very fast | Medium | Slow |
| Cost to write and maintain | Low | Medium | High |
| Points to the broken line | ✓ | Sometimes | ✗ |
| Catches wiring and config bugs | ✗ | ✓ | ✓ |
| Chance of flaky failures | Low | Medium | High |
A failing unit test usually names the exact function that broke. A failing E2E test only tells you that something, somewhere in the stack, went wrong.
What each layer catches#
Cost is only half the picture. Each layer also finds a different kind of bug, so you need all three.
- Unit tests catch logic errors: off-by-one bugs, wrong branches, bad edge-case handling, wrong math.
- Integration tests catch mistakes at the boundary: a SQL query with a typo, a wrong column type, a bad serializer, a missing migration, a wrong timeout setting.
- E2E tests catch problems that only appear when everything runs together: a broken login flow, a misrouted request, a front end that calls the wrong endpoint.
A useful rule: if a bug escapes to production, write the test at the lowest layer that would have caught it.
The ice-cream cone anti-pattern#
Many teams end up with the opposite shape. The ice-cream cone is the pyramid turned upside down. The team has many manual and E2E tests, some integration tests, and almost no unit tests.
This shape causes predictable problems:
- The suite takes hours, so people run it rarely and find bugs late.
- Failures are hard to debug because each test covers so much code.
- Flaky failures become normal, and people start to ignore red builds.
- Releases wait on a manual test pass.
To fix it, take each slow test and ask which behavior it checks. Move that check down to a unit or integration test. Keep only a few E2E tests for the most important user journeys.
Flaky tests#
Flaky failures came up in both tables above, and they deserve their own section. A flaky test passes and fails on the same code without any change. Flaky tests are expensive because they teach the team to re-run builds instead of reading failures. Most come from a handful of causes:
| Cause | Fix |
|---|---|
Fixed sleep() calls that wait for async work |
Wait for a condition with a timeout, or make the work synchronous in tests |
| Shared state between tests (globals, a shared database row) | Give each test its own data, and reset state after each test |
| Test order matters | Run tests in random order to find it, then remove the shared state |
| Real time, time zones or dates | Inject a clock, or freeze time in the test |
| Randomness | Seed the random generator, or pass the random source in |
| Real network calls to outside services | Replace them with a fake or a local container |
When you find a flaky test, don't add retries and move on. Retries hide the bug, and sometimes the bug is in your code, not the test. Quarantine the test (move it out of the blocking suite), open a ticket, and fix the cause soon.
What to test#
Whatever the layer, a test is only useful if it checks the right thing. Test behavior, not implementation. A good test checks what a unit does from the outside: given these inputs, it returns this output or makes this change. It doesn't check which private helper the code calls, or in what order.
- Test through the public interface of a module.
- Assert on results and visible side effects, not on internal steps.
- Cover the happy path, the edge cases, and the error cases.
- Write a test for each bug you fix, so it can't come back.
- Skip trivial code with no logic, such as plain getters and generated code.
Tests that are tied to the implementation break every time you refactor, even when the behavior is still correct. That makes refactoring slow, which is the opposite of what tests are for.
Coverage is a signal, not a goal#
A common way to judge whether you've tested enough is coverage. Code coverage is the percentage of lines (or branches) that run during your tests. Tools such as coverage.py and the pytest-cov plugin measure it.
Low coverage is a useful warning: it shows code that no test runs at all. High coverage proves much less. A test can run a line without checking its result, so 100% coverage can still miss real bugs.
If a team sets a hard coverage target, people start writing tests that run code without asserting anything. Use coverage reports to find untested areas, then decide if they matter.
Key takeaways#
- Write many fast unit tests, fewer integration tests, and only a few E2E tests.
- Push each check to the lowest layer that can catch the bug.
- The ice-cream cone (mostly E2E and manual tests) gives slow, flaky feedback.
- Fix the cause of a flaky test; do not hide it with retries.
- Test behavior through public interfaces, so refactors do not break tests.
- Use coverage to find gaps, not as a target.