As a product grows, one type of test stops being enough. Screenshot, UI, API, performance: each one is built for a different purpose and catches a different kind of problem.
Mature projects usually have most of them. Either way, you should know what each one does.
| Type | What it checks | Speed | Catches | Example tools |
|---|---|---|---|---|
| Screenshot | Page renders like the approved image | Medium | Visual and CSS regressions | Playwright, Percy |
| UI | User flows through the real interface | Slow | Broken flows across the whole stack | Playwright, Cypress, Selenium |
| API | Responses of the business layer | Fast | Logic, contract and error-handling bugs | Postman and Newman, Supertest |
| Performance | Speed and stability under load | Depends on the test | Bottlenecks, leaks, capacity limits | k6, JMeter |
What is screenshot testing?
Screenshot testing is an end-to-end technique that captures a screenshot of a given URL and compares it with an expected image. Most often it checks that the user interface isn’t broken and the page renders properly.
Why run screenshot tests?
Your application has to render in browsers from old Internet Explorer to the newest Chrome, on mobile and desktop, across operating systems. Checking all of that by hand is unrealistic. And a developer can break something by accident: a small CSS change is enough to wreck the design.
How to run screenshot tests
You capture screenshots across desktop and mobile devices on different operating systems, and compare each one with an expected image (a snapshot). If the comparison passes, you can deploy the new version. If it doesn’t, the flow can ask a manager or designer to accept or reject the change. The diagram below shows it.
You need to work closely with designers and developers here. Accepting a UI change, or sending it back to the developer because something broke, is their call.
Screenshot testing in the deployment pipeline
Popular screenshot testing tools
- Playwright visual comparisons, built into
@playwright/testwithexpect(page).toHaveScreenshot() - Percy, a hosted service with a review workflow for accepting or rejecting changes
In 2020 I listed Wraith and PhantomCSS here. Both repositories are now archived, so I removed them.
What is UI testing?
UI testing is an end-to-end technique that checks how the application handles user actions made with a mouse, keyboard and other input devices. It also checks that the elements on the page respond correctly.
Why run UI tests?
Imagine a broken UI element that your business depends on. Users can’t book an appointment or buy a book. You lose money, and you lose clients, because people who don’t trust the application don’t stay.
UI tests behave like a real user. They interact with the application and check the result. They also cover more than the presentation layer, because a complex flow needs the business and data layers working too.
How to run UI tests
There are many tools for controlling browsers, listed at the end of this section. You write tests that act like a real user and run them on every deploy, so you know right away when something breaks. You can wire them into the deployment pipeline.
The downside: UI tests are slow. Each one loads a page, interacts with it, probably reloads it. Many tests add up to a long run, so run only the core ones on each deploy. I wrote about that here: smoke, sanity and regression testing.
Slow is not a reason to skip them. You still need to know the whole application works.
UI testing in the deployment pipeline
Popular UI testing tools
What is API testing?
API testing sends calls to an API and validates the response against defined input parameters. It checks that the API returns the correct response, and that it reacts correctly to failures and unexpected input. The input comes from software instead of a user.
Unlike UI testing, it ignores the presentation layer and the feel of the application. It checks the business layer.
Why run API tests?
When you change backend code, automated UI tests are slow and repetitive. API tests skip the UI, so they run much faster, and you find out sooner whether your change broke business behavior.
How to run API tests
You run them much like UI tests. You build a fresh API, run the tests against it, and based on the result you deploy the new version or reject the change.
Many tools can do this, a few are below. API tests matter most when your architecture is built on microservices, because you can check each service separately.
API testing in the deployment pipeline
Popular API testing tools
What is performance testing?
Performance testing is a non-functional technique that measures the speed, scalability, stability and responsiveness of an application under a given workload. It evaluates throughput, data transfer speed, network bandwidth usage and response times.
Why run performance tests?
You need to know whether the application is fast enough on production. Slow software loses users.
Performance tests find bottlenecks, the single points or components that hold back the whole system. Those give users a poor experience, and that costs you money. You can also use the tests to compare two or more systems and pick the faster one.
How to run performance tests
It depends on your organization and application: what you need, and what the business cares about most.
Performance testing in the deployment pipeline
A performance check in CI needs a pass/fail rule, or nobody reads the numbers. In k6 that rule is a threshold in the script, and a crossed threshold fails the run (exit code 99), so the pipeline stops. I show the scripts in load, stress, soak and spike testing with k6. SpeedCurve’s article on breaking the build on performance covers the same idea for front-end metrics.
Popular performance testing tools
Which type of test do you need?
Each type has pros and cons, and which ones you need depends on the project:
- Screenshot testing checks that pages render correctly.
- UI testing checks that users can complete core flows.
- API testing checks the business layer, faster than the UI.
- Performance testing checks speed and stability under load.
If you automate more than one, decide which run on every deploy and which run less often. That is the subject of smoke, sanity and regression testing. To pick tools for the UI layer, start with the Playwright tutorial.