BlogTesting & quality

Testing & quality

Browser testing for complete AI user journeys

Verify the whole AI experience, from submitting a request to reviewing and saving a result. Browser tests catch failures that model evaluations cannot see.

A useful AI response is only one step in a working user journey. A customer can still lose work if a loading state never ends, an approval button stays disabled, or a saved result disappears after refresh. Browser testing checks those connections. Consider a hypothetical report assistant: the user uploads a file, requests a summary, edits the result, and saves it to a project. Every transition needs to behave predictably.

Write journeys around user outcomes

Choose a small set of important paths, including at least one recovery path. For the report assistant, test a successful upload and save, an unsupported file, a failed generation followed by a retry, and leaving the page with unsaved edits. Assert visible outcomes such as a saved status or an actionable error. Avoid making every check depend on the exact phrasing of generated prose when the wording is allowed to vary.

Control the AI response during routine tests

Use a test adapter that can return known results, delayed responses, and failures on demand. This makes it possible to verify the interface without paying for a live generation on every run or depending on variable language. Keep a separate, limited live journey to confirm that the real integration still connects. Clearly identify which checks use fixtures so a passing browser suite is not mistaken for evidence of model quality.

Test transitions and user interruptions

Try a second submission while the first is pending, cancel an operation, and refresh after a result has been saved. Check that stale responses do not overwrite newer work. Include keyboard interaction and a narrow viewport. In Playwright, web-first assertions can wait for expected page conditions; use meaningful conditions instead of arbitrary long sleeps. The test should fail because an outcome never occurred, not because a machine happened to be slower.

Inspect persistence and permissions

Verify that the saved report appears in the correct project and remains available after a new session. Test that another user without permission cannot open it. Confirm that an error does not leave a misleading success message or a half-created record. Keep test accounts and fixtures isolated. When a journey fails, capture enough context to locate the broken step while avoiding real user content in screenshots and diagnostic artifacts.

Sources & further reading