11 min read

Lessons learned after one year of Playwright

It’s been a year since my occasional interest in Playwright turned into a full-time job. I had to build a test automation solution from scratch and constantly make decisions on how to solve one problem after another.

These practices are pretty opinionated. They worked for me in my circumstances and addressed real test automation pains, so you might find them helpful as well. Some of them are Playwright-specific, while others are general test automation principles.

0. Setup strict linting and formatting

Once you’ve finished your initial boilerplate setup, make sure to configure strict linting for your project. It’s okay if you can’t envision everything in advance; you will most certainly come back and adjust your rules as you go.

It helps to enforce a unified code style across your project and between collaborators. But even more important are the test automation rules you will enforce. For example, there is a decent ESLint plugin for Playwright. With its rules, you can facilitate basic sanity checks on your test code. Red underlines and linter warnings will remind you to avoid logical statements inside test structures or to remove .only from tests after debugging.

1. Use entities for logic above models

In this great article about the so-called “Component-Based Approach,” I stumbled upon this quote:

The solution is to align our automation design with how modern applications are built (…)

Richa Pandey

I really liked the idea and applied it to my project. The problem was that with traditional POMs, I didn’t know where to place business logic that resides above any particular model. For example, validation constraints for a user entity are identical across the user registration form for guests, the profile form for users, and the user edit form for admins.

By aligning automation and application designs, I introduced entities as a new abstract concept. They hold all the constants, types, and interfaces I’m gonna need inside my app logic or inside my tests:

// lib/entities/user.ts
// Constants
export const MIN_PASSWORD_LENGTH = 8

export const USER_ROLES = {
  ADMIN: { value: 'Admin' },
  USER: { value: 'User' },
  GUEST: { value: 'Guest' },
} as const

// Types
export type UserRole = (typeof USER_ROLES)[keyof typeof USER_ROLES]['value']

// Interfaces
export interface UserData {
  name: string
  email: string
  password: string
  role: UserRole
}

2. Randomize whenever possible

This principle is so fundamental that it is one of the 7 principles presented in the ISTQB FL Syllabus. It states “Tests wear out,” i.e., tests that are repeated with the same test data will lose efficiency over time due to the “pesticide paradox.”

A test should not verify every time that a registration form is working correctly with “John Doe” input. It should verify that the registration form is working correctly with any valid input. And introducing randomness within proper limits is much simpler with test automation, especially with packages like Faker.

// tests/registration.spec.ts
test('Check that validation error is shown for too long user name', async ({
  registrationPage,
}) => {
  const tooLongName = faker.string.alpha({
    length: { min: MAX_NAME_LENGTH + 1, max: MAX_NAME_LENGTH + 100 },
  })
  await registrationPage.nameField.fill(tooLongName)
  await registrationPage.submitButton.click()
  await expect(registrationPage.errorMessage).toHaveText(
    text.errors.tooLongName,
  )
})

Combined with DRYed business logic constants, we have a test case with a good level of random noise within required constraints. If name length validation is changed, we replace it in one place.

3. Build fluent test flows

Unlike humans, Playwright cannot consider something outside of immediate or timed-out successful actions. It means that once the previous action is finished with any result, the next action is executed right away. When building tests, you need to make sure that the test flow is stable, meaning it’s the only way Playwright can execute your scenario. Sometimes it might require deviating from the manual test flow — it’s okay as long as your tests avoid flakiness. Add an extra comment as to why you deviated and move on.

This test, for instance, might fail because the header is a common element across all pages, so its pageTitle locator will be immediately resolved milliseconds after clicking the button. Since redirection might occur far less immediately, the page title at that moment might remain “Dashboard” instead of “Categories.”

// tests/dashboard.spec.ts
test('Check that user is redirected to categories page', async ({
  header,
  dashboardPage,
}) => {
  await dashboardPage.showCategoriesButton.click()
  await expect(header.pageTitle).toHaveText('Categories')
})

What would prevent this? Adding an action or assertion that is possible only after our expected redirection happened, such as the visibility of an element specific to the new page:

// tests/dashboard.spec.ts
test('Check that user is redirected to categories page', async ({
  header,
  categoriesPage,
  dashboardPage,
}) => {
  await dashboardPage.showCategoriesButton.click()
  await expect(categoriesPage.categoriesList).toBeVisible()
  await expect(header.pageTitle).toHaveText('Categories')
})

4. No, really, avoid waitForTimeout…

Unless super-duper necessary, of course. Having built half a thousand automated tests, I only encountered maybe 20 cases where waitForTimeout() was justified. And they were all related to waiting for something happening deep under the hood without any visible consequences, like switching entity states via API calls.

Why are hardcoded timeouts bad? Because you introduce a superficial setback for your test execution, which goes against the core principle of Playwright itself with all its auto-awaits. Because you introduce a flaky point in your tests. 5 seconds might suffice on an external CI machine but might fail on your localhost.

If you absolutely have to hardcode a timer, give it some extra buffer. For me, the formula is the time it takes to execute on my machine (which is a benchmark of the slowest machine possible 😥) plus 25% extra for occasional universal anomalies.

5. …or at least name its call

I don’t know about you, but my mental pedestrian gets hit by a wall when I go through test code lines and see waitForTimeout() somewhere between actions and assertions:

// tests/dashboard.spec.ts
test('Check that user is redirected to categories page', async ({
  header,
  categoriesPage,
  dashboardPage,
}) => {
  await dashboardPage.gotoThisPage()
  await waitForTimeout(15000)
  await dashboardPage.showCategoriesButton.click()
  await waitForTimeout(30000)
  await expect(categoriesPage.categoriesList).toBeVisible()
})

“Why a timeout? Is it because something heavy started right now, or because something was already happening and needs extra time to finish? Why 30000? It’s 30 seconds, right? Why not 15000 or 5000?” Something like this.

But then I make a declarative function wrapper around timeouts:

// tests/dashboard.spec.ts
test('Check that user is redirected to categories page', async ({
  header,
  categoriesPage,
  dashboardPage,
}) => {
  await dashboardPage.gotoThisPage()
  await dashboardPage.waitForDBLoad()
  await dashboardPage.showCategoriesButton.click()
  await categoriesPage.waitForCategoriesFetch()
  await expect(categoriesPage.categoriesList).toBeVisible()
})

Without any additional comments, I know exactly what we are expecting. And if something changes, I know exactly where to go to adjust the timer.

6. Orchestrate your sessions within test suites

There are several valid designs for robust user session management. Some tests want to run sessionless as guests, some as regular users, and some as admins. Some tests might introduce changes to user information and/or state, which will influence or even break other tests if they are executed in parallel for the same user session.

For this problem, I came up with the second most straightforward approach: create a new context for every session I need and store them in different storage states:

// tests/setup/auth.setup.ts
setup('Authentication Setup', async ({ authPages, page, browser }) => {
  await authPages.gotoThisPage()
  await authPages.auth(creds.admin.email, creds.admin.password)

  await page.waitForURL('/admin')

  // Save admin session to storage state
  await page.context().storageState({
    path: 'test-data/storage-state-admin.json',
  })

  // Create another session based on empty storage state
  const anotherAdminContext = await browser.newContext({
    storageState: 'test-data/storage-state-empty.json',
  })
  const anotherAdminPage = await anotherAdminContext.newPage()

  await anotherAdminPage.gotoThisPage()
  await anotherAdminPage.auth(
    creds.anotherAdmin.email,
    creds.anotherAdmin.password,
  )

  await page.waitForURL('/admin')

  // Save another admin session to storage state
  await anotherAdminContext.storageState({
    path: 'test-data/storage-state-another-admin.json',
  })

  // Don't forget to close another admin context
  await anotherAdminPage.close()
  await anotherAdminContext.close()

  // ...and so on for other sessions
})

Now we have separate sessions we can use and re-use for different tests. The only restriction is that test.use() statements work only with test.describe() or whole test file scopes, i.e., you can’t switch different sessions within the same test suite. That was never an issue for me; we just introduced several test.describe() blocks within a single file. But if you have too many sessions, you should probably consider the first most straightforward approach: test pre- and post-conditions.

7. Preserve test independence

When building a test automation solution, you might encounter situations where it would be so much easier to write dependent or even sequential tests. If for some reason you need to min-max test data generation, it would also tempt you to write tests that reuse the same pieces of data, i.e., dependent tests.

And yes, of course, there are projects and contexts where it won’t matter or would even be a win, but as a general rule of thumb, you should always strive to preserve test independence. Why? Because you might want to run particular tests, not the whole scope of them. Because you might want to increase the number of workers to make CI faster. Because, while there are signs that Playwright simply follows the project order from your config, you still don’t have total control over execution order. Unless, of course, you introduce dependencies to every test project in the Playwright config, but then it will become one big automated test scenario, which does not make much sense.

8. Generate test data at the lowest level possible

When you are designing your automation solution, you should consider how you will create test data for your test preconditions. And while considering, strive for the lowest level possible.

  1. Seeds? Perfect.
  2. Database queries? Fast but might be brittle due to skipping back-end validations.
  3. API calls? Not so perfect due to flakiness but still good.
  4. UI via Playwright setup? Straight terrible.

The interesting part here, though, is that you also should keep an eye on what we’ll call “test cleanliness.” Because the further you are from a “legal” way of creating data as a real user would do it, the more possibilities you have to introduce artifacts, discrepancies, and simply unreal data. And what’s more interesting, the chart for “test cleanliness” is reversed!

  1. UI via Playwright setup? The most natural way to create something.
  2. API calls? Less natural, but it’s still what’s happening under the hood after you do something in the UI, so it works fine.
  3. Database queries? Dangerous due to the aforementioned skip of back-end validations and data normalization.
  4. Seeds? The most “insider” way to create data, thus the most risky.

So, by saying “at the lowest level possible,” I mean the lowest level where you do not break your tests by creating something that is not possible in a real user session.

9. Stay declarative & consistent

When building a test automation solution, make sure to name models, elements, and actions in a way that is easy to understand and read. Do not obfuscate your code with abstract or temporary names, because you will be the first one to get lost in it.

It goes a little further: make your functions do exactly what they say they do and your variables hold exactly what they say they hold. If you have a getUserName() function, do not add implicit assertions to it. If you have a userRegistrationData object, do not add user posts, comments, or other post-registration data to it.

Stick to a consistent naming convention, not only formatting-wise but also terminology-wise and wording-wise. If you have resendPasswordButton somewhere in your models, don’t suddenly start using short versions like sendBtn or switching the word order like buttonSend. It should be sendButton, consistent with what you already have.

This example might sound pretty irrelevant, but when you have to read tests while debugging and constantly jump between emailField, pwdFld, buttonSubmit, name, etc., adjusting your mental parser every time, you’ll understand how much better it is to remain consistent.

10. Implement global setup and teardown as tests

Currently, the Playwright documentation states pretty clearly that global setup and teardown should be implemented as test projects, similarly to tests.

Otherwise, you won’t be able to use fixtures in them, debug them with the Playwright reporter, or check traces. Pretty obvious decision, right? But a couple of years ago, globalSetup from the config seemed like a more “default” solution. Nevermind, just wanted to point this out.

Enjoyed this post?

Subscribe to get notified when I publish another one.