AI can accelerate Playwright test creation, but generated code is only a draft. A model does not automatically know the live DOM, product requirements, authentication setup or acceptable side effects. The reliable workflow is observe, generate, review, execute and stabilize.

Use this approach to turn an approved user-flow specification into maintainable browser tests. For the wider strategy—where browser tests fit alongside model evaluations and agent traces—start with the AI Testing & Evaluation hub.

What AI should and should not generate

AI is useful for:

• drafting test structure and repeated setup;
• turning acceptance criteria into candidate scenarios;
• suggesting boundary and failure paths;
• explaining a trace or failed assertion;
• refactoring duplicated test helpers.

Do not let it invent:

• product requirements;
• selectors without inspecting the application;
• test credentials or secrets;
• assertions that merely repeat the implementation;
• production actions that are unsafe to replay.

A generated test that passes for the wrong reason is worse than a missing test because it creates false confidence.

Begin with a test contract

Give the model a structured specification rather than “write a login test”:

name: assistant requires approval before sending
start: authenticated conversation page
fixture: pending-message.json
steps:
  - enter a request to email the account owner
  - start the assistant
expected:
  - approval dialog appears
  - recipient and subject are visible
  - no send request occurs before approval
  - cancelling returns to an editable draft
forbidden:
  - real email delivery
  - production credentials

This separates intent from generated code and gives a human reviewer a stable reference.

Ground the model in the real interface

Provide only the context needed for the test:

• the acceptance criteria;
• relevant accessible names or a focused DOM snapshot;
• approved fixtures and API contracts;
• existing test conventions;
• the Playwright version and project configuration.

Prefer Playwright’s locator generator or direct inspection for selectors. Playwright’s test generator prioritizes user-facing locators such as role, text and test ID, but generated actions and assertions still require review.

Avoid sending private production HTML, session tokens or user data to an external model. Local models reduce external disclosure, but they do not remove the need to protect files, logs and credentials.

Prompt for a constrained draft

Create a Playwright Test TypeScript draft for the supplied test contract.

Requirements:
- use existing fixtures and helpers shown in context;
- prefer getByRole/getByLabel over CSS selectors;
- mock the model and tool endpoints specified below;
- assert observable user behavior and network side effects;
- do not use waitForTimeout;
- do not invent credentials, routes, labels or expected copy;
- mark missing information with TODO_REQUIRED_CONTEXT;
- output one test file, without changing application code.

Requiring an explicit marker for missing context is safer than encouraging the model to guess.

Example: control the nondeterministic model boundary

Browser E2E tests should usually control the model response. Evaluate open-ended answer quality in a separate eval suite.

import { test, expect } from '@playwright/test';

test('requires approval before the agent sends a message', async ({ page }) => {
  let sendCalls = 0;

  await page.route('**/api/agent/run', route => route.fulfill({
    status: 200,
    contentType: 'application/json',
    body: JSON.stringify({
      status: 'approval_required',
      action: { type: 'send_email', recipient: 'owner@example.test' }
    })
  }));

  await page.route('**/api/email/send', route => {
    sendCalls += 1;
    return route.fulfill({ status: 202, body: '{}' });
  });

  await page.goto('/assistant');
  await page.getByLabel('Request').fill('Email the account owner');
  await page.getByRole('button', { name: 'Run assistant' }).click();

  await expect(page.getByRole('dialog', { name: 'Approve action' })).toBeVisible();
  expect(sendCalls).toBe(0);
});

The useful assertion is not that the model produced a particular sentence. It is that the application preserved the approval boundary and did not execute the side effect.

Human review checklist

Before accepting generated code, verify:

1. Requirement: every assertion maps to an approved behavior.
2. Selector: locators use stable user-facing semantics or deliberate test IDs.
3. Isolation: data, sessions and side effects cannot leak between tests.
4. Model boundary: provider output is controlled where determinism matters.
5. Failure path: the test would fail if the product behavior broke.
6. Security: no secret, production token or private user data entered the prompt or test.
7. Maintainability: helpers reduce repetition without hiding important behavior.

Review generated changes like any coding-agent contribution. The Coding Agents hub covers scope control and human approval for repository changes.

Reduce flaky tests instead of retrying blindly

AI-generated drafts often contain fixed sleeps or overly broad selectors. Replace them with:

• web-first assertions that wait for the required state;
• explicit waits for the relevant response or event;
• controlled test data and isolated accounts;
• mocked provider responses for deterministic workflow tests;
• stable locators based on accessibility semantics;
• assertions around both UI state and consequential network calls.

Retries can expose intermittent failures, but they should not redefine a flaky test as healthy. Capture a trace for failed or retried CI runs. Playwright traces provide actions, DOM snapshots and network activity that help distinguish a slow model boundary from a broken UI transition.

See Playwright timeout errors in AI browser agents for diagnosis and Playwright vs Cypress for AI applications for the framework decision.

Run the reviewed suite in CI

A safe workflow should:

• install a pinned dependency set and browser version;
• start an isolated application environment;
• inject only the minimum test credentials;
• block external side effects or use sandbox services;
• run reviewed tests, not freshly generated code;
• retain reports and traces on failure;
• prevent untrusted pull requests from accessing deployment secrets.

Generate or revise tests in a separate, reviewable step. Do not ask a model to create new tests and immediately execute them with privileged credentials in the same job.

Connect the pipeline to CI/CD for AI applications, AI Security and AI Operations.

Where generated E2E tests fit

Playwright verifies observable browser behavior: streaming states, tool approvals, retries, navigation and recovery. It does not establish whether an open-ended answer is factually correct or useful.

Use:

• deterministic E2E tests for application behavior;
• evaluation datasets for answer quality;
• trace assertions for agent tool use;
• human review for ambiguous or high-impact decisions;
• production monitoring for failure modes that offline tests missed.

AI test generation is valuable when it shortens the path from a clear requirement to a reviewed executable test. The quality gate remains the test’s evidence—not the fact that an AI produced it.