chore: allow storing aria snapshots in files (#33919)

docs/src/api/class-locator.md

@@ -155,7 +155,7 @@ Additional locator to match.
 - returns: <[string]>
 
 Captures the aria snapshot of the given element.
-Read more about [aria snapshots](../aria-snapshots.md) and [`method: LocatorAssertions.toMatchAriaSnapshot`] for the corresponding assertion.
+Read more about [aria snapshots](../aria-snapshots.md) and [`method: LocatorAssertions.toMatchAriaSnapshot#2`] for the corresponding assertion.
 
 **Usage**
 

docs/src/api/class-locatorassertions.md

@@ -446,7 +446,7 @@ Expected options currently selected.
 * since: v1.49
 * langs: python
 
-The opposite of [`method: LocatorAssertions.toMatchAriaSnapshot`].
+The opposite of [`method: LocatorAssertions.toMatchAriaSnapshot#2`].
 
 ### param: LocatorAssertions.NotToMatchAriaSnapshot.expected
 * since: v1.49
@@ -2121,7 +2121,58 @@ Expected options currently selected.
 * since: v1.23
 
 
-## async method: LocatorAssertions.toMatchAriaSnapshot
+## async method: LocatorAssertions.toMatchAriaSnapshot#1
+* since: v1.50
+* langs:
+  - alias-java: matchesAriaSnapshot
+
+Asserts that the target element matches the given [accessibility snapshot](../aria-snapshots.md).
+
+**Usage**
+
+```js
+await expect(page.locator('body')).toMatchAriaSnapshot();
+await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'snapshot' });
+await expect(page.locator('body')).toMatchAriaSnapshot({ path: '/path/to/snapshot.yml' });
+```
+
+```python async
+await expect(page.locator('body')).to_match_aria_snapshot(path='/path/to/snapshot.yml')
+```
+
+```python sync
+expect(page.locator('body')).to_match_aria_snapshot(path='/path/to/snapshot.yml')
+```
+
+```csharp
+await Expect(page.Locator("body")).ToMatchAriaSnapshotAsync(new { Path = "/path/to/snapshot.yml" }); 
+```
+
+```java
+assertThat(page.locator("body")).matchesAriaSnapshot(new LocatorAssertions.MatchesAriaSnapshotOptions().setPath("/path/to/snapshot.yml"));
+```
+
+### option: LocatorAssertions.toMatchAriaSnapshot#1.name
+* since: v1.50
+* langs: js
+- `name` <[string]>
+
+Name of the snapshot to store in the snapshot folder corresponding to this test. Generates ordinal name if not specified.
+
+### option: LocatorAssertions.toMatchAriaSnapshot#1.path
+* since: v1.50
+- `path` <[string]>
+
+Path to the YAML snapshot file.
+
+### option: LocatorAssertions.toMatchAriaSnapshot#1.timeout = %%-js-assertions-timeout-%%
+* since: v1.50
+
+### option: LocatorAssertions.toMatchAriaSnapshot#1.timeout = %%-csharp-java-python-assertions-timeout-%%
+* since: v1.50
+
+
+## async method: LocatorAssertions.toMatchAriaSnapshot#2
 * since: v1.49
 * langs:
   - alias-java: matchesAriaSnapshot
@@ -2170,12 +2221,12 @@ assertThat(page.locator("body")).matchesAriaSnapshot("""
 """);
 ```
 
-### param: LocatorAssertions.toMatchAriaSnapshot.expected
+### param: LocatorAssertions.toMatchAriaSnapshot#2.expected
 * since: v1.49
 - `expected` <string>
 
-### option: LocatorAssertions.toMatchAriaSnapshot.timeout = %%-js-assertions-timeout-%%
+### option: LocatorAssertions.toMatchAriaSnapshot#2.timeout = %%-js-assertions-timeout-%%
 * since: v1.49
 
-### option: LocatorAssertions.toMatchAriaSnapshot.timeout = %%-csharp-java-python-assertions-timeout-%%
+### option: LocatorAssertions.toMatchAriaSnapshot#2.timeout = %%-csharp-java-python-assertions-timeout-%%
 * since: v1.49

docs/src/aria-snapshots.md

@@ -154,7 +154,7 @@ structure of a page, use the [Chrome DevTools Accessibility Pane](https://develo
 
 ## Snapshot matching
 
-The [`method: LocatorAssertions.toMatchAriaSnapshot`] assertion method in Playwright compares the accessible
+The [`method: LocatorAssertions.toMatchAriaSnapshot#2`] assertion method in Playwright compares the accessible
 structure of the locator scope with a predefined aria snapshot template, helping validate the page's state against
 testing requirements.
 

docs/src/release-notes-csharp.md

@@ -9,7 +9,7 @@ toc_max_heading_level: 2
 
 ### Aria snapshots
 
-New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
+New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
 
 ```csharp
 await page.GotoAsync("https://playwright.dev");

docs/src/release-notes-java.md

@@ -8,7 +8,7 @@ toc_max_heading_level: 2
 
 ### Aria snapshots
 
-New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
+New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
 
 ```java
 page.navigate("https://playwright.dev");

docs/src/release-notes-js.md

@@ -15,7 +15,7 @@ import LiteYouTube from '@site/src/components/LiteYouTube';
 
 ### Aria snapshots
 
-New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
+New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
 
 ```js
 await page.goto('https://playwright.dev');

docs/src/release-notes-python.md

@@ -8,7 +8,7 @@ toc_max_heading_level: 2
 
 ### Aria snapshots
 
-New assertion [`method: LocatorAssertions.toMatchAriaSnapshot`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
+New assertion [`method: LocatorAssertions.toMatchAriaSnapshot#2`] verifies page structure by comparing to an expected accessibility tree, represented as YAML.
 
 ```python
 page.goto("https://playwright.dev")

packages/playwright-core/types/types.d.ts

@@ -12428,7 +12428,7 @@ export interface Locator {
 
   /**
    * Captures the aria snapshot of the given element. Read more about [aria snapshots](https://playwright.dev/docs/aria-snapshots) and
-   * [expect(locator).toMatchAriaSnapshot(expected[, options])](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-match-aria-snapshot)
+   * [expect(locator).toMatchAriaSnapshot(expected[, options])](https://playwright.dev/docs/api/class-locatorassertions#locator-assertions-to-match-aria-snapshot-2)
    * for the corresponding assertion.
    *
    * **Usage**

packages/playwright/src/matchers/toMatchAriaSnapshot.ts

@@ -18,19 +18,25 @@
 import type { LocatorEx } from './matchers';
 import type { ExpectMatcherState } from '../../types/test';
 import { kNoElementsFoundError, matcherHint, type MatcherResult } from './matcherHint';
-import { colors } from 'playwright-core/lib/utilsBundle';
 import { EXPECTED_COLOR } from '../common/expectBundle';
-import { callLogText } from '../util';
+import { callLogText, sanitizeFilePathBeforeExtension, trimLongString } from '../util';
 import { printReceivedStringContainExpectedSubstring } from './expect';
 import { currentTestInfo } from '../common/globals';
 import type { MatcherReceived } from '@injected/ariaSnapshot';
-import { escapeTemplateString } from 'playwright-core/lib/utils';
+import { escapeTemplateString, isString, sanitizeForFilePath } from 'playwright-core/lib/utils';
+import fs from 'fs';
+import path from 'path';
+
+type ToMatchAriaSnapshotExpected = {
+  name?: string;
+  path?: string;
+} | string;
 
 export async function toMatchAriaSnapshot(
   this: ExpectMatcherState,
   receiver: LocatorEx,
-  expected: string,
+  expectedParam: ToMatchAriaSnapshotExpected,
-  options: { timeout?: number, matchSubstring?: boolean } = {},
+  options: { timeout?: number } = {},
 ): Promise<MatcherResult<string | RegExp, string>> {
   const matcherName = 'toMatchAriaSnapshot';
 
@@ -39,7 +45,7 @@ export async function toMatchAriaSnapshot(
     throw new Error(`toMatchAriaSnapshot() must be called during the test`);
 
   if (testInfo._projectInternal.ignoreSnapshots)
-    return { pass: !this.isNot, message: () => '', name: 'toMatchAriaSnapshot', expected };
+    return { pass: !this.isNot, message: () => '', name: 'toMatchAriaSnapshot', expected: '' };
 
   const updateSnapshots = testInfo.config.updateSnapshots;
 
@@ -48,12 +54,25 @@ export async function toMatchAriaSnapshot(
     promise: this.promise,
   };
 
-  if (typeof expected !== 'string') {
+  let expected: string;
-    throw new Error([
+  let expectedPath: string | undefined;
-      matcherHint(this, receiver, matcherName, receiver, expected, matcherOptions),
+  if (isString(expectedParam)) {
-      `${colors.bold('Matcher error')}: ${EXPECTED_COLOR('expected',)} value must be a string`,
+    expected = expectedParam;
-      this.utils.printWithType('Expected', expected, this.utils.printExpected)
+  } else {
-    ].join('\n\n'));
+    if (expectedParam?.path) {
+      expectedPath = expectedParam.path;
+    } else if (expectedParam?.name) {
+      expectedPath = testInfo.snapshotPath(sanitizeFilePathBeforeExtension(expectedParam.name));
+    } else {
+      let snapshotNames = (testInfo as any)[snapshotNamesSymbol] as SnapshotNames;
+      if (!snapshotNames) {
+        snapshotNames = { anonymousSnapshotIndex: 0 };
+        (testInfo as any)[snapshotNamesSymbol] = snapshotNames;
+      }
+      const fullTitleWithoutSpec = [...testInfo.titlePath.slice(1), ++snapshotNames.anonymousSnapshotIndex].join(' ');
+      expectedPath = testInfo.snapshotPath(sanitizeForFilePath(trimLongString(fullTitleWithoutSpec)) + '.yml');
+    }
+    expected = await fs.promises.readFile(expectedPath, 'utf8').catch(() => '');
   }
 
   const generateMissingBaseline = updateSnapshots === 'missing' && !expected;
@@ -102,8 +121,24 @@ export async function toMatchAriaSnapshot(
     if ((updateSnapshots === 'all') ||
         (updateSnapshots === 'changed' && pass === this.isNot) ||
         generateMissingBaseline) {
+      if (expectedPath) {
+        await fs.promises.mkdir(path.dirname(expectedPath), { recursive: true });
+        await fs.promises.writeFile(expectedPath, typedReceived.regex, 'utf8');
+        const relativePath = path.relative(process.cwd(), expectedPath);
+        if (updateSnapshots === 'missing') {
+          const message = `A snapshot doesn't exist at ${relativePath}, writing actual.`;
+          testInfo._hasNonRetriableError = true;
+          testInfo._failWithError(new Error(message));
+        } else {
+          const message = `A snapshot is generated at ${relativePath}.`;
+          /* eslint-disable no-console */
+          console.log(message);
+        }
+        return { pass: true, message: () => '', name: 'toMatchAriaSnapshot' };
+      } else {
         const suggestedRebaseline = `toMatchAriaSnapshot(\`\n${escapeTemplateString(indent(typedReceived.regex, '{indent}  '))}\n{indent}\`)`;
-      return { pass: this.isNot, message: () => '', name: 'toMatchAriaSnapshot', suggestedRebaseline };
+        return { pass: false, message: () => '', name: 'toMatchAriaSnapshot', suggestedRebaseline };
+      }
     }
   }
 
@@ -134,3 +169,9 @@ function unshift(snapshot: string): string {
 function indent(snapshot: string, indent: string): string {
   return snapshot.split('\n').map(line => indent + line).join('\n');
 }
+
+const snapshotNamesSymbol = Symbol('snapshotNames');
+
+type SnapshotNames = {
+  anonymousSnapshotIndex: number;
+};

packages/playwright/types/test.d.ts

@@ -8430,6 +8430,37 @@ interface LocatorAssertions {
     timeout?: number;
   }): Promise<void>;
 
+  /**
+   * Asserts that the target element matches the given [accessibility snapshot](https://playwright.dev/docs/aria-snapshots).
+   *
+   * **Usage**
+   *
+   * ```js
+   * await expect(page.locator('body')).toMatchAriaSnapshot();
+   * await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'snapshot' });
+   * await expect(page.locator('body')).toMatchAriaSnapshot({ path: '/path/to/snapshot.yml' });
+   * ```
+   *
+   * @param options
+   */
+  toMatchAriaSnapshot(options?: {
+    /**
+     * Name of the snapshot to store in the snapshot folder corresponding to this test. Generates ordinal name if not
+     * specified.
+     */
+    name?: string;
+
+    /**
+     * Path to the YAML snapshot file.
+     */
+    path?: string;
+
+    /**
+     * Time to retry the assertion for in milliseconds. Defaults to `timeout` in `TestConfig.expect`.
+     */
+    timeout?: number;
+  }): Promise<void>;
+
   /**
    * Asserts that the target element matches the given [accessibility snapshot](https://playwright.dev/docs/aria-snapshots).
    *

tests/playwright-test/aria-snapshot-file.spec.ts

@@ -0,0 +1,172 @@
+/**
+ * Copyright Microsoft Corporation. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import { test, expect } from './playwright-test-fixtures';
+import fs from 'fs';
+import path from 'path';
+
+test.describe.configure({ mode: 'parallel' });
+
+test('should match snapshot with name', async ({ runInlineTest }, testInfo) => {
+  const result = await runInlineTest({
+    'playwright.config.ts': `
+      export default {
+        snapshotPathTemplate: '__snapshots__/{testFilePath}/{arg}{ext}',
+      };
+    `,
+    '__snapshots__/a.spec.ts/test.yml': `
+      - heading "hello world"
+    `,
+    'a.spec.ts': `
+      import { test, expect } from '@playwright/test';
+      test('test', async ({ page }) => {
+        await page.setContent(\`<h1>hello world</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'test.yml' });
+      });
+    `
+  });
+
+  expect(result.exitCode).toBe(0);
+});
+
+test('should match snapshot with path', async ({ runInlineTest }, testInfo) => {
+  const result = await runInlineTest({
+    'test.yml': `
+      - heading "hello world"
+    `,
+    'a.spec.ts': `
+      import { test, expect } from '@playwright/test';
+      import path from 'path';
+      test('test', async ({ page }) => {
+        await page.setContent(\`<h1>hello world</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot({ path: path.resolve(__dirname, 'test.yml') });
+      });
+    `
+  });
+
+  expect(result.exitCode).toBe(0);
+});
+
+test('should generate multiple missing', async ({ runInlineTest }, testInfo) => {
+  const result = await runInlineTest({
+    'playwright.config.ts': `
+      export default {
+        snapshotPathTemplate: '__snapshots__/{testFilePath}/{arg}{ext}',
+      };
+    `,
+    'a.spec.ts': `
+      import { test, expect } from '@playwright/test';
+      test('test', async ({ page }) => {
+        await page.setContent(\`<h1>hello world</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'test-1.yml' });
+        await page.setContent(\`<h1>hello world 2</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'test-2.yml' });
+      });
+    `
+  });
+
+  expect(result.exitCode).toBe(1);
+  expect(result.output).toContain(`A snapshot doesn't exist at __snapshots__${path.sep}a.spec.ts${path.sep}test-1.yml, writing actual`);
+  expect(result.output).toContain(`A snapshot doesn't exist at __snapshots__${path.sep}a.spec.ts${path.sep}test-2.yml, writing actual`);
+  const snapshot1 = await fs.promises.readFile(testInfo.outputPath('__snapshots__/a.spec.ts/test-1.yml'), 'utf8');
+  expect(snapshot1).toBe('- heading "hello world" [level=1]');
+  const snapshot2 = await fs.promises.readFile(testInfo.outputPath('__snapshots__/a.spec.ts/test-2.yml'), 'utf8');
+  expect(snapshot2).toBe('- heading "hello world 2" [level=1]');
+});
+
+test('should rebaseline all', async ({ runInlineTest }, testInfo) => {
+  const result = await runInlineTest({
+    'playwright.config.ts': `
+      export default {
+        snapshotPathTemplate: '__snapshots__/{testFilePath}/{arg}{ext}',
+      };
+    `,
+    '__snapshots__/a.spec.ts/test-1.yml': `
+      - heading "foo"
+    `,
+    '__snapshots__/a.spec.ts/test-2.yml': `
+      - heading "bar"
+    `,
+    'a.spec.ts': `
+      import { test, expect } from '@playwright/test';
+      test('test', async ({ page }) => {
+        await page.setContent(\`<h1>hello world</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'test-1.yml' });
+        await page.setContent(\`<h1>hello world 2</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'test-2.yml' });
+      });
+    `
+  }, { 'update-snapshots': 'all' });
+
+  expect(result.exitCode).toBe(0);
+  expect(result.output).toContain(`A snapshot is generated at __snapshots__${path.sep}a.spec.ts${path.sep}test-1.yml`);
+  expect(result.output).toContain(`A snapshot is generated at __snapshots__${path.sep}a.spec.ts${path.sep}test-2.yml`);
+  const snapshot1 = await fs.promises.readFile(testInfo.outputPath('__snapshots__/a.spec.ts/test-1.yml'), 'utf8');
+  expect(snapshot1).toBe('- heading "hello world" [level=1]');
+  const snapshot2 = await fs.promises.readFile(testInfo.outputPath('__snapshots__/a.spec.ts/test-2.yml'), 'utf8');
+  expect(snapshot2).toBe('- heading "hello world 2" [level=1]');
+});
+
+test('should not rebaseline matching', async ({ runInlineTest }, testInfo) => {
+  const result = await runInlineTest({
+    'playwright.config.ts': `
+      export default {
+        snapshotPathTemplate: '__snapshots__/{testFilePath}/{arg}{ext}',
+      };
+    `,
+    '__snapshots__/a.spec.ts/test.yml': `
+      - heading "hello world"
+    `,
+    'a.spec.ts': `
+      import { test, expect } from '@playwright/test';
+      test('test', async ({ page }) => {
+        await page.setContent(\`<h1>hello world</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot({ name: 'test.yml' });
+      });
+    `
+  }, { 'update-snapshots': 'changed' });
+
+  expect(result.exitCode).toBe(0);
+  const snapshot1 = await fs.promises.readFile(testInfo.outputPath('__snapshots__/a.spec.ts/test.yml'), 'utf8');
+  expect(snapshot1.trim()).toBe('- heading "hello world"');
+});
+
+test('should generate snapshot name', async ({ runInlineTest }, testInfo) => {
+  const result = await runInlineTest({
+    'playwright.config.ts': `
+      export default {
+        snapshotPathTemplate: '__snapshots__/{testFilePath}/{arg}{ext}',
+      };
+    `,
+    'a.spec.ts': `
+      import { test, expect } from '@playwright/test';
+      test('test name', async ({ page }) => {
+        await page.setContent(\`<h1>hello world</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot();
+        await page.setContent(\`<h1>hello world 2</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot();
+      });
+    `
+  });
+
+  expect(result.exitCode).toBe(1);
+  expect(result.output).toContain(`A snapshot doesn't exist at __snapshots__${path.sep}a.spec.ts${path.sep}test-name-1.yml, writing actual`);
+  expect(result.output).toContain(`A snapshot doesn't exist at __snapshots__${path.sep}a.spec.ts${path.sep}test-name-2.yml, writing actual`);
+  const snapshot1 = await fs.promises.readFile(testInfo.outputPath('__snapshots__/a.spec.ts/test-name-1.yml'), 'utf8');
+  expect(snapshot1).toBe('- heading "hello world" [level=1]');
+  const snapshot2 = await fs.promises.readFile(testInfo.outputPath('__snapshots__/a.spec.ts/test-name-2.yml'), 'utf8');
+  expect(snapshot2).toBe('- heading "hello world 2" [level=1]');
+});

tests/playwright-test/update-aria-snapshot.spec.ts

@@ -116,6 +116,55 @@ test('should update missing snapshots', async ({ runInlineTest }, testInfo) => {
   expect(result2.exitCode).toBe(0);
 });
 
+test('should update multiple missing snapshots', async ({ runInlineTest }, testInfo) => {
+  const result = await runInlineTest({
+    '.git/marker': '',
+    'a.spec.ts': `
+      import { test, expect } from '@playwright/test';
+      test('test', async ({ page }) => {
+        await page.setContent(\`<h1>hello</h1>\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot(\`\`);
+        await expect(page.locator('body')).toMatchAriaSnapshot(\`\`);
+      });
+    `
+  });
+
+  expect(result.exitCode).toBe(0);
+
+  expect(stripAnsi(result.output).replace(/\\/g, '/')).toContain(`New baselines created for:
+
+  a.spec.ts
+
+  git apply test-results/rebaselines.patch
+`);
+
+  const patchPath = testInfo.outputPath('test-results/rebaselines.patch');
+  const data = fs.readFileSync(patchPath, 'utf-8');
+  expect(trimPatch(data)).toBe(`diff --git a/a.spec.ts b/a.spec.ts
+--- a/a.spec.ts
++++ b/a.spec.ts
+@@ -2,7 +2,11 @@
+       import { test, expect } from '@playwright/test';
+       test('test', async ({ page }) => {
+         await page.setContent(\`<h1>hello</h1>\`);
+-        await expect(page.locator('body')).toMatchAriaSnapshot(\`\`);
+-        await expect(page.locator('body')).toMatchAriaSnapshot(\`\`);
++        await expect(page.locator('body')).toMatchAriaSnapshot(\`
++          - heading "hello" [level=1]
++        \`);
++        await expect(page.locator('body')).toMatchAriaSnapshot(\`
++          - heading "hello" [level=1]
++        \`);
+       });
+
+\\ No newline at end of file
+`);
+
+  execSync(`patch -p1 < ${patchPath}`, { cwd: testInfo.outputPath() });
+  const result2 = await runInlineTest({});
+  expect(result2.exitCode).toBe(0);
+});
+
 test('should generate baseline with regex', async ({ runInlineTest }, testInfo) => {
   const result = await runInlineTest({
     '.git/marker': '',