The spirit of this change is reverting #23153. Since that time, we have
moved tracing and `artifactsDir` lifetime into the test runner, so the
reason for revert is mitigated.
Fixes#30287, fixes#30718, fixes#30959.
Previously, screenshot was taken after hooks and fixtures teardown.
However, hooks can easily modify the state of the page, and
screenshot would not reflect the moment of failure.
Instead, we take screenshots immediately after the test function
finishes with an error.
Reland: worker.stop() before worker.run() was hanging because
`_runFinished` promise was not initially resolved.
---
This moves `beforeAll`, `afterAll` and some modifiers from running
as a separate entity into running inside a test.
Pros:
- All errors are reported as test errors.
- All artifacts are collected as test artifacts.
- Reporters support this out of the box.
Details:
- Each test computes the necessary hooks to run and runs them.
- Teardown is usually performed during the test (on test failure or worker stop).
- `skipRemaining` is added to `DonePayload` to preserve the behavior
where `beforeAll` hook failure skips subsequent tests.
This behavior can now be improved to only target tests affected by this hook.
* Revert "fix(hooks): separate test timeout from beforeAll/afterAll timeouts (#12413)"
This reverts commit 73dee69558.
* Revert "fix(test-runner): rely on test title paths instead of ordinal (#12414)"
This reverts commit d744a87aee.
* Revert "chore(test runner): run hooks/modifiers as a part of the test (#12329)"
This reverts commit 47045ba48d.
chore(test runner): run hooks/modifiers as a part of the test
This moves `beforeAll`, `afterAll` and some modifiers from running
as a separate entity into running inside a test.
Pros:
- All errors are reported as test errors.
- All artifacts are collected as test artifacts.
- Reporters support this out of the box.
Details:
- Each test computes the necessary hooks to run and runs them.
- Teardown is usually performed during the test (on test failure or worker stop).
- `skipRemaining` is added to `DonePayload` to preserve the behavior
where `beforeAll` hook failure skips subsequent tests.
This behavior can now be improved to only target tests affected by this hook.
When sharing a context between tests and using `'on-first-retry'` we
could end up with tracing still running in non-retried tests. That's
extra overhead without a reason.
- Uses some auto fixtures to set default options and instrumentation on BrowserType.
- Moves screenshot, trace and video to worker-scoped fixtures.
- Throws in page/context when used from beforeAll/afterAll.
- Plumbs around BrowserType to be accessible from Browser and BrowserContext.
Simon Knott