Add Playwright E2E testing infrastructure
- Add playwright-web-flake to flake.nix for NixOS browser support - Pin @playwright/test@1.56.1 to match nixpkgs version - Create playwright.config.ts with Chromium-only, auto-start dev server - Add e2e/smoke.spec.ts with initial smoke tests - Add .mcp.json for Claude browser control via MCP - Update .gitignore for playwright artifacts - Remove E2E test skip from spec.md Known Limitations - Update specs/testing.md to require three-tier testing approach Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
130
specs/testing.md
130
specs/testing.md
@@ -6,7 +6,7 @@ When I make changes to the codebase, I want automated tests to catch regressions
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
PhaseFlow uses **unit and integration tests** with Vitest. End-to-end tests are not required for MVP (authorized skip).
|
||||
PhaseFlow uses a **three-tier testing approach**: unit tests, integration tests, and end-to-end tests.
|
||||
|
||||
### Test Types
|
||||
|
||||
@@ -14,6 +14,7 @@ PhaseFlow uses **unit and integration tests** with Vitest. End-to-end tests are
|
||||
|------|-------|-------|----------|
|
||||
| Unit | Pure functions, utilities | Vitest | Colocated `*.test.ts` |
|
||||
| Integration | API routes, PocketBase interactions | Vitest + supertest | Colocated `*.test.ts` |
|
||||
| E2E | Full user flows, browser interactions | Playwright | `e2e/*.spec.ts` |
|
||||
|
||||
## Framework
|
||||
|
||||
@@ -148,9 +149,96 @@ describe('GET /api/today', () => {
|
||||
});
|
||||
```
|
||||
|
||||
## End-to-End Tests
|
||||
|
||||
Test complete user flows through the browser using Playwright.
|
||||
|
||||
### Framework
|
||||
|
||||
**Playwright** - Cross-browser E2E testing with auto-waiting and tracing.
|
||||
|
||||
**Configuration (`playwright.config.ts`):**
|
||||
```typescript
|
||||
import { defineConfig, devices } from "@playwright/test";
|
||||
|
||||
export default defineConfig({
|
||||
testDir: "./e2e",
|
||||
fullyParallel: true,
|
||||
forbidOnly: !!process.env.CI,
|
||||
retries: process.env.CI ? 2 : 0,
|
||||
reporter: [["html", { open: "never" }], ["list"]],
|
||||
use: {
|
||||
baseURL: "http://localhost:3000",
|
||||
trace: "on-first-retry",
|
||||
screenshot: "only-on-failure",
|
||||
},
|
||||
projects: [
|
||||
{ name: "chromium", use: { ...devices["Desktop Chrome"] } },
|
||||
],
|
||||
webServer: {
|
||||
command: "pnpm dev",
|
||||
url: "http://localhost:3000",
|
||||
reuseExistingServer: !process.env.CI,
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Priority Targets
|
||||
|
||||
| Flow | Tests |
|
||||
|------|-------|
|
||||
| Authentication | Login page loads, login redirects, logout works |
|
||||
| Dashboard | Shows decision, displays cycle info, override toggles work |
|
||||
| Settings | Garmin token paste, preferences save |
|
||||
| Calendar | ICS feed accessible, calendar view renders |
|
||||
| Period logging | "Period Started" updates cycle |
|
||||
|
||||
### Example Test
|
||||
|
||||
```typescript
|
||||
// e2e/dashboard.spec.ts
|
||||
import { expect, test } from "@playwright/test";
|
||||
|
||||
test.describe("dashboard", () => {
|
||||
test("shows training decision for authenticated user", async ({ page }) => {
|
||||
// Login first (or use auth state)
|
||||
await page.goto("/");
|
||||
|
||||
// Verify decision card is visible
|
||||
await expect(page.getByTestId("decision-card")).toBeVisible();
|
||||
|
||||
// Verify cycle day is displayed
|
||||
await expect(page.getByText(/Day \d+ of \d+/)).toBeVisible();
|
||||
});
|
||||
|
||||
test("override toggles update decision", async ({ page }) => {
|
||||
await page.goto("/");
|
||||
|
||||
// Enable flare mode
|
||||
await page.getByLabel("Flare Mode").click();
|
||||
|
||||
// Decision should change to gentle
|
||||
await expect(page.getByText(/GENTLE/)).toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### NixOS Setup
|
||||
|
||||
E2E tests require browser binaries. On NixOS, use `playwright-web-flake`:
|
||||
|
||||
```nix
|
||||
# In flake.nix inputs
|
||||
inputs.playwright-web-flake.url = "github:pietdevries94/playwright-web-flake/1.56.1";
|
||||
|
||||
# In devShell
|
||||
PLAYWRIGHT_BROWSERS_PATH = "${playwright-driver.browsers}";
|
||||
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD = "1";
|
||||
```
|
||||
|
||||
## File Naming
|
||||
|
||||
Tests colocated with source files:
|
||||
Tests colocated with source files (unit/integration) or in `e2e/` directory (E2E):
|
||||
|
||||
```
|
||||
src/
|
||||
@@ -164,22 +252,33 @@ src/
|
||||
today/
|
||||
route.ts
|
||||
route.test.ts
|
||||
e2e/
|
||||
smoke.spec.ts
|
||||
dashboard.spec.ts
|
||||
auth.spec.ts
|
||||
settings.spec.ts
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
npm test
|
||||
|
||||
# Run with coverage
|
||||
npm run test:coverage
|
||||
# Run unit/integration tests
|
||||
pnpm test:run
|
||||
|
||||
# Run in watch mode
|
||||
npm run test:watch
|
||||
pnpm test
|
||||
|
||||
# Run specific file
|
||||
npm test -- src/lib/cycle.test.ts
|
||||
# Run E2E tests (headless)
|
||||
pnpm test:e2e
|
||||
|
||||
# Run E2E tests (visible browser)
|
||||
pnpm test:e2e:headed
|
||||
|
||||
# Run E2E tests with UI mode
|
||||
pnpm test:e2e:ui
|
||||
|
||||
# Run all tests (unit + E2E)
|
||||
pnpm test:run && pnpm test:e2e
|
||||
```
|
||||
|
||||
## Coverage Expectations
|
||||
@@ -190,15 +289,20 @@ No strict coverage thresholds for MVP, but aim for:
|
||||
|
||||
## Success Criteria
|
||||
|
||||
1. All tests pass in CI before merge
|
||||
2. Core decision engine logic has comprehensive tests
|
||||
1. All tests (unit, integration, E2E) pass in CI before merge
|
||||
2. Core decision engine logic has comprehensive unit tests
|
||||
3. Phase scaling tested for multiple cycle lengths
|
||||
4. API auth tested for protected routes
|
||||
5. Critical user flows covered by E2E tests
|
||||
|
||||
## Acceptance Tests
|
||||
|
||||
- [ ] `npm test` runs without errors
|
||||
- [ ] `pnpm test:run` runs without errors
|
||||
- [ ] `pnpm test:e2e` runs without errors
|
||||
- [ ] Unit tests cover decision engine logic
|
||||
- [ ] Unit tests cover cycle phase calculations
|
||||
- [ ] Integration tests verify API authentication
|
||||
- [ ] E2E tests verify login flow
|
||||
- [ ] E2E tests verify dashboard displays correctly
|
||||
- [ ] E2E tests verify period logging works
|
||||
- [ ] Tests run in CI pipeline
|
||||
|
||||
Reference in New Issue
Block a user