- Add playwright-web-flake to flake.nix for NixOS browser support - Pin @playwright/test@1.56.1 to match nixpkgs version - Create playwright.config.ts with Chromium-only, auto-start dev server - Add e2e/smoke.spec.ts with initial smoke tests - Add .mcp.json for Claude browser control via MCP - Update .gitignore for playwright artifacts - Remove E2E test skip from spec.md Known Limitations - Update specs/testing.md to require three-tier testing approach Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
309 lines
7.7 KiB
Markdown
309 lines
7.7 KiB
Markdown
# Testing Specification
|
|
|
|
## Job to Be Done
|
|
|
|
When I make changes to the codebase, I want automated tests to catch regressions, so that I can deploy with confidence.
|
|
|
|
## Testing Strategy
|
|
|
|
PhaseFlow uses a **three-tier testing approach**: unit tests, integration tests, and end-to-end tests.
|
|
|
|
### Test Types
|
|
|
|
| Type | Scope | Tools | Location |
|
|
|------|-------|-------|----------|
|
|
| Unit | Pure functions, utilities | Vitest | Colocated `*.test.ts` |
|
|
| Integration | API routes, PocketBase interactions | Vitest + supertest | Colocated `*.test.ts` |
|
|
| E2E | Full user flows, browser interactions | Playwright | `e2e/*.spec.ts` |
|
|
|
|
## Framework
|
|
|
|
**Vitest** - Fast, Vite-native test runner with TypeScript support.
|
|
|
|
**Configuration (`vitest.config.ts`):**
|
|
```typescript
|
|
import { defineConfig } from 'vitest/config';
|
|
|
|
export default defineConfig({
|
|
test: {
|
|
globals: true,
|
|
environment: 'node',
|
|
include: ['src/**/*.test.ts'],
|
|
coverage: {
|
|
provider: 'v8',
|
|
reporter: ['text', 'html'],
|
|
},
|
|
},
|
|
});
|
|
```
|
|
|
|
## Unit Tests
|
|
|
|
Test pure functions in isolation without external dependencies.
|
|
|
|
### Priority Targets
|
|
|
|
| Module | Functions | Priority |
|
|
|--------|-----------|----------|
|
|
| `src/lib/cycle.ts` | `getCycleDay`, `getPhase`, `getPhaseConfig` | High |
|
|
| `src/lib/decision-engine.ts` | `getTrainingDecision` | High |
|
|
| `src/lib/nutrition.ts` | `getNutritionGuidance`, `getSeeds`, `getMacros` | Medium |
|
|
| `src/lib/ics.ts` | `generatePhaseEvents`, `generateCalendarFeed` | Medium |
|
|
| `src/lib/encryption.ts` | `encrypt`, `decrypt` | Medium |
|
|
|
|
### Example Test
|
|
|
|
```typescript
|
|
// src/lib/cycle.test.ts
|
|
import { describe, it, expect } from 'vitest';
|
|
import { getCycleDay, getPhase } from './cycle';
|
|
|
|
describe('getCycleDay', () => {
|
|
it('returns 1 on period start date', () => {
|
|
const lastPeriod = new Date('2024-01-01');
|
|
const today = new Date('2024-01-01');
|
|
expect(getCycleDay(lastPeriod, 31, today)).toBe(1);
|
|
});
|
|
|
|
it('handles cycle rollover', () => {
|
|
const lastPeriod = new Date('2024-01-01');
|
|
const today = new Date('2024-02-01'); // Day 32
|
|
expect(getCycleDay(lastPeriod, 31, today)).toBe(1);
|
|
});
|
|
});
|
|
|
|
describe('getPhase', () => {
|
|
it('returns MENSTRUAL for days 1-3', () => {
|
|
expect(getPhase(1, 31)).toBe('MENSTRUAL');
|
|
expect(getPhase(3, 31)).toBe('MENSTRUAL');
|
|
});
|
|
|
|
it('scales correctly for 28-day cycle', () => {
|
|
// LATE_LUTEAL should be days 22-28 for 28-day cycle
|
|
expect(getPhase(22, 28)).toBe('LATE_LUTEAL');
|
|
expect(getPhase(21, 28)).toBe('EARLY_LUTEAL');
|
|
});
|
|
});
|
|
```
|
|
|
|
## Integration Tests
|
|
|
|
Test API routes and PocketBase interactions.
|
|
|
|
### Setup
|
|
|
|
Use a test PocketBase instance or PocketBase's testing utilities.
|
|
|
|
```typescript
|
|
// src/test/setup.ts
|
|
import PocketBase from 'pocketbase';
|
|
|
|
export const testPb = new PocketBase(process.env.TEST_POCKETBASE_URL);
|
|
|
|
beforeAll(async () => {
|
|
// Setup test data
|
|
});
|
|
|
|
afterAll(async () => {
|
|
// Cleanup
|
|
});
|
|
```
|
|
|
|
### Priority Targets
|
|
|
|
| Route | Tests |
|
|
|-------|-------|
|
|
| `GET /api/today` | Returns decision with valid auth |
|
|
| `GET /api/cycle/current` | Returns correct phase info |
|
|
| `POST /api/cycle/period` | Updates user record |
|
|
| `GET /api/user` | Returns authenticated user |
|
|
| `PATCH /api/user` | Updates user fields |
|
|
| `GET /api/health` | Returns health status |
|
|
|
|
### Example Test
|
|
|
|
```typescript
|
|
// src/app/api/today/route.test.ts
|
|
import { describe, it, expect } from 'vitest';
|
|
import { GET } from './route';
|
|
|
|
describe('GET /api/today', () => {
|
|
it('returns 401 without auth', async () => {
|
|
const request = new Request('http://localhost/api/today');
|
|
const response = await GET(request);
|
|
expect(response.status).toBe(401);
|
|
});
|
|
|
|
it('returns decision with valid auth', async () => {
|
|
// Setup authenticated request
|
|
const request = new Request('http://localhost/api/today', {
|
|
headers: { Cookie: 'pb_auth=...' },
|
|
});
|
|
const response = await GET(request);
|
|
expect(response.status).toBe(200);
|
|
|
|
const data = await response.json();
|
|
expect(data).toHaveProperty('decision');
|
|
expect(data).toHaveProperty('cycleDay');
|
|
});
|
|
});
|
|
```
|
|
|
|
## End-to-End Tests
|
|
|
|
Test complete user flows through the browser using Playwright.
|
|
|
|
### Framework
|
|
|
|
**Playwright** - Cross-browser E2E testing with auto-waiting and tracing.
|
|
|
|
**Configuration (`playwright.config.ts`):**
|
|
```typescript
|
|
import { defineConfig, devices } from "@playwright/test";
|
|
|
|
export default defineConfig({
|
|
testDir: "./e2e",
|
|
fullyParallel: true,
|
|
forbidOnly: !!process.env.CI,
|
|
retries: process.env.CI ? 2 : 0,
|
|
reporter: [["html", { open: "never" }], ["list"]],
|
|
use: {
|
|
baseURL: "http://localhost:3000",
|
|
trace: "on-first-retry",
|
|
screenshot: "only-on-failure",
|
|
},
|
|
projects: [
|
|
{ name: "chromium", use: { ...devices["Desktop Chrome"] } },
|
|
],
|
|
webServer: {
|
|
command: "pnpm dev",
|
|
url: "http://localhost:3000",
|
|
reuseExistingServer: !process.env.CI,
|
|
},
|
|
});
|
|
```
|
|
|
|
### Priority Targets
|
|
|
|
| Flow | Tests |
|
|
|------|-------|
|
|
| Authentication | Login page loads, login redirects, logout works |
|
|
| Dashboard | Shows decision, displays cycle info, override toggles work |
|
|
| Settings | Garmin token paste, preferences save |
|
|
| Calendar | ICS feed accessible, calendar view renders |
|
|
| Period logging | "Period Started" updates cycle |
|
|
|
|
### Example Test
|
|
|
|
```typescript
|
|
// e2e/dashboard.spec.ts
|
|
import { expect, test } from "@playwright/test";
|
|
|
|
test.describe("dashboard", () => {
|
|
test("shows training decision for authenticated user", async ({ page }) => {
|
|
// Login first (or use auth state)
|
|
await page.goto("/");
|
|
|
|
// Verify decision card is visible
|
|
await expect(page.getByTestId("decision-card")).toBeVisible();
|
|
|
|
// Verify cycle day is displayed
|
|
await expect(page.getByText(/Day \d+ of \d+/)).toBeVisible();
|
|
});
|
|
|
|
test("override toggles update decision", async ({ page }) => {
|
|
await page.goto("/");
|
|
|
|
// Enable flare mode
|
|
await page.getByLabel("Flare Mode").click();
|
|
|
|
// Decision should change to gentle
|
|
await expect(page.getByText(/GENTLE/)).toBeVisible();
|
|
});
|
|
});
|
|
```
|
|
|
|
### NixOS Setup
|
|
|
|
E2E tests require browser binaries. On NixOS, use `playwright-web-flake`:
|
|
|
|
```nix
|
|
# In flake.nix inputs
|
|
inputs.playwright-web-flake.url = "github:pietdevries94/playwright-web-flake/1.56.1";
|
|
|
|
# In devShell
|
|
PLAYWRIGHT_BROWSERS_PATH = "${playwright-driver.browsers}";
|
|
PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD = "1";
|
|
```
|
|
|
|
## File Naming
|
|
|
|
Tests colocated with source files (unit/integration) or in `e2e/` directory (E2E):
|
|
|
|
```
|
|
src/
|
|
lib/
|
|
cycle.ts
|
|
cycle.test.ts
|
|
decision-engine.ts
|
|
decision-engine.test.ts
|
|
app/
|
|
api/
|
|
today/
|
|
route.ts
|
|
route.test.ts
|
|
e2e/
|
|
smoke.spec.ts
|
|
dashboard.spec.ts
|
|
auth.spec.ts
|
|
settings.spec.ts
|
|
```
|
|
|
|
## Running Tests
|
|
|
|
```bash
|
|
# Run unit/integration tests
|
|
pnpm test:run
|
|
|
|
# Run in watch mode
|
|
pnpm test
|
|
|
|
# Run E2E tests (headless)
|
|
pnpm test:e2e
|
|
|
|
# Run E2E tests (visible browser)
|
|
pnpm test:e2e:headed
|
|
|
|
# Run E2E tests with UI mode
|
|
pnpm test:e2e:ui
|
|
|
|
# Run all tests (unit + E2E)
|
|
pnpm test:run && pnpm test:e2e
|
|
```
|
|
|
|
## Coverage Expectations
|
|
|
|
No strict coverage thresholds for MVP, but aim for:
|
|
- 80%+ coverage on `src/lib/` (core logic)
|
|
- Key API routes tested for auth and happy path
|
|
|
|
## Success Criteria
|
|
|
|
1. All tests (unit, integration, E2E) pass in CI before merge
|
|
2. Core decision engine logic has comprehensive unit tests
|
|
3. Phase scaling tested for multiple cycle lengths
|
|
4. API auth tested for protected routes
|
|
5. Critical user flows covered by E2E tests
|
|
|
|
## Acceptance Tests
|
|
|
|
- [ ] `pnpm test:run` runs without errors
|
|
- [ ] `pnpm test:e2e` runs without errors
|
|
- [ ] Unit tests cover decision engine logic
|
|
- [ ] Unit tests cover cycle phase calculations
|
|
- [ ] Integration tests verify API authentication
|
|
- [ ] E2E tests verify login flow
|
|
- [ ] E2E tests verify dashboard displays correctly
|
|
- [ ] E2E tests verify period logging works
|
|
- [ ] Tests run in CI pipeline
|