animaltrack/PLAN.md

AnimalTrack Implementation Plan

Check off items as completed. Each phase builds on the previous.

---

Phase 1: Foundation

Step 1.1: Project Structure & Configuration

• Create Python package structure (src/animaltrack/)
• Create pyproject.toml with dependencies
• Create flake.nix with dev environment
• Create .envrc, .gitignore
• Implement config.py with all env vars from spec §18
• Write tests for config loading and validation
• Commit checkpoint (61f704c)

Step 1.2: Database Connection & Pragmas

• Create db.py with connection factory
• Set pragmas: WAL, synchronous=FULL, foreign_keys=ON, busy_timeout=5000
• Create transaction context manager (BEGIN IMMEDIATE)
• Create constants.py with END_OF_TIME_UTC
• Write tests for pragmas and transactions
• Commit checkpoint (d8910d6)

Step 1.3: Migration Framework

• Create migrations.py using FastMigrate patterns
• Create cli.py with migrate and create-migration commands
• Create initial migration for schema_migrations table
• Write tests for migration discovery and application
• Commit checkpoint (7d7cd2b)

Step 1.4: Reference Tables Schema

• Create migration for species, locations, products, feed_types, users tables
• Create Pydantic models for each reference type
• Write tests for model validation and constraints
• Commit checkpoint (2e28827)

Step 1.5: Reference Data Repository & Seeds

• Create repositories for each reference type (CRUD operations)
• Create seeds.py with idempotent seeder
• Seed: users (ppetru, ines, guest), locations (Strip 1-4, Nursery 1-4)
• Seed: species (duck, goose active; sheep inactive)
• Seed: products (egg.duck, meat/offal/fat/bones/feathers/down)
• Seed: feed types (starter, grower, layer - 20kg default)
• Create id_gen.py for ULID generation
• Write tests for repositories and seeding idempotency
• Commit checkpoint (4579229)

---

Phase 2: Event Infrastructure

Step 2.1: Events Table Schema

• Create migration for events, event_revisions, event_tombstones
• Create migration for idempotency_nonces, event_animals tables
• Create Pydantic models for Event, EventRevision, EventTombstone
• Write tests for table constraints (ULID checks, JSON validation)
• Commit checkpoint (262739d)

Step 2.2: Event Store Core

• Create events/store.py with append_event, get_event, list_events
• Implement nonce validation (reject duplicates)
• Implement clock skew guard (reject ts_utc > now + 5 min)
• Create events/types.py with event type constants
• Write tests for event storage, nonce rejection, clock skew
• Commit checkpoint (29d5d3f)

Step 2.3: Event Type Payloads

• Create Pydantic models for all event payloads (spec §6):
  • LocationCreated, LocationRenamed, LocationArchived
  • AnimalCohortCreated, AnimalPromoted, AnimalMoved
  • AnimalAttributesUpdated, AnimalTagged, AnimalTagEnded
  • HatchRecorded, AnimalOutcome
  • ProductCollected, ProductSold
  • FeedPurchased, FeedGiven
  • AnimalMerged, AnimalStatusCorrected
• Create events/validation.py for payload validation
• Implement tag normalization (spec §20)
• Write tests for each payload model
• Commit checkpoint (80784ff)

Step 2.4: Projection Infrastructure

• Create projections/base.py with Projection base class
• Create ProjectionRegistry
• Create events/processor.py with process_event function
• Write tests for projection registration and processing
• Commit checkpoint (42cb1ed)

---

Phase 3: Animal Domain

Step 3.1: Animal Registry Schema

• Create migration for animal_registry table with indexes
• Create migration for live_animals_by_location table
• Create migration for animal_aliases table
• Create Pydantic models: Animal, enums (Sex, ReproStatus, LifeStage, Status, Origin)
• Write tests for table constraints and unique nickname constraint
• Commit checkpoint (739b7bf)

Step 3.2: Interval Projections Schema

• Create migration for animal_location_intervals
• Create migration for animal_tag_intervals
• Create migration for animal_attr_intervals
• Create Pydantic models for interval types
• Write tests for CHECK constraints (end_utc > start_utc)
• Commit checkpoint (e3d6528)

Step 3.3: Animal Cohort Creation

• Create projections/animal_registry.py for cohort creation
• Create projections/intervals.py for location and attribute intervals
• Create services/animal.py with create_cohort function
• Populate animal_registry, live_animals_by_location, event_animals
• Create initial location and attribute intervals
• Write tests: cohort validation, correct animal count, all projections updated
• Commit checkpoint (876e817)

Step 3.4: Animal Movement

• Update projections for AnimalMoved event
• Close old location interval, open new one
• Update animal_registry and live_animals_by_location
• Add move_animals to services/animal.py
• Write tests: move validation (to!=from, single from_location), interval updates
• Commit checkpoint (b89ea41)

Step 3.5: Animal Attributes Update

• Update projections for AnimalAttributesUpdated event
• Close old attribute intervals, open new ones for changed attrs
• Add update_attributes to services/animal.py
• Write tests: only changed attrs create new intervals
• Commit checkpoint (dc7700d)

Step 3.6: Animal Tagging

• Create migration for tag_suggestions table
• Create projections/tags.py for tag intervals and suggestions
• Update live_animals_by_location tags JSON
• Add add_tag, end_tag to services/animal.py
• Write tests: tag normalization, interval open/close, no-op handling
• Commit checkpoint (0511ed7)

---

Phase 4: Products & Feed

Step 4.1: Feed Inventory Schema & Purchase

• Create migration for feed_inventory table
• Create projections/feed.py with FeedInventoryProjection
• Implement apply_feed_purchased (increment purchased_kg, update price)
• Implement revert_feed_purchased
• Create services/feed.py with purchase_feed function
• Write tests: purchase increments, price stored as cents, revert works
• Commit checkpoint (5c10a75)

Step 4.2: Feed Given Event

• Implement apply_feed_given (increment given_kg, decrement balance)
• Implement revert_feed_given
• Add give_feed to services/feed.py
• Block if no purchase <= ts_utc
• Write tests: give updates inventory, blocked without purchase, revert
• Commit checkpoint (2eb0ca7)

Step 4.3: Product Collection Event

• Create projections/products.py for ProductCollected
• Create services/products.py with collect_product function
• Create selection/resolver.py with basic resolve_selection
• Populate event_animals for each resolved animal
• Write tests: event created, event_animals populated
• Commit checkpoint (d53decd)

Step 4.4: 30-Day Stats Computation

• Create migration for egg_stats_30d_by_location table
• Implement bird-days calculation from intervals
• Implement layer-eligible days filtering
• Implement feed proration formula
• Implement cost calculations (integer cents)
• Create services/stats.py with get_egg_stats (compute on read)
• Write tests: verify E2E test #1 baseline values (spec §21)
• Commit checkpoint (c08fa47)

Step 4.5: Product Sold Event

• Implement apply_product_sold projection
• Add sell_product to services/products.py
• Calculate unit_price_cents = floor(total/qty)
• Write tests: event stored, unit price calculated
• Commit checkpoint (b48fab5)

---

Phase 5: Selection & Historical Queries

Step 5.1: Selection Filter DSL Parser

• Create selection/parser.py for filter parsing
• Support: AND (default), OR (|), negate (-), quotes
• Fields: location, species, sex, life_stage, identified, tag
• Create selection/ast.py for filter AST nodes
• Write tests for all filter syntax variations
• Commit checkpoint

Step 5.2: Historical State Resolver

• Update selection/resolver.py for point-in-time resolution
• Use interval tables for historical state (spec §7 query pattern)
• Create selection/hash.py with roster hash (xxhash64)
• Write tests: resolver returns correct animals before/after events
• Commit checkpoint

Step 5.3: Optimistic Locking

• Create selection/validation.py for selection validation
• Re-resolve on submit, compute hash, return diff on mismatch
• Create SelectionContext and SelectionDiff models
• Write tests: mismatch detected, diff correct, confirmed bypasses
• Commit checkpoint

---

Phase 6: Event Lifecycle

Step 6.1: Event Editing

• Create events/edit.py with edit_event function
• Store old version in event_revisions
• Increment version
• Implement projection updates using revert/apply pattern
• Write tests: revision stored, version incremented, projections updated
• Write test: E2E test #5 (edit egg event)
• Commit checkpoint

Step 6.2: Event Deletion

• Create events/delete.py with delete_event function
• Create tombstone, trigger replay/revert
• Create events/dependencies.py for finding dependents
• Implement recorder vs admin rules (spec §10)
• Write tests: tombstone created, projections updated
• Write tests: recorder blocked if dependents (409), admin cascade
• Write test: E2E test #6
• Commit checkpoint (282d3d0)

Step 6.3: Animal Lifecycle Events

• Implement HatchRecorded (creates hatchlings)
• Implement AnimalOutcome (death/harvest/sold with yields)
• Implement AnimalPromoted (identified=true, nickname)
• Implement AnimalMerged (status=merged_into, aliases)
• Implement AnimalStatusCorrected (admin-only with reason)
• Write tests for each event type
• Write test: E2E test #7 (harvest with yields)
• Commit checkpoint (1153f6c)

---

Phase 7: HTTP API

Step 7.1: FastHTML App Shell

• Create web/app.py with FastHTML setup
• Configure HTMX extensions (head-support, preload, etc.)
• Create web/middleware.py:
  • Auth middleware (X-Oidc-Username, TRUSTED_PROXY_IPS)
  • CSRF middleware (cookie + header + Origin/Referer)
  • Request logging (NDJSON format)
  • Request ID generation
• Create web/auth.py with get_current_user, require_role
• Write tests: auth extraction, CSRF validation, untrusted IP rejection
• Commit checkpoint (84225d8)

Step 7.2: Health & Static Assets

• Create web/routes/health.py with /healthz (DB writable check)
• /metrics with Prometheus format (enabled by default, env gated)
• Configure static file serving (/static/vN/... with immutable cache)
• Vendor MonsterUI assets (via Theme.slate.headers() CDN)
• Create web/templates/base.py with base HTML template
• Create bottom nav component (Egg • Feed • Move • Registry)
• Write tests: healthz returns 200/503, static cache headers
• Commit checkpoint (6cdf48f)

Step 7.3: Egg Quick Capture ✓

• Create web/routes/eggs.py:
  • GET / - Egg Quick Capture form
  • POST /actions/product-collected
• Create web/templates/eggs.py with form
• Implement UX defaults (integer only, min=1, location sticks, qty clears)
• Write tests: form renders, POST creates event, validation errors (422)
• Commit checkpoint (e9804cd)

Step 7.4: Feed Quick Capture ✓

• Create web/routes/feed.py:
  • GET /feed - Feed Quick Capture form
  • POST /actions/feed-given
  • POST /actions/feed-purchased
• Create web/templates/feed.py with forms
• Implement defaults per spec §22
• Write tests: form renders, POST creates events, blocked without purchase
• Adopt hx-boost pattern (idiomatic FastHTML/HTMX)
• Commit checkpoint (68e1a59)

Step 7.5: Move Animals

• Create web/routes/move.py:
  • GET /move - Move Animals form
  • POST /actions/animal-move
• Create web/templates/move.py with selection UI
• Implement mismatch confirmation flow
• Write tests: form renders, move validation, mismatch (409)
• Write test: E2E test #8 (optimistic lock)
• Commit checkpoint (ff4fa86)

---

Phase 8: Registry & Event Log

Step 8.1: Animal Registry View

• Create web/routes/registry.py:
  • GET /registry with filters and pagination
• Create web/templates/registry.py:
  • Table: ID, species, sex, life_stage, location, tags, last event, status
  • Facet sidebar with counts
• Create repositories/animals.py with list_animals, get_facet_counts
• Implement infinite scroll (50/page, cursor=base64)
• Write tests: renders with animals, pagination works, filters apply
• Add status filter field to selection DSL (parser + resolver)
• Commit checkpoint

Step 8.2: Event Log Projection & View

• Create migration for event_log_by_location table with cap trigger
• Create projections/event_log.py for event summaries
• Create web/routes/events.py:
  • GET /event-log?location_id=...
• Create web/templates/events.py
• Write tests: events appear, capped at 500, ordered by ts_utc DESC
• Commit checkpoint

Step 8.3: Animal Detail Drawer

• Create web/routes/animals.py:
  • GET /animals/{animal_id}
• Create web/templates/animal_detail.py:
  • Header summary
  • Timeline (newest first)
  • Quick actions
• Create repositories/animal_timeline.py
• Write tests: detail renders, timeline shows events, merge info shown
• Commit checkpoint

---

Phase 9: Remaining Actions

Step 9.1: Animal Actions Routes ✅

• POST /actions/animal-attrs
• POST /actions/hatch-recorded
• POST /actions/animal-outcome
• POST /actions/animal-cohort
• POST /actions/animal-promote
• POST /actions/animal-tag-add
• POST /actions/animal-tag-end
• POST /actions/animal-status-correct (admin-only)
• Create form templates for each
• Write tests for each action
• Commit checkpoint: 29ea3e2

Step 9.2: Product Sold Route ✅

• POST /actions/product-sold
• Create form template
• Write tests: sale creates event, unit price calculated
• Commit checkpoint

Step 9.3: User Defaults

• Create migration for user_defaults table
• Create repositories/user_defaults.py
• Integrate defaults into form rendering
• Write tests: defaults saved and loaded
• Commit checkpoint

---

Phase 10: Polish & E2E

Step 10.1: Full E2E Test Suite

• E2E test #1: Baseline eggs+feed+costs
• E2E test #2: Mixed group proration
• E2E test #3: Split flock, per-location stats
• E2E test #4: Backdated eggs use historical roster
• E2E test #5: Edit egg event (if not already done)
• E2E test #6: Deletes: recorder vs admin cascade (if not already done)
• E2E test #7: Harvest with yields (if not already done)
• E2E test #8: Optimistic lock with confirm (if not already done)
• Commit checkpoint

Step 10.2: Location Events & Error Handling

• Implement LocationCreated (idempotent for seeding)
• Implement LocationRenamed
• Implement LocationArchived
• Standardize error responses (422, 409, 401/403)
• Toast via HX-Trigger
• Write tests for location events and error rendering
• Commit checkpoint

Step 10.3: CLI, Docker & Deployment

• Complete CLI: serve, seed, migrate commands
• Update flake.nix for Docker image build
• Create docker.nix
• Document deployment configuration
• Write tests: CLI commands work
• Final commit

---

Summary

Total steps: 40 steps across 10 phases Each step: Self-contained, testable, ends with integration

Key Milestones

• Phase 1 complete: Project boots, DB works
• Phase 3 complete: Animals can be created and tracked
• Phase 4 complete: Feed/egg collection works with costs
• Phase 7 complete: Web UI is functional
• Phase 10 complete: Production ready

Notes

• TDD: Write tests first
• Each step should be committable
• Refer to spec.md for authoritative behavior
• Check CLAUDE.md for framework patterns