backend architecture estimatic chat

# Estimatic AI - Backend Architecture

> Complete technical reference for the FastAPI + OpenAI Agents SDK backend that powers the Estimatic AI construction estimating platform.

---

## Table of Contents

1. [System Overview](#1-system-overview)
2. [Entry Points & Application Bootstrap](#2-entry-points--application-bootstrap)
3. [Agent Pipeline Architecture](#3-agent-pipeline-architecture)
4. [Agent System Prompts (Complete)](#4-agent-system-prompts-complete)
5. [Agent Tools Reference](#5-agent-tools-reference)
6. [Domain Layer: Trade Profiles & Rules](#6-domain-layer-trade-profiles--rules)
7. [Service Layer](#7-service-layer)
8. [Database Models & Schema](#8-database-models--schema)
9. [WebSocket Protocol & Real-Time Events](#9-websocket-protocol--real-time-events)
10. [REST API Routes](#10-rest-api-routes)
11. [Estimate Lifecycle & State Machine](#11-estimate-lifecycle--state-machine)
12. [Quality Assurance Pipeline](#12-quality-assurance-pipeline)
13. [Plan Mode: Compile & Execute](#13-plan-mode-compile--execute)
14. [Pricing & Material Intelligence](#14-pricing--material-intelligence)
15. [Configuration & Environment](#15-configuration--environment)
16. [Testing Strategy](#16-testing-strategy)

---

## 1. System Overview

Estimatic AI is a multi-agent construction estimating platform. Users describe work via chat or image uploads, and AI agents produce structured estimates with trade groups, line items (labor + materials), and real market-priced materials from Google Shopping.

### High-Level Data Flow

```
User (browser)
  |
  |-- WebSocket ws://localhost:8000/ws/{estimate_id}
  |
  v
FastAPI WebSocket Handler (routes/ws.py)
  |
  |-- Restores session, conversation history, pending plans
  |
  v
Orchestrator Agent (agents/orchestrator.py)
  |
  |-- Routes to specialist based on user intent
  |
  +---> Scope Agent          -- creates/modifies groups, line items, materials
  +---> Pricing Agent        -- searches real prices, attaches materials
  +---> Blueprint Agent      -- analyzes photos/blueprints, extracts dimensions
  |
  v
Post-Processing Pipeline (automatic, silent)
  |
  +---> Product Intelligence Agent  -- validates material-to-scope matches
  +---> QA Agent                    -- adds missing permits, cleanup, contingency
  +---> Feedback Loop               -- retries if quality score < 70
  |
  v
WebSocket Events --> Frontend Zustand stores --> UI update
```

### Technology Stack

| Layer | Technology |
|-------|-----------|
| Runtime | Python 3.13+, async/await throughout |
| Framework | FastAPI with WebSocket support |
| Agent SDK | OpenAI Agents SDK (`agents` package) with handoffs |
| LLM | GPT-4.1 (main agents), GPT-4.1-mini (product intelligence) |
| Database | SQLite + aiosqlite (async), SQLAlchemy 2.0 ORM |
| Price Search | SearchAPI.io Google Shopping integration |
| Package Manager | uv |

---

## 2. Entry Points & Application Bootstrap

### `main.py` - FastAPI Application

```python
# Lifespan context manager
@asynccontextmanager
async def lifespan(app):
    await init_db()          # Create tables + run lightweight migrations
    set_default_openai_key() # Configure OpenAI SDK with env key
    set_tracing_disabled()   # Disable OpenAI tracing
    yield
```

**Startup sequence:**
1. Initialize async SQLite database (create tables, run column migrations)
2. Set OpenAI API key for the Agents SDK
3. Register CORS middleware (configured origins from settings)
4. Mount route modules: estimates, uploads, WebSocket

**Middleware:**
- CORS: Allows configured frontend origins
- Request timeout: 900 seconds (long-running agent pipelines)

**Health check:** `GET /api/health` returns `{"status": "ok"}`

### `config.py` - Settings

```python
class Settings(BaseSettings):
    openai_api_key: str           # Required - GPT-4.1 access
    searchapi_key: str = ""       # Optional - Google Shopping price search
    database_url: str = "sqlite+aiosqlite:///./estimatic.db"
    cors_origins: list[str] = ["http://localhost:5173"]
```

Loaded from `backend/.env` via pydantic-settings.

### `database.py` - Async Database

- SQLAlchemy async engine with `aiosqlite` dialect
- `async_session()` context manager for all DB access
- Lightweight migration functions (`_migrate_add_*`) for column additions
- No Alembic -- uses `create_all()` + incremental column additions at startup

---

## 3. Agent Pipeline Architecture

### SDK Import Shim (`agents/_sdk.py`)

The project has a local `agents/` package that collides with the installed `agents` SDK. The shim temporarily hides the local package from `sys.path` and `sys.modules`, imports the real SDK, then restores the local package.

All agent modules import SDK symbols from this shim:
```python
from agents._sdk import Agent, Runner, function_tool, handoff, RunContextWrapper
```

**Exported symbols:** `Agent`, `Runner`, `RunContextWrapper`, `function_tool`, `handoff`, `RunResultStreaming`, `set_default_openai_key`, `set_default_openai_client`, `set_tracing_disabled`, `ModelSettings`, `WebSearchTool`

### Shared Context (`agents/_shared_context.py`)

`PLATFORM_CONTEXT` is a large string constant prepended to all agent system prompts. It defines:

- **What Estimatic AI is:** Professional construction estimating platform for all trades
- **What a complete estimate includes:**
  1. Assembly Line Items (labor + materials with markups)
  2. Standalone Material Line Items
  3. Equipment rentals
  4. Permits
  5. Cleanup & Disposal
  6. Contingency (5%)
  7. Exclusions/Assumptions

- **Scope Parsing Rules:** How to interpret "labor only", "material only", "customer provides"
- **Scope Precedence:** User-specified items always override defaults, never set quantity=0
- **Price Interpretation:** Per-unit vs lump-sum determination
- **Markups:** labor_markup=20%, material_markup=15%
- **Cost Model (three patterns):**
  - **Type A:** Labor-only assembly (demo, cleanup, testing) -- rate = labor cost, no materials
  - **Type B:** Installed assembly with materials -- rate = labor per unit, materials attached separately
  - **Type C:** Standalone material -- rate = material unit price, no labor
- **Critical Violation:** Never double-cost the same product (as assembly material AND standalone)

### EstimateContext (dataclass)

Passed to all tool functions via the SDK's `RunContextWrapper`:

```python
@dataclass
class EstimateContext:
    estimate_id: int
    websocket: WebSocket
    session_id: str
    zipcode: str | None = None
    location_city: str | None = None
    location_state: str | None = None
```

### Agent Definitions

| Agent | Model | Temperature | Max Turns | Purpose |
|-------|-------|------------|-----------|---------|
| Orchestrator | gpt-4.1 | 0.1 | 1 (router) | Routes to specialists, manages pipeline |
| Scope Agent | gpt-4.1 | 0.2 | 50 | Creates/modifies estimate structure |
| Pricing Agent | gpt-4.1 | 0.0 | 30 | Searches prices, attaches materials |
| Blueprint Agent | gpt-4.1 | 0.2 | 30 | Analyzes photos/blueprints |
| QA Agent | gpt-4.1 | 0.0 | 15 | Scoring, safe structural additions |
| PI Consultant | gpt-4.1-mini | 0.0 | 5 | Real-time product questions |
| PI Reviewer | gpt-4.1-mini | 0.0 | 30 | Post-pipeline material validation |
| Execution Agent | gpt-4.1 | 0.0 | 120 | Executes approved plans with LLM judgment |

### Orchestrator Pipeline Flow (`run_orchestrator()`)

```
1. Build message content (text + image attachments as base64)
2. Load conversation history from DB
3. Run Orchestrator agent (routes to Scope/Pricing/Blueprint via handoffs)
4. Stream assistant chunks to frontend via WebSocket
5. Save assistant response to chat history
6. IF mode == "agent" (not "plan"):
   a. Run Product Intelligence reviewer pass
   b. Run QA Agent pass (silent, no text streaming)
   c. Compute quality score (0-100)
   d. IF score < 70: run feedback loop with targeted retries
   e. Send estimate_quality_updated event
   f. Send token_usage_updated event
```

### Mode Support

- **"plan" mode:** Agent only generates a plan via `generate_plan()`. No line items created. User reviews and approves.
- **"agent" mode:** Full pipeline with QA, PI, and retry feedback loops.

---

## 4. Agent System Prompts (Complete)

### 4.1 Orchestrator Agent

**Role:** Central router that delegates to specialist agents.

**Routing Rules:**
| User Intent | Route To |
|------------|----------|
| Create, modify, remove estimate content | Scope Agent |
| Look up or compare prices only | Pricing Agent |
| Photo/blueprint references, import mode | Blueprint Agent |

**Key Behaviors:**
- Never modifies estimates directly
- Maintains conversation context across handoffs
- Supports plan mode (restricts to `generate_plan()` only)

---

### 4.2 Scope Agent

**Role:** Senior construction estimator (20+ years experience) that breaks projects into groups and line items.

**Model:** gpt-4.1, temperature=0.2

<details>
<summary><strong>Full System Prompt (click to expand)</strong></summary>

**MANDATORY FIRST STEPS:**
1. Call `get_estimate_summary()` to see current state
2. Call `get_reference_data("all")` for regional labor rates
3. Check if estimate is empty or has existing items
4. Never create duplicates

**LOCATION HANDLING:**
- User-specified location overrides stored zipcode
- Update estimate with location_city and location_state
- Use location in all search_price calls

**SCOPE PRECEDENCE:**
- User-explicit items always included with full pricing
- Never set quantity=0 on user-requested items
- Add text note if code doesn't require something but user requested it

**STRUCTURAL CONSISTENCY:**
- Pick ONE approach (e.g., deck blocks OR poured concrete footings, never both)

**PLAN MODE:**
- Must call `generate_plan()` tool (not text)
- No dollar amounts or rates in plan
- Plan is scope preview only
- When approved, switch to agent mode and execute

**CORRECTIONS & UPDATES:**
- Call get_estimate_summary to find items
- Use update_line_item or update_material to fix
- Never create new items to fix existing ones

**COST STRUCTURE CLASSIFICATION:**
- **Type A** (labor-only): demolition, removal, cleanup, testing, startup
- **Type B** (labor + materials): any work installing physical products
- **Type C** (standalone material): customer-purchased items not installed by assembly

**THE ASSEMBLY MODEL:**
- rate = labor cost per unit (NOT hourly wage)
- materials[] = actual products attached via search_price + add_material
- labor_markup = 20%, material_markup = 15%
- CRITICAL: Assembly rate is ALWAYS labor. Materials attached separately.

**HOW TO GET CORRECT LABOR RATE:**
- MUST call `lookup_labor_rate(trade, uom, work_description)` (not mental math)
- Returns rate_per_unit from formula
- Never use raw hourly rate as per-sqft or per-linear-ft rate

**WORKFLOW FOR EACH ASSEMBLY ITEM:**
1. Call `get_assembly_template(work_description, quantity, uom)`
2. If no template, call `lookup_labor_rate(trade, uom, work_description)`
3. Call `validate_before_creation(title, type, uom, rate, quantity)`
4. Create: `add_priced_line_item(type="assembly", rate=..., quantity=..., uom=...)`
5. Call `get_material_requirements(title, group_name, quantity, uom)` for checklist
6. For each required material:
   - Search: `search_price("specific product query", location=...)`
   - Calculate: `calculate_material_quantity(product_name, work_quantity, work_uom)`
   - Attach: `add_material(line_item_id=..., name=..., price=..., quantity=packages_needed)`
7. Small consumables (<$10): `add_material(...)` directly without search

**ALLOWED UOMs FOR ASSEMBLY ITEMS:**
`each` | `sq_ft` | `linear_ft` | `hour` | `day` | `week` | `lump_sum` | `yard` | `cubic_yards` | `pounds` | `tons` | `gallon` | `box` | `roll` | `bag` | `pair` | `set` | `piece` | `count` | `unit`

**PACKAGE-BASED VS PER-UNIT PRICING:**
- Material quantity != line item work quantity
- MUST call `calculate_material_quantity()` before add_material
- Example: 1,650 sq ft drywall -> 57 sheets (not 1,650)

**SEARCH QUERY RULES:**
- Include specific product with specs, dimensions, material type
- Include brand, grade, size when known
- Bad: "panel" | Good: "200 amp main breaker panel square d"
- Bad: "pipe" | Good: "3/4 inch type L copper pipe 10 ft"
- Omit search_query for labor-only items

**WASTE & OVERAGE:**
- Flooring +10%, Tile +15%, Drywall +10%, Lumber +10%, Paint +15%

**PHASED APPROACH:**
1. Check for templates
2. Get regional data
3. Build group by group
4. Verify after each group
5. Add final groups (permits, cleanup, contingency)
6. Final quality check

**NON-NEGOTIABLE RULES (15 total):**
1. Every assembly with physical work MUST have materials
2. MUST include cleanup & disposal
3. Regulated trades MUST include permits
4. Assembly rate = labor only, materials separate
5. "Labor only" on one item != whole estimate labor only
6. Never create duplicate groups
7. Never override user-provided prices
8. Never set assembly rate to material price
9. Every line item MUST have non-zero rate (except text)
10. High/difficult access MUST include equipment
11. Material quantity vs product unit -- read product description
12. ALWAYS call lookup_labor_rate (never raw hourly rate)
13. Material >$10 MUST use search_price
14. ALWAYS call calculate_material_quantity before add_material
15. ALWAYS call get_material_requirements for materials checklist

</details>

**Tools Available:**
- Estimate structure: `add_group`, `remove_group`, `add_line_item`, `update_line_item`, `remove_line_item`
- Materials: `add_material`, `update_material`, `remove_material`
- Pricing: `search_price`, `get_estimate_summary`, `add_priced_line_item`
- Reference: `get_reference_data`, `get_estimate_template`
- Calculations: `calculate_area`, `calculate_material_quantity`
- Knowledge: `get_material_requirements`, `validate_before_creation`, `get_assembly_template`, `lookup_labor_rate`
- Advanced: `consult_product_expert`, `generate_plan`

---

### 4.3 Pricing Agent

**Role:** Construction material pricing specialist. Zero temperature for deterministic behavior.

**Model:** gpt-4.1, temperature=0.0

<details>
<summary><strong>Full System Prompt (click to expand)</strong></summary>

**MANDATORY FIRST STEPS:**
1. Call `get_reference_data("regional_info")` for search location
2. Call `get_estimate_summary()` to see every line item

**STEP 1: CLASSIFY EACH LINE ITEM**

| Type | What rate means | Action |
|------|-----------------|--------|
| `assembly` | Labor cost per unit | NEVER change rate. Check if materials need attaching. |
| `material` with rate=$0 | Unpriced standalone | Search for price, attach material, set rate. |
| `material` with rate>$0 | Already priced | Only update if user explicitly asked. |
| `equipment`/`permit`/`text` | Not material costs | Leave alone unless user asks. |

**CRITICAL RULE:** assembly rate = labor cost. NEVER overwrite it.

**STEP 2: DECIDE WHETHER TO ATTACH MATERIALS**

Attach when assembly work consumes a physical product:
- "Install Ceramic Tile" -> tile, thinset, grout
- "Hang and Finish Drywall" -> sheets, compound, tape, screws
- "Run Electrical Circuit" -> wire, conduit, wire nuts
- "Apply Roof Shingles" -> shingles, nails

Do NOT attach for pure labor:
- "Demo Labor", "Demolition", "Tear-out"
- "Site Supervision", "Project Management"
- "Final Cleanup", "Debris Removal"
- "Inspection", "Testing", "Commissioning"

**STEP 3: CRAFT SEARCH QUERIES**

Trade-specific patterns:
- Electrical: "200A main breaker panel", "12/2 NM-B wire 250 ft roll"
- Plumbing: "1/2 inch copper pipe type L 10 ft", "PVC 3 inch DWV coupling"
- HVAC: "2 ton split system condenser", "flex duct 6 inch 25 ft"
- Roofing: "architectural shingles 3-tab 25-year bundle"
- Flooring: "porcelain floor tile 12x24 inch"
- Drywall: "1/2 inch drywall 4x8 sheet", "joint compound 5 gallon bucket"
- Painting: "interior latex paint 1 gallon satin"
- Insulation: "R-19 fiberglass batt 15 inch wide"
- Cabinetry: "kitchen base cabinet 36 inch"

**STEP 4: PACKAGING VS SCOPE UNITS**

1. Read the product title for coverage
2. Use normalized data from search_price (packages_needed, pricing_mode, coverage_sqft)
3. Formula: packages_needed = ceil(scope_qty / coverage_per_package)
4. NEVER set quantity = scope_qty when product not sold per that unit

**STEP 5: VALIDATE BEFORE ATTACHING**

Confirm product category matches, NOT a tool/PPE/sample/swatch, confidence HIGH or MEDIUM.

**STEP 6: EXECUTE**

For material items (rate=$0): search -> validate -> add_material -> update_line_item(rate)
For assembly items: search -> validate -> add_material -> DO NOT change rate

**WHEN NO VALID RESULT:** Create Pricing Allowance with conservative estimate.

**QUALITY TIERS:** Default to mid-grade. Home Depot / Lowe's preferred.

</details>

**Tools Available:**
- `search_price`, `update_line_item`, `add_material`
- `get_estimate_summary`, `get_reference_data`
- `consult_product_expert`, `calculate_material_quantity`

---

### 4.4 Blueprint Agent

**Role:** Photo and blueprint analysis specialist.

**Model:** gpt-4.1, temperature=0.2

<details>
<summary><strong>Full System Prompt (click to expand)</strong></summary>

**MANDATORY FIRST STEP:** Call `get_reference_data("all")` for regional labor rates.

**IMAGE ANALYSIS:**

From Site Photos, extract:
- Room dimensions, current condition, existing materials
- Fixtures, features, scope indicators, structural concerns

From Blueprints/Floor Plans, extract:
- Room dimensions (read directly), room count & layout
- Door & window locations, wall lengths
- Plumbing locations, electrical layout, total square footage

**WORKFLOW:**
1. Analyze image(s) in detail
2. Extract measurements
3. Identify scope of work
4. Ask 1-2 clarifying questions if critical info missing
5. Build estimate or hand off to Scope Agent

**IMPORT MODE** (user says "import", "recreate", "organize this bid"):
1. Extract ALL line items from PDF with EXACT values
2. Extract financial summary (subtotal, overhead %, profit %, grand total)
3. Preserve trade/category groupings
4. Generate plan with EXACT source values (set `source_backed=true`)
5. Do NOT call search_price or compute labor rates
6. After execution: "Import complete. Compare against market rates?"

**MEASUREMENT CHEAT SHEET:**
- Standard interior door: 6'8" x 2'8" or 3'0"
- Standard window: 3'0" x 4'0"
- Kitchen counter height: 36"
- Ceiling height: 8'0" or 9'0"
- Standard bathtub: 60" x 30"

</details>

**Tools Available:**
- `add_group`, `add_line_item`, `update_line_item`, `remove_line_item`
- `add_material`, `remove_material`, `add_priced_line_item`
- `get_estimate_summary`, `search_price`
- `get_reference_data`, `calculate_area`, `generate_plan`
- **Handoff:** Can hand off to Scope Agent for complex estimates

---

### 4.5 QA Agent

**Role:** Quality assurance reviewer and scorer. Runs silently after main agent.

**Model:** gpt-4.1, temperature=0.0

<details>
<summary><strong>Full System Prompt (click to expand)</strong></summary>

**PRIMARY JOB:** SCORING and REPORTING
- ValidationGate enforces material correctness at creation time
- QA does NOT re-validate pricing, quantities, or unit conversions

**SECONDARY JOB:** Add SAFE STRUCTURAL items (cleanup, permits, contingency)

**ABSOLUTE RULE: RESPECT USER'S INTENT**
- If user removed something -> NEVER re-add
- If user changed something -> do not revert
- When in doubt, do NOT add or change

**SAFETY LIMITS:**
- NEVER auto-fix if fix would reduce estimate total >15%
- NEVER remove or replace materials
- NEVER change rate on assembly items
- NEVER change non-zero rates on material items
- NEVER change material quantities or prices

**MAY AUTO-FIX (safe structural additions only):**
1. $0 rate items -> update_line_item with reasonable rate
2. Missing cleanup/disposal group -> add_line_item
3. Missing permits for regulated trades -> add_line_item
4. Missing contingency -> add 5% lump_sum if estimate >$2,000

**MUST NOT DO:**
- Change material quantities, prices, names
- Remove or add materials
- Reorganize groups or rename items
- Create pricing allowances

**WORKFLOW:**
1. Call `get_estimate_summary()` and `get_reference_data("all")`
2. Review user's message for intent
3. Apply ONLY safe structural additions
4. Verify with `get_estimate_summary()`
5. Work silently via tool calls only -- NO text streaming

</details>

**Tools Available:**
- `get_estimate_summary`, `add_group`, `add_line_item`, `update_line_item`
- `get_reference_data`, `calculate_area`
- `validate_before_creation`, `lookup_labor_rate`

---

### 4.6 Product Intelligence Agent (Two Modes)

#### PI Consultant (Lightweight)
**Role:** Real-time product questions from other agents.
**Model:** gpt-4.1-mini, temperature=0.0, max_turns=5
**Tools:** `WebSearchTool()`, `search_price`

#### PI Reviewer (Full)
**Role:** Post-pipeline material validation.
**Model:** gpt-4.1-mini, temperature=0.0, max_turns=30
**Tools:** `WebSearchTool()`, `get_estimate_summary`, `update_line_item`, `add_material`, `remove_material`, `search_price`

<details>
<summary><strong>Shared Knowledge Base (click to expand)</strong></summary>

**Coverage-based products:**
- Flooring (LVP, laminate, hardwood): per BOX (~20-25 sq ft/box)
- Underlayment: per ROLL (~100 sq ft/roll)
- Roofing shingles: per BUNDLE (~33.3 sq ft, 3/square)
- Tile: per BOX or per sq ft
- Drywall: per SHEET (4x8=32 sq ft, 4x12=48 sq ft)

**Linear products:**
- Electrical wire: per ROLL (25-1000 ft)
- Conduit: per STICK (10 ft)
- Copper/PVC pipe: per STICK (10-20 ft)
- Lumber: per BOARD (8-20 ft lengths)

**Board lumber:**
- Nominal -> actual: 2"->1.5", 4"->3.5", 6"->5.5", 8"->7.25", 10"->9.25", 12"->11.25"
- Coverage: (actual_width / 12) x length_ft = sq ft per board

**Critical Rules:**
1. NEVER multiply per-package price by area/linear quantity
2. Formula: packages_needed = ceil(required_qty / coverage_per_package)
3. ALWAYS use web search when uncertain
4. Reject samples, swatches, discontinued items
5. Flag insane prices (>5x typical range)

**REVIEWER SAFETY RULES:**
- Do NOT remove materials unless clearly wrong
- Do NOT change prices unless clearly insane (>5x typical)
- Do NOT reduce estimate total >30% -- flag instead
- If material seems unusual but plausible, LEAVE IT

</details>

---

### 4.7 Execution Agent

**Role:** Executes approved plans with LLM judgment for material selection.

**Model:** gpt-4.1, temperature=0.0, max_turns=120

<details>
<summary><strong>Full System Prompt (click to expand)</strong></summary>

**WORKFLOW:**
1. Call `get_reference_data("all")`
2. Process each group and item in plan IN ORDER
3. After all items, add Overhead & Profit line items
4. Call `get_estimate_summary` for final quality check

**FOR EACH LINE ITEM:**

Step 1: Get Assembly Template (MANDATORY for assembly items)
- Call `get_assembly_template(work_description, quantity, uom)`
- ALWAYS use assembly engine rate over plan's rate

Step 2: Determine Type and Rate
- Labor-only -> use plan rate or `lookup_labor_rate`
- Assembly -> use assembly engine rate, minimum $35/hr x hours if both return 0

Step 3: Create the Line Item

Step 4: Search and Attach Materials (assembly items)
- For each BOM material:
  - skip_search=true: add_material directly
  - skip_search=false: search_price -> pick best HIGH/MEDIUM confidence -> add_material
  - Compute correct quantity from BOM or coverage
  - No good product: flag as "REQUIRES REVIEW"

**AFTER ALL ITEMS -- OVERHEAD & PROFIT:**
- Overhead: 10% of total labor cost, lump_sum
- Profit: 10% of total estimate, lump_sum

**CRITICAL RULES:**
1. NEVER rate=$0 for assembly items
2. NEVER use scope qty as material qty
3. REJECT garbage products (specs, BIM, samples, CAD)
4. Flag uncertain items with "REQUIRES REVIEW:" prefix
5. Use assembly BOM quantities (pre-computed)
6. Process ALL items (never skip)

</details>

**Tools Available:**
- Read: `get_estimate_summary`, `get_reference_data`, `get_assembly_template`, `lookup_labor_rate`
- Search: `search_price` (returns all 10 results)
- Compute: `calculate_material_quantity`, `calculate_area`
- Write: `add_group`, `add_line_item`, `add_priced_line_item`, `add_material`, `update_line_item`, `update_suggestions`

---

## 5. Agent Tools Reference

### Tool Categories & Locations

All tools are decorated with `@function_tool` from the OpenAI Agents SDK and receive `EstimateContext` via `RunContextWrapper`.

#### Estimate Structure (`agents/tools/estimate_structure_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `add_group` | name | Create a trade group. Emits `group_added` WebSocket event. |
| `remove_group` | group_name | Case-insensitive delete with cascade. Emits `group_removed`. |

#### Line Items (`agents/tools/estimate_line_item_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `add_line_item` | group_name, title, line_item_type, quantity, uom, rate, description, labor_markup, material_markup | Creates line item. Auto-creates group if needed. Auto-computes labor rate when rate=0. |
| `update_line_item` | item_id, title?, description?, line_item_type?, quantity?, uom?, rate?, labor_markup?, material_markup? | Partial update. Recalculates extended cost. |
| `remove_line_item` | item_id | Delete with ownership validation. |
| `add_priced_line_item` | group_name, title, type, quantity, uom, rate, description, markups, search_query?, search_location? | **Most complex tool.** Three-phase: BUILD (normalize + search + score) -> VALIDATE (ValidationGate) -> PERSIST (DB + WebSocket). |

**`add_priced_line_item` Pipeline:**
1. Normalize scope item (trade detection, labor-only check, waste factor)
2. Auto-compute labor rate if rate=0
3. Create/find group with locking
4. Search Google Shopping if search_query provided
5. Score product candidates for compatibility
6. Normalize product pricing (package detection, coverage)
7. Run ValidationGate checks
8. Persist or mark as unresolved
9. Emit WebSocket event

#### Materials (`agents/tools/estimate_material_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `add_material` | line_item_id, name, price, quantity, supplier_name?, image_url?, product_url?, package_unit?, package_qty?, normalized_unit_price?, confidence_score? | Attaches material to line item. **Critical:** price x quantity = product retail unit, NOT work quantity. Prevents duplicates. |
| `update_material` | material_id, name?, price?, quantity? | Partial update. Recalculates parent extended cost. |
| `remove_material` | material_id | Delete. Recalculates parent extended cost. |

#### Read Tools (`agents/tools/estimate_read_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `get_estimate_summary` | (none) | Returns formatted text with all groups, items, IDs, costs, and WARNINGS section (zero rates, missing labor, missing permits, unresolved items). |

#### Plan Tools (`agents/tools/estimate_plan_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `generate_plan` | groups_json (JSON array), summary | Creates EstimatePlan, compiles with trade metadata, stores as pending_plan, emits `plan_proposed` WebSocket event. |

#### Price Search (`agents/tools/price_search.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `search_price` | query, location?, required_qty?, required_unit? | Google Shopping search via SearchAPI.io. Returns top 10 results with confidence scores, normalized pricing, packages_needed. |

#### Calculations (`agents/tools/calc_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `calculate_area` | calculation_type, dimensions (JSON) | Supports: rectangle_area, triangle_area, circle_area, perimeter, volume, paint_coverage, flooring_with_waste, concrete_cubic_yards, roofing_squares, wall_area, board_count, railing_linear_ft, footing_count, material_quantity |

#### Quantity Tools (`agents/tools/quantity_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `calculate_material_quantity` | product_name, work_quantity, work_uom, waste_percent? | Deterministic package math. Looks up product in registry, calculates `ceil((work_qty x waste) / coverage_per_unit)`. Returns packages_needed. |

#### Labor Tools (`agents/tools/labor_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `lookup_labor_rate` | trade, uom, work_description, hours_per_unit? | Resolves trade -> gets regional multiplier -> computes per-unit rate from productivity tables. Returns rate, formula, work type. |

#### Reference Data (`agents/tools/reference_data.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `get_reference_data` | category (labor_rates/waste_factors/permit_costs/equipment/regional_info/all) | Returns regionally-adjusted data. 20+ trades, waste factors, permit costs, equipment rentals. |
| `get_product_info` | product_title | Product coverage, packaging, price range, waste factor from registry. |

**Regional Data (`_ZIPCODE_REGIONS`):**
Maps 3-digit zipcode prefixes to (region, cost_multiplier, search_location). Example: "900" -> ("West Coast", 1.20, "Los Angeles, CA").

**Base Labor Rates (national average):**
- General Labor: $32/hr | Carpenter: $58/hr | Electrician: $82/hr | Plumber: $90/hr
- HVAC Tech: $80/hr | Tile Setter: $70/hr | Painter: $45/hr | Roofer: $65/hr
- Drywall: $52/hr | Framer: $55/hr | Finish Carpenter: $62/hr
- Concrete Mason: $60/hr | Landscaper: $42/hr | Flooring: $55/hr | Siding: $50/hr | Insulation: $45/hr

#### Trade Knowledge (`agents/tools/trade_knowledge_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `get_material_requirements` | line_item_title, group_name, work_quantity, work_uom | Returns material checklist with search queries, computed quantities, accessories, permit requirements. |
| `validate_before_creation` | title, line_item_type, uom, rate, quantity, group_name? | Pre-creation validation. Catches hourly-rate-as-unit-rate confusion (THE #1 ERROR). Returns issues and suggestions. |
| `get_assembly_template` | work_description, quantity, uom | Complete assembly specification: UOM, labor rate, BOM materials, accessories. Integrates with assembly engine for YAML-based profiles. |

#### Templates (`agents/tools/template_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `get_estimate_template` | job_type (bathroom_remodel/kitchen_remodel/roof_replacement/interior_painting/deck_build) | Returns professional checklist with groups, items, rate ranges, search queries. |

#### Product Expert (`agents/tools/product_expert_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `consult_product_expert` | product_title, required_qty?, required_unit?, line_item_title?, question? | Fast path for known products (coverage, pricing). AI path for unknowns (invokes PI Agent with web search). |

#### Suggestions (`agents/tools/suggestion_tools.py`)

| Tool | Parameters | Description |
|------|-----------|-------------|
| `update_suggestions` | suggestions (list of 3-4 strings) | Sends contextual follow-up action chips to frontend via WebSocket. |

---

## 6. Domain Layer: Trade Profiles & Rules

### Directory Structure

```
domain/trades/
  models.py          -- Pure data classes (TradeProfile, ValidationIssue, etc.)
  registry.py        -- 13 Python-defined profiles + YAML loading + alias index
  rules.py           -- Canonical rule functions (labor-only, permits, etc.)
  validators.py      -- Estimate/line item validation, quality scoring
  validator.py       -- Trade profile completeness validation (runs at startup)
  profile_loader.py  -- YAML profile loader for domain expert authoring
  profiles/          -- YAML trade profiles (20 files)
```

### Trade Profile Structure (`models.py`)

```python
@dataclass
class TradeProfile:
    trade_id: str                    # e.g., "electrical_service"
    name: str                        # e.g., "Electrical Service & Panel"
    version: str                     # e.g., "2.0"
    aliases: list[str]               # keyword matching ["electrical", "panel", ...]
    work_items: list[WorkItemRule]    # pattern-based work item definitions
    labor_only_patterns: list[str]   # keywords that indicate labor-only items
    material_requirements: list[MaterialRequirementRule]  # BOM rules
    accessory_rules: list[AccessoryRule]
    compatibility_rules: list[CompatibilityRule]
    permit_rules: list[PermitRule]
    equipment_rules: list[EquipmentRule]
    pricing_sanity: list[PricingSanityRange]
    default_waste_factor: float
    allowed_uoms: list[str]
    labor_rate_key: str              # maps to base labor rates
    labor_productivity: list[LaborProductivityRule]
    assemblies: list[AssemblyTemplate]  # YAML-defined complete assemblies
```

### Registered Trades (Python-defined, `registry.py`)

| Trade ID | Name | Labor Rate Key | Base Rate |
|----------|------|---------------|-----------|
| general_remodel | General Remodel | general_labor | $32/hr |
| drywall | Drywall Installation & Finishing | drywall | $52/hr |
| painting | Interior/Exterior Painting | painter | $45/hr |
| flooring_tile | Flooring & Tile | tile_setter | $70/hr |
| framing_deck | Framing & Deck Construction | framer | $55/hr |
| roofing | Roofing | roofer | $65/hr |
| electrical_service | Electrical Service & Panel | electrician | $82/hr |
| plumbing_fixture | Plumbing Fixtures & Rough-in | plumber | $90/hr |
| hvac_split_system | HVAC Split System | hvac_tech | $80/hr |
| irrigation_residential | Residential Irrigation | landscaper | $42/hr |
| finish_carpentry | Finish Carpentry | finish_carpenter | $62/hr |
| siding | Siding Installation | siding_installer | $50/hr |
| insulation | Insulation | insulation | $45/hr |

### YAML Trade Profiles (`profiles/`)

20 YAML files for domain-expert-authored profiles. These override Python definitions when both exist. Example structure (from `electrical.yaml`):

```yaml
trade_id: electrical_service
name: "Electrical Service"
version: "2.0"
aliases: [electrical, electric, outlet, switch, ...]

labor_rate_key: electrician   # base rate $82/hr

labor_productivity:
  - keywords: [install outlet, outlet install, ...]
    uom: each
    units_per_hour: 0.67
    description: "Duplex outlet installation"

work_items:
  - pattern: outlet_install
    keywords: [install outlet, ...]
    required_uoms: [each]
    default_uom: each
    search_query_template: "duplex electrical outlet receptacle 15 amp"

assemblies:
  - pattern: outlet_install
    keywords: [install outlet, ...]
    uom: each
    labor:
      trade_key: electrician
      base_hours_per_each: 1.5
      setup_hours: 0.5
      cleanup_hours: 0.25
      complexity_factors:
        residential_new: 1.0
        residential_retrofit: 1.20
      min_rate_per_unit: 120.00
      max_rate_per_unit: 350.00
    materials:
      - role: primary
        name: "Duplex receptacle outlet"
        search_query_template: "duplex electrical outlet receptacle 15 amp"
        quantity_formula: "qty"
        quantity_unit: each
      - role: consumable
        name: "Wire nuts"
        skip_search: true
        estimated_price: 5.00
        quantity_formula: "qty"
```

Available YAML profiles: concrete_flatwork, fencing, drywall, electrical, plumbing, hvac, insulation, flooring, painting, finish_carpentry, siding, roofing, framing, commercial_ti, fire_protection, security_glazing, ada_plumbing, low_voltage, millwork

### Registry Loading Order

1. 13 Python-defined profiles loaded into `TRADE_REGISTRY`
2. YAML profiles loaded from `profiles/` directory
3. YAML overrides Python if same `trade_id` exists
4. All profiles validated at import time (`validate_registry()`)
5. Alias index built for fast text matching

### Canonical Rules (`rules.py`)

Functions used across agents, tools, and services:

| Function | Purpose |
|----------|---------|
| `is_labor_only_item(title, group_name)` | Universal + trade-specific labor-only detection |
| `group_requires_materials(group_name)` | Whether a group should have material items |
| `item_requires_materials(title, type)` | Whether specific line item needs materials |
| `trade_requires_permit(title, group_name)` | Identifies required permits |
| `requires_cleanup(groups)` | Whether estimate needs cleanup group |
| `requires_contingency(total)` | Whether estimate needs contingency (>$2,000) |
| `is_valid_product_for_line_item(product, title)` | Trade-specific compatibility check |
| `is_unit_conversion_plausible(scope_qty, scope_uom, material_qty, material_uom)` | Catches quantity errors |
| `compute_packages_needed(coverage, required_qty, waste)` | Coverage-based package calculation |

### Validators (`validators.py`)

| Function | Returns | Description |
|----------|---------|-------------|
| `validate_line_item(item)` | `list[ValidationIssue]` | Per-item: zero rate, missing materials, compatibility |
| `validate_estimate(estimate)` | `list[ValidationIssue]` | Full estimate: group-level materials, labor ratio, permits, contingency, plausibility |
| `compute_quality_score(issues)` | `int (0-100)` | Score from issue counts and severities |

---

## 7. Service Layer

### Build Pipeline (`services/build_pipeline.py`)

Deterministic, testable pipeline for constructing estimate line items:

```
normalize_scope_item()     -- Creates ScopeItem with trade detection
    |
score_product_compatibility()  -- Scores products 0-100 against line items
    |
select_best_product()      -- Chooses highest-scoring viable product
    |
build_material_attachment()  -- Converts to MaterialAttachment with quantity calc
    |
validate_built_item()      -- Final validation
    |
build_provenance()         -- Tracks origin + assumptions for audit
```

**Key data types:**
- `ScopeItem`: Normalized line item with trade_id, is_labor_only, waste_factor
- `CandidateProduct`: Search result with normalized pricing and compatibility score
- `MaterialAttachment`: Ready-to-persist material with computed quantity
- `BuildResult`: Final item + material + provenance bundle

### Validation Gate (`services/validation_gate.py`)

Pre-persist validation barrier. Single static method: `ValidationGate.check()`

**Four possible decisions:**

| Decision | Meaning |
|----------|---------|
| `PERSIST_VERIFIED` | All checks passed, high confidence |
| `PERSIST_ESTIMATED` | Used registry defaults, under cost threshold |
| `PERSIST_UNRESOLVED` | Explicit unresolved state with visual flag |
| `REJECT` | Do NOT persist -- return error to agent with fix instructions |

**Key checks:**
- Scope quantity leakage (line item qty leaked into purchase qty)
- Cross-unit conversion with unknown coverage
- Unit mismatch without valid coverage
- Catastrophic material totals (>$50,000 single item)
- Implausible unit conversions

### Plan Compiler (`services/plan_compiler.py`)

Enriches raw plans with trade-aware metadata:

```python
compile_plan(plan, zipcode) -> EstimatePlan
```

For each plan item:
1. Detect trade from title + group name
2. Apply work item rules (waste factor, labor-only flag)
3. Compute labor rate from productivity tables
4. Set search queries from trade templates
5. Mark plan as `compiled=True`

### Plan Executor (`services/plan_executor.py`)

Deterministic execution of approved plans WITHOUT LLM involvement:

```
execute_plan(plan, estimate_id, websocket, ...) -> PlanExecutionResult
```

For each item:
1. Find or create group (with per-estimate locking)
2. Check for source-backed items (imported) -- preserve values
3. Check for assembly templates -> use assembly engine for multi-material BOM
4. Normalize scope item with compiled trade data
5. Compute labor rate if needed
6. Search Google Shopping for materials
7. Score and select best product
8. Build material attachment
9. Run ValidationGate
10. Persist or fall back to material estimator
11. Emit WebSocket events

**Fallback chain:** Search -> Estimated material from registry -> Persist as unresolved

### Assembly Engine (`services/assembly_engine.py`)

Pure computation (no DB, no async) for assembly pricing:

```python
price_assembly(template, quantity, zipcode) -> AssemblyPriceResult
```

Computes:
- Labor rate with setup, cleanup, helper ratio, complexity factors
- All material quantity formulas evaluated
- Returns `AssemblyPriceResult` with labor_rate_per_unit and materials list

### Unit Normalizer (`services/unit_normalizer.py`)

Converts product titles + prices into normalized per-unit costs:

```python
normalize_product_price(title, price, required_qty, required_unit) -> NormalizedPrice
```

Detection pipeline:
1. `_detect_package()`: Extract package info from title ("10 ft stick" -> qty=10, unit="ft")
2. `_detect_coverage()`: Parse coverage ("covers 22 sq ft" -> 22)
3. `_detect_dimensional_coverage()`: Parse dimensions ("9 ft x 100 ft" -> 900 sq ft)
4. Compute packages_needed, normalized_unit_price, pricing_mode

### Price Search (`services/price_search.py`)

Google Shopping integration via SearchAPI.io:

```python
search_material_price(query, location) -> dict  # shopping_results list
compute_confidence(query, result, price) -> str  # "high"/"medium"/"low"
```

- 15-minute TTL cache (bounded, prevents OOM)
- Preferred suppliers: Home Depot, Lowe's, Menards, Ferguson, Ace Hardware
- Confidence scoring based on keyword overlap and supplier tier

### Product Intelligence Registry (`services/product_intelligence.py`)

Central registry of construction material metadata:

```python
PRODUCT_REGISTRY: list[ProductInfo]  # All known products
lookup_product(title) -> ProductInfo | None  # Keyword matching
get_coverage_default(product) -> float  # sq_ft/linear_ft per package
is_board_product(title) -> bool
get_board_coverage(title) -> float  # sq ft per board from dimensions
```

Categories: flooring, roofing, drywall, paint, insulation, electrical, plumbing, lumber, concrete, tile, cabinetry, etc.

### Labor Rate Service (`services/labor_rate_service.py`)

```python
compute_labor_rate(title, group_name, uom, zipcode) -> tuple[float, str]
```

1. Detect trade from title + group
2. Get regional multiplier from zipcode
3. Look up productivity for UOM
4. Return (rate_per_unit, formula_string)

### Material Estimator (`services/material_estimator.py`)

Fallback when Google Shopping search fails:

```python
estimate_material(title, search_query, quantity, uom, waste_factor) -> EstimatedMaterial | None
```

- Uses product registry coverage + price ranges
- Falls back to industry-standard estimates
- Always marked `confidence_score="estimated"` with "Est: " prefix

### Estimate Service (`services/estimate_service.py`)

Core CRUD layer:

- `list_estimates()`, `get_estimate()`, `create_estimate()`, `update_estimate()`, `delete_estimate()`
- `add_group()`, `remove_group()`
- `add_line_item()`, `update_line_item()`, `remove_line_item()`
- `add_material()`, `update_material()`, `remove_material()`
- `recompute_estimate_state()`: Recalculates lifecycle state from validation results
- `accumulate_token_usage()`: Tracks API usage and costs per pipeline phase
- `transition_lifecycle()`: State machine transitions

### Session Manager (`services/session_manager.py`)

WebSocket session state:

```python
@dataclass
class SessionState:
    estimate_id: int
    conversation: list[dict]          # Message history
    groups_completed: set[str]        # Completed group names
    groups_pending: set[str]          # Pending group names
    created_group_ids: dict[str, int] # For idempotent retries
    created_item_ids: list[int]       # Audit trail

@dataclass
class Checkpoint:
    phase: str       # "scope" | "pricing" | "qa" | etc.
    status: str      # "started" | "completed" | "failed"
    details: dict    # Phase-specific data
```

---

## 8. Database Models & Schema

### Entity Relationship

```
Estimate (1) ----< Group (N) ----< LineItem (N) ----< Material (N)
Estimate (1) ----< ChatMessage (N)
```

All relationships use cascade delete.

### Estimate

| Column | Type | Description |
|--------|------|-------------|
| id | Integer PK | |
| title | String(255) | Estimate name |
| status | String(50) | "draft" / "active" / "archived" |
| labor_markup | Float | Default 20% |
| material_markup | Float | Default 15% |
| tax_rate | Float | Default 0% |
| zipcode | String(10) | Regional pricing |
| location_city | String(100) | City for search |
| location_state | String(50) | State for search |
| total_input_tokens | Integer | Cumulative API usage |
| total_output_tokens | Integer | |
| total_cost_usd | Float | API cost tracking |
| quality_score | Integer | 0-100 from validators |
| lifecycle_state | String(50) | State machine state |
| pending_plan | JSON | Stored plan awaiting approval |
| created_at | DateTime | |
| updated_at | DateTime | |

### Group

| Column | Type | Description |
|--------|------|-------------|
| id | Integer PK | |
| estimate_id | Integer FK | Parent estimate |
| name | String(255) | Trade group name |
| sort_order | Integer | Display ordering |

### LineItem

| Column | Type | Description |
|--------|------|-------------|
| id | Integer PK | |
| group_id | Integer FK | Parent group |
| title | String(255) | Item description |
| description | Text | Extended description |
| line_item_type | String(50) | assembly/material/equipment/permit/text |
| quantity | Float | Work quantity |
| rate | Float | Per-unit rate (labor for assembly, price for material) |
| uom | String(50) | Unit of measure |
| sort_order | Integer | Display ordering |
| labor_markup | Float | Override or default 20% |
| material_markup | Float | Override or default 15% |
| **Correctness Fields** | | |
| confidence_level | String(50) | verified/estimated/unresolved |
| pricing_state | String(50) | verified/estimated/unresolved/labor_only/source_backed |
| coverage_source | String(100) | How coverage was determined |
| calculator_source | String(100) | What computed the values |
| blocking_issues | JSON | Unresolved validation issues |
| unresolved_reason | Text | Human-readable explanation |
| provenance_json | JSON | Full audit trail |
| validation_issues_json | JSON | All validation results |

### Material

| Column | Type | Description |
|--------|------|-------------|
| id | Integer PK | |
| line_item_id | Integer FK | Parent line item |
| name | String(255) | Product name |
| price | Float | Per-package price |
| quantity | Float | Number of packages |
| supplier_name | String(255) | Retailer name |
| image_url | Text | Product image |
| supplier_image_url | Text | Supplier logo |
| product_url | Text | Purchase link |
| package_unit | String(50) | e.g., "roll", "box", "sheet" |
| package_qty | Float | Units per package |
| normalized_unit_price | Float | Price per work unit (sq_ft, linear_ft) |
| confidence_score | String(50) | high/medium/low/estimated/source_backed |

### ChatMessage

| Column | Type | Description |
|--------|------|-------------|
| id | Integer PK | |
| estimate_id | Integer FK | |
| role | String(50) | user/assistant/system |
| content | Text | Message content |
| created_at | DateTime | |

---

## 9. WebSocket Protocol & Real-Time Events

### Connection

```
ws://localhost:8000/ws/{estimate_id}
```

On connect:
1. Restore estimate from DB
2. Send existing pending_plan if any (`plan_proposed` event)
3. Load conversation history as `chat_message_added` events

### Client -> Server Messages

| Event | Payload | Trigger |
|-------|---------|---------|
| `user_message` | `{text, attachments?}` | User sends chat message |
| `approve_plan` | `{}` | User approves proposed plan |

### Server -> Client Events

| Event | Payload | When |
|-------|---------|------|
| `typing_started` | `{}` | Agent begins processing |
| `typing_stopped` | `{}` | Agent finishes processing |
| `assistant_chunk` | `{delta}` | Streaming text token |
| `assistant_message` | `{content}` | Complete response |
| `chat_message_added` | `{role, content}` | History replay on connect |
| `group_added` | `{id, name, sort_order}` | New group created |
| `group_removed` | `{id}` | Group deleted |
| `line_item_added` | `{id, group_id, title, type, quantity, rate, uom, extended_cost, materials[], unresolved?}` | New line item |
| `line_item_updated` | `{id, ...changed_fields}` | Line item modified |
| `line_item_removed` | `{id}` | Line item deleted |
| `material_added` | `{line_item_id, material: {...}}` | Material attached |
| `material_removed` | `{id, line_item_id}` | Material deleted |
| `plan_proposed` | `{plan: {groups, summary}}` | Plan ready for review |
| `estimate_quality_updated` | `{quality_score, lifecycle_state}` | After QA pass |
| `token_usage_updated` | `{input_tokens, output_tokens, cost_usd}` | API usage update |
| `suggestions_updated` | `{suggestions: string[]}` | Contextual action chips |
| `error` | `{message}` | Error occurred |

---

## 10. REST API Routes

### Estimates (`routes/estimates.py`)

| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/estimates` | List estimates (with optional search) |
| POST | `/api/estimates` | Create new estimate |
| GET | `/api/estimates/{id}` | Get full detail with groups, items, materials |
| PUT | `/api/estimates/{id}` | Update metadata (title, markups, location) |
| DELETE | `/api/estimates/{id}` | Soft delete (archive) |
| POST | `/api/estimates/{id}/duplicate` | Clone estimate |
| PUT | `/api/line-items/{id}` | Update line item fields |

### Uploads (`routes/uploads.py`)

| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/uploads` | Upload image or PDF (20 MB limit) |
| GET | `/api/uploads/{filename}` | Serve uploaded file |

**Security:** Path traversal prevention, MIME type validation via magic bytes.
**Allowed types:** JPEG, PNG, WebP, GIF, PDF

---

## 11. Estimate Lifecycle & State Machine

```
building  ──>  review_required  ──>  validated  ──>  sent
   ^                 |                    |
   |                 v                    |
   +─── (user edits) ────────────────────+
```

| State | Meaning | Transition Trigger |
|-------|---------|--------------------|
| `building` | Estimate being constructed | Initial state |
| `review_required` | Has unresolved items or low quality score | Unresolved items detected |
| `validated` | All items verified, quality score acceptable | All issues resolved |
| `sent` | User exported/sent the estimate | User action |

`recompute_estimate_state()` runs after each pipeline pass and recalculates based on:
- Number of unresolved items
- Overall quality score
- Blocking validation issues

---

## 12. Quality Assurance Pipeline

### Pipeline Flow (after main agent execution)

```
1. Product Intelligence Pass
   - PI Reviewer agent validates ALL materials
   - Checks category match, quantity accuracy, price sanity
   - Removes samples/swatches, fixes wrong products
   - Safety: won't reduce estimate >30%

2. QA Agent Pass (silent)
   - Adds missing cleanup/disposal
   - Adds missing permits for regulated trades
   - Adds missing contingency (5% if >$2,000)
   - Fixes $0 rate items
   - Safety: won't reduce estimate >15%

3. Quality Score Computation
   - validate_estimate() returns ValidationIssue[]
   - compute_quality_score() returns 0-100
   - Score = 100 - (blocking * 15) - (warnings * 5) - (info * 2)

4. Feedback Loop (if score < 70)
   - Classify issues: "pricing" | "scope" | "qa_only"
   - Pick best retry target (Pricing > Scope)
   - Run single targeted agent retry
   - Re-score
   - Guard: max 15% estimate reduction, token budget limits
```

### Quality Score Thresholds

| Score | Status | Action |
|-------|--------|--------|
| 70-100 | Acceptable | Pipeline complete |
| 50-69 | Needs improvement | One targeted retry |
| 0-49 | Critical issues | Flag as review_required |

---

## 13. Plan Mode: Compile & Execute

### Flow

```
User message ("build me a deck estimate")
    |
    v
Orchestrator routes to Scope Agent (mode="plan")
    |
    v
Scope Agent calls generate_plan(groups_json, summary)
    |
    v
Plan Compiler enriches with trade metadata
    |
    v
WebSocket: plan_proposed event -> Frontend shows plan UI
    |
    v
User reviews and approves (approve_plan message)
    |
    v
Execution Agent OR Plan Executor processes the plan
    |
    v
Line items created with real pricing
```

### Plan Schema

```json
{
  "groups": [
    {
      "name": "Demolition",
      "items": [
        {
          "title": "Demo existing deck",
          "line_item_type": "assembly",
          "quantity": 1,
          "uom": "lump_sum",
          "rate": 0,
          "description": "Remove old deck structure",
          "search_query": ""
        }
      ]
    }
  ],
  "summary": "Complete deck replacement estimate",
  "compiled": false
}
```

### Plan Compilation (what gets added)

For each item, the compiler:
- Detects trade_id from title + group
- Sets `labor_only` flag based on trade rules
- Computes rate from labor productivity tables
- Sets waste_factor from trade profile
- Adds search_query from work item templates
- Sets work_item_type for routing

### Two Execution Paths

1. **Plan Executor** (`services/plan_executor.py`): Fully deterministic. No LLM. Uses build pipeline directly. Faster, more predictable.

2. **Execution Agent** (`agents/execution.py`): LLM-powered. Better material selection with judgment. Uses assembly engine for complex BOMs. Adds Overhead & Profit. max_turns=120.

The WebSocket handler (`routes/ws.py`) uses the Execution Agent when `approve_plan` is received.

---

## 14. Pricing & Material Intelligence

### Price Search Pipeline

```
Agent calls search_price(query, location, required_qty, required_unit)
    |
    v
SearchAPI.io Google Shopping API
    |
    v
Results cached (15-min TTL, bounded size)
    |
    v
For each result:
  - Extract price (handles various formats)
  - Compute confidence (keyword overlap + supplier tier)
  - Detect package (unit normalizer)
  - Normalize pricing (per-unit cost)
  - Calculate packages_needed
    |
    v
Return top 10 results with:
  - name, price, supplier, image, URL
  - confidence: high/medium/low
  - packages_needed, total_cost
  - pricing_mode: coverage/per-unit/unknown
  - normalized_unit_price
```

### Material Attachment Rules

1. **Assembly items:** Materials are attached as children. Rate stays as labor. Materials tracked separately with `material_markup`.
2. **Material items:** Rate IS the material price. `add_material` also sets the rate.
3. **Labor-only items:** No materials attached. Rate is pure labor.

### Package-Based Quantity Calculation

The #1 source of errors. The system enforces:

```
WRONG: 650 sq_ft of tile -> quantity=650, price=$3.50/sq_ft
RIGHT: 650 sq_ft of tile -> quantity=33 boxes, price=$45.00/box (20 sq_ft/box + 10% waste)
```

Three enforcement layers:
1. `calculate_material_quantity()` tool for agents
2. `normalize_product_price()` in unit normalizer
3. `ValidationGate.check()` catches scope-quantity leakage

### Fallback Chain

```
1. Google Shopping search -> verified product with real price
2. Material estimator -> registry-based estimate with "Est:" prefix
3. Pricing Allowance -> agent creates placeholder for user review
4. Persist as UNRESOLVED -> flagged in UI for user action
```

---

## 15. Configuration & Environment

### Required Environment Variables

```bash
OPENAI_API_KEY=sk-...          # GPT-4.1 access (required)
```

### Optional Environment Variables

```bash
SEARCHAPI_KEY=...              # Google Shopping price search
DATABASE_URL=sqlite+aiosqlite:///./estimatic.db
```

### File Upload Storage

Uploaded files stored in `backend/uploads/` directory. Served via `/api/uploads/{filename}`.

### CORS Configuration

Default: `["http://localhost:5173"]` (Vite dev server)

---

## 16. Testing Strategy

### Test Files

```
tests/
  conftest.py                    -- Shared fixtures
  test_assembly_engine.py        -- Assembly pricing computation
  test_build_pipeline.py         -- Line item build pipeline
  test_formula_parser.py         -- Quantity formula evaluation
  test_lifecycle_state.py        -- Estimate state transitions
  test_material_estimator.py     -- Fallback material estimation
  test_material_semantics.py     -- Product-scope compatibility
  test_plan_compiler.py          -- Plan compilation with trade data
  test_plan_executor.py          -- Deterministic plan execution
  test_plausibility_regression.py -- Quantity/price sanity regression
  test_product_intelligence.py   -- Product registry lookups
  test_profile_loader.py         -- YAML profile loading
  test_state_model_integrity.py  -- DB model state consistency
  test_trade_rules.py            -- Canonical rule functions
  test_trade_validator.py        -- Trade profile completeness
  test_unit_normalizer.py        -- Price normalization
  test_validation_gate.py        -- Pre-persist validation
  golden/
    conftest.py                  -- Golden test fixtures
    test_golden_estimates.py     -- End-to-end estimate snapshots
```

### Running Tests

```bash
cd backend
pytest tests/                          # All tests
pytest tests/test_trade_rules.py -v    # Single file
pytest -k "normalize" -v               # Filter by name
```

### Test Categories

| Category | What's Tested |
|----------|--------------|
| Domain rules | Labor-only detection, permit requirements, material compatibility |
| Build pipeline | Scope normalization, product scoring, material attachment |
| Validation gate | Persist decisions, blocking issues, quantity leakage |
| Unit normalizer | Package detection, coverage parsing, pricing normalization |
| Plan execution | Compile -> execute -> persist -> WebSocket events |
| Assembly engine | Labor computation, BOM quantity formulas |
| Product intelligence | Registry lookup, coverage defaults, board products |
| Profile loader | YAML parsing, validation, error handling |
| Golden tests | Full end-to-end estimate snapshots |