Yes. Here is the final Week 6 plan I would follow to make ClaimFlow AI production-grade for your portfolio/demo.

The goal is not to add more claim logic. Your repo already has Week 1–5 scripts for extraction, review, RAG, agent, and memory evals. Your DB already has extraction runs, review tasks, RAG policy chunks, agent action logs, follow-up drafts, workflow memories, memory hits, and memory updates.

Week 6 should make the system observable, traceable, evaluable, and demo-ready.


Week 6 final goal

By the end, your demo should say:

ClaimFlow AI is a governed agentic workflow.

Every AI call goes through a gateway.
Every model, prompt version, latency, cost, and error is logged.
Every extraction, RAG answer, agent action, memory hit, review decision, and memory update is traceable.
Eval results from Week 1–6 are visible in a dashboard.
Gateway failure cases prove the system handles production AI failures safely.

Your README is currently still mostly describing the early extraction/validation workflow, so final docs are a critical part of this week.


Day 1 — Add AI Gateway schema and package

Goal

Create the production control layer for all model calls.

Your current schema tracks workflow events, agent actions, memory, and review, but it does not yet have a dedicated AI gateway call log. Add that first.

Files to create

packages/gateway/
  package.json
  index.ts
  types.ts
  ai-gateway.ts
  errors.ts
  cost-policy.ts
  latency-policy.ts
  prompt-registry.ts
  response-parser.ts
  scripts/smoke-test-gateway-log.ts

Prisma additions

Add enums:

enum AiCallKind {
  EXTRACTION
  VALIDATION_ASSIST
  RAG_QUERY_REWRITE
  RAG_ANSWER
  AGENT_PLANNER
  MEMORY_WRITER
  MEMORY_SUMMARIZER
  EVAL_JUDGE
  SYNTHETIC_GATEWAY_TEST
}

enum AiCallStatus {
  STARTED
  SUCCEEDED
  FAILED
  RETRYABLE
  BLOCKED
}

enum AiGatewayFailureType {
  MODEL_TIMEOUT
  INVALID_JSON_RESPONSE
  PROVIDER_ERROR
  COST_LIMIT_EXCEEDED
  LATENCY_SPIKE
  PROMPT_VERSION_REGRESSION
  EVAL_SCORE_DROPPED
  MISSING_TRACE_ID
  MISSING_MODEL_VERSION
  UNKNOWN
}

Add model:

model AiCallLog {
  id String @id @default(cuid())

  traceId String
  runId String?
  run ExtractionRun? @relation(fields: [runId], references: [id], onDelete: SetNull)

  kind AiCallKind
  status AiCallStatus

  provider String
  model String
  modelVersion String?
  promptVersion String?
  schemaVersion String?

  inputJson Json?
  outputJson Json?
  parsedOutputJson Json?

  errorType AiGatewayFailureType?
  errorMessage String?
  retryable Boolean @default(false)

  latencyMs Int?
  inputTokens Int?
  outputTokens Int?
  totalTokens Int?
  estimatedCostUsd Float?

  metadata Json?

  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt

  @@index([traceId])
  @@index([runId])
  @@index([kind])
  @@index([status])
  @@index([errorType])
  @@index([createdAt])
  @@map("ai_call_logs")
}