Yes. Here is the final Week 6 plan I would follow to make ClaimFlow AI production-grade for your portfolio/demo.
The goal is not to add more claim logic. Your repo already has Week 1–5 scripts for extraction, review, RAG, agent, and memory evals. Your DB already has extraction runs, review tasks, RAG policy chunks, agent action logs, follow-up drafts, workflow memories, memory hits, and memory updates.
Week 6 should make the system observable, traceable, evaluable, and demo-ready.
By the end, your demo should say:
ClaimFlow AI is a governed agentic workflow.
Every AI call goes through a gateway.
Every model, prompt version, latency, cost, and error is logged.
Every extraction, RAG answer, agent action, memory hit, review decision, and memory update is traceable.
Eval results from Week 1–6 are visible in a dashboard.
Gateway failure cases prove the system handles production AI failures safely.
Your README is currently still mostly describing the early extraction/validation workflow, so final docs are a critical part of this week.
Create the production control layer for all model calls.
Your current schema tracks workflow events, agent actions, memory, and review, but it does not yet have a dedicated AI gateway call log. Add that first.
packages/gateway/
package.json
index.ts
types.ts
ai-gateway.ts
errors.ts
cost-policy.ts
latency-policy.ts
prompt-registry.ts
response-parser.ts
scripts/smoke-test-gateway-log.ts
Add enums:
enum AiCallKind {
EXTRACTION
VALIDATION_ASSIST
RAG_QUERY_REWRITE
RAG_ANSWER
AGENT_PLANNER
MEMORY_WRITER
MEMORY_SUMMARIZER
EVAL_JUDGE
SYNTHETIC_GATEWAY_TEST
}
enum AiCallStatus {
STARTED
SUCCEEDED
FAILED
RETRYABLE
BLOCKED
}
enum AiGatewayFailureType {
MODEL_TIMEOUT
INVALID_JSON_RESPONSE
PROVIDER_ERROR
COST_LIMIT_EXCEEDED
LATENCY_SPIKE
PROMPT_VERSION_REGRESSION
EVAL_SCORE_DROPPED
MISSING_TRACE_ID
MISSING_MODEL_VERSION
UNKNOWN
}
Add model:
model AiCallLog {
id String @id @default(cuid())
traceId String
runId String?
run ExtractionRun? @relation(fields: [runId], references: [id], onDelete: SetNull)
kind AiCallKind
status AiCallStatus
provider String
model String
modelVersion String?
promptVersion String?
schemaVersion String?
inputJson Json?
outputJson Json?
parsedOutputJson Json?
errorType AiGatewayFailureType?
errorMessage String?
retryable Boolean @default(false)
latencyMs Int?
inputTokens Int?
outputTokens Int?
totalTokens Int?
estimatedCostUsd Float?
metadata Json?
createdAt DateTime @default(now())
updatedAt DateTime @updatedAt
@@index([traceId])
@@index([runId])
@@index([kind])
@@index([status])
@@index([errorType])
@@index([createdAt])
@@map("ai_call_logs")
}