← cd ../projects
2026 · Solo · design + build
Dispute Triage Agent
An LLM agent that helps a merchant decide whether to fight or accept a card chargeback, and drafts the letter when it's worth fighting.
$ triage --case #4471
classifydone
assessdone
decideFIGHT · 90%
draftletter ready
_
context
Fraud and chargeback was the domain I used to work in at PicPay, mostly fraud. I wanted to see what an LLM adds to it (honestly measured, not assumed) and to close my own Python gap, so I built this in Python on purpose instead of in the Node stack I already know.
It's decision support on synthetic sample data, not a production platform. A LangGraph pipeline runs four steps (classify → assess → decide → draft) and a Next.js front end shows the verdict and the letter.
stack
PythonFastAPILangGraphClaudeNext.js
decisions
01
LangGraph instead of one big prompt
- considered
- A single mega-prompt, or a hand-rolled tool loop on the raw SDK.
- why
- A mega-prompt has no structure, is hard to test and can't branch. Explicit nodes let me test each step and route between them, and LangGraph is a skill worth being able to show.
02
The fight/accept decision is code, not the model
- considered
- Letting Claude make the final call.
- why
- The model weighs conflicting signals: that's judgment, not
if/else, and it's where AI earns its place. The verdict itself is a threshold on its estimate, so the decision boundary is testable without mocking an LLM.
03
The letter can only cite evidence the merchant has
- considered
- Trusting the prompt alone to keep the model from inventing evidence.
- why
- A representment letter that cites evidence you don't have loses the dispute. The prompt separates available from unavailable evidence, and the eval set fails any letter that mentions the second list.
outcome
- ▸Built a rules-only baseline first, then upgraded it in vertical slices, so I can say exactly what the AI adds over plain rules.
- ▸22 tests with Claude mocked, plus a 10-case eval set that checks every drafted letter for invented evidence.
- ▸If a Claude call fails, each model-driven step falls back to rules instead of failing the request.
- ▸Deployed as two services: FastAPI on Render and Next.js on Vercel, with per-IP rate limiting on the analysis endpoint.