An AI lead-qualification agent for mortgage lenders, reshaped per lender and proven with evals
A borrower submits a refinance inquiry; the agent texts them within seconds, qualifies the lead, handles objections, stays compliant, and warm-transfers qualified borrowers to a licensed loan officer with a structured handoff. Every behavior lives in a lender config, and every change is tested against simulated borrowers before it ships.
Latest results
| Lender | Conversations | Correct handoffs | Compliance-clean | Borrower experience | Clear next step |
|---|---|---|---|---|---|
| Harborline Home Loans | 30 | 100% | 100% | 4.6 / 5 | 100% |
| Bayview Community Credit Union | 30 | 100% | 100% | 4.6 / 5 | 100% |
159 simulated conversations run so far across all eval runs. Scores from simulated borrowers and an LLM judge; not real-world results.
Explore
Live chat
Text with either lender's agent as a borrower. See the fields it captures and the handoff it writes. Rate-limited.
Transcripts and replays
Recorded eval conversations with the judge's notes inline. Replays run without any API calls.
Eval scorecards
Rule-based checks plus an LLM judge, scored per behavior, and how the agent improved run over run.
Reshaping for a second lender
A credit union with stricter rules: 56 config settings changed, zero lines of agent code, same eval suite.
Account health
A weekly-review view over synthetic call data, with three recommended agent changes, each tied to a number.
How it works
1. The lender lives in config
Persona, qualification fields, objection playbook, compliance rules, and handoff rules are YAML. The prompt and tools are generated from it.
2. Hard rules are enforced in code
"STOP" ends the chat without a model call. The transfer tool checks LTV, required fields, hours, and membership, and refuses with instructions.
3. Simulated borrowers test every change
Ten personas with hidden ground truth. The expected outcome comes from each lender's own rules, so a stricter lender changes what "correct" means.
4. Rules first, judge second
Regex and field comparisons catch what they can, for free and repeatably. An LLM judge, blind to ground truth, scores the judgment calls.