RecruitX

SHIPPED

Independent

Evidence-first verification — every conclusion traces to evidence, a human decides

The problem

Hiring tools hand you a score for a candidate without showing you where it came from, so you're trusting a number you can't check.

What I built

RecruitX runs a 6-stage multi-agent LangGraph pipeline with checkpointing, pausing for human review rather than issuing a verdict. There is deliberately no scoring model: every conclusion traces back to the evidence it came from, and the human stays the decision-maker. A golden-dataset harness measuring precision, recall and F1 gates every change before it ships.

LangGraphFastAPIPostgreSQLQLoRALangfuseStripe

Pipeline

6-stage multi-agent LangGraph orchestration
Checkpointing — runs resume rather than restart
Human-in-the-loop review stage

Evaluation

Golden-dataset harness — precision, recall, F1
Every pipeline change gated on the harness
Fine-tuned Qwen2.5-3B matched teacher-model F1

Observability & resilience

Langfuse span tracing across every run
Per-run token and cost tracking
Multi-provider LLM abstraction with automatic fallback

Architecture

in / out
Candidate evidence in
Claims, documents, and supporting artifacts.
reasoning
LangGraph orchestration — 6 stages
Multi-agent, each stage with an explicit contract.
state
Checkpointer
A failed run resumes from the last good stage instead of restarting.
reasoning
Fine-tuned Qwen2.5-3B
QLoRA, ~800 distilled examples. High-volume structured stages.
reasoning
Frontier model
Kept for the genuinely open-ended stages.
gate
No scoring model. No auto-verdict.
Every conclusion ships with the evidence it came from, or it doesn't ship.
human
Human review
The decision-maker — by design, not as a fallback.

Across every step

Langfuse span tracing
Per-run token and cost tracking across every stage.
Multi-provider fallback
Automatic failover — zero user-facing failures during provider outages.
Golden-dataset eval gate
Precision / recall / F1 measured before any pipeline change ships.

reasoningtoolsstategatehuman

Six checkpointed stages that end in a human, not a verdict. Model choice is routed per stage: distilled small model where the work is repetitive, frontier model where the reasoning is open-ended.

Hard decisions & trade-offs

I refused to add a scoring model

Chose: The system surfaces evidence and stops. No composite score, no auto-verdict, no ranking.
Rejected: A single 0-100 candidate score — the feature every comparable product ships and the one buyers ask for first.
Why: A score is the part of the system nobody can audit, and it's the part that determines the outcome. Once a number exists, it gets treated as the answer and the evidence underneath it stops being read — so the tool quietly becomes the decision-maker while the human keeps the liability. Refusing to compute it keeps the conclusion traceable and keeps the responsibility where it legally and ethically belongs. It cost the most-requested feature.

When a fine-tuned small model beats a frontier API call

Chose: Fine-tuned Qwen2.5-3B with QLoRA on a distilled ~800-example dataset for the high-volume structured stages, keeping frontier models where reasoning is genuinely open-ended.
Rejected: Calling a frontier model for every stage — better zero-shot quality, no training pipeline, no dataset to maintain.
Why: The high-volume stages are narrow and repetitive: the same extraction shape, over and over. That's exactly the shape distillation captures well, and the tuned 3B matched the teacher's F1 at roughly a tenth the inference cost. It's the wrong trade for the open-ended stages, where a small model degrades and the volume is too low for the savings to matter — so those still call frontier models.

Multi-provider abstraction bought before it was needed

Chose: Every LLM call goes through a provider-agnostic layer with automatic fallback, built before any outage forced it.
Rejected: Coding directly against one provider's SDK and dealing with portability if it ever became a problem.
Why: Provider outages are not rare and not announced in advance. Building the seam early cost a small indirection; retrofitting it during an outage means shipping the abstraction and the migration under time pressure with users watching. It paid for itself — provider outages have produced zero user-facing failures.

Verified numbers

pipeline stages, checkpointed: 6
lower inference cost at matched F1: ~10x
distilled training examples: ~800
user-facing failures during provider outages: 0

Open RecruitX →All work →Get in touch →