AI coding routes tested on real engineering work.
Faros Route Index compares complete coding routes on the same task cohort. See quality, cost, cache use, effort, and recorded runtime together—then inspect the evidence behind every result.
Powered by Faros AI Simulator
Current leader——Routes are ranked by mean task-specific quality. Cost, cache use, task wins, and runtime when recorded remain separate observed measures.
Mean quality across the same task cohort · select a route for details
| Rank | Route | Quality score | Task wins | Cost / task | Cache hit | Runtime |
|---|
Quality whiskers are marginal 95% confidence intervals for each route’s mean across the task cohort. They describe score uncertainty; they are not rank ranges or paired-significance tests. Green cells mark the best observed value in each column.
Every published route is evaluated on the same task cohort and scoring rules.
—
—
Set your current route and monthly task volume. Cost and quality stay separate.
Uses Faros Engineering Benchmark averages. It is not observed production savings or a customer-specific evaluation.
No composite score hides the choice. The frontier shows quality against observed cost; the scorecard shows the rubric dimensions behind quality.
Up is better. Left is cheaper. The chart area is divided into four equal quadrants.
Average rubric dimensions by route
A tie counts as half a win. Every comparison uses the exact same task cohort.
Uses the selected cohort. Route A and Route B are highlighted across the matrix.
Slice the same official cohort without recomputing or reinterpreting the source files.
Official quality scores and public-safe task evidence are served from the catalog.
| Task | Description | Domain | Complexity | Best route | Quality | Spread |
|---|
Every route receives the same task, repository state, and rubric. Read the essentials here or inspect the complete evaluation framework.
Direct answers about the workload, ranking, evaluation, cost, and what this public leaderboard does—and does not—represent.
Faros Route Index is a public catalog that compares complete AI coding routes using the same set of real Faros engineering tasks. It publishes each official benchmark run with quality, cost, cache use, task-level evidence, and runtime when the source recorded it.
No. It is evidence for the Faros Engineering Benchmark. A different organization, repository mix, or task distribution may produce a different ranking.
A route is the complete configuration used to do the work: coding harness, model, provider, effort level, and relevant runtime settings. Faros Route Index compares routes because changing the harness or effort can change the outcome even when the model is the same.
Each benchmark version contains a stable cohort of anonymized tasks sampled from real Faros engineering work across product domains, task intents, and complexity levels. Every route in a run receives that same cohort.
Each generated change is graded against a task-specific rubric by a blinded judge. The quality score is the mean percentage of rubric credit earned; it is not blended with cost or runtime.
Cost uses observed token usage with the provider pricing configured for the run. Runtime is the observed end-to-end duration when captured; a dash means the source did not record it. Cache hit is the share of eligible input served from cache; it is not itself a discount and does not determine total cost.
New models, harnesses, and providers can be tested against the stable cohort as they become relevant. Every official benchmark run is dated, preserved, and selectable so results do not silently change.
No. This public page does not accept customer repositories. It publishes the Faros benchmark. Customer-specific optimization—connecting engineering data, identifying opportunities, and validating policies—is a separate Faros product workflow.