# LLM Bidding System

> A task-aware router that compares model confidence, price, risk fit, and observed results before selecting a coding agent.

Published: 2026-06-10
Updated: 2026-08-03
Canonical URL: https://aidanmarshall.ai/projects/llm-bidding-system/
Repository: https://github.com/Aidan2111/llm-bidding-system

## Context

The best coding model depends on the task. A low-risk copy change and a destructive migration should not use the same price and quality tradeoff.


## Problem

Static model preferences ignore task risk, cost, and whether a model's confidence has matched its past results. A useful router also needs to abstain when no bid clears the policy.


## Approach

Each provider receives the task and returns a structured bid with confidence, approach, token estimate, and effort class. Agent Autonomy Score supplies deterministic risk context before utility scoring selects a winner.


```text
quality  = 0.6 × calibrated confidence + 0.4 × success rate
price    = 1 - min(estimated cost / cost ceiling, 1)
risk fit = success rate within the task's risk band
utility  = 0.5 × quality + 0.2 × price + 0.3 × risk fit
```

## Implementation

The router supports Anthropic, OpenAI-compatible providers, and local Ollama models. SQLite stores auctions, bids, costs, diffs, and reported outcomes so future auctions can use calibration and per-risk-band success rates.

A selected actor can propose a patch, but the system never applies it. A human or supervising agent reviews, tests, and reports the outcome.


## Tradeoffs and limitations

Self-reported confidence is weak before enough outcomes exist. Neutral priors and calibration reduce that problem, but provider prices and model behavior still need maintenance. The supervised boundary is intentional.


## What it demonstrates

The system makes routing decisions inspectable and connects model selection to cost, risk, and observed performance. It can run deterministic dry auctions without API keys and route work to local hardware through Ollama.


## Technology

Python, Multi-model routing, Agentic AI, Ollama, SQLite

## Verification

- Public repository: https://github.com/Aidan2111/llm-bidding-system