Public LLM benchmarks have a quiet problem: they test from someone else's servers, against someone else's account tier, at a time of day that may have nothing to do with your traffic patterns. The numbers look authoritative, but they are not your numbers. If you are running Llama 3.1 70B in production through Fireworks, Together AI, or Groq, the latency your users experience depends on your API key, your network path, and how your account is provisioned — not on a global average collected by a third-party tool.

This is the problem Tesseract solves.

What Tesseract actually does

Tesseract is a benchmarking and routing tool for engineers who run open-source LLMs in production with their own provider accounts. You connect your existing API keys for Fireworks, Together AI, Groq, Baseten, and OpenRouter. Tesseract then fires the same prompt to every connected provider serving your chosen model — in parallel — and measures time to first token, tokens per second, total latency, and cost per million tokens. The results are specific to your keys and your traffic, not a synthetic composite.

The output is a live scoreboard that answers a simple question most engineering teams struggle to answer precisely: which provider is fastest and cheapest for this model, right now, on my account?

Why hand-rolled scripts are not enough

Most ML engineers who feel this pain have already written a benchmarking script at some point. A few asyncio calls, some timing logic, a CSV export. The script works once, gets committed, and is stale within a week. Provider performance shifts. A new model version rolls out. Pricing changes. The script does not know about any of it, and nobody has time to rerun it on a schedule while also shipping features.

That stale script is worse than no data in one specific way: it creates false confidence. You make routing decisions based on a snapshot from three weeks ago, and you only discover the performance has changed after a user complains or an incident retrospective forces the question.

Tesseract replaces that workflow. The Pro tier runs automated benchmarks on an hourly, 6-hour, or daily schedule and maintains a rolling 7-day scoreboard with trend arrows so you can see not just who is winning right now but whether a provider's latency has been drifting upward over the past week.

From benchmark to shipped config

Seeing who is fastest is useful. Being able to act on it immediately is better. Tesseract's routing rules do the second part. You define a rule for a model with an objective — lowest latency, highest throughput, or lowest cost — and Tesseract generates a ready-to-paste configuration: the provider base URL, the model identifier, a curl command, and a Python snippet. One click to copy, paste it into your app, and your code is pointing at the current best provider for that objective.

This matters because the distance between "I know Groq is faster this week" and "my app is actually using Groq" is usually several manual steps: look up the base URL in the provider docs, update the environment variable, test the change, deploy. Tesseract collapses those steps into one copy-paste action.

When the best provider changes — because it always eventually does — the routing rule updates automatically and the config snippet reflects the new recommendation the next time you check.

Alerts that find the problem before your users do

Latency spikes and cost creep are easy to miss until they are impossible to ignore. Tesseract's alert system lets you set a threshold — say, p95 latency over 800 milliseconds, or cost over a certain ceiling — and sends you an email the moment the condition is met, naming the model, the condition, and the better provider that is currently available. The alert fires once when the condition is triggered and stays suppressed until the metric recovers and then degrades again, so you do not get flooded with repeated notifications for the same ongoing issue.

For engineers who have discovered a provider problem through a user complaint or a late-night Slack ping, this is the feature that changes the experience from reactive to proactive.

Free tier and pricing

The free tier on Tesseract is functional, not just a demo. You can connect up to two providers, run five manual benchmarks per day, view a scoreboard built from your own runs, and generate one routing rule with a copy-ready config. No credit card required to start.

The Pro tier at $39 per month removes those limits: unlimited connected providers, unlimited manual benchmarks, automated scheduled benchmarks, unlimited routing rules, latency and cost alerts by email, and the 7-day trend charts. It is priced for an individual engineer to expense without a procurement process, and most engineers who upgrade do so after the first time an alert catches a degradation they would otherwise have missed.

Who Tesseract is built for

The product is built for ML engineers and backend engineers at startups that have already committed to running open-source models in production. If you are still evaluating which model to use, or if all your inference goes through a single managed provider you do not control, Tesseract is not the right tool yet. But if you hold accounts at two or more inference providers, track p95 latency and cost per million tokens as real operational metrics, and feel the friction of making routing decisions based on stale data, Tesseract is the tool that fills that gap.

You can connect your first providers and run a benchmark at tesseract.click. The free tier requires no payment information, and a benchmark run takes under 30 seconds.

Frequently asked questions

Does Tesseract store my API keys? Keys are encrypted at rest and never returned to the client in full. Only the last four characters are shown after a key is saved. You can delete a key at any time and it is removed from all future benchmark runs immediately.

Which providers does Tesseract support? Currently Fireworks, Together AI, Groq, Baseten, and OpenRouter. Support for additional providers is on the roadmap.

Which models can I benchmark? Tesseract maintains a catalog of open-source models and which providers serve each one. You pick the model, and the benchmark runs against every connected provider that serves it. Common choices include Llama 3.1 70B, Mistral 7B, Mixtral 8x7B, and Qwen 2.5.

How is this different from public leaderboards like LMSpeed or ProviderBench? Public leaderboards run benchmarks from their own infrastructure and aggregate the results globally. Tesseract runs benchmarks using your actual API keys from your network path, which means the numbers reflect what your application will actually experience. Provider performance can vary significantly between accounts and regions.

What is the accuracy of the latency measurements? Time to first token is measured from the moment the streaming request is initiated to the moment the first token arrives in the response stream. Tokens per second is computed across the full streamed response. These are application-layer measurements, not network-layer, which is the number that matters for end-user experience.