skip to content
projects

multi-llm client

05.2026

One async Python client that puts OpenAI, Anthropic and Gemini behind a single interface.

source
The Multi-LLM Client REPL running one prompt against OpenAI, Anthropic and Gemini at once
One prompt, three providers, one interface.

A provider-agnostic async Python client that normalizes messages, streaming, token usage and errors across OpenAI, Anthropic and Gemini behind one interface. The same core ships four ways: as a library, a CLI, an interactive REPL and a FastAPI service.

architecture

One interface normalizes messages, streaming, token usage and errors across three providers, so switching between them is a config change rather than a rewrite. Requests and responses are Pydantic v2 models under mypy-strict typing, provider calls are wrapped in tenacity retries, and a concurrent compare() fans a single prompt out to every provider at once.

how i built it

  • Normalized messages, streaming, token usage and errors, so switching provider is a config change rather than a rewrite.
  • Modelled every request and response with Pydantic v2 under mypy-strict typing.
  • Wrapped provider calls in tenacity retries.
  • Shipped the same core as a library, a CLI, an interactive REPL and a FastAPI service.

results

  • 40 tests, under mypy strict.

stack

pythonasyncioopenaianthropicgeminifastapipydantictenacity