MultiLLM
and MultiLLMAsync
classes in our Python SDK.
Single LLM Clients
Before diving into the Multi-LLM clients, it’s helpful to have a quick recap on how theUniLLM
clients,
Unify
and AsyncUnify
, can be used:
AsyncUnify
is given below:
Multi-LLM Clients
BothMultiLLM
and MultiLLMAsync
wrap AsyncUnify
instances under the hood,
such that the LLMs are queried in parallel. The distinction between MultiLLM
and
MultiLLMAsync
refers to whether the .generate()
method is itself also an
asynchronous function, which can be nested inside a broader outer program orchestrated
by asyncio.run
.
An interactive session with several LLM can be spun up in Python like so:
MultiLLMAsync
, as follows:
Unify
client, setters can also be chained for multi-llm clients, like so: