MultiLLM
and MultiLLMAsync
classes in our Python SDK.
UniLLM
clients,
Unify
and AsyncUnify
, can be used:
AsyncUnify
is given below:
MultiLLM
and MultiLLMAsync
wrap AsyncUnify
instances under the hood,
such that the LLMs are queried in parallel. The distinction between MultiLLM
and
MultiLLMAsync
refers to whether the .generate()
method is itself also an
asynchronous function, which can be nested inside a broader outer program orchestrated
by asyncio.run
.
An interactive session with several LLM can be spun up in Python like so:
MultiLLMAsync
, as follows:
Unify
client, setters can also be chained for multi-llm clients, like so: