Parameters can be used to represent the settable parameters of an experiment or a logging task. For example, if you’re evaluating an LLM, then parameters can be used to represent the: system message, temperature, tools available etc.

Under the hood, parameters are effectively just versioned entries.

log = unify.log(
    params={
        "system_message": "You are a helpful assistant.",
        "tool_use": True,
        "temperature": 0.5,
    },
    question="What is the capital of France?",
    answer="The capital of France is Paris.",
    score=1.0
)

They are always shown on the left hand side of the table.

Parameter versions are of type string, and the version is simply an incrementing integer. In future, we plan to support custom version naming, thus the choice of a string type rather than an integer. You can see the version of a paramter in the view pane.

​
Parameter Contexts

As with entries, parameters can be used as a context manager.

with unify.Params(
    system_message="You are a helpful assistant.",
    tool_use=True,
    temperature=0.5,
):
    for _ in range(5):
        unify.log(score=random.random())

This will show in the table like so:

Like the other context managers, parameter context managers can also be nested:

with unify.Params(
    system_message="You are a helpful assistant.",
):  
    for temperature in [0.5, 0.7]:
        with unify.Params(temperature=temperature):
            for tool_use in [False, True]:
                with unify.Params(tool_use=tool_use):
                    [
                        unify.log(score=random.random())
                        for _ in range(2)
                    ]

This will show in the table like so:

As with entries, unify.Parameter("...") also changes the behaviour of get_logs, returning only the logs which include the parameter. This behaviour can be change by setting unify.Parameter("...", mode="read") which will only impact log getting, and unify.Parameter("...", mode="write") which will only impact log setting.

with unify.Params(
    system_message="You are a helpful assistant.",
):  
    for temperature in [0.5, 0.7]:
        with unify.Params(temperature=temperature):
            for tool_use in [False, True]:
                with unify.Params(tool_use=tool_use):
                    [
                        unify.log(score=random.random())
                        for _ in range(2)
                    ]
                    assert len(unify.get_logs()) == 2
            assert len(unify.get_logs()) == 4
    assert len(unify.get_logs()) == 8
EntriesExperiments