Evaluator

class Evaluator

source code

​

properties

​

name

source code

def name(self) -> Optional[str]:

​

score_config

source code

def score_config(self) -> Dict[float, str]:

​

setters

​

set_name

source code

def set_name(self, value: str) -> Self:

​

set_score_config

source code

def set_score_config(self, value: Dict[float, str]) -> Self:

​

methods

​

evaluate

source code

def evaluate(self, *args, **kwargs) -> float:

Evaluate the given response for this input prompt, with optional extra data.

Arguments:

• prompt - The user message or the full input prompt being responded to.
• response - The response which is being evaluated, either as just the most
• kwargs - Extra information relevant to the prompt, as is stored in the Datum.

Returns:

The score, either as a boolean, a float, or the full Score instance.

​

dunder_methods

​

__init__

source code

def __init__(
        self,
        score_config: Optional[Dict[float, str]],
        name: Optional[str] = None,
        api_key: Optional[str] = None,
    ):

Create an Evaluator.

Arguments:

• name - The name for this evaluator.
• score_config - Either a derived Score subclass, or the configuration for the
• api_key - API key for accessing the Unify API. If None, it attempts to

Raises:

• UnifyError: If the API key is missing.