Language Models

Language models are used to generate agent responses to questions and can be specified when running a survey. API keys are required in order to access the available models, and should be stored in your private .env file. See the API Keys page for instructions on storing your API keys.

Available services

We can see all of the available services (model providers) by calling the services() method of the Model class:

from edsl import Model

Model.services()

This will return a list of the services we can choose from:

Service Name
openai
anthropic
deep_infra
google
groq
bedrock
azure
ollama
test
together
perplexity
mistral

Available models

We can see all of the available models by calling the available() method of the Model class:

from edsl import Model

Model.available()

This will return a list of the models we can choose from (not shown below–run the code on yor own to see an up-to-date list).

Adding a model

Newly available models for these services are added automatically. If you do not see a publicly available model that you want to work with, please send us a feature request to add it or add it yourself by calling the add_model() method:

from edsl import Model

Model.add_model(service_name = "anthropic", model_name = "new_model")

This will add the model new_model to the anthropic service. You can then see the model in the list of available models, and search by service name:

Model.available("anthropic")

Check models

Check the models for which you have already properly stored API keys by calling the check_models() method:

Model.check_models()

This will return a list of the available models and a confirmation message whether a valid key exists. The output will look like this (note that the keys are not shown):

Checking all available models...

Now checking: <model name>
OK!

Etc.

Specifying a model

We specify a model to use with a survey by creating a Model object and passing it the name of an available model. We can optionally set other model parameters as well (temperature, etc.). For example, the following code creates a Model object for Claude 3.5 Sonnet with default model parameters:

from edsl import Model

model = Model('gpt-4o')

We can see that the object consists of a model name and a dictionary of parameters:

model

This will show the default parameters of the model:

key	value
model	gpt-4o
parameters:temperature	0.5
parameters:max_tokens	1000
parameters:top_p	1
parameters:frequency_penalty	0
parameters:presence_penalty	0
parameters:logprobs	False
parameters:top_logprobs	3

Running a survey with models

Similar to how we specify Agents and Scenarios in running a survey, we specify the models to use by adding them to a survey with the by() method when the survey is run. We can pass either a single Model object or a list of models to the by() method. If multiple models are to be used they are passed as a list or as a ModelList object. For example, the following code specifies that a survey be run with each of GPT 4 and Gemini Pro:

from edsl import Model, Survey

models = [Model('gpt-4o'), Model('gemini-pro')]

survey = Survey.example()

results = survey.by(models).run()

This code uses ModelList instead of a list of Model objects:

from edsl import Model, ModelList, Survey

models = ModelList(Model(m) for m in ['gpt-4o', 'gemini-pro'])

survey = Survey.example()

results = survey.by(models).run()

This will generate a result for each question in the survey with each model. If agents and/or scenarios are also specified, the responses will be generated for each combination of agents, scenarios and models. Each component is added with its own by() method, the order of which does not matter. The following commands are equivalent:

results = survey.by(scenarios).by(agents).by(models).run()

results = survey.by(models).by(agents).by(scenarios).run()

Default model

If no model is specified, a survey is automatically run with the default model. Run Model() to check the current default model. For example, the following code runs the example survey with the default model (and no agents or scenarios) without needing to import the Model class:

from edsl import Survey

results = Survey.example().run()

Inspecting model details in results

If a survey has been run, we can inspect the models that were used by calling the models method on the Results object. For example, we can verify the default model when running a survey without specifying a model:

from edsl import Survey

survey = Survey.example()

results = survey.run()

results.models

This will return the following information about the default model that was used (note the default model may have changed since this page was last updated):

model	temperature	max_tokens	top_p	frequency_penalty	presence_penalty	logprobs	top_logprobs
gpt-4o	0.5	1000	1	0	0	False	3

To learn more about all the components of a Results object, please see the Results section.

Printing model attributes

If multiple models were used to generate results, we can print the attributes in a table. For example, the following code prints a table of the model names and temperatures for some results:

from edsl import Survey, ModelList, Model

models = ModelList(
   Model(m) for m in ['gpt-4o', 'gemini-1.5-pro']
)

survey = Survey.example()

results = survey.by(models).run()

results.select("model", "temperature") # This is equivalent to: results.select("model.model", "model.temperature")

Output:

model.model	model.temperature
gpt-4o	0.5
gemini-1.5-pro	0.5

We can also print model attributes together with other components of results. We can see a list of all components by calling the columns method on the results:

results.columns

Output:

0
agent.agent_instruction
agent.agent_name
answer.q0
answer.q1
answer.q2
comment.q0_comment
comment.q1_comment
comment.q2_comment
generated_tokens.q0_generated_tokens
generated_tokens.q1_generated_tokens
generated_tokens.q2_generated_tokens
iteration.iteration
model.frequency_penalty
model.logprobs
model.maxOutputTokens
model.max_tokens
model.model
model.presence_penalty
model.stopSequences
model.temperature
model.topK
model.topP
model.top_logprobs
model.top_p
prompt.q0_system_prompt
prompt.q0_user_prompt
prompt.q1_system_prompt
prompt.q1_user_prompt
prompt.q2_system_prompt
prompt.q2_user_prompt
question_options.q0_question_options
question_options.q1_question_options
question_options.q2_question_options
question_text.q0_question_text
question_text.q1_question_text
question_text.q2_question_text
question_type.q0_question_type
question_type.q1_question_type
question_type.q2_question_type
raw_model_response.q0_cost
raw_model_response.q0_one_usd_buys
raw_model_response.q0_raw_model_response
raw_model_response.q1_cost
raw_model_response.q1_one_usd_buys
raw_model_response.q1_raw_model_response
raw_model_response.q2_cost
raw_model_response.q2_one_usd_buys
raw_model_response.q2_raw_model_response

The following code will display a table of the model names together with the simulated answers:

results.select("model", "answer.*")

Output:

model.model	answer.q0	answer.q1	answer.q2
gpt-4o	no	killer bees in cafeteria
gemini-1.5-pro	yes		other

To learn more about methods of inspecting and printing results, please see the Results section.

ModelList class

class edsl.language_models.ModelList.ModelList(data: list | None = None)

Bases: Base, UserList

__init__(data: list | None = None)

Initialize the ScenarioList class.

>>> from edsl import Model
>>> m = ModelList(Model.available())

code()

This method should be implemented by subclasses.

classmethod example(randomize: bool = False) → ModelList

Returns an example ModelList instance.

Parameters:

randomize – If True, uses Model’s randomize method.

classmethod from_dict(data)

Create a ModelList from a dictionary.

>>> newm = ModelList.from_dict(ModelList.example().to_dict())
>>> assert ModelList.example() == newm

classmethod from_names(*args, **kwargs)

A a model list from a list of names

property names

>>> ModelList.example().names
{'...'}

rich_print()

table(*fields, tablefmt: str | None = None, pretty_labels: dict | None = None)

>>> ModelList.example().table('model')
model
-------
gpt-4o
gpt-4o
gpt-4o

to_dict(sort=False, add_edsl_version=True)

This method should be implemented by subclasses.

to_list()

to_scenario_list()

tree(node_list: List[str] | None = None)

LanguageModel class

This module contains the LanguageModel class, which is an abstract base class for all language models.

Terminology:

raw_response: The JSON response from the model. This has all the model meta-data about the call.

edsl_augmented_response: The JSON response from model, but augmented with EDSL-specific information, such as the cache key, token usage, etc.

generated_tokens: The actual tokens generated by the model. This is the output that is used by the user. edsl_answer_dict: The parsed JSON response from the model either {‘answer’: …} or {‘answer’: …, ‘comment’: …}

class edsl.language_models.LanguageModel.LanguageModel(tpm: float | None = None, rpm: float | None = None, omit_system_prompt_if_empty_string: bool = True, key_lookup: 'KeyLookup' | None = None, **kwargs)

Bases: PersistenceMixin, RepresentationMixin, HashingMixin, ABC

ABC for Language Models.

DEFAULT_RPM = 100

DEFAULT_TPM = 1000

__init__(tpm: float | None = None, rpm: float | None = None, omit_system_prompt_if_empty_string: bool = True, key_lookup: 'KeyLookup' | None = None, **kwargs)

Initialize the LanguageModel.

property api_token: str

ask_question(question: QuestionBase) → str

Ask a question and return the response.

Parameters:

question – The question to ask.

abstract async async_execute_model_call(system_prompt: str)

Execute the model call and returns a coroutine.

async async_get_response(user_prompt: str, system_prompt: str, cache: Cache, iteration: int = 1, files_list: List[FileStore] | None = None, **kwargs) → dict

Get response, parse, and return as string.

Parameters:

• user_prompt – The user’s prompt.

• system_prompt – The system’s prompt.

• cache – The cache to use.

• iteration – The iteration number.

• files_list – The list of files to use.

cost(raw_response: dict[str, Any]) → float | str

Return the dollar cost of a raw response.

Parameters:

raw_response – The raw response from the model.

classmethod example(test_model: bool = False, canned_response: str = 'Hello world', throw_exception: bool = False) → LanguageModel

Return a default instance of the class.

>>> from edsl.language_models import LanguageModel
>>> m = LanguageModel.example(test_model = True, canned_response = "WOWZA!")
>>> isinstance(m, LanguageModel)
True
>>> from edsl import QuestionFreeText
>>> q = QuestionFreeText(question_text = "What is your name?", question_name = 'example')
>>> q.by(m).run(cache = False, disable_remote_cache = True, disable_remote_inference = True).select('example').first()
'WOWZA!'
>>> m = LanguageModel.example(test_model = True, canned_response = "WOWZA!", throw_exception = True)
>>> r = q.by(m).run(cache = False, disable_remote_cache = True, disable_remote_inference = True, print_exceptions = True)
Exception report saved to ...
Also see: ...

execute_model_call(**kwargs)

classmethod from_dict(data: dict) → Type[LanguageModel]

Convert dictionary to a LanguageModel child instance.

classmethod get_generated_token_string(raw_response: dict[str, Any]) → str

Return the generated token string from the raw response.

>>> m = LanguageModel.example(test_model = True)
>>> raw_response = m.execute_model_call("Hello, model!", "You are a helpful agent.")
>>> m.get_generated_token_string(raw_response)
'Hello world'

get_response(user_prompt: str, system_prompt: str, cache: Cache, iteration: int = 1, files_list: List[FileStore] | None = None, **kwargs) → dict

Get response, parse, and return as string.

Parameters:

• user_prompt – The user’s prompt.

• system_prompt – The system’s prompt.

• cache – The cache to use.

• iteration – The iteration number.

• files_list – The list of files to use.

classmethod get_usage_dict(raw_response: dict[str, Any]) → dict[str, Any]

Return the usage dictionary from the raw response.

has_valid_api_key() → bool

Check if the model has a valid API key.

>>> LanguageModel.example().has_valid_api_key() : 
True

This method is used to check if the model has a valid API key.

hello(verbose=False)

Runs a simple test to check if the model is working.

key_sequence = None

classmethod parse_response(raw_response: dict[str, Any]) → EDSLOutput

Parses the API response and returns the response text.

async remote_async_execute_model_call(user_prompt: str, system_prompt: str)

Execute the model call and returns the result as a coroutine, using Coop.

response_handler = <edsl.language_models.RawResponseHandler.RawResponseHandler object>

property rpm

set_key_lookup(key_lookup: KeyLookup) → None

simple_ask(question: QuestionBase, system_prompt='You are a helpful agent pretending to be a human.', top_logprobs=2)

Ask a question and return the response.

to_dict(add_edsl_version: bool = True) → dict[str, Any]

Convert instance to a dictionary

Parameters:

add_edsl_version – Whether to add the EDSL version to the dictionary.

>>> m = LanguageModel.example()
>>> m.to_dict()
{'model': '...', 'parameters': {'temperature': ..., 'max_tokens': ..., 'top_p': ..., 'frequency_penalty': ..., 'presence_penalty': ..., 'logprobs': False, 'top_logprobs': ...}, 'edsl_version': '...', 'edsl_class_name': 'LanguageModel'}

property tpm

class edsl.language_models.LanguageModel.classproperty(method)

Bases: object

__init__(method)

edsl.language_models.LanguageModel.handle_key_error(func)

Handle KeyError exceptions.

Other methods

class edsl.language_models.registry.Meta

Bases: type

class edsl.language_models.registry.Model(model_name=None, registry=None, service_name=None, *args, **kwargs)

Bases: object

classmethod add_model(service_name, model_name)

classmethod available(search_term: str = None, name_only: bool = False, registry=None, service: str | None = None)

classmethod check_models(verbose=False)

default_model = 'gpt-4o'

classmethod example(randomize: bool = False) → Model

Returns an example Model instance.

Parameters:

randomize – If True, the temperature is set to a random decimal between 0 and 1.

classmethod key_info()

classmethod service_classes(registry=None)

classmethod services(registry=None)

edsl.language_models.registry.get_model_class(model_name, registry=None)

edsl.language_models.registry.random() → x in the interval [0, 1).