EDSL basics
This notebook provides an example of each of the basic components of EDSL. In a series of steps we show how to:
Create a
Question
Parameterize it with a
Scenario
to create different versions of the questionAdd the question to a
Survey
Create an AI
Agent
to answer the question versionsSelect a language
Model
to generate responsesInspect the
Results
of the survey
Learn more about each of the base classes in the docs: Questions, Scenarios, Agents and Models.
[1]:
# ! pip install edsl
[2]:
# Import the tools
from edsl.questions import QuestionLinearScale
from edsl import Scenario, Survey, Agent, Model
# Create a question that takes a parameter
q = QuestionLinearScale(
question_name="example",
question_text="On a scale from 0 to 5, how much do you enjoy {{ activity }}?",
question_options=[0, 1, 2, 3, 4, 5],
)
# Identify parameter values
activities = ["exercising", "reading", "cooking"]
scenarios = [Scenario({"activity": a}) for a in activities]
# Add the base question to a survey
survey = Survey(questions=[q])
# Create agents that will respond to the survey
personas = ["You are an athlete", "You are a student", "You are a chef"]
agents = [Agent(traits={"persona": p}) for p in personas]
# Select large language models
models = [Model("gpt-3.5-turbo"), Model("gpt-4-1106-preview")]
# Administer the survey and store the results
results = survey.by(scenarios).by(agents).by(models).run()
# Inspect components of the results
results.select(
"model.model", "scenario.activity", "agent.persona", "answer.example"
).print()
model.model | scenario.activity | agent.persona | answer.example |
---|---|---|---|
gpt-3.5-turbo | cooking | You are an athlete | 2 |
gpt-3.5-turbo | exercising | You are a student | 3 |
gpt-3.5-turbo | cooking | You are a student | 3 |
gpt-3.5-turbo | exercising | You are an athlete | 5 |
gpt-3.5-turbo | reading | You are a chef | 5 |
gpt-3.5-turbo | reading | You are a student | 4 |
gpt-3.5-turbo | reading | You are an athlete | 1 |
gpt-3.5-turbo | cooking | You are a chef | 5 |
gpt-3.5-turbo | exercising | You are a chef | 1 |
gpt-4-1106-preview | cooking | You are an athlete | 3 |
gpt-4-1106-preview | cooking | You are a student | 4 |
gpt-4-1106-preview | reading | You are a student | 5 |
gpt-4-1106-preview | reading | You are an athlete | 4 |
gpt-4-1106-preview | exercising | You are a student | 4 |
gpt-4-1106-preview | exercising | You are a chef | 3 |
gpt-4-1106-preview | exercising | You are an athlete | 5 |
gpt-4-1106-preview | reading | You are a chef | 4 |
gpt-4-1106-preview | cooking | You are a chef | 5 |