Exploring free text responses
This notebook provide sample EDSL code demonstrating ways of modifying instructions for free response questions.
EDSL is an open-source library for simulating surveys, experiments and other research with AI agents and large language models. Before running the code below, please ensure that you have installed the EDSL library and either activated remote inference from your Coop account or stored API keys for the language models that you want to use with EDSL. Please also see our documentation page for tips and tutorials on getting started using EDSL.
Creating free text questions
[1]:
from edsl import QuestionFreeText
q1 = QuestionFreeText(
question_name="pasttime",
question_text="What is your favorite pasttime? {{ instruction }}",
)
q2 = QuestionFreeText(
question_name="vacation",
question_text="What is your favorite vacation destination? {{ instruction }}",
)
Parameterizing the questions with special instructions
[2]:
from edsl import ScenarioList, Scenario
special_instructions = [
"Be as specific as possible.",
"Be concise!",
"Wax poetic here.",
]
scenarios = ScenarioList.from_list("instruction", special_instructions)
Creating agent personas with specific survey contexts
[3]:
from edsl import AgentList, Agent
personas = [
"You are middle-aged.",
"You are a senior citizen.",
"You are a young adult.",
]
instructions = [
"You are handwriting answers in a paper survey.",
"You are typing answers in an online survey.",
"You are providing answers verbally to a researcher in a live interview.",
]
agents = AgentList(
Agent(traits={"persona": p}, instruction=i) for p in personas for i in instructions
)
Selecting LLMs
[4]:
from edsl import ModelList, Model
models = ModelList(
Model(m) for m in ["gpt-4o", "gemini-pro"]
)
Administering the survey
[5]:
from edsl import Survey
survey = Survey(questions = [q1, q2])
results = survey.by(scenarios).by(agents).by(models).run()
[6]:
(
results
.filter("instruction == 'Be concise!' and model.model == 'gpt-4o'")
.sort_by("model", "persona", "agent_instruction")
.select("model", "persona", "agent_instruction", "scenario.*", "answer.*")
.print(format="rich")
)
┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓ ┃ model ┃ agent ┃ agent ┃ scenario ┃ answer ┃ answer ┃ ┃ .model ┃ .persona ┃ .agent_instruction ┃ .instruction ┃ .pasttime ┃ .vacation ┃ ┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩ │ gpt-4o │ You are a senior │ You are handwriting │ Be concise! │ Reading historical │ I love visiting the │ │ │ citizen. │ answers in a paper │ │ novels. │ peaceful │ │ │ │ survey. │ │ │ countryside. │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are a senior │ You are providing │ Be concise! │ I enjoy gardening; │ I love the │ │ │ citizen. │ answers verbally to │ │ it keeps me active │ peacefulness of the │ │ │ │ a researcher in a │ │ and connected to │ countryside, │ │ │ │ live interview. │ │ nature. │ especially the │ │ │ │ │ │ │ rolling hills and │ │ │ │ │ │ │ quaint villages of │ │ │ │ │ │ │ the English │ │ │ │ │ │ │ countryside. │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are a senior │ You are typing │ Be concise! │ Gardening. │ The Grand Canyon. │ │ │ citizen. │ answers in an online │ │ │ │ │ │ │ survey. │ │ │ │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are a young │ You are handwriting │ Be concise! │ Playing video │ Japan. │ │ │ adult. │ answers in a paper │ │ games. │ │ │ │ │ survey. │ │ │ │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are a young │ You are providing │ Be concise! │ I love playing │ I love visiting │ │ │ adult. │ answers verbally to │ │ video games. It's a │ Japan for its mix of │ │ │ │ a researcher in a │ │ fun way to relax │ traditional culture │ │ │ │ live interview. │ │ and connect with │ and modern tech │ │ │ │ │ │ friends. │ vibes. │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are a young │ You are typing │ Be concise! │ Playing video │ Japan. │ │ │ adult. │ answers in an online │ │ games. │ │ │ │ │ survey. │ │ │ │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are │ You are handwriting │ Be concise! │ Reading historical │ The mountains. │ │ │ middle-aged. │ answers in a paper │ │ novels. │ │ │ │ │ survey. │ │ │ │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are │ You are providing │ Be concise! │ I enjoy reading │ I love visiting the │ │ │ middle-aged. │ answers verbally to │ │ historical novels. │ serene beaches of │ │ │ │ a researcher in a │ │ │ the Greek Islands. │ │ │ │ live interview. │ │ │ │ ├────────┼─────────────────────┼──────────────────────┼──────────────┼─────────────────────┼──────────────────────┤ │ gpt-4o │ You are │ You are typing │ Be concise! │ Reading historical │ The Mediterranean. │ │ │ middle-aged. │ answers in an online │ │ fiction. │ │ │ │ │ survey. │ │ │ │ └────────┴─────────────────────┴──────────────────────┴──────────────┴─────────────────────┴──────────────────────┘