Exploring free text responses

This notebook provide sample EDSL code demonstrating ways of modifying instructions for free response questions.

EDSL is an open-source library for simulating surveys, experiments and other research with AI agents and large language models. Before running the code below, please ensure that you have installed the EDSL library and either activated remote inference from your Coop account or stored API keys for the language models that you want to use with EDSL. Please also see our documentation page for tips and tutorials on getting started using EDSL.

Creating free text questions

[1]:

from edsl import QuestionFreeText

q1 = QuestionFreeText(
    question_name="pasttime",
    question_text="What is your favorite pasttime? {{ scenario.instruction }}",
)

q2 = QuestionFreeText(
    question_name="vacation",
    question_text="What is your favorite vacation destination? {{ scenario.instruction }}",
)

Parameterizing the questions with special instructions

[2]:

from edsl import ScenarioList

special_instructions = [
    "Be as specific as possible.",
    "Be concise!",
    "Wax poetic here.",
]

scenarios = ScenarioList.from_list("instruction", special_instructions)

Creating agent personas with specific survey contexts

[3]:

from edsl import AgentList, Agent

personas = [
    "You are middle-aged.",
    "You are a senior citizen.",
    "You are a young adult.",
]

instructions = [
    "You are handwriting answers in a paper survey.",
    "You are typing answers in an online survey.",
    "You are providing answers verbally to a researcher in a live interview.",
]

agents = AgentList(
    Agent(traits={"persona": p}, instruction=i) for p in personas for i in instructions
)

Selecting LLMs

[4]:

from edsl import ModelList, Model

models = ModelList(
    Model(m) for m in ["gpt-4o", "gemini-1.5-flash"]
)

Administering the survey

[5]:

from edsl import Survey

survey = Survey(questions = [q1, q2])

results = survey.by(scenarios).by(agents).by(models).run()

▼ Job Status (2025-03-03 10:10:34)

Job UUID	cf8c6b4b-5077-46e1-8fa2-0a5fb0f36c93
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/cf8c6b4b-5077-46e1-8fa2-0a5fb0f36c93
Exceptions Report URL	None
Results UUID	d8380efb-8455-4944-bf64-6716c261126a
Results URL	https://www.expectedparrot.com/content/d8380efb-8455-4944-bf64-6716c261126a

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/d8380efb-8455-4944-bf64-6716c261126a

[6]:

(
    results
    .filter("instruction == 'Be concise!' and model.model == 'gpt-4o'")
    .sort_by("model", "persona", "agent_instruction")
    .select("model", "persona", "agent_instruction", "scenario.*", "answer.*")
)

[6]:

	model.model	agent.persona	agent.agent_instruction	scenario.scenario_index	scenario.instruction	answer.pasttime	answer.vacation
0	gpt-4o	You are a senior citizen.	You are handwriting answers in a paper survey.	1	Be concise!	Reading historical novels.	I love the peacefulness of the mountains.
1	gpt-4o	You are a senior citizen.	You are providing answers verbally to a researcher in a live interview.	1	Be concise!	I love gardening; it brings me peace and joy.	I love visiting the serene countryside, especially places with beautiful gardens and historic sites.
2	gpt-4o	You are a senior citizen.	You are typing answers in an online survey.	1	Be concise!	Reading historical novels.	I love visiting national parks for the tranquility and natural beauty.
3	gpt-4o	You are a young adult.	You are handwriting answers in a paper survey.	1	Be concise!	Playing video games.	Japan!
4	gpt-4o	You are a young adult.	You are providing answers verbally to a researcher in a live interview.	1	Be concise!	I love playing video games.	I love going to Japan. The mix of modern cities and traditional culture is amazing!
5	gpt-4o	You are a young adult.	You are typing answers in an online survey.	1	Be concise!	Exploring new music.	Tokyo, for its vibrant culture and delicious food.
6	gpt-4o	You are middle-aged.	You are handwriting answers in a paper survey.	1	Be concise!	Reading novels.	Italy.
7	gpt-4o	You are middle-aged.	You are providing answers verbally to a researcher in a live interview.	1	Be concise!	Reading historical fiction.	I love visiting the serene landscapes of the Scottish Highlands.
8	gpt-4o	You are middle-aged.	You are typing answers in an online survey.	1	Be concise!	Reading historical novels.	Italy.

Posting to Coop

[8]:

from edsl import Notebook

nb = Notebook("free_responses.ipynb")

if refresh := False:
    n.push(
        description = "Free responses example",
        alias = "free-responses-example-notebook",
        visibility = "public"
    )

else:
    nb.patch('279df9e3-e9b2-4d17-b747-def84f86ca15', value = nb)