EDSL Overview

EDSL lets you design and run studies in Python, specifying what to ask, what information to provide, who or what should respond, and how to organize the results. You can vary individual elements of a study, such as question wording, respondent profiles, or language models, and examine how those changes affect the findings.

​

The main components

For example, a message-testing study might use a survey to ask about clarity and persuasiveness, scenarios to supply different messages, agents to describe audience profiles, and several models to generate responses. (Note: In EDSL, an Agent represents a simulated respondent, not an AI assistant that performs tasks. Coding agents and Expected Parrot’s Research Agent can use EDSL to design and run studies.)

​

Running studies with AI

Combine a survey with the agents, scenarios, and models you want to use. EDSL handles model calls, parallel execution, survey logic, and response collection. EDSL calls one pass through a survey an interview. In a model run, each interview uses one agent, one scenario, and one model. Its result contains the answers to all questions reached on that pass. A job can contain many interviews, such as one for every message and audience combination in a message test. This makes it possible to repeat a study across experimental conditions without writing separate loops or provider integrations. For data labeling, ask survey questions about each input and use the answers as its labels. You can translate an existing codebook into structured questions, or first use exploratory questions to develop the categories those questions will offer. Scenarios supply the items being labeled. You can run model calls from your own environment or use Expected Parrot’s remote execution. See Managing Keys, Remote Inference, and Costs for configuration and usage details.

​

Collecting human responses

You can also publish an EDSL survey for human participants. Calling survey.humanize() creates a hosted web survey and returns a respondent link. After participants complete it, you can retrieve their responses for analysis in Python. Use this workflow to evaluate simulated findings against human evidence or to conduct a human survey directly. When comparing human and AI responses, retain their source so that the analysis clearly distinguishes observed responses from simulations. The Humanize guide covers publishing surveys and retrieving data. See Prolific studies for participant recruitment.

​

Working with data

EDSL provides methods for selecting columns, filtering responses, sorting observations, and comparing groups. You can export data to CSV or a pandas DataFrame for further analysis. Responses remain associated with study information such as agent traits, scenarios, and model settings. This lets you compare conditions and trace observations back to the inputs used to generate them.

​

Inspecting and reproducing a study

A study defined in code makes its questions, logic, inputs, and model choices available for inspection. EDSL’s open-source implementation also lets you examine and extend the underlying functionality. EDSL jobs can be saved, shared, and rerun. Caching lets a rerun reuse the same model responses when the inputs match. See Saving objects and Remote caching. For a complete example, follow the Run your first survey tutorial.