Skip to main content
EDSL is an open-source Python library for simulating surveys, experiments and research tasks with AI agents and language models, and validating results with human respondents. It is fully integrated with Coop </en/latest/indexen/latest/coop.html>, a platform for creating and sharing AI and hybrid AI-human research projects.
1

Create an account

Create an account at Expected Parrot. This allows you to access special features for storing, sharing and collaborating on your EDSL projects. Your account comes with an API key to access all available language models (see information about current models) and free credits for getting started.
2

Install the EDSL package

Open your terminal or command prompt, navigate to the directory where you want to install the EDSL library, and run the following command:
pip install edsl
Prerequisites: You must also have Python version 3.9-3.13 and pip installed on your computer. See the Installation section for more details.
3

Open a Python environment

After installing EDSL, you can start using it in a Python IDE or text editor of your choice, such as Jupyter Notebook or Visual Studio Code. Save a new notebook in the directory where you installed EDSL and open it.You can also start working in EDSL by downloading an example notebook and modifying the code as needed for your research. See the How-to Guides and Notebooks for various use cases listed in the sidebar of the documentation page, and a Starter Tutorial.
Note: If you are using Google Colab, please see special instructions on storing and accessing your Expected Parrot API key.
4

Design your research

EDSL allows you to design AI research projects as surveys of questions administered to language models (optionally) using AI agents to answer the questions. A typical workflow consists of the following steps:
  • Create questions: Construct Questions you want to ask a language model. EDSL provides many common question types that you can choose from based on the form of the response that you want to get back from a model (multiple choice, free text, linear scale, etc.). You can also use built-in methods for importing data to use as scenarios of your questions (CSV, PDF, PNG, MP4, DOC, tables, lists, dicts, etc.). This can be efficient for data labeling and other workflows where you want to repeat questions with different data inputs.
  • Construct a survey: Combine your questions into Surveys to administer them together, and add desired logic, such as skip/stop rules, randomization, piping answers, and more.
  • Design AI agents: Specify relevant personas and instructions for AI agents to answer the questions (this is optional).
  • Select language models: Choose the Language Models you want to use to generate the answers.
  • Run experiments: Administer a survey with AI agents and selected language models. This generates a formatted dataset of results.
  • Analyze results: Use built-in methods to analyze your Results.
  • Validate with humans: Launch your survey with human respondents to compare the AI-generated responses. Iterate on your survey design and rerun.
5

Example code

Below is example code for creating a survey, running it with AI agents and language models, launching it with humans and comparing results. You can also view the code and output in a downloadable notebook at Coop.Start by creating a survey:
from edsl import QuestionMultipleChoice, QuestionNumerical, Survey, Agent, AgentList, Model, ModelList

# Create questions
q1 = QuestionMultipleChoice(
    question_name="preferred_source",
    question_text="What is your preferred source of national news?",
    question_options=["television", "social media", "online news", "newspaper", "radio", "podcast", "other"],
)
q2 = QuestionNumerical(
    question_name="hours_per_week",
    question_text="How many hours per week on average do you consume national news via {{ preferred_source.answer }}?" # piping answer from previous question
)

# Create a survey
survey = Survey(questions = [q1, q2])
Run it with AI agents and language models:
# Create AI agents
a = AgentList(Agent(traits = {"persona":p}) for p in ["college student", "retired professional"])

# Select language models
m = ModelList([
    Model("gpt-4o", service_name = "openai"),
    Model("claude-3-sonnet-20240229", service_name = "anthropic")
])

# Run the survey with the agent and model
results = survey.by(a).by(m).run()
To see all columns of the results:
# results.columns
To inspect components of results:
results.select("model", "persona", "preferred_source", "hours_per_week")
Example output:
model.modelagent.personaanswer.preferred_sourceanswer.hours_per_week
gpt-4ocollege studentonline news5.0
claude-3-sonnet-20240229college studentonline news10.0
gpt-4oretired professionalnewspaper2.5
claude-3-sonnet-20240229retired professionalonline news10.0
Use built-in methods to analyze results. For example:
(
    results
        .filter("{{ model.model }} == 'gpt-4o' and {{ agent.persona }} == 'college student'")
        .sort_by("hours_per_week")
        .select("model", "persona", "preferred_source", "hours_per_week", "hours_per_week_comment")
)
Example output:
results.sql("""
select
    model,
    persona,
    preferred_source,
    preferred_source_comment
from self
where 1=1
    and persona == 'retired professional'
    and model == 'claude-3-sonnet-20240229'
""")
Example output:
modelpersonapreferred_sourcepreferred_source_comment
claude-3-sonnet-20240229retired professionalonline news# As a retired professional, I prefer online news sources as they allow me to easily access a wide variety of reputable national and international news outlets at my convenience. Online news is also frequently updated throughout the day.
To generate a web-based version to share with human respondents:
web_info = survey.humanize()
web_info
Example output:
{'project_name': 'Project',
'uuid': 'cd3dff38-9979-4966-b595-ed9fc6e61362',
'admin_url': 'https://www.expectedparrot.com/home/projects/cd3dff38-9979-4966-b595-ed9fc6e61362',
'respondent_url': 'https://www.expectedparrot.com/respond/cd3dff38-9979-4966-b595-ed9fc6e61362'}
Import the responses:
from edsl import Coop

coop = Coop()

human_results = coop.get_project_human_responses(info["uuid"])
Combine human and AI results:
combined_results = results + human_results

combined_results.select("model", "agent_name", "preferred_source", "hours_per_week")
Example output:
model.modelagent.agent_nameanswer.preferred_sourceanswer.hours_per_week
gpt-4oAgent_0online news5.000000
claude-3-sonnet-20240229Agent_1online news10.000000
gpt-4oAgent_2newspaper2.500000
claude-3-sonnet-20240229Agent_3online news10.000000
test50c21352-0c94-4370-b50f-32b7847895e3newspaper7.000000
testa2765047-02c3-4040-ab90-549d12778d96social media5.000000
6

Share and collaborate

Results are automatically stored at your account. You can also post any other EDSL objects to your account from your workspace. For example, to share a notebook of your code:
from edsl import Notebook

nb = Notebook("my_notebook.ipynb")

nb.push(
    description="My EDSL notebook for AI research",
    alias = "my-notebook",
    visibility = "public"  # or "private" or "unlisted" by defauilt
)
Support and resources For more information about using EDSL, please see the documentation page. You can also join our Discord channel to post questions and get help from the community.
⌘I