# EDSL Documentation ## Docs - [Agents](https://docs.expectedparrot.com/en/latest/agents.md): *Agent* objects are used to simulate survey responses for target audiences. They can be created with specified traits, such as personas and relevant attributes for a survey, that are used together with language models to generate answers to questions. - [Managing Keys](https://docs.expectedparrot.com/en/latest/api_keys.md): API keys are required to access large language models. To use EDSL with models you can either provide your own keys from service providers (*Anthropic, Azure, Bedrock, Deep Infra, DeepSeek, Google, Groq, Mistral, OpenAI, Perplexity, Together, Xai*) or use an **Expected Parrot key** to access all ava… - [Base](https://docs.expectedparrot.com/en/latest/base.md) - [Survey logic checklist](https://docs.expectedparrot.com/en/latest/checklist.md): This page provides a checklist for reviewing the logic and contents of an EDSL survey and ensuring that it will run as you intend it to. - [Citation](https://docs.expectedparrot.com/en/latest/citation.md): If you use the EDSL package in your research or projects, please use the following citation: - [Colab Notebooks](https://docs.expectedparrot.com/en/latest/colab_notebooks.md): All of the methods for working with notebooks in EDSL are available in Colab. - [Colab Setup](https://docs.expectedparrot.com/en/latest/colab_setup.md): All examples, tutorials, and demo notebooks are executable in any Python notebook environment. - [Contributing](https://docs.expectedparrot.com/en/latest/contributing.md) - [Expected Parrot](https://docs.expectedparrot.com/en/latest/coop.md): Expected Parrot provides a free platform for storing, sharing and collaborating on AI-based research, fully integrated with EDSL via the Coop module. - [Estimating & Tracking Costs](https://docs.expectedparrot.com/en/latest/costs.md): EDSL comes with built-in methods for estimating costs before running your survey jobs, and tracking actual costs after running them. This is useful for budgeting and understanding the costs of running surveys with different models and inference services, and determining which models are most efficie… - [Credits](https://docs.expectedparrot.com/en/latest/credits.md): Credits are required in order to run surveys with models using your Expected Parrot API key. They can be purchased at your account (see details below). Credits are deducted to cover API call costs, which depend on token rates set by language model service providers. You can view current pricing for… - [Custom CSS](https://docs.expectedparrot.com/en/latest/custom_css.md): CSS classes and HTML structure for styling human-facing surveys with custom CSS. - [Caching LLM Calls](https://docs.expectedparrot.com/en/latest/data.md): The *Cache* class is used to store responses from a language model so that they can easily be retrieved, reused and shared. - [Dataset](https://docs.expectedparrot.com/en/latest/dataset.md): The `Dataset` class is a versatile data container for tabular data with powerful manipulation capabilities. It represents data in a column-oriented format, providing methods for analysis, transformation, and visualization.The `Dataset` class is a versatile data container for tabular data with powerf… - [Enums](https://docs.expectedparrot.com/en/latest/enums.md): Enums module wrapper for backward compatibility. - [Exceptions & Debugging](https://docs.expectedparrot.com/en/latest/exceptions.md): An exception is an error that occurs during the execution of a question or survey. When an exception is raised, EDSL will display a message about the error and an interactive report with more details in a new browser tab. - [File Store](https://docs.expectedparrot.com/en/latest/filestore.md): *FileStore* is a module for storing and sharing data files at Expected Parrot. It allows you to post and retrieve files of various types to use in EDSL surveys, such as survey data, PDFs, CSVs, docs or images. It can also be used to create Scenario objects for questions or traits for Agent objects f… - [Firecrawl Integration](https://docs.expectedparrot.com/en/latest/firecrawl.md): The EDSL Firecrawl integration provides seamless access to the Firecrawl web scraping platform, allowing you to scrape, crawl, search, and extract structured data from web content directly into EDSL Scenarios and ScenarioLists. - [Getting Started](https://docs.expectedparrot.com/en/latest/getting_started.md): To use EDSL, you need to install the package and choose how to access language models. - [Humanize](https://docs.expectedparrot.com/en/latest/humanize.md): EDSL comes with built-in methods for generating web versions of your surveys and collecting and analyzing responses from humans. - [Humanize schema](https://docs.expectedparrot.com/en/latest/humanize_schema.md): Reference for the humanize schema: a dict that configures how survey questions are presented in web surveys with human respondents. - [Expected Parrot: Tools for AI-Powered Research](https://docs.expectedparrot.com/en/latest/index.md): Expected Parrot delivers powerful tools for conducting research with human and artificial intelligences. - [Installation](https://docs.expectedparrot.com/en/latest/installation.md): EDSL is an open-source Python library. It is compatible with Python 3.9 - 3.13 and available at [PyPI](https://pypi.org/project/edsl/) and [GitHub](https://github.com/expectedparrot/edsl). - [Jobs](https://docs.expectedparrot.com/en/latest/jobs.md): A `Job` is a `Survey` object combined with one or more `Model` objects and any `Agent` and `Scenario` objects that have been added to the survey. It is used to manage the execution of a survey with the specified models, agents and scenarios. - [Language Models](https://docs.expectedparrot.com/en/latest/language_models.md): EDSL works with many models from popular inference service providers, including Anthropic, Azure, Bedrock, Deep Infra, DeepSeek, Google, Mistral, OpenAI, Perplexity, Together and Xai. Current pricing and performance information is on the [models page](https://www.expectedparrot.com/getting-started/m… - [Log Management](https://docs.expectedparrot.com/en/latest/logging.md): EDSL provides comprehensive log management capabilities through the **LogManager** class. This powerful tool allows you to filter, analyze, and manage EDSL log files with advanced features for debugging, monitoring, and data analysis. - [Notebooks](https://docs.expectedparrot.com/en/latest/notebooks.md): The Notebook object allows you to share your notebooks and scripts (*.ipynb* and *.py* files) by uploading them to Expected Parrot. You can also view and pull notebooks that other users have shared publicly or privately with you. - [Adding metadata to survey results](https://docs.expectedparrot.com/en/latest/notebooks/adding_metadata.md): This notebook provides sample [EDSL](/en/latest/index) code for adding metadata to survey [results](/en/latest/results). This can be useful when you are using EDSL to conduct [data labeling](/en/latest/notebooks/data_labeling_example) or similar tasks and want to include information about the data o… - [Agent fatigue](https://docs.expectedparrot.com/en/latest/notebooks/agent_fatigue.md): This notebook presents some examples of ways to explore simulating agent fatigue and memory capacity using EDSL. - [Analyzing a transcript](https://docs.expectedparrot.com/en/latest/notebooks/analyze_customer_call.md): This notebook provides example EDSL code for analyzing a transcript using AI. In the steps below we construct a survey of questions about a transcript of a fictional call between a business development representative and a potential customer, and then use a language model to answer the questions in… - [Analyzing course evaluations](https://docs.expectedparrot.com/en/latest/notebooks/analyze_evaluations.md): This notebook provides sample [EDSL](https://pypi.org/project/edsl/) code for using a language model to analyze course evaluations. The analysis is designed as a survey of questions about a set of evaluations that we prompt an AI agent to answer, using a language model to generate the responses as a… - [Answering instructions](https://docs.expectedparrot.com/en/latest/notebooks/answering_instructions_example.md): This notebook demonstrates how you can modify default instructions for question types and answer comments. - [Batching results](https://docs.expectedparrot.com/en/latest/notebooks/batching_results.md): This notebook provides sample [EDSL](/en/latest/index) code for combining survey [results](/en/latest/results) into a single `Results` object. This can be useful when you are running a survey with batches of [scenarios](/en/latest/scenarios), such as when completing a large-scale [data labeling](/en… - [Working with CES data in EDSL](https://docs.expectedparrot.com/en/latest/notebooks/ces_data_edsl.md): This notebook provides code for reconstructing the [CES 2022 Pre-election Questionnaire](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi%3A10.7910/DVN/PR4L8P) in EDSL. - [Concept induction](https://docs.expectedparrot.com/en/latest/notebooks/concept_induction.md): This notebook provides example [EDSL](/en/latest/index) code for using language models to perform “concept induction”: identify concepts in unstructured texts; generate criteria for the concepts; and then apply the criteria to evaluate the texts. This idea is inspired by the recent paper: [Concept I… - [Simulate a qualitative interview](https://docs.expectedparrot.com/en/latest/notebooks/conduct_interview.md): This notebook provides sample [EDSL](/en/latest/index) code for simulating an interview between a researcher and a subject, with instructions for modifying the interviewer, interview subject or topic. - [Data cleaning](https://docs.expectedparrot.com/en/latest/notebooks/data_cleaning.md): This notebook provides sample [EDSL](/en/latest/index) code for using a language model to conduct a data cleaning task. In a series of steps we use EDSL to prompt a language model to generate appropriate sense checks for a dataset and then run the sense checks in the form of a survey about the data,… - [Data labeling agents](https://docs.expectedparrot.com/en/latest/notebooks/data_labeling_agent.md): This notebook shows how to conduct data labeling tasks using [EDSL](/en/latest/index), an open-source library for simulating surveys, experiments and other research with AI agents and large language models. - [Data labeling](https://docs.expectedparrot.com/en/latest/notebooks/data_labeling_example.md): This notebook provides example code for conducting data labeling and content analysis in EDSL, an open-source library for simulating surveys, experiments and research tasks with AI agents and large language models. - [Data labeling with LLMs, validating with humans](https://docs.expectedparrot.com/en/latest/notebooks/data_labeling_validation_example.md): This notebook provides example [EDSL](https://github.com/expectedparrot/edsl) code for conducting a data labeling task with large language models and validating responses with humans. - [Creating a digital twin](https://docs.expectedparrot.com/en/latest/notebooks/digital_twin.md): This notebook contains sample [EDSL](/en/latest/index) code for creating an AI agent and prompting it to critique some content. The code is readily editable to create other agents and survey questions with any available language models. - [Using EDSL for AI polling](https://docs.expectedparrot.com/en/latest/notebooks/edsl_polling.md): This notebook demonstrates methods for conducting surveys with AI agents using [EDSL](https://github.com/expectedparrot/edsl), an open-source Python library for simulating social science research with large language models. EDSL is developed by [Expected Parrot](https://www.expectedparrot.com/), a s… - [Azure example](https://docs.expectedparrot.com/en/latest/notebooks/edsl_with_cloud_providers.md) - [EDSL with Offline Inference Services](https://docs.expectedparrot.com/en/latest/notebooks/edsl_with_offline_inference_services.md): Learn how to use EDSL with offline inference services like Ollama. - [Estimating costs](https://docs.expectedparrot.com/en/latest/notebooks/estimating_costs.md) - [Evaluating job posts](https://docs.expectedparrot.com/en/latest/notebooks/evaluating_job_posts.md): This notebook provides sample code for conducting a text analysis using [EDSL](/en/latest/index), an open-source library for simulating surveys, experiments and other research with AI agents and large language models. - [Agent dynamic traits](https://docs.expectedparrot.com/en/latest/notebooks/example_agent_dynamic_traits.md): This notebook provides a demonstration of the optional `Agent` parameter `dynamic_traits_function` that can be used to generate agent traits dynamically based on the question being asked or the scenario in which the question is asked. - [Cognitive testing & LLM biases](https://docs.expectedparrot.com/en/latest/notebooks/explore_llm_biases.md): This notebook provides example code for using [EDSL](/en/latest/index) to investigate biases of large language models. - [Exploring real world survey contexts](https://docs.expectedparrot.com/en/latest/notebooks/explore_survey_contexts.md): This notebook explores ways of specifying “contexts” in which an [EDSL](/en/latest/index) survey is administered to AI agents in order to investigate potential impacts to simulated responses. We show how to do this in three different ways by modifying our `Question` texts and `Agent` traits in order… - [FileStore examples](https://docs.expectedparrot.com/en/latest/notebooks/filestore_examples_new.md) - [Exploring free text responses](https://docs.expectedparrot.com/en/latest/notebooks/free_responses.md): This notebook provide sample [EDSL](/en/latest/index) code demonstrating ways of modifying instructions for free response questions. - [Google Form -> EDSL](https://docs.expectedparrot.com/en/latest/notebooks/google_form_to_edsl.md): This notebook provides example EDSL code for converting a non-EDSL survey into in an EDSL survey. This can be useful for accessing EDSL’s built-in methods for analyzing survey data, and extending it with responses simulated with AI agents and diverse large language models. - [Using images in a survey](https://docs.expectedparrot.com/en/latest/notebooks/image_scenario_example.md) - [Importing agents](https://docs.expectedparrot.com/en/latest/notebooks/import_agents.md): This notebook provides sample EDSL code for simulating surveys with agent personas imported from [Persona Hub](https://github.com/tencent-ailab/persona-hub), an open-source repository provided by [Hugging Face](https://huggingface.co/datasets/proj-persona/PersonaHub). - [Looping & piping questions](https://docs.expectedparrot.com/en/latest/notebooks/looping_and_piping.md): This notebook provides example EDSL code for automatically looping (repeating) a question with content piped from other questions and answers. - [Comparing model performance](https://docs.expectedparrot.com/en/latest/notebooks/models_scoring_models.md): In this notebook we show how to use EDSL to prompt a set of models to answer the same survey at once and compare their responses. - [NPS survey in EDSL](https://docs.expectedparrot.com/en/latest/notebooks/nps_survey.md): This notebook provides sample [EDSL](https://github.com/expectedparrot/edsl) code for simulating a Net Promoter Score (NPS) survey with AI agents and large language models. In the steps below we show how to construct an EDSL survey, create personas for AI agents to answer the questions, and then adm… - [Piping question answers & comments](https://docs.expectedparrot.com/en/latest/notebooks/piping_comments.md): This notebook demonstrates how to pipe components of a question and answer into follow-on questions. When piping is used, survey questions are automatically administered in the order required to facilitate the piping (i.e., a question with components piped from another question will be administered… - [Launch an interview](https://docs.expectedparrot.com/en/latest/notebooks/qualitative_research.md): This notebook provides sample [EDSL](/en/latest/index) code for simulating qualitative research interviews. In a series of steps we construct an AI agent representing a qualitative researcher and launch an interview with specified topics and interview subjects. - [Creating question variants](https://docs.expectedparrot.com/en/latest/notebooks/question_loop_scenarios.md): EDSL comes with a variety of features for efficiently generating different versions of questions in surveys. This notebook demonstrates methods for doing this with `Scenario` objects. - [Simulating randomness](https://docs.expectedparrot.com/en/latest/notebooks/random_numbers.md): This notebook provides example EDSL code for exploring ways of prompting AI agents to return “random” numbers. We do this by passing agents different instructions for picking a number at random, and by including and excluding memories of prior selections. - [Using EDSL with a reasoning model](https://docs.expectedparrot.com/en/latest/notebooks/reasoning_model_example.md) - [Cognitive testing & creating new methods](https://docs.expectedparrot.com/en/latest/notebooks/research_methods.md): This notebook shows some ways of using EDSL to conduct research, including data labeling, cognitive testing and creating new methods. - [Run a job in the background](https://docs.expectedparrot.com/en/latest/notebooks/run_background.md): You can run a remote survey in the background by passing a parameter `background=True` to the `run()` method. This allows you to continue working (or stop working) while your survey job completes. - [Save & load objects](https://docs.expectedparrot.com/en/latest/notebooks/save_load_objects_locally.md): This notebook demonstrates methods for [saving](https://github.com/expectedparrot/edsl/blob/69429fe1dba63867eab57973edb39930b2a7d510/edsl/base/Base.py#L104) and [loading](https://github.com/expectedparrot/edsl/blob/69429fe1dba63867eab57973edb39930b2a7d510/edsl/base/Base.py#L145) EDSL objects locally… - [Using PDFs in a survey](https://docs.expectedparrot.com/en/latest/notebooks/scenario_from_pdf.md): This notebook provides sample [EDSL](/en/latest/index) code for using PDFs with surveys. - [Using Wikipedia tables in a survey](https://docs.expectedparrot.com/en/latest/notebooks/scenario_list_wikipedia.md) - [Using EDSL to sense check data](https://docs.expectedparrot.com/en/latest/notebooks/scenariolist_unpivot.md): This notebook provides example code for sense checking survey data using [EDSL](/en/latest/index), an open-source library for simulating surveys, experiments and market research with AI agents and large language models. - [Using data with surveys: FileStore](https://docs.expectedparrot.com/en/latest/notebooks/scenarios_filestore_example.md): This notebook provides example [EDSL](https://github.com/expectedparrot/edsl) code for methods for using data with an EDSL survey. In the steps below we show how to use the [FileStore](/en/latest/filestore) module to upload, share and retrieve data files at the [Expected Parrot](/en/latest/coop), an… - [Skip logic & scenarios](https://docs.expectedparrot.com/en/latest/notebooks/skip_logic_scenarios.md): This notebook provides example EDSL code for using a language model to simulate a survey that uses skip logic: rules for determining which questions are administered based on responses to other questions in the survey. - [Starter Tutorial](https://docs.expectedparrot.com/en/latest/notebooks/starter_tutorial.md): This tutorial provides step-by-step instructions for getting started using EDSL (*Expected Parrot Domain-Specific Language*), an open-source Python library for simulating surveys, experiments and other research tasks using AI agents and large language models. EDSL is developed by [Expected Parrot](h… - [Analyzing customer service calls](https://docs.expectedparrot.com/en/latest/notebooks/summarizing_transcripts.md): This notebook provides sample [EDSL](/en/latest/index) code for conducting an analysis of textual content with AI agents and large language models. Using a set of transcripts of mock customer calls as an example, we use EDSL to prompt an AI agent to identify the callers’ names, emails, questions and… - [Testing model training data](https://docs.expectedparrot.com/en/latest/notebooks/testing_training_data.md): This notebook provides a template for prompting an AI agent to answer questions from a point in time and testing that knowledge for data leaks. - [Updating agent traits](https://docs.expectedparrot.com/en/latest/notebooks/updating_agents.md): This notebook demonstrates how to update an agent’s traits with new answers to questions using the `add traits()` and `to_agent_list()` methods. - [Using videos in a survey](https://docs.expectedparrot.com/en/latest/notebooks/video_scenario_example.md) - [Stylized content](https://docs.expectedparrot.com/en/latest/notebooks/writing_style.md): This notebook provides a quick example of how to use [EDSL](/en/latest/index) to prompt an AI agent to immitate a writing style in drafting some content. - [Simulate a feedback survey](https://docs.expectedparrot.com/en/latest/notebooks/yoga_studio_name_survey.md): This notebook provides example code for simulating a feedback survey with AI agents in [EDSL](https://github.com/expectedparrot/edsl), an open-source library for simulating surveys, experiments and research tasks with AI agents and large language models. We also show how EDSL can be used to generate… - [Overview](https://docs.expectedparrot.com/en/latest/overview.md): *Expected Parrot Domain-Specific Language* (EDSL) is an open-source package for conducting AI-powered research. EDSL simplifies the creation and execution of surveys, experiments, data labeling tasks, and other research activities involving large numbers of AI agents and language models. - [Papers & Citations](https://docs.expectedparrot.com/en/latest/papers.md): This page provides a list of papers that use or cite EDSL. *Please let us know of any omissions!* - [Prolific studies](https://docs.expectedparrot.com/en/latest/prolific.md): EDSL allows you to launch surveys with human participants using Prolific, a platform for recruiting participants for research studies. You can choose whether to create and manage Prolific studies directly in your EDSL code or interactivately at your Expected Parrot account, enabling you to gather hu… - [Prompts](https://docs.expectedparrot.com/en/latest/prompts.md) - [Questions](https://docs.expectedparrot.com/en/latest/questions.md): EDSL provides templates for many common question types, including free text, multiple choice, checkbox, linear scale, matrix, numerical, dictionary and others. The Question class has subclasses for each of these types: QuestionFreeText, QuestionMultipleChoice, QuestionCheckBox, QuestionLinearScale,… - [Remote Caching](https://docs.expectedparrot.com/en/latest/remote_caching.md): Remote caching automatically stores responses from language models at the Expected Parrot server, and retrieves stored responses to questions that have already been run. This makes it free and instant to reproduce results from any survey that has been run remotely. - [Remote Inference](https://docs.expectedparrot.com/en/latest/remote_inference.md): Remote inference allows you to run surveys at the Expected Parrot server instead of locally on your own machine, and to use [Remote Caching](/en/latest/remote_caching) to store survey results and logs at your account. - [Results](https://docs.expectedparrot.com/en/latest/results.md): A *Results* object represents the outcome of running a *Survey*. It contains a list of individual *Result* objects, where each *Result* corresponds to a response to the survey for a unique combination of *Agent*, Model, and Scenario objects used with the survey. - [Scenarios](https://docs.expectedparrot.com/en/latest/scenarios.md): A Scenario is a dictionary containing one or more key/value pairs that is used to add data or content to questions in a survey, replacing a parameter in a question with a specific value (e.g., numerical or textual) or content (e.g., an image or PDF). A ScenarioList is a list of Scenario objects. - [Starter Tutorial](https://docs.expectedparrot.com/en/latest/starter_tutorial.md): This tutorial provides step-by-step instructions for getting started using EDSL (*Expected Parrot Domain-Specific Language*), an open-source Python library for simulating surveys, experiments and other research tasks using AI agents and large language models. EDSL is developed by [Expected Parrot](h… - [Survey Builder](https://docs.expectedparrot.com/en/latest/survey_builder.md) - [Surveys](https://docs.expectedparrot.com/en/latest/surveys.md): A Survey is a collection of [Questions](/en/latest/questions) administered to one or more AI [Agents](/en/latest/agents) and [Language Models](/en/latest/language_models). Questions run asynchronously by default, or according to rules like skip/stop logic, with optional memory of prior answers. - [Teaching guide](https://docs.expectedparrot.com/en/latest/teaching.md): This page provides a short guide for getting started using EDSL (_Expected Parrot Domain-Specific Language_) to conduct and collaborate on AI research. - [Token Usage](https://docs.expectedparrot.com/en/latest/token_usage.md): EDSL comes with a variety of features for monitoring token usage. - [Whitepaper](https://docs.expectedparrot.com/en/latest/whitepaper.md): In progress! ## OpenAPI Specs - [openapi](https://docs.expectedparrot.com/api-reference/openapi.json) ## Optional - [GitHub](https://github.com/expectedparrot/edsl) - [Discord](https://discord.gg/mxAYkjfy9m) - [Blog](https://blog.expectedparrot.com)