Using EDSL for AI polling
This notebook demonstrates methods for conducting surveys with AI agents using EDSL, an open-source Python library for simulating social science research with large language models. EDSL is developed by Expected Parrot, a start-up building tools for AI-powered research.
Contents
The sections below include:
Creating an AI agent: Basic steps to construct an AI agent.
Administering questions: How to create questions and prompt agents to answer them.
Selecting language models: Specify language models that you want to use to generate responses.
Analyzing results: Examples of built-in methods for analyzing responses as datasets.
Designing agent traits: How to construct agents with complex personas and traits.
Converting surveys into EDSL: Import other surveys into EDSL to analyze and extend them with agents.
Constructing agents from survey data: Use survey responses to construct agents representing respondents.
Sample data: Cooperative Election Study Common Content, 2022
For purposes of demonstration, we use data from the Cooperative Election Study Common Content, 2022 in several ways:
In this notebook we use lists of respondent attributes from the Breakdown of National Vote for U.S. House (CES validated voters) (CES Guide 2022 pp.24-25) to design agents with combinations of the attributes, and then administer questions to them.
Companion notebooks:
In Working with CES data in EDSL we reconstruct the 2022 CES Pre-Election Questionnaire in EDSL. The formatted questions can be modified and administered to agents in new surveys as desired. The notebook also contains dictionaries of the CES questions and answer codebook for convenience in working with the data.
In Creating agents for survey respondents we use the dataset of responses to the 2022 CES Pre-Election Questionnaire to create agents representing the respondents, and administer follow-on questions with them.
Reference & contact
Documentation for the EDSL package is available at https://docs/expectedparrot.com. You can also find example code, tutorials and notebooks for a variety of use cases.
Please let us know if you have any questions or encounter issues working with this data:
Email: info@expectedparrot.com
Technical setup
EDSL is compatible with Python 3.9-3.12.
See instructions on installing the EDSL library and storing API keys for the language models that you want to use. In examples below where no model is specified, EDSL will use GPT 4 by default (an API key for OpenAI is required). We also show how to use different models.
Creating an AI agent
In this section we show how to create an AI agent and give it desired attributes. For more details on constructing and using AI agents please see our documentation page on agents.
We start by importing the tools for creating agents:
[1]:
from edsl import Agent
Here we create a simple Agent and pass it a dictionary of traits. We optionally include a narrative persona and also specify traits individually for use in segmenting and analyzing survey responses:
[2]:
agent = Agent(
traits={
"persona": "You are 55-year-old research scientist living in Cambridge, Massachusetts.",
"occupation": "Research scientist",
"location": "Cambridge, Massachusetts",
"age": 55,
}
)
We can access the traits directly:
[3]:
agent.location
[3]:
'Cambridge, Massachusetts'
Designing agent panels
We can also create panels of agents in an AgentList and administer surveys to all of the agents at once. Here we construct combinations of traits from lists of respondent attributes in the CES Guide (see source details above). (Information can be imported from a variety of data source types; see documentation for details.)
[4]:
sex = ["Male", "Female"]
race = ["White", "Black", "Hispanic", "Asian", "Other"]
age = ["18-29", "30-44", "45-64", "65 and over"]
education = [
"High school or less",
"Some college/assoc. degree",
"College/graduate",
"Postgraduate study",
]
income = [
"Under $30,000",
"$30,000 to $49,999",
"$50,000 to $99,999",
"$100,000 to $199,999",
"$200,000 or more",
]
party_affiliation = ["Democrat", "Republican", "Independent/Other"]
political_ideology = ["Liberal", "Moderate", "Conservative", "Unsure"]
religion = [
"Protestant/other Christian",
"Catholic",
"Jewish",
"Something else",
"None",
]
evangelical = ["Yes", "No"]
married = ["Yes", "No"]
lgbt = ["Yes", "No"]
Here we create a method to generate a list of agents with randomly selected combinations of traits:
[5]:
from edsl import AgentList
import random
def generate_random_agents(num_agents):
agents = []
for _ in range(num_agents):
agent_traits = {
"sex": random.choice(sex),
"race": random.choice(race),
"age": random.choice(age),
"education": random.choice(education),
"income": random.choice(income),
"party_affiliation": random.choice(party_affiliation),
"political_ideology": random.choice(political_ideology),
"religion": random.choice(religion),
"evangelical": random.choice(evangelical),
"married": random.choice(married),
"lgbt": random.choice(lgbt),
}
agents.append(Agent(traits=agent_traits))
return AgentList(agents)
Example usage:
[6]:
num_agents = 3
agents = generate_random_agents(num_agents)
Agent instructions
If we want to give all the agents a special instruction, we can optionally pass an instruction to the agents (this can also be done when the agents are created):
[7]:
for agent in agents:
agent.instruction = "Today is July 1, 2022."
We can inspect the agents that have been created:
[8]:
agents
[8]:
[
{
"traits": {
"sex": "Female",
"race": "Other",
"age": "45-64",
"education": "College/graduate",
"income": "Under $30,000",
"party_affiliation": "Independent/Other",
"political_ideology": "Liberal",
"religion": "Protestant/other Christian",
"evangelical": "No",
"married": "No",
"lgbt": "Yes"
},
"instruction": "Today is July 1, 2022.",
"edsl_version": "0.1.29.dev6",
"edsl_class_name": "Agent"
},
{
"traits": {
"sex": "Female",
"race": "Asian",
"age": "65 and over",
"education": "College/graduate",
"income": "$100,000 to $199,999",
"party_affiliation": "Independent/Other",
"political_ideology": "Conservative",
"religion": "Catholic",
"evangelical": "Yes",
"married": "Yes",
"lgbt": "No"
},
"instruction": "Today is July 1, 2022.",
"edsl_version": "0.1.29.dev6",
"edsl_class_name": "Agent"
},
{
"traits": {
"sex": "Female",
"race": "Black",
"age": "18-29",
"education": "High school or less",
"income": "$50,000 to $99,999",
"party_affiliation": "Democrat",
"political_ideology": "Moderate",
"religion": "Protestant/other Christian",
"evangelical": "No",
"married": "No",
"lgbt": "Yes"
},
"instruction": "Today is July 1, 2022.",
"edsl_version": "0.1.29.dev6",
"edsl_class_name": "Agent"
}
]
Creating questions
An Agent is designed to be assigned questions to answer. In this section we construct questions in the form of Question objects, combine them into a Survey, administer it to some sample agents (from above), and inspect the responses in the dataset of Results that is generated.
EDSL comes with many question types that we can select from based on the form of the response that we want to get back from the language model (free text, linear scale, checkbox, etc.). See examples of all question types.
Here we create a multiple choice question from the CES Pre-Election Questionnaire (the response will be a selection from the list of options that we include) and compose a follow-up free text question (the response will be unstructured text):
[9]:
from edsl import QuestionMultipleChoice, QuestionFreeText
# From the CES pre-election questionnaire
q_pid3 = QuestionMultipleChoice(
question_name="pid3",
question_text="Generally speaking, do you think of yourself as a ...?",
question_options=["Democrat", "Republican", "Independent", "Other", "Not sure"],
)
# Potential follow-up question
q_views = QuestionFreeText(
question_name="views", question_text="Describe your political views."
)
We combine the questions into a Survey to administer them together:
[10]:
from edsl import Survey
survey = Survey([q_pid3, q_views])
Administering a survey
We administer a survey by calling the run method, after (optionally) adding agents with the by method:
[11]:
results = survey.by(agents).run()
We can show a list of all the components of the Results that have been generated, and see that the results include information about the agents, questions, models, prompts and responses:
[12]:
results.columns
[12]:
['agent.age',
'agent.agent_instruction',
'agent.agent_name',
'agent.education',
'agent.evangelical',
'agent.income',
'agent.lgbt',
'agent.married',
'agent.party_affiliation',
'agent.political_ideology',
'agent.race',
'agent.religion',
'agent.sex',
'answer.pid3',
'answer.views',
'comment.pid3_comment',
'iteration.iteration',
'model.frequency_penalty',
'model.logprobs',
'model.max_tokens',
'model.model',
'model.presence_penalty',
'model.temperature',
'model.top_logprobs',
'model.top_p',
'prompt.pid3_system_prompt',
'prompt.pid3_user_prompt',
'prompt.views_system_prompt',
'prompt.views_user_prompt',
'question_options.pid3_question_options',
'question_options.views_question_options',
'question_text.pid3_question_text',
'question_text.views_question_text',
'question_type.pid3_question_type',
'question_type.views_question_type',
'raw_model_response.pid3_raw_model_response',
'raw_model_response.views_raw_model_response']
We can select and print components of the Results in a table (see examples of all methods for analyzing results):
[13]:
(results.select("age", "education", "pid3", "views").print(format="rich"))
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ agent ┃ agent ┃ answer ┃ answer ┃ ┃ .age ┃ .education ┃ .pid3 ┃ .views ┃ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ 18-29 │ High school or less │ Democrat │ My political views are moderate within the Democratic party. │ │ │ │ │ I believe in the importance of balancing progressive ideas │ │ │ │ │ with practical solutions that can be realistically │ │ │ │ │ implemented. I support equal rights for all, including the │ │ │ │ │ LGBTQ+ community, and I advocate for social justice and │ │ │ │ │ economic policies that aim to reduce inequality. As a young │ │ │ │ │ black woman, issues like police reform, affordable │ │ │ │ │ healthcare, and education are particularly important to me. I │ │ │ │ │ value the principles of democracy and strive for a government │ │ │ │ │ that represents the diversity and needs of its people. │ ├─────────────┼─────────────────────┼─────────────┼───────────────────────────────────────────────────────────────┤ │ 65 and over │ College/graduate │ Independent │ As a conservative individual, my political views are guided │ │ │ │ │ by a belief in limited government, personal responsibility, │ │ │ │ │ and traditional values. I value the importance of family, the │ │ │ │ │ sanctity of life, and the importance of religious freedom. I │ │ │ │ │ am fiscally conservative, supporting policies that encourage │ │ │ │ │ economic growth and fiscal responsibility. While I am open to │ │ │ │ │ hearing different perspectives, I tend to support policies │ │ │ │ │ that I believe will uphold the nation's moral fabric, protect │ │ │ │ │ its sovereignty, and foster an environment where individuals │ │ │ │ │ and businesses can thrive without excessive government │ │ │ │ │ intervention. As an independent voter, I carefully consider │ │ │ │ │ each issue on its own merits rather than following a party │ │ │ │ │ line. │ ├─────────────┼─────────────────────┼─────────────┼───────────────────────────────────────────────────────────────┤ │ 45-64 │ College/graduate │ Independent │ As an independent with liberal leanings, my political views │ │ │ │ │ are centered around the importance of social equality, │ │ │ │ │ environmental protection, and the defense of individual │ │ │ │ │ rights, particularly concerning the LGBTQ+ community, given │ │ │ │ │ my personal identification. I believe in a government that │ │ │ │ │ actively works to reduce economic disparities, supports │ │ │ │ │ comprehensive healthcare for all, and invests in education. │ │ │ │ │ While I am not strictly tied to any political party, I tend │ │ │ │ │ to support policies and candidates that prioritize social │ │ │ │ │ justice, civil liberties, and a more inclusive society. As a │ │ │ │ │ Protestant Christian, I also value compassion and community │ │ │ │ │ support, which inform my stance on issues like welfare and │ │ │ │ │ immigration. My views are reflective of a broader perspective │ │ │ │ │ that values diversity and progressive change. │ └─────────────┴─────────────────────┴─────────────┴───────────────────────────────────────────────────────────────┘
Answer commentary
Question types other than free text automatically include a comment field for the agent to provide any unstructured commentary on its response to a question. This is useful in ensuring that responses are formatted as specified, providing an outlet for model verbosity. For example, in results.columns we can see that there is a field comment.pid3_comment. We can inspect this field as we do any other component of results. Here we also apply some pretty_labels to our table for
readability:
[14]:
(
results.select("pid3", "pid3_comment").print(
pretty_labels={"answer.pid3": "Party", "comment.pid3_comment": "Comment"},
format="rich",
)
)
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Party ┃ Comment ┃ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ Democrat │ I align with the Democratic Party and its values, and I identify as a Democrat. │ ├─────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Independent │ As an independent thinker with conservative views, I align myself with the Independent category, │ │ │ as I do not strictly adhere to the platforms of the major political parties. │ ├─────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Independent │ As someone who identifies as an Independent with liberal views, I align most with the Independent │ │ │ category. │ └─────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────┘
Selecting language models
As mentioned above, if we do not specify a language model GPT 4 is used by default. We can also specify other language models to use in generating results, and compare responses for them.
To see a list of all available models (uncomment the code):
[15]:
from edsl import Model, ModelList
# Model.available()
To check models you have stored API keys for (uncomment the code):
[16]:
# Model.check_models()
To select models for a survey, pass the model names to Model objects:
[17]:
models = ModelList(Model(m) for m in ["gpt-3.5-turbo", "gpt-4"])
We add a Model or list of models to a survey with the by method, the same as we do agents:
[18]:
results = survey.by(agents).by(models).run()
results.select("model", "pid3", "pid3_comment").print(format="rich")
┏━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ model ┃ answer ┃ comment ┃ ┃ .model ┃ .pid3 ┃ .pid3_comment ┃ ┡━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ gpt-3.5-turbo │ Other │ I consider myself as an Independent/Other due to my conservative views and │ │ │ │ independent thinking. │ ├───────────────┼─────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ gpt-3.5-turbo │ Democrat │ I think of myself as a Democrat based on my party affiliation. │ ├───────────────┼─────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ gpt-3.5-turbo │ Independent │ I think of myself as an Independent because I do not align strictly with either │ │ │ │ the Democratic or Republican party. I prefer to evaluate each issue independently │ │ │ │ and make decisions based on my own beliefs and values. │ ├───────────────┼─────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ gpt-4 │ Independent │ As an individual who identifies as Independent/Other in terms of party │ │ │ │ affiliation, I would classify myself as an Independent. Although my political │ │ │ │ ideology leans towards liberal, I prefer not to align myself strictly with the │ │ │ │ Democratic party, believing in the importance of evaluating each issue and │ │ │ │ candidate independently. │ ├───────────────┼─────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ gpt-4 │ Independent │ As an independent, I do not align myself strictly with either the Democratic or │ │ │ │ Republican party. I prefer to evaluate each issue on its own merits rather than │ │ │ │ adhering to a party line. My conservative leanings influence my decisions, but I │ │ │ │ maintain my independence in political matters. │ ├───────────────┼─────────────┼───────────────────────────────────────────────────────────────────────────────────┤ │ gpt-4 │ Democrat │ I identify myself as a Democrat, aligning with the party's values and principles. │ └───────────────┴─────────────┴───────────────────────────────────────────────────────────────────────────────────┘
Learn more about specifying language models.
Question context & memory
Survey questions are administered asynchronously by default, for efficiency. If we want an agent to have the context of one or more prior questions when presented a new question we can apply a rule specifying the questions and answers to add to the new question prompt:
[19]:
from edsl import QuestionMultipleChoice, QuestionFreeText, Survey
# From the CES pre-election questionnaire
q_CC22_309e = QuestionMultipleChoice(
question_name="CC22_309e",
question_text="Would you say that in general your health is...",
question_options=["Excellent", "Very good", "Good", "Fair", "Poor"],
)
q_CC22_309f = QuestionMultipleChoice(
question_name="CC22_309f",
question_text="Would you say that in general your mental health is...",
question_options=["Excellent", "Very good", "Good", "Fair", "Poor"],
)
survey = Survey([q_CC22_309e, q_CC22_309f])
Here we add a memory of q_CC22_309e when administering q_CC22_309f and show the prompts that were administered:
[20]:
survey = survey.add_targeted_memory(q_CC22_309f, q_CC22_309e)
[21]:
results = survey.run()
[22]:
(
results.select(
"CC22_309e_user_prompt", "CC22_309e", "CC22_309f_user_prompt", "CC22_309f"
).print(format="rich")
)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ prompt ┃ prompt ┃ answer ┃ answer ┃ ┃ .CC22_309e_user_prompt ┃ .CC22_309f_user_prompt ┃ .CC22_309e ┃ .CC22_309f ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ You are being asked the following │ You are being asked the following │ Good │ Good │ │ question: Would you say that in general │ question: Would you say that in general │ │ │ │ your health is... │ your mental health is... │ │ │ │ The options are │ The options are │ │ │ │ │ │ │ │ │ 0: Excellent │ 0: Excellent │ │ │ │ │ │ │ │ │ 1: Very good │ 1: Very good │ │ │ │ │ │ │ │ │ 2: Good │ 2: Good │ │ │ │ │ │ │ │ │ 3: Fair │ 3: Fair │ │ │ │ │ │ │ │ │ 4: Poor │ 4: Poor │ │ │ │ │ │ │ │ │ Return a valid JSON formatted like this, │ Return a valid JSON formatted like this, │ │ │ │ selecting only the number of the option: │ selecting only the number of the option: │ │ │ │ {"answer": <put answer code here>, │ {"answer": <put answer code here>, │ │ │ │ "comment": "<put explanation here>"} │ "comment": "<put explanation here>"} │ │ │ │ Only 1 option may be selected. │ Only 1 option may be selected. │ │ │ │ │ Before the question you are now │ │ │ │ │ answering, you already answered the │ │ │ │ │ following question(s): │ │ │ │ │ Question: Would you say │ │ │ │ │ that in general your health is... │ │ │ │ │ Answer: Good │ │ │ └───────────────────────────────────────────┴───────────────────────────────────────────┴────────────┴────────────┘
See examples of all methods for applying question context and memories (e.g., full memory of all prior questions, or a subset of questions).
Piping questions
We can also pipe individual components of questions into other questions. Here we use the answer to inputstate in the question text for CC22_320d:
[23]:
from edsl import QuestionMultipleChoice, Survey
q_inputstate = QuestionMultipleChoice(
question_name="inputstate",
question_text="What is your State of Residence?",
question_options=[
"Alabama",
"Alaska",
"Arizona",
"Arkansas",
"California",
"Colorado",
"Connecticut",
"Delaware",
"District of Columbia",
"Florida",
"Georgia",
"Hawaii",
"Idaho",
"Illinois",
"Indiana",
"Iowa",
"Kansas",
"Kentucky",
"Louisiana",
"Maine",
"Maryland",
"Massachusetts",
"Michigan",
"Minnesota",
"Mississippi",
"Missouri",
"Montana",
"Nebraska",
"Nevada",
"New Hampshire",
"New Jersey",
"New Mexico",
"New York",
"North Carolina",
"North Dakota",
"Ohio",
"Oklahoma",
"Oregon",
"Pennsylvania",
"Rhode Island",
"South Carolina",
"South Dakota",
"Tennessee",
"Texas",
"Utah",
"Vermont",
"Virginia",
"Washington",
"West Virginia",
"Wisconsin",
"Wyoming",
],
)
q_CC22_320d = QuestionMultipleChoice(
question_name="CC22_320d",
question_text="Do you approve of the way the Governor of {{ inputstate.answer }} is doing their job?",
question_options=[
"Strongly approve",
"Somewhat approve",
"Somewhat disapprove",
"Strongly disapprove",
"Not sure",
],
)
survey = Survey([q_inputstate, q_CC22_320d])
results = survey.by(agents).run()
results.select("inputstate", "CC22_320d_user_prompt", "CC22_320d").print(format="rich")
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ answer ┃ answer ┃ prompt ┃ ┃ .inputstate ┃ .CC22_320d ┃ .CC22_320d_user_prompt ┃ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ Washington │ Not sure │ You are being asked the following question: Do you approve of the way the │ │ │ │ Governor of Washington is doing their job? │ │ │ │ The options are │ │ │ │ │ │ │ │ 0: Strongly approve │ │ │ │ │ │ │ │ 1: Somewhat approve │ │ │ │ │ │ │ │ 2: Somewhat disapprove │ │ │ │ │ │ │ │ 3: Strongly disapprove │ │ │ │ │ │ │ │ 4: Not sure │ │ │ │ │ │ │ │ Return a valid JSON formatted like this, selecting only the number of the │ │ │ │ option: │ │ │ │ {"answer": <put answer code here>, "comment": "<put explanation here>"} │ │ │ │ Only 1 option may be selected. │ ├─────────────┼─────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ California │ Not sure │ You are being asked the following question: Do you approve of the way the │ │ │ │ Governor of California is doing their job? │ │ │ │ The options are │ │ │ │ │ │ │ │ 0: Strongly approve │ │ │ │ │ │ │ │ 1: Somewhat approve │ │ │ │ │ │ │ │ 2: Somewhat disapprove │ │ │ │ │ │ │ │ 3: Strongly disapprove │ │ │ │ │ │ │ │ 4: Not sure │ │ │ │ │ │ │ │ Return a valid JSON formatted like this, selecting only the number of the │ │ │ │ option: │ │ │ │ {"answer": <put answer code here>, "comment": "<put explanation here>"} │ │ │ │ Only 1 option may be selected. │ ├─────────────┼─────────────────────┼─────────────────────────────────────────────────────────────────────────────┤ │ California │ Strongly disapprove │ You are being asked the following question: Do you approve of the way the │ │ │ │ Governor of California is doing their job? │ │ │ │ The options are │ │ │ │ │ │ │ │ 0: Strongly approve │ │ │ │ │ │ │ │ 1: Somewhat approve │ │ │ │ │ │ │ │ 2: Somewhat disapprove │ │ │ │ │ │ │ │ 3: Strongly disapprove │ │ │ │ │ │ │ │ 4: Not sure │ │ │ │ │ │ │ │ Return a valid JSON formatted like this, selecting only the number of the │ │ │ │ option: │ │ │ │ {"answer": <put answer code here>, "comment": "<put explanation here>"} │ │ │ │ Only 1 option may be selected. │ └─────────────┴─────────────────────┴─────────────────────────────────────────────────────────────────────────────┘
Survey rules & conditions
We can apply survey rules and conditions to administer relevant follow-up questions based on responses to questions. For example, here we add skip rules to a set of questions by calling the method add_skip_rule() and passing the target question and the condition to evaluate (questions not administered will show a None response):
[24]:
from edsl import QuestionCheckBox, QuestionMultipleChoice, Survey
# From the CES pre-election questionnaire
q_CC22_300 = (
QuestionCheckBox( # Use checkbox to allow the agent to select multiple options
question_name="CC22_300",
question_text="In the past 24 hours have you... (check all that apply)",
question_options=[
"Used social media (such as Facebook or Youtube)",
"Watched TV news",
"Read a newspaper in print or online",
"Listened to a radio news program or talk radio",
"None of these",
],
)
)
# Skip this question if the agent does not select "Watched TV news"
q_CC22_300a = QuestionMultipleChoice(
question_name="CC22_300a",
question_text="Did you watch local news, national news, or both?",
question_options=["Local Newscast", "National Newscast", "Both"],
)
# Skip this question if the agent does not select "Watched TV news"
q_CC22_300b = QuestionMultipleChoice(
question_name="CC22_300b",
question_text="Which of these networks did you watch?",
question_options=["ABC", "CBS", "NBC", "CNN", "Fox News", "MSNBC", "PBS", "Other"],
)
# Skip this question if the agent does not select "Read a newspaper..."
q_CC22_300c = QuestionMultipleChoice(
question_name="CC22_300c",
question_text="Did you read a print newspaper, an online newspaper, or both?",
question_options=["Print", "Online", "Both"],
)
# Skip this question if the agent does not select "Used social media..."
q_CC22_300d = QuestionMultipleChoice(
question_name="CC22_300d",
question_text="In the past 24 hours, did you do any of the following on social media (such as Facebook, Youtube or Twitter)?",
question_options=[
"Posted a story, photo, video or link about politics",
"Posted a comment about politics",
"Read a story or watched a video about politics",
"Followed a political event",
"Forwarded a story, photo, video or link about politics to friends",
"None of the above",
],
)
survey_CC22_300 = (
Survey([q_CC22_300, q_CC22_300a, q_CC22_300b, q_CC22_300c, q_CC22_300d])
.add_skip_rule(q_CC22_300a, "'Watched TV news' not in CC22_300")
.add_skip_rule(q_CC22_300b, "'Watched TV news' not in CC22_300")
.add_skip_rule(q_CC22_300c, "'Read a newspaper in print or online' not in CC22_300")
.add_skip_rule(
q_CC22_300d, "'Used social media (such as Facebook or Youtube)' not in CC22_300"
)
)
results_CC22_300 = survey_CC22_300.by(agents).run()
results_CC22_300.select(
"CC22_300", "CC22_300a", "CC22_300b", "CC22_300c", "CC22_300d"
).print(format="rich")
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ answer ┃ answer ┃ answer ┃ answer ┃ answer ┃ ┃ .CC22_300 ┃ .CC22_300a ┃ .CC22_300b ┃ .CC22_300c ┃ .CC22_300d ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ ['Used social media (such as │ None │ None │ None │ Read a story or watched a video │ │ Facebook or Youtube)'] │ │ │ │ about politics │ ├────────────────────────────────────┼────────────┼────────────┼────────────┼─────────────────────────────────────┤ │ ['Watched TV news', 'Read a │ Both │ Fox News │ Both │ None │ │ newspaper in print or online'] │ │ │ │ │ ├────────────────────────────────────┼────────────┼────────────┼────────────┼─────────────────────────────────────┤ │ ['Used social media (such as │ Both │ CNN │ Online │ Read a story or watched a video │ │ Facebook or Youtube)', 'Watched TV │ │ │ │ about politics │ │ news', 'Read a newspaper in print │ │ │ │ │ │ or online'] │ │ │ │ │ └────────────────────────────────────┴────────────┴────────────┴────────────┴─────────────────────────────────────┘
Here we use add_stop_rule() to end the survey based on the response to an initial question (an option selection that makes the second question unnecessary):
[25]:
from edsl import QuestionMultipleChoice, QuestionYesNo, Survey
# From the CES pre-election questionnaire
q_employ = QuestionMultipleChoice(
question_name="employ",
question_text="Which of the following best describes your current employment status?",
question_options=[
"Working full time now",
"Working part time now",
"Temporarily laid off",
"Unemployed",
"Retired",
"Permanently disabled",
"Taking care of home or family",
"Student",
"Other",
],
)
q_hadjob = QuestionYesNo(
question_name="hadjob",
question_text="At any time over the past five years, have you had a job?",
)
survey = Survey([q_employ, q_hadjob]).add_stop_rule(
q_employ,
"employ in ['Working full time now', 'Working part time now', 'Temporarily laid off']",
)
results = survey.by(agents).run()
results.select("employ", "hadjob").print(format="rich")
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓ ┃ answer ┃ answer ┃ ┃ .employ ┃ .hadjob ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩ │ Working part time now │ None │ ├───────────────────────┼─────────┤ │ Working full time now │ None │ ├───────────────────────┼─────────┤ │ Retired │ Yes │ └───────────────────────┴─────────┘
Combining survey methods
Here we apply multiple methods at once: we add a memory of region to the prompt for inputzip, then pipe the answer to inputzip into the question text of votereg_f, and a stop rule if the answer to votereg is not “Yes” (i.e., do not administer votereg_f):
[26]:
from edsl import QuestionMultipleChoice, QuestionList, QuestionYesNo, Survey
# From the CES pre-election questionnaire
q_region = QuestionMultipleChoice(
question_name="region",
question_text="In which census region do you live?",
question_options=["Northeast", "Midwest", "South", "West"],
)
q_inputzip = QuestionList(
question_name="inputzip",
question_text="So that we can ask you about the news and events in your area, in what zip code do you currently reside?",
max_list_items=1,
)
q_votereg = QuestionMultipleChoice(
question_name="votereg",
question_text="Are you registered to vote?",
question_options=["Yes", "No", "Don't know"],
)
q_votereg_f = QuestionYesNo(
question_name="votereg_f",
question_text="Is {{ inputzip.answer[0] }} the zip code where you are registered to vote?",
)
survey = (
Survey([q_region, q_inputzip, q_votereg, q_votereg_f])
.add_targeted_memory(q_inputzip, q_region)
.add_stop_rule(q_votereg, "votereg != 'Yes'")
)
results = survey.by(agents).run()
results.select(
"region", "inputzip", "votereg", "votereg_f_user_prompt", "votereg_f"
).print(format="rich")
┏━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ answer ┃ answer ┃ answer ┃ answer ┃ prompt ┃ ┃ .region ┃ .inputzip ┃ .votereg ┃ .votereg_f ┃ .votereg_f_user_prompt ┃ ┡━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ South │ ['30309'] │ Yes │ No │ You are being asked the following question: Is 30309 the zip code │ │ │ │ │ │ where you are registered to vote? │ │ │ │ │ │ The options are │ │ │ │ │ │ │ │ │ │ │ │ 0: Yes │ │ │ │ │ │ │ │ │ │ │ │ 1: No │ │ │ │ │ │ │ │ │ │ │ │ Return a valid JSON formatted like this, selecting only the │ │ │ │ │ │ number of the option: │ │ │ │ │ │ {"answer": <put answer code here>, "comment": "<put explanation │ │ │ │ │ │ here>"} │ │ │ │ │ │ Only 1 option may be selected. │ ├─────────┼───────────┼──────────┼────────────┼───────────────────────────────────────────────────────────────────┤ │ West │ ['90001'] │ Yes │ No │ You are being asked the following question: Is 90001 the zip code │ │ │ │ │ │ where you are registered to vote? │ │ │ │ │ │ The options are │ │ │ │ │ │ │ │ │ │ │ │ 0: Yes │ │ │ │ │ │ │ │ │ │ │ │ 1: No │ │ │ │ │ │ │ │ │ │ │ │ Return a valid JSON formatted like this, selecting only the │ │ │ │ │ │ number of the option: │ │ │ │ │ │ {"answer": <put answer code here>, "comment": "<put explanation │ │ │ │ │ │ here>"} │ │ │ │ │ │ Only 1 option may be selected. │ ├─────────┼───────────┼──────────┼────────────┼───────────────────────────────────────────────────────────────────┤ │ South │ ['30309'] │ Yes │ No │ You are being asked the following question: Is 30309 the zip code │ │ │ │ │ │ where you are registered to vote? │ │ │ │ │ │ The options are │ │ │ │ │ │ │ │ │ │ │ │ 0: Yes │ │ │ │ │ │ │ │ │ │ │ │ 1: No │ │ │ │ │ │ │ │ │ │ │ │ Return a valid JSON formatted like this, selecting only the │ │ │ │ │ │ number of the option: │ │ │ │ │ │ {"answer": <put answer code here>, "comment": "<put explanation │ │ │ │ │ │ here>"} │ │ │ │ │ │ Only 1 option may be selected. │ └─────────┴───────────┴──────────┴────────────┴───────────────────────────────────────────────────────────────────┘
See more details on all survey methods.
Parameterizing questions
We can create variations of questions using Scenario objects for content that we want to add to questions. This allows us to efficiently administer multiple versions of questions at once. We start by using a {{ parameter }} in a question:
[27]:
from edsl import QuestionMultipleChoice, Survey
# Modified from the CES pre-election questionnaire
q_votereg = QuestionMultipleChoice(
question_name="votereg",
question_text="Are you {{ status }}?",
question_options=["Yes", "No", "Don't know"],
)
survey = Survey([q_votereg])
Next we create a Scenario for each text that we want to insert in the question:
[28]:
from edsl import Scenario, ScenarioList
statuses = [
"registered to vote", # original CES question
"enrolled in school",
"employed full- or part-time",
"married or in a domestic partnership",
"licensed to drive",
]
scenarios = ScenarioList(Scenario({"status": s}) for s in statuses)
[29]:
results = survey.by(scenarios).by(agents).run()
[30]:
(
results.sort_by("status", "age")
.select("age", "education", "sex", "status", "votereg")
.print(format="rich")
)
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓ ┃ agent ┃ agent ┃ agent ┃ scenario ┃ answer ┃ ┃ .age ┃ .education ┃ .sex ┃ .status ┃ .votereg ┃ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩ │ 18-29 │ High school or less │ Female │ employed full- or part-time │ Yes │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 45-64 │ College/graduate │ Female │ employed full- or part-time │ No │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 65 and over │ College/graduate │ Female │ employed full- or part-time │ No │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ High school or less │ Female │ enrolled in school │ No │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 45-64 │ College/graduate │ Female │ enrolled in school │ No │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 65 and over │ College/graduate │ Female │ enrolled in school │ No │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ High school or less │ Female │ licensed to drive │ Yes │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 45-64 │ College/graduate │ Female │ licensed to drive │ Yes │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 65 and over │ College/graduate │ Female │ licensed to drive │ Yes │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ High school or less │ Female │ married or in a domestic partnership │ No │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 45-64 │ College/graduate │ Female │ married or in a domestic partnership │ No │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 65 and over │ College/graduate │ Female │ married or in a domestic partnership │ Yes │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ High school or less │ Female │ registered to vote │ Yes │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 45-64 │ College/graduate │ Female │ registered to vote │ Yes │ ├─────────────┼─────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 65 and over │ College/graduate │ Female │ registered to vote │ Yes │ └─────────────┴─────────────────────┴────────┴──────────────────────────────────────┴──────────┘
Using the CES responses to create agents
See the companion notebooks for complete sets of CES questions reformatted in EDSL and examples of using survey responses to construct agents representing respondents: