Using EDSL for AI polling
This notebook demonstrates methods for conducting surveys with AI agents using EDSL, an open-source Python library for simulating social science research with large language models. EDSL is developed by Expected Parrot, a start-up building tools for AI-powered research.
Contents
The sections below include:
Creating an AI agent: Basic steps to construct an AI agent.
Administering questions: How to create questions and prompt agents to answer them.
Selecting language models: Specify language models that you want to use to generate responses.
Analyzing results: Examples of built-in methods for analyzing responses as datasets.
Designing agent traits: How to construct agents with complex personas and traits.
Converting surveys into EDSL: Import other surveys into EDSL to analyze and extend them with agents.
Constructing agents from survey data: Use survey responses to construct agents representing respondents.
Sample data: Cooperative Election Study Common Content, 2022
For purposes of demonstration, we use data from the Cooperative Election Study Common Content, 2022 in several ways:
In this notebook we use lists of respondent attributes from the Breakdown of National Vote for U.S. House (CES validated voters) (CES Guide 2022 pp.24-25) to design agents with combinations of the attributes, and then administer questions to them.
Companion notebooks:
In Working with CES data in EDSL we reconstruct the 2022 CES Pre-Election Questionnaire in EDSL. The formatted questions can be modified and administered to agents in new surveys as desired. The notebook also contains dictionaries of the CES questions and answer codebook for convenience in working with the data.
In Creating agents for survey respondents we use the dataset of responses to the 2022 CES Pre-Election Questionnaire to create agents representing the respondents, and administer follow-on questions with them.
Reference & contact
Documentation for the EDSL package is available at https://docs/expectedparrot.com. You can also find example code, tutorials and notebooks for a variety of use cases.
Please let us know if you have any questions or encounter issues working with this data:
Email: info@expectedparrot.com
Technical setup
EDSL is compatible with Python 3.9-3.12.
See instructions on installing the EDSL library and storing API keys for the language models that you want to use. In examples below where no model is specified, EDSL will use GPT 4 by default (an API key for OpenAI is required). We also show how to use different models.
Creating an AI agent
In this section we show how to create an AI agent and give it desired attributes. For more details on constructing and using AI agents please see our documentation page on agents.
We start by importing the tools for creating agents:
[1]:
from edsl import Agent
Here we create a simple Agent and pass it a dictionary of traits. We optionally include a narrative persona and also specify traits individually for use in segmenting and analyzing survey responses:
[2]:
agent = Agent(
traits={
"persona": "You are 55-year-old research scientist living in Cambridge, Massachusetts.",
"occupation": "Research scientist",
"location": "Cambridge, Massachusetts",
"age": 55,
}
)
We can access the traits directly:
[3]:
agent.location
[3]:
'Cambridge, Massachusetts'
Designing agent panels
We can also create panels of agents in an AgentList and administer surveys to all of the agents at once. Here we construct combinations of traits from lists of respondent attributes in the CES Guide (see source details above). (Information can be imported from a variety of data source types; see documentation for details.)
[4]:
sex = ["Male", "Female"]
race = ["White", "Black", "Hispanic", "Asian", "Other"]
age = ["18-29", "30-44", "45-64", "65 and over"]
education = [
"High school or less",
"Some college/assoc. degree",
"College/graduate",
"Postgraduate study",
]
income = [
"Under $30,000",
"$30,000 to $49,999",
"$50,000 to $99,999",
"$100,000 to $199,999",
"$200,000 or more",
]
party_affiliation = ["Democrat", "Republican", "Independent/Other"]
political_ideology = ["Liberal", "Moderate", "Conservative", "Unsure"]
religion = [
"Protestant/other Christian",
"Catholic",
"Jewish",
"Something else",
"None",
]
evangelical = ["Yes", "No"]
married = ["Yes", "No"]
lgbt = ["Yes", "No"]
Here we create a method to generate a list of agents with randomly selected combinations of traits:
[5]:
from edsl import AgentList
import random
def generate_random_agents(num_agents):
agents = []
for _ in range(num_agents):
agent_traits = {
"sex": random.choice(sex),
"race": random.choice(race),
"age": random.choice(age),
"education": random.choice(education),
"income": random.choice(income),
"party_affiliation": random.choice(party_affiliation),
"political_ideology": random.choice(political_ideology),
"religion": random.choice(religion),
"evangelical": random.choice(evangelical),
"married": random.choice(married),
"lgbt": random.choice(lgbt),
}
agents.append(Agent(traits=agent_traits))
return AgentList(agents)
Example usage:
[6]:
num_agents = 3
agents = generate_random_agents(num_agents)
Agent instructions
If we want to give all the agents a special instruction, we can optionally pass an instruction to the agents (this can also be done when the agents are created):
[7]:
for agent in agents:
agent.instruction = "Today is July 1, 2022."
We can inspect the agents that have been created:
[8]:
agents
[8]:
[
{
"traits": {
"sex": "Female",
"race": "Black",
"age": "30-44",
"education": "College/graduate",
"income": "$50,000 to $99,999",
"party_affiliation": "Independent/Other",
"political_ideology": "Conservative",
"religion": "Something else",
"evangelical": "No",
"married": "No",
"lgbt": "Yes"
},
"instruction": "Today is July 1, 2022.",
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Agent"
},
{
"traits": {
"sex": "Female",
"race": "Black",
"age": "30-44",
"education": "High school or less",
"income": "$200,000 or more",
"party_affiliation": "Democrat",
"political_ideology": "Moderate",
"religion": "Something else",
"evangelical": "Yes",
"married": "No",
"lgbt": "Yes"
},
"instruction": "Today is July 1, 2022.",
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Agent"
},
{
"traits": {
"sex": "Female",
"race": "Other",
"age": "18-29",
"education": "Some college/assoc. degree",
"income": "Under $30,000",
"party_affiliation": "Republican",
"political_ideology": "Conservative",
"religion": "Protestant/other Christian",
"evangelical": "No",
"married": "Yes",
"lgbt": "Yes"
},
"instruction": "Today is July 1, 2022.",
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Agent"
}
]
Creating questions
An Agent is designed to be assigned questions to answer. In this section we construct questions in the form of Question objects, combine them into a Survey, administer it to some sample agents (from above), and inspect the responses in the dataset of Results that is generated.
EDSL comes with many question types that we can select from based on the form of the response that we want to get back from the language model (free text, linear scale, checkbox, etc.). See examples of all question types.
Here we create a multiple choice question from the CES Pre-Election Questionnaire (the response will be a selection from the list of options that we include) and compose a follow-up free text question (the response will be unstructured text):
[9]:
from edsl import QuestionMultipleChoice, QuestionFreeText
# From the CES pre-election questionnaire
q_pid3 = QuestionMultipleChoice(
question_name="pid3",
question_text="Generally speaking, do you think of yourself as a ...?",
question_options=["Democrat", "Republican", "Independent", "Other", "Not sure"],
)
# Potential follow-up question
q_views = QuestionFreeText(
question_name="views", question_text="Describe your political views."
)
We combine the questions into a Survey to administer them together:
[10]:
from edsl import Survey
survey = Survey([q_pid3, q_views])
Administering a survey
We administer a survey by calling the run method, after (optionally) adding agents with the by method:
[11]:
results = survey.by(agents).run()
We can show a list of all the components of the Results that have been generated, and see that the results include information about the agents, questions, models, prompts and responses:
[12]:
results.columns
[12]:
['agent.age',
'agent.agent_instruction',
'agent.agent_name',
'agent.education',
'agent.evangelical',
'agent.income',
'agent.lgbt',
'agent.married',
'agent.party_affiliation',
'agent.political_ideology',
'agent.race',
'agent.religion',
'agent.sex',
'answer.pid3',
'answer.views',
'comment.pid3_comment',
'comment.views_comment',
'generated_tokens.pid3_generated_tokens',
'generated_tokens.views_generated_tokens',
'iteration.iteration',
'model.frequency_penalty',
'model.logprobs',
'model.max_tokens',
'model.model',
'model.presence_penalty',
'model.temperature',
'model.top_logprobs',
'model.top_p',
'prompt.pid3_system_prompt',
'prompt.pid3_user_prompt',
'prompt.views_system_prompt',
'prompt.views_user_prompt',
'question_options.pid3_question_options',
'question_options.views_question_options',
'question_text.pid3_question_text',
'question_text.views_question_text',
'question_type.pid3_question_type',
'question_type.views_question_type',
'raw_model_response.pid3_cost',
'raw_model_response.pid3_one_usd_buys',
'raw_model_response.pid3_raw_model_response',
'raw_model_response.views_cost',
'raw_model_response.views_one_usd_buys',
'raw_model_response.views_raw_model_response']
We can select and print components of the Results in a table (see examples of all methods for analyzing results):
[13]:
(results.select("age", "education", "pid3", "views").print(format="rich"))
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ agent ┃ agent ┃ answer ┃ answer ┃ ┃ .age ┃ .education ┃ .pid3 ┃ .views ┃ ┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ 18-29 │ Some college/assoc. degree │ Republican │ As a conservative and a member of the Republican Party, I │ │ │ │ │ tend to support policies that emphasize limited government, │ │ │ │ │ individual liberties, free-market principles, and │ │ │ │ │ traditional values. I believe in the importance of personal │ │ │ │ │ responsibility and the role of private enterprise in driving │ │ │ │ │ economic growth. My religious beliefs as a Protestant also │ │ │ │ │ influence my views, particularly on social issues. While I │ │ │ │ │ am not an evangelical, my faith plays a significant role in │ │ │ │ │ shaping my perspectives on topics like family values and │ │ │ │ │ moral conduct. │ ├───────┼────────────────────────────┼─────────────┼──────────────────────────────────────────────────────────────┤ │ 30-44 │ College/graduate │ Independent │ As an Independent with conservative leanings, my political │ │ │ │ │ views tend to emphasize the importance of individual │ │ │ │ │ responsibility, limited government, and free-market │ │ │ │ │ principles. I believe in maintaining fiscal discipline and │ │ │ │ │ reducing government intervention in the economy to promote │ │ │ │ │ innovation and entrepreneurship. │ │ │ │ │ │ │ │ │ │ On social issues, my views may vary. While I value │ │ │ │ │ traditional values and the importance of community and │ │ │ │ │ family, I also support individual freedoms and rights. Being │ │ │ │ │ part of the LGBT community, I advocate for equal rights and │ │ │ │ │ protections under the law for all individuals, regardless of │ │ │ │ │ their sexual orientation or gender identity. │ ├───────┼────────────────────────────┼─────────────┼──────────────────────────────────────────────────────────────┤ │ 30-44 │ High school or less │ Democrat │ As a moderate Democrat, I believe in a balanced approach to │ │ │ │ │ governance that incorporates both progressive and pragmatic │ │ │ │ │ solutions. I support policies that aim to reduce income │ │ │ │ │ inequality, improve access to quality healthcare, and ensure │ │ │ │ │ a robust social safety net. Education is a priority for me, │ │ │ │ │ and I advocate for increased funding and reforms that make │ │ │ │ │ quality education accessible to all. │ │ │ │ │ │ │ │ │ │ I also believe in the importance of addressing climate │ │ │ │ │ change through sustainable practices and policies that │ │ │ │ │ promote clean energy. On social issues, I support LGBTQ+ │ │ │ │ │ rights, gender equality, and racial justice, striving for a │ │ │ │ │ society where everyone has equal opportunities and │ │ │ │ │ protections under the law. │ └───────┴────────────────────────────┴─────────────┴──────────────────────────────────────────────────────────────┘
Answer commentary
Question types other than free text automatically include a comment field for the agent to provide any unstructured commentary on its response to a question. This is useful in ensuring that responses are formatted as specified, providing an outlet for model verbosity. For example, in results.columns we can see that there is a field comment.pid3_comment. We can inspect this field as we do any other component of results. Here we also apply some pretty_labels to our table for
readability:
[14]:
(
results.select("pid3", "pid3_comment").print(
pretty_labels={"answer.pid3": "Party", "comment.pid3_comment": "Comment"},
format="rich",
)
)
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Party ┃ Comment ┃ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ Republican │ Given my party affiliation and political ideology, I identify as a Republican. │ ├─────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Independent │ I chose this option because I align most closely with Independent/Other party affiliation, as I │ │ │ don't fully agree with the platforms of the major parties. │ ├─────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────┤ │ Democrat │ I chose this option because my party affiliation is Democrat. │ └─────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────┘
Selecting language models
As mentioned above, if we do not specify a language model GPT 4 is used by default. We can also specify other language models to use in generating results, and compare responses for them.
To see a list of all available models (uncomment the code):
[15]:
from edsl import Model, ModelList
# Model.available()
To check models you have stored API keys for (uncomment the code):
[16]:
# Model.check_models()
To select models for a survey, pass the model names to Model objects:
[17]:
models = ModelList(Model(m) for m in ["gemini-pro", "gpt-4o"])
We add a Model or list of models to a survey with the by method, the same as we do agents:
[18]:
results = survey.by(agents).by(models).run()
results.select("model", "pid3", "pid3_comment").print(format="rich")
┏━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ model ┃ answer ┃ comment ┃ ┃ .model ┃ .pid3 ┃ .pid3_comment ┃ ┡━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ gpt-4o │ Independent │ I chose this option because I align most closely with Independent/Other party │ │ │ │ affiliation, as I don't fully agree with the platforms of the major parties. │ ├────────────┼─────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ │ gpt-4o │ Democrat │ I chose this option because my party affiliation is Democrat. │ ├────────────┼─────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ │ gpt-4o │ Republican │ Given my party affiliation and political ideology, I identify as a Republican. │ ├────────────┼─────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ │ gemini-pro │ Republican │ I am a Republican because I believe in the principles of individual liberty, limited │ │ │ │ government, and free markets. I believe that the government should play a limited │ │ │ │ role in our lives and that we should be free to make our own choices. I also believe │ │ │ │ that the free market is the best way to create jobs and prosperity. │ ├────────────┼─────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ │ gemini-pro │ Democrat │ I am a Democrat because I believe in the party's values of equality, social justice, │ │ │ │ and economic fairness. I believe that the government should play a role in helping │ │ │ │ people meet their basic needs, such as healthcare, education, and housing. I also │ │ │ │ believe that we need to take action to address climate change and protect our │ │ │ │ environment. │ ├────────────┼─────────────┼──────────────────────────────────────────────────────────────────────────────────────┤ │ gemini-pro │ Independent │ I am an Independent because I do not fully align with either the Democratic or │ │ │ │ Republican parties. I believe in fiscal responsibility and limited government, but I │ │ │ │ also support social programs that help those in need. │ └────────────┴─────────────┴──────────────────────────────────────────────────────────────────────────────────────┘
Learn more about specifying language models.
Question context & memory
Survey questions are administered asynchronously by default, for efficiency. If we want an agent to have the context of one or more prior questions when presented a new question we can apply a rule specifying the questions and answers to add to the new question prompt:
[19]:
from edsl import QuestionMultipleChoice, QuestionFreeText, Survey
# From the CES pre-election questionnaire
q_CC22_309e = QuestionMultipleChoice(
question_name="CC22_309e",
question_text="Would you say that in general your health is...",
question_options=["Excellent", "Very good", "Good", "Fair", "Poor"],
)
q_CC22_309f = QuestionMultipleChoice(
question_name="CC22_309f",
question_text="Would you say that in general your mental health is...",
question_options=["Excellent", "Very good", "Good", "Fair", "Poor"],
)
survey = Survey([q_CC22_309e, q_CC22_309f])
Here we add a memory of q_CC22_309e when administering q_CC22_309f and show the prompts that were administered:
[20]:
survey = survey.add_targeted_memory(q_CC22_309f, q_CC22_309e)
[21]:
results = survey.run()
[22]:
(
results.select(
"CC22_309e_user_prompt", "CC22_309e", "CC22_309f_user_prompt", "CC22_309f"
).print(format="rich")
)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ prompt ┃ answer ┃ prompt ┃ answer ┃ ┃ .CC22_309e_user_prompt ┃ .CC22_309e ┃ .CC22_309f_user_prompt ┃ .CC22_309f ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ │ Very good │ │ Very good │ │ Would you say that in general your health │ │ Would you say that in general your mental │ │ │ is... │ │ health is... │ │ │ │ │ │ │ │ │ │ │ │ │ Excellent │ │ Excellent │ │ │ │ │ │ │ │ Very good │ │ Very good │ │ │ │ │ │ │ │ Good │ │ Good │ │ │ │ │ │ │ │ Fair │ │ Fair │ │ │ │ │ │ │ │ Poor │ │ Poor │ │ │ │ │ │ │ │ │ │ │ │ │ Only 1 option may be selected. │ │ Only 1 option may be selected. │ │ │ │ │ │ │ │ Respond only with a string corresponding │ │ Respond only with a string corresponding │ │ │ to one of the options. │ │ to one of the options. │ │ │ │ │ │ │ │ │ │ │ │ │ After the answer, you can put a comment │ │ After the answer, you can put a comment │ │ │ explaining why you chose that option on │ │ explaining why you chose that option on │ │ │ the next line. │ │ the next line. │ │ │ │ │ Before the question you are now │ │ │ │ │ answering, you already answered the │ │ │ │ │ following question(s): │ │ │ │ │ Question: Would you say │ │ │ │ │ that in general your health is... │ │ │ │ │ Answer: Very good │ │ └───────────────────────────────────────────┴────────────┴───────────────────────────────────────────┴────────────┘
See examples of all methods for applying question context and memories (e.g., full memory of all prior questions, or a subset of questions).
Piping questions
We can also pipe individual components of questions into other questions. Here we use the answer to inputstate in the question text for CC22_320d:
[23]:
from edsl import QuestionMultipleChoice, Survey
q_inputstate = QuestionMultipleChoice(
question_name="inputstate",
question_text="What is your State of Residence?",
question_options=[
"Alabama",
"Alaska",
"Arizona",
"Arkansas",
"California",
"Colorado",
"Connecticut",
"Delaware",
"District of Columbia",
"Florida",
"Georgia",
"Hawaii",
"Idaho",
"Illinois",
"Indiana",
"Iowa",
"Kansas",
"Kentucky",
"Louisiana",
"Maine",
"Maryland",
"Massachusetts",
"Michigan",
"Minnesota",
"Mississippi",
"Missouri",
"Montana",
"Nebraska",
"Nevada",
"New Hampshire",
"New Jersey",
"New Mexico",
"New York",
"North Carolina",
"North Dakota",
"Ohio",
"Oklahoma",
"Oregon",
"Pennsylvania",
"Rhode Island",
"South Carolina",
"South Dakota",
"Tennessee",
"Texas",
"Utah",
"Vermont",
"Virginia",
"Washington",
"West Virginia",
"Wisconsin",
"Wyoming",
],
)
q_CC22_320d = QuestionMultipleChoice(
question_name="CC22_320d",
question_text="Do you approve of the way the Governor of {{ inputstate.answer }} is doing their job?",
question_options=[
"Strongly approve",
"Somewhat approve",
"Somewhat disapprove",
"Strongly disapprove",
"Not sure",
],
)
survey = Survey([q_inputstate, q_CC22_320d])
results = survey.by(agents).run()
results.select("inputstate", "CC22_320d_user_prompt", "CC22_320d").print(format="rich")
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓ ┃ answer ┃ prompt ┃ answer ┃ ┃ .inputstate ┃ .CC22_320d_user_prompt ┃ .CC22_320d ┃ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩ │ Florida │ │ Strongly approve │ │ │ Do you approve of the way the Governor of Florida is doing their job? │ │ │ │ │ │ │ │ │ │ │ │ Strongly approve │ │ │ │ │ │ │ │ Somewhat approve │ │ │ │ │ │ │ │ Somewhat disapprove │ │ │ │ │ │ │ │ Strongly disapprove │ │ │ │ │ │ │ │ Not sure │ │ │ │ │ │ │ │ │ │ │ │ Only 1 option may be selected. │ │ │ │ │ │ │ │ Respond only with a string corresponding to one of the options. │ │ │ │ │ │ │ │ │ │ │ │ After the answer, you can put a comment explaining why you chose that │ │ │ │ option on the next line. │ │ ├─────────────┼─────────────────────────────────────────────────────────────────────────────┼─────────────────────┤ │ California │ │ Somewhat disapprove │ │ │ Do you approve of the way the Governor of California is doing their job? │ │ │ │ │ │ │ │ │ │ │ │ Strongly approve │ │ │ │ │ │ │ │ Somewhat approve │ │ │ │ │ │ │ │ Somewhat disapprove │ │ │ │ │ │ │ │ Strongly disapprove │ │ │ │ │ │ │ │ Not sure │ │ │ │ │ │ │ │ │ │ │ │ Only 1 option may be selected. │ │ │ │ │ │ │ │ Respond only with a string corresponding to one of the options. │ │ │ │ │ │ │ │ │ │ │ │ After the answer, you can put a comment explaining why you chose that │ │ │ │ option on the next line. │ │ ├─────────────┼─────────────────────────────────────────────────────────────────────────────┼─────────────────────┤ │ California │ │ Somewhat approve │ │ │ Do you approve of the way the Governor of California is doing their job? │ │ │ │ │ │ │ │ │ │ │ │ Strongly approve │ │ │ │ │ │ │ │ Somewhat approve │ │ │ │ │ │ │ │ Somewhat disapprove │ │ │ │ │ │ │ │ Strongly disapprove │ │ │ │ │ │ │ │ Not sure │ │ │ │ │ │ │ │ │ │ │ │ Only 1 option may be selected. │ │ │ │ │ │ │ │ Respond only with a string corresponding to one of the options. │ │ │ │ │ │ │ │ │ │ │ │ After the answer, you can put a comment explaining why you chose that │ │ │ │ option on the next line. │ │ └─────────────┴─────────────────────────────────────────────────────────────────────────────┴─────────────────────┘
Survey rules & conditions
We can apply survey rules and conditions to administer relevant follow-up questions based on responses to questions. For example, here we add skip rules to a set of questions by calling the method add_skip_rule() and passing the target question and the condition to evaluate (questions not administered will show a None response):
[24]:
from edsl import QuestionCheckBox, QuestionMultipleChoice, Survey
# From the CES pre-election questionnaire
q_CC22_300 = (
QuestionCheckBox( # Use checkbox to allow the agent to select multiple options
question_name="CC22_300",
question_text="In the past 24 hours have you... (check all that apply)",
question_options=[
"Used social media (such as Facebook or Youtube)",
"Watched TV news",
"Read a newspaper in print or online",
"Listened to a radio news program or talk radio",
"None of these",
],
)
)
# Skip this question if the agent does not select "Watched TV news"
q_CC22_300a = QuestionMultipleChoice(
question_name="CC22_300a",
question_text="Did you watch local news, national news, or both?",
question_options=["Local Newscast", "National Newscast", "Both"],
)
# Skip this question if the agent does not select "Watched TV news"
q_CC22_300b = QuestionMultipleChoice(
question_name="CC22_300b",
question_text="Which of these networks did you watch?",
question_options=["ABC", "CBS", "NBC", "CNN", "Fox News", "MSNBC", "PBS", "Other"],
)
# Skip this question if the agent does not select "Read a newspaper..."
q_CC22_300c = QuestionMultipleChoice(
question_name="CC22_300c",
question_text="Did you read a print newspaper, an online newspaper, or both?",
question_options=["Print", "Online", "Both"],
)
# Skip this question if the agent does not select "Used social media..."
q_CC22_300d = QuestionMultipleChoice(
question_name="CC22_300d",
question_text="In the past 24 hours, did you do any of the following on social media (such as Facebook, Youtube or Twitter)?",
question_options=[
"Posted a story, photo, video or link about politics",
"Posted a comment about politics",
"Read a story or watched a video about politics",
"Followed a political event",
"Forwarded a story, photo, video or link about politics to friends",
"None of the above",
],
)
survey_CC22_300 = (
Survey([q_CC22_300, q_CC22_300a, q_CC22_300b, q_CC22_300c, q_CC22_300d])
.add_skip_rule(q_CC22_300a, "'Watched TV news' not in CC22_300")
.add_skip_rule(q_CC22_300b, "'Watched TV news' not in CC22_300")
.add_skip_rule(q_CC22_300c, "'Read a newspaper in print or online' not in CC22_300")
.add_skip_rule(
q_CC22_300d, "'Used social media (such as Facebook or Youtube)' not in CC22_300"
)
)
results_CC22_300 = survey_CC22_300.by(agents).run()
results_CC22_300.select(
"CC22_300", "CC22_300a", "CC22_300b", "CC22_300c", "CC22_300d"
).print(format="rich")
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ answer ┃ answer ┃ answer ┃ answer ┃ answer ┃ ┃ .CC22_300 ┃ .CC22_300a ┃ .CC22_300b ┃ .CC22_300c ┃ .CC22_300d ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ ['Used social media (such as │ Both │ Fox News │ Online │ None of the above │ │ Facebook or Youtube)', 'Watched TV │ │ │ │ │ │ news', 'Read a newspaper in print │ │ │ │ │ │ or online'] │ │ │ │ │ ├────────────────────────────────────┼────────────┼────────────┼────────────┼─────────────────────────────────────┤ │ ['Used social media (such as │ Both │ Fox News │ None │ None of the above │ │ Facebook or Youtube)', 'Watched TV │ │ │ │ │ │ news'] │ │ │ │ │ ├────────────────────────────────────┼────────────┼────────────┼────────────┼─────────────────────────────────────┤ │ ['Used social media (such as │ Both │ CNN │ Online │ Read a story or watched a video │ │ Facebook or Youtube)', 'Watched TV │ │ │ │ about politics │ │ news', 'Read a newspaper in print │ │ │ │ │ │ or online'] │ │ │ │ │ └────────────────────────────────────┴────────────┴────────────┴────────────┴─────────────────────────────────────┘
Here we use add_stop_rule() to end the survey based on the response to an initial question (an option selection that makes the second question unnecessary):
[25]:
from edsl import QuestionMultipleChoice, QuestionYesNo, Survey
# From the CES pre-election questionnaire
q_employ = QuestionMultipleChoice(
question_name="employ",
question_text="Which of the following best describes your current employment status?",
question_options=[
"Working full time now",
"Working part time now",
"Temporarily laid off",
"Unemployed",
"Retired",
"Permanently disabled",
"Taking care of home or family",
"Student",
"Other",
],
)
q_hadjob = QuestionYesNo(
question_name="hadjob",
question_text="At any time over the past five years, have you had a job?",
)
survey = Survey([q_employ, q_hadjob]).add_stop_rule(
q_employ,
"employ in ['Working full time now', 'Working part time now', 'Temporarily laid off']",
)
results = survey.by(agents).run()
results.select("employ", "hadjob").print(format="rich")
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓ ┃ answer ┃ answer ┃ ┃ .employ ┃ .hadjob ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩ │ Working full time now │ None │ ├───────────────────────┼─────────┤ │ Working full time now │ None │ ├───────────────────────┼─────────┤ │ Student │ Yes │ └───────────────────────┴─────────┘
Combining survey methods
Here we apply multiple methods at once: we add a memory of region to the prompt for inputzip, then pipe the answer to inputzip into the question text of votereg_f, and a stop rule if the answer to votereg is not “Yes” (i.e., do not administer votereg_f):
[26]:
from edsl import QuestionMultipleChoice, QuestionList, QuestionYesNo, Survey
# From the CES pre-election questionnaire
q_region = QuestionMultipleChoice(
question_name="region",
question_text="In which census region do you live?",
question_options=["Northeast", "Midwest", "South", "West"],
)
q_inputzip = QuestionList(
question_name="inputzip",
question_text="So that we can ask you about the news and events in your area, in what zip code do you currently reside?",
max_list_items=1,
)
q_votereg = QuestionMultipleChoice(
question_name="votereg",
question_text="Are you registered to vote?",
question_options=["Yes", "No", "Don't know"],
)
q_votereg_f = QuestionYesNo(
question_name="votereg_f",
question_text="Is {{ inputzip.answer[0] }} the zip code where you are registered to vote?",
)
survey = (
Survey([q_region, q_inputzip, q_votereg, q_votereg_f])
.add_targeted_memory(q_inputzip, q_region)
.add_stop_rule(q_votereg, "votereg != 'Yes'")
)
results = survey.by(agents).run()
results.select(
"region", "inputzip", "votereg", "votereg_f_user_prompt", "votereg_f"
).print(format="rich")
┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ answer ┃ answer ┃ answer ┃ prompt ┃ answer ┃ ┃ .region ┃ .inputzip ┃ .votereg ┃ .votereg_f_user_prompt ┃ .votereg_f ┃ ┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ South │ [30301] │ Yes │ │ No │ │ │ │ │ Is 30301 the zip code where you are registered to vote? │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ No │ │ │ │ │ │ │ │ │ │ │ │ Yes │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ Only 1 option may be selected. │ │ │ │ │ │ Please reponse with just your answer. │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ After the answer, you can put a comment explaining your │ │ │ │ │ │ reponse. │ │ ├───────────┼───────────┼──────────┼─────────────────────────────────────────────────────────────────┼────────────┤ │ South │ [30301] │ Yes │ │ No │ │ │ │ │ Is 30301 the zip code where you are registered to vote? │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ No │ │ │ │ │ │ │ │ │ │ │ │ Yes │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ Only 1 option may be selected. │ │ │ │ │ │ Please reponse with just your answer. │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ After the answer, you can put a comment explaining your │ │ │ │ │ │ reponse. │ │ ├───────────┼───────────┼──────────┼─────────────────────────────────────────────────────────────────┼────────────┤ │ Northeast │ [10001] │ Yes │ │ No │ │ │ │ │ Is 10001 the zip code where you are registered to vote? │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ No │ │ │ │ │ │ │ │ │ │ │ │ Yes │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ Only 1 option may be selected. │ │ │ │ │ │ Please reponse with just your answer. │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ │ After the answer, you can put a comment explaining your │ │ │ │ │ │ reponse. │ │ └───────────┴───────────┴──────────┴─────────────────────────────────────────────────────────────────┴────────────┘
See more details on all survey methods.
Parameterizing questions
We can create variations of questions using Scenario objects for content that we want to add to questions. This allows us to efficiently administer multiple versions of questions at once. We start by using a {{ parameter }} in a question:
[27]:
from edsl import QuestionMultipleChoice, Survey
# Modified from the CES pre-election questionnaire
q_votereg = QuestionMultipleChoice(
question_name="votereg",
question_text="Are you {{ status }}?",
question_options=["Yes", "No", "Don't know"],
)
survey = Survey([q_votereg])
Next we create a Scenario for each text that we want to insert in the question:
[28]:
from edsl import Scenario, ScenarioList
statuses = [
"registered to vote", # original CES question
"enrolled in school",
"employed full- or part-time",
"married or in a domestic partnership",
"licensed to drive",
]
scenarios = ScenarioList(Scenario({"status": s}) for s in statuses)
[29]:
results = survey.by(scenarios).by(agents).run()
[30]:
(
results.sort_by("status", "age")
.select("age", "education", "sex", "status", "votereg")
.print(format="rich")
)
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓ ┃ agent ┃ agent ┃ agent ┃ scenario ┃ answer ┃ ┃ .age ┃ .education ┃ .sex ┃ .status ┃ .votereg ┃ ┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩ │ 18-29 │ Some college/assoc. degree │ Female │ employed full- or part-time │ No │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ High school or less │ Female │ employed full- or part-time │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ College/graduate │ Female │ employed full- or part-time │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ Some college/assoc. degree │ Female │ enrolled in school │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ High school or less │ Female │ enrolled in school │ No │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ College/graduate │ Female │ enrolled in school │ No │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ Some college/assoc. degree │ Female │ licensed to drive │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ High school or less │ Female │ licensed to drive │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ College/graduate │ Female │ licensed to drive │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ Some college/assoc. degree │ Female │ married or in a domestic partnership │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ College/graduate │ Female │ married or in a domestic partnership │ No │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ High school or less │ Female │ married or in a domestic partnership │ No │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 18-29 │ Some college/assoc. degree │ Female │ registered to vote │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ College/graduate │ Female │ registered to vote │ Yes │ ├───────┼────────────────────────────┼────────┼──────────────────────────────────────┼──────────┤ │ 30-44 │ High school or less │ Female │ registered to vote │ Yes │ └───────┴────────────────────────────┴────────┴──────────────────────────────────────┴──────────┘