Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.expectedparrot.com/llms.txt

Use this file to discover all available pages before exploring further.

In the first example below we construct a survey of questions and then add a rule to skip one question based on the response to another question. In the second example we add some complexity. We first create different “scenarios” (versions) of questions and combine them in a survey. Then we add multiple rules to skip specific versions of the questions based on responses to a particular version of a question. EDSL is an open-source library for simulating surveys, experiments and other research with AI agents and large language models. Before running the code below, please ensure that you have installed the EDSL library and either activated remote inference from your Expected Parrot account or stored API keys for the language models that you want to use with EDSL. Please also see our documentation page for tips and tutorials on getting started using EDSL.

​
Example 1

In the first example below we construct questions, combine them in a survey, and add a rule to skip the second question based on the response to the first question. Then we create Scenario objects for contents that will be added to the questions when the survey is run. The effect of this is that the second question will be skipped based on the response to the first question for each individual scenario. We start by constructing questions: 
from edsl import QuestionYesNo, QuestionNumerical, QuestionMultipleChoice

q1 = QuestionYesNo(
    question_name = "recent_purchase",
    question_text = "In the last year have you or anyone in your household purchased any `{{ scenario.item }}`?",
)

q2 = QuestionNumerical(
    question_name = "amount",
    question_text = "In the last year, how much did your household spend on `{{ scenario.item }}` (in USD)?"
)

q3 = QuestionMultipleChoice(
    question_name = "next_purchase",
    question_text = "When do you next expect to purchase `{{ scenario.item }}`?",
    question_options = [
        "Never",
        "Within the next month",
        "Within the next year",
        "I do not know"
    ]
)
We combine the questions in a survey to administer them together: 
from edsl import Survey

survey = Survey(questions = [q1, q2, q3])
survey
Survey # questions: 3; question_name list: [‘recent_purchase’, ‘amount’, ‘next_purchase’];
question_optionsquestion_namequestion_typequestion_text
0[‘No’, ‘Yes’]recent_purchaseyes_noIn the last year have you or anyone in your household purchased any {{ scenario.item }}?
1nanamountnumericalIn the last year, how much did your household spend on {{ scenario.item }} (in USD)?
2[‘Never’, ‘Within the next month’, ‘Within the next year’, ‘I do not know’]next_purchasemultiple_choiceWhen do you next expect to purchase {{ scenario.item }}?
Here we add a rule to skip q2 based on the response to q1: 
survey = survey.add_skip_rule(q2, "`{{ recent_purchase.answer }}` == 'No'")
Next we create scenarios for the “item” to be used with each question: 
from edsl import Scenario, ScenarioList

s = ScenarioList(
    Scenario({"item":item}) for item in ["electronics", "phones"]
)
Note:Note that we could also use a method for the data type that we are using–this is equivalent:
s = ScenarioList.from_list("item", ["electronics", "phones"])
s

ScenarioList scenarios: 2; keys: ['item'];
item
0electronics
1phones
We can inspect the flow of the survey that has been created with the scenarios that we’re using: 
survey.by(s).show_flow()
../_images/notebooks_skip_logic_scenarios_13_0.png
Next we create some agent personas to answer the questions: 
from edsl import Agent, AgentList

income_levels = ["under $100,000", "$100,000-250,000", "above $250,000"]
ages = [30, 50, 70]

a = AgentList(
    Agent({"annual_income":income, "age":age}) for income in income_levels for age in ages
)
a

AgentList agents: 9;
annual_incomeage
0under $100,00030
1under $100,00050
2under $100,00070
3$100,000-250,00030
4$100,000-250,00050
5$100,000-250,00070
6above $250,00030
7above $250,00050
8above $250,00070
Next we select a model to generate the responses (check available models and pricing): 
from edsl import Model

m = Model("gemini-1.5-flash")
We can inspect (or modify) the default parameters of the model that will be used: 
m

gemini-1.5-flash
keyvalue
0modelgemini-1.5-flash
1parameters:temperature0.500000
2parameters:topP1
3parameters:topK1
4parameters:maxOutputTokens2048
5parameters:stopSequences[]
6inference_servicegoogle
We run the survey by adding any scenarios, agents and models and then calling the run: 
results = survey.by(s).by(a).by(m).run()
We can inspect a list of the columns of the dataset of results that has been generated: 
results.columns
0
0agent.age
1agent.agent_index
2agent.agent_instruction
3agent.agent_name
4agent.annual_income
5answer.amount
6answer.next_purchase
7answer.recent_purchase
8cache_keys.amount_cache_key
9cache_keys.next_purchase_cache_key
10cache_keys.recent_purchase_cache_key
11cache_used.amount_cache_used
12cache_used.next_purchase_cache_used
13cache_used.recent_purchase_cache_used
14comment.amount_comment
15comment.next_purchase_comment
16comment.recent_purchase_comment
17generated_tokens.amount_generated_tokens
18generated_tokens.next_purchase_generated_tokens
19generated_tokens.recent_purchase_generated_tokens
20iteration.iteration
21model.inference_service
22model.maxOutputTokens
23model.model
24model.model_index
25model.stopSequences
26model.temperature
27model.topK
28model.topP
29prompt.amount_system_prompt
30prompt.amount_user_prompt
31prompt.next_purchase_system_prompt
32prompt.next_purchase_user_prompt
33prompt.recent_purchase_system_prompt
34prompt.recent_purchase_user_prompt
35question_options.amount_question_options
36question_options.next_purchase_question_options
37question_options.recent_purchase_question_options
38question_text.amount_question_text
39question_text.next_purchase_question_text
40question_text.recent_purchase_question_text
41question_type.amount_question_type
42question_type.next_purchase_question_type
43question_type.recent_purchase_question_type
44raw_model_response.amount_cost
45raw_model_response.amount_input_price_per_million_tokens
46raw_model_response.amount_input_tokens
47raw_model_response.amount_one_usd_buys
48raw_model_response.amount_output_price_per_million_tokens
49raw_model_response.amount_output_tokens
50raw_model_response.amount_raw_model_response
51raw_model_response.next_purchase_cost
52raw_model_response.next_purchase_input_price_per_million_tokens
53raw_model_response.next_purchase_input_tokens
54raw_model_response.next_purchase_one_usd_buys
55raw_model_response.next_purchase_output_price_per_million_tokens
56raw_model_response.next_purchase_output_tokens
57raw_model_response.next_purchase_raw_model_response
58raw_model_response.recent_purchase_cost
59raw_model_response.recent_purchase_input_price_per_million_tokens
60raw_model_response.recent_purchase_input_tokens
61raw_model_response.recent_purchase_one_usd_buys
62raw_model_response.recent_purchase_output_price_per_million_tokens
63raw_model_response.recent_purchase_output_tokens
64raw_model_response.recent_purchase_raw_model_response
65reasoning_summary.amount_reasoning_summary
66reasoning_summary.next_purchase_reasoning_summary
67reasoning_summary.recent_purchase_reasoning_summary
68scenario.item
69scenario.scenario_index
We can select and inspect any components of the results. We can see by a “None” response that a question was skipped: 
(
    results
    .sort_by("annual_income", "age", "item")
    .select("model", "annual_income", "age", "item", "recent_purchase", "amount", "next_purchase")
)
model.modelagent.annual_incomeagent.agescenario.itemanswer.recent_purchaseanswer.amountanswer.next_purchase
0gemini-1.5-flash$100,000-250,00030electronicsYes2500.000000Within the next year
1gemini-1.5-flash$100,000-250,00030phonesYes1200.000000Within the next year
2gemini-1.5-flash$100,000-250,00050electronicsYes2500.000000Within the next year
3gemini-1.5-flash$100,000-250,00050phonesYes1200.000000Within the next year
4gemini-1.5-flash$100,000-250,00070electronicsYes500.000000Within the next year
5gemini-1.5-flash$100,000-250,00070phonesNonanWithin the next year
6gemini-1.5-flashabove $250,00030electronicsYes5000.000000Within the next year
7gemini-1.5-flashabove $250,00030phonesYes0.000000Within the next year
8gemini-1.5-flashabove $250,00050electronicsYes5000.000000Within the next year
9gemini-1.5-flashabove $250,00050phonesYes0.000000Within the next year
10gemini-1.5-flashabove $250,00070electronicsYes5000.000000Within the next year
11gemini-1.5-flashabove $250,00070phonesNonanNever
12gemini-1.5-flashunder $100,00030electronicsYes250.000000Within the next year
13gemini-1.5-flashunder $100,00030phonesNonanWithin the next year
14gemini-1.5-flashunder $100,00050electronicsNonanWithin the next year
15gemini-1.5-flashunder $100,00050phonesNonanWithin the next year
16gemini-1.5-flashunder $100,00070electronicsNonanWithin the next year
17gemini-1.5-flashunder $100,00070phonesNonanNever

​
Example 2

In the next example, we use the same scenarios to create versions of the questions before we combine them in a survey. This allows us to add a skip rule based on a question/scenario combination, as opposed to skipping a question for all scenarios: 
q1 = QuestionYesNo(
    question_name = "recent_purchase_`{{ scenario.item }}`",
    question_text = "In the last year have you or anyone in your household purchased any `{{ scenario.item }}`?",
)

q2 = QuestionNumerical(
    question_name = "amount_`{{ item }}`",
    question_text = "In the last year, how much did your household spend on `{{ scenario.item }}` (in USD)?"
)

q3 = QuestionMultipleChoice(
    question_name = "next_purchase_`{{ scenario.item }}`",
    question_text = "When do you next expect to purchase `{{ scenario.item }}`?",
    question_options = [
        "Never",
        "Within the next month",
        "Within the next year",
        "I do not know"
    ]
)
The loop method creates new versions of questions with scenarios already inserted: 
questions = q1.loop(s) + q2.loop(s) + q3.loop(s)
questions

[Question('yes_no', question_name = """recent_purchase_electronics""", question_text = """In the last year have you or anyone in your household purchased any electronics?""", question_options = ['No', 'Yes']),
 Question('yes_no', question_name = """recent_purchase_phones""", question_text = """In the last year have you or anyone in your household purchased any phones?""", question_options = ['No', 'Yes']),
 Question('numerical', question_name = """amount_electronics""", question_text = """In the last year, how much did your household spend on electronics (in USD)?""", min_value = None, max_value = None),
 Question('numerical', question_name = """amount_phones""", question_text = """In the last year, how much did your household spend on phones (in USD)?""", min_value = None, max_value = None),
 Question('multiple_choice', question_name = """next_purchase_electronics""", question_text = """When do you next expect to purchase electronics?""", question_options = ['Never', 'Within the next month', 'Within the next year', 'I do not know']),
 Question('multiple_choice', question_name = """next_purchase_phones""", question_text = """When do you next expect to purchase phones?""", question_options = ['Never', 'Within the next month', 'Within the next year', 'I do not know'])]
We combine the questions in a survey to administer them together the same as before: 
survey = Survey(questions)
Here we add different rules specifying that questions with one scenario (phones) should be administered or skipped based on the answer to a question with another scenario (electronics): 
survey = (
    survey
    .add_skip_rule("recent_purchase_phones", "`{{ recent_purchase_electronics.answer }}` == 'No'")
    .add_skip_rule("amount_phones", "`{{ recent_purchase_electronics.answer }}` == 'No'")
    .add_skip_rule("next_purchase_phones", "`{{ recent_purchase_electronics.answer }}` == 'No'")
)

survey.show_flow()
../_images/notebooks_skip_logic_scenarios_34_0.png
Here we run the survey with the scenarios, agents and model: 
results = survey.by(a).by(m).run()
There is no “scenario” field in results because the scenarios were already added to questions. Instead, there are separate columns for each version of a question: 
(
    results
    .sort_by("annual_income", "age")
    .select("model", "annual_income", "age", "recent_purchase_electronics", "amount_electronics", "next_purchase_electronics", "recent_purchase_phones", "amount_phones", "next_purchase_phones")
)
model.modelagent.annual_incomeagent.ageanswer.recent_purchase_electronicsanswer.amount_electronicsanswer.next_purchase_electronicsanswer.recent_purchase_phonesanswer.amount_phonesanswer.next_purchase_phones
0gemini-1.5-flash$100,000-250,00030Yes2500Within the next yearYes1200.000000Within the next year
1gemini-1.5-flash$100,000-250,00050Yes2500Within the next yearYes1200.000000Within the next year
2gemini-1.5-flash$100,000-250,00070Yes500Within the next yearNo0.000000Within the next year
3gemini-1.5-flashabove $250,00030Yes5000Within the next yearYes0.000000Within the next year
4gemini-1.5-flashabove $250,00050Yes5000Within the next yearYes0.000000Within the next year
5gemini-1.5-flashabove $250,00070Yes5000Within the next yearNo0.000000Never
6gemini-1.5-flashunder $100,00030Yes250Within the next yearNo300.000000Within the next year
7gemini-1.5-flashunder $100,00050No250Within the next yearnannannan
8gemini-1.5-flashunder $100,00070No0Within the next yearnannannan

​
Posting to Expected Parrot

Here we post this notebook to Expected Parrot, a free platform for creating and sharing AI-based research (learn more about how it works): 
from edsl import Notebook

nb = Notebook(path = "skip_logic_scenarios.ipynb")

nb.push(
    description = "Using skip logic with question scenarios",
    alias = "skip-logic-scenarios",
    visibility = "public"
)

{'description': 'Using skip logic with question scenarios',
 'object_type': 'notebook',
 'url': 'https://www.expectedparrot.com/content/d91d29a1-607e-4a98-a962-a662c7f94399',
 'alias_url': 'https://www.expectedparrot.com/content/RobinHorton/skip-logic-scenarios',
 'uuid': 'd91d29a1-607e-4a98-a962-a662c7f94399',
 'version': '0.1.62.dev1',
 'visibility': 'public'}