Google Form -> EDSL

EDSL is an open-source library for simulating surveys, experiments and other research with AI agents and large language models. Before running the code below, please ensure that you have installed the EDSL library and either activated remote inference from your Expected Parrot account or stored API keys for the language models that you want to use with EDSL. Please also see our documentation page for tips and tutorials on getting started using EDSL.

Designing the task as an EDSL survey

We design the task as an EDSL survey about the survey to be converted: a series of questions prompting a language model to read and reformat the contents of a given survey. The formatted responses of the language model are readily usable components of a new EDSL survey that can be administered to AI agents and/or human audiences.

Creating a meta-survey

We start by selecting appropriate question types for reformatting the contents of a given survey. EDSL comes with many common question types that we can choose from based on the form of the response that we want to get back from a model: multiple choice, checkbox, free text, linear scale, etc. Here we use QuestionList to return information about all the questions in the survey at once, as a list. We create a sequence of questions, using the response to one question as an input to the next question. This step-wise approach can improve performance by allowing a model to focus on distinct tasks, and also allow us to pinpoint modifications to instructions as needed (some models will perform better and need fewer instructions than others). Note that we use a {{ placeholder }} for the text of the survey to be reformatted that we want to add to the initial question, which allows us to reuse it with other content (e.g., another survey):

from edsl import QuestionList

First we ask the model to return just the questions from the survey:

q1 = QuestionList(
    question_name="q_text",
    question_text="""
    You are being asked to extract questions from the text of a survey.
    Read the text and then return a list of all the questions in the
    order that you find them. Return only the list of questions.
    Survey: {{ scenario.text }}
    """
)

Next we ask the model to format the questions as dictionaries, and specify the question text and type:

q2 = QuestionList(
    question_name="q_type",
    question_text="""
    Now create a dictionary for each question, using keys 'question_text' and 'question_type'.
    The value for 'question_text' is the question text you already identified.
    The value for 'question_type' should be the most appropriate of the following types:
    'multiple_choice', 'checkbox', 'linear_scale' or 'free_text'.
    Return only the list of dictionaries you have created, with the 2 key/value pairs for each question.
    """
)

Next we ask the model to add the question options (if any):

q3 = QuestionList(
    question_name="q_options",
    question_text="""
    Now add a key 'question_options' to each dictionary for all questions that are not free text,
    with a value that is a list of the answer options for the question.
    Preserve any integer options as integers, not strings.
    If there are labels for linear scale answer options then add another key 'option_labels'
    with a value that is a dictionary: the keys are the relevant integers and the values are the labels.
    Return only the list of dictionaries you have created with all relevant key/value pairs for each question.
    """
)

Finally, we ask the model to give each question a name:

q4 = QuestionList(
    question_name="q_name",
    question_text="""
    Now add a key 'question_name' to each dictionary.
    The value should be a unique short pythonic string.
    Return only the list of dictionaries that you have created,
    with all the key/value pairs for each question.
    """
)

Next we combine the questions into a Survey in order to administer them together. We add a “memory” of each prior question in the survey so that the model will have the context and its answers on hand when answering each successive question:

from edsl import Survey

survey = Survey(questions = [q1, q2, q3, q4]).set_full_memory_mode()

Adding content to questions

Next we create a Scenario object for the contents of a (non-EDSL) survey to be inserted in the first question. This allows us to reuse the questions with other content. Learn more about using scenarios to scale data labeling and other tasks. Here we create a scenario for a Google Form (a customer feedback survey) that we have stored as a publicly-accessible PDF at Expected Parrot. Code for posting a PDF to Expected Parrot (rerun with your own file):

from edsl import FileStore

if refresh := False:
    fs = FileStore("customer_feedback_survey.pdf") # file type is automatically inferred
    info = fs.push(
    description = "Example customer feedback survey",
    alias = "mock-survey",
    visibility = "public"
    )
    info

info = {'description': 'Example customer feedback survey',
 'object_type': 'scenario',
 'url': 'https://www.expectedparrot.com/content/d8bdddd2-7af9-49b1-86b7-51ebed2f71e1',
 'uuid': 'd8bdddd2-7af9-49b1-86b7-51ebed2f71e1',
 'version': '0.1.47.dev1',
 'visibility': 'public'}

Retrieving a file (replace with UUID of any desired object at Expected Parrot):

pdf_file = FileStore.pull(info["uuid"])

Creating a scenario for the content:

from edsl import Scenario

s = Scenario({"text":pdf_file.to_tempfile()})
s

Scenario

	key	value
0	text	/var/folders/j0/xq1nxxt51j7_1dgv8s116fmh0000gn/T/tmp5cqcqowt.pdf

Selecting language models

EDSL works with many popular language models that we can select to use in generating survey responses. You can provide your own API keys for models or activate remote inference to run surveys at the Expected Parrot server with any available models. Learn more about working with language models and using remote inference.

from edsl import ModelList, Model

A list of all available models can be viewed at the model pricing and performance page.

# Model.available()

Here we select several models to compare their responses:

models = ModelList(
    Model(m) for m in ["gemini-1.5-flash", "gpt-4o", "claude-3-5-sonnet-20240620"]
)

Running a survey

Next we add the scenario and models to the survey and run it. This generates a dataset of Results that we can access with built-in methods for analysis. Learn more about working with results.

results = survey.by(s).by(models).run()

To see a list of all the components of the results that have been generated:

results.columns

	0
0	agent.agent_index
1	agent.agent_instruction
2	agent.agent_name
3	answer.q_name
4	answer.q_options
5	answer.q_text
6	answer.q_type
7	cache_keys.q_name_cache_key
8	cache_keys.q_options_cache_key
9	cache_keys.q_text_cache_key
10	cache_keys.q_type_cache_key
11	cache_used.q_name_cache_used
12	cache_used.q_options_cache_used
13	cache_used.q_text_cache_used
14	cache_used.q_type_cache_used
15	comment.q_name_comment
16	comment.q_options_comment
17	comment.q_text_comment
18	comment.q_type_comment
19	generated_tokens.q_name_generated_tokens
20	generated_tokens.q_options_generated_tokens
21	generated_tokens.q_text_generated_tokens
22	generated_tokens.q_type_generated_tokens
23	iteration.iteration
24	model.frequency_penalty
25	model.inference_service
26	model.logprobs
27	model.maxOutputTokens
28	model.max_tokens
29	model.model
30	model.model_index
31	model.presence_penalty
32	model.stopSequences
33	model.temperature
34	model.topK
35	model.topP
36	model.top_logprobs
37	model.top_p
38	prompt.q_name_system_prompt
39	prompt.q_name_user_prompt
40	prompt.q_options_system_prompt
41	prompt.q_options_user_prompt
42	prompt.q_text_system_prompt
43	prompt.q_text_user_prompt
44	prompt.q_type_system_prompt
45	prompt.q_type_user_prompt
46	question_options.q_name_question_options
47	question_options.q_options_question_options
48	question_options.q_text_question_options
49	question_options.q_type_question_options
50	question_text.q_name_question_text
51	question_text.q_options_question_text
52	question_text.q_text_question_text
53	question_text.q_type_question_text
54	question_type.q_name_question_type
55	question_type.q_options_question_type
56	question_type.q_text_question_type
57	question_type.q_type_question_type
58	raw_model_response.q_name_cost
59	raw_model_response.q_name_one_usd_buys
60	raw_model_response.q_name_raw_model_response
61	raw_model_response.q_options_cost
62	raw_model_response.q_options_one_usd_buys
63	raw_model_response.q_options_raw_model_response
64	raw_model_response.q_text_cost
65	raw_model_response.q_text_one_usd_buys
66	raw_model_response.q_text_raw_model_response
67	raw_model_response.q_type_cost
68	raw_model_response.q_type_one_usd_buys
69	raw_model_response.q_type_raw_model_response
70	scenario.scenario_index
71	scenario.text

We can filter, sort, select and print components in a table:

(
    results
    .sort_by("model")
    .select("model", "q_name") #"q_text", "q_type", "q_options", "q_name")
)

	model.model	answer.q_name
0	claude-3-5-sonnet-20240620	[{'question_text': 'How many times have you visited this store in the past month?', 'question_type': 'multiple_choice', 'question_options': [0, 1, 2, 3, '4 or more'], 'question_name': 'visit_frequency'}, {'question_text': 'How satisfied were you with your overall shopping experience?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very Dissatisfied', '5': 'Very Satisfied'}, 'question_name': 'overall_satisfaction'}, {'question_text': 'How likely are you to recommend this store to a friend or family member?', 'question_type': 'linear_scale', 'question_options': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 'option_labels': {'0': 'Not at all likely', '10': 'Extremely likely'}, 'question_name': 'recommendation_likelihood'}, {'question_text': 'How would you rate the cleanliness of the store?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'store_cleanliness'}, {'question_text': 'How would you rate the friendliness of our staff?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'staff_friendliness'}, {'question_text': 'How would you rate the variety of products available?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'product_variety'}, {'question_text': 'How would you rate the prices of our products?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'product_prices'}, {'question_text': 'What is your age range?', 'question_type': 'multiple_choice', 'question_options': ['Under 18', '18-24', '25-34', '35-44', '45-54', '55-64', '65 or older'], 'question_name': 'age_range'}, {'question_text': 'What is your gender?', 'question_type': 'multiple_choice', 'question_options': ['Male', 'Female', 'Non-binary', 'Prefer not to say'], 'question_name': 'gender'}, {'question_text': 'Do you have any additional comments or suggestions for improvement?', 'question_type': 'free_text', 'question_name': 'additional_comments'}]
1	gemini-1.5-flash	[{'question_text': 'What is your age?', 'question_type': 'free_text', 'question_name': 'age'}, {'question_text': 'What is your gender?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'gender'}, {'question_text': 'What is your highest level of education?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'education'}, {'question_text': 'What is your annual household income?', 'question_type': 'free_text', 'question_name': 'income'}, {'question_text': 'How many people live in your household?', 'question_type': 'free_text', 'question_name': 'household_size'}, {'question_text': 'How many children live in your household?', 'question_type': 'free_text', 'question_name': 'children'}, {'question_text': 'What is your marital status?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'marital_status'}, {'question_text': 'What is your race/ethnicity?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'race'}, {'question_text': 'How would you rate your overall health?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Poor', '2': 'Fair', '3': 'Good', '4': 'Very Good', '5': 'Excellent'}, 'question_name': 'health'}, {'question_text': 'Do you have any chronic health conditions?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'chronic_conditions'}, {'question_text': 'If yes, please specify.', 'question_type': 'free_text', 'question_name': 'chronic_conditions_details'}, {'question_text': 'Do you have health insurance?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'health_insurance'}, {'question_text': 'If yes, what type of health insurance do you have?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'insurance_type'}, {'question_text': 'How often do you exercise?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'exercise'}, {'question_text': 'How many hours of sleep do you get per night?', 'question_type': 'free_text', 'question_name': 'sleep'}, {'question_text': 'How would you rate your stress level?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very Low', '2': 'Low', '3': 'Moderate', '4': 'High', '5': 'Very High'}, 'question_name': 'stress'}, {'question_text': 'Do you smoke?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'smoke'}, {'question_text': 'If yes, how many cigarettes do you smoke per day?', 'question_type': 'free_text', 'question_name': 'cigarettes_per_day'}, {'question_text': 'Do you drink alcohol?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'alcohol'}, {'question_text': 'If yes, how many drinks do you consume per week?', 'question_type': 'free_text', 'question_name': 'drinks_per_week'}, {'question_text': 'How would you rate your overall satisfaction with life?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very Dissatisfied', '2': 'Dissatisfied', '3': 'Neutral', '4': 'Satisfied', '5': 'Very Satisfied'}, 'question_name': 'life_satisfaction'}]
2	gpt-4o	[{'question_text': 'What is your age?', 'question_type': 'free_text', 'question_name': 'age'}, {'question_text': 'How satisfied are you with our service?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very dissatisfied', '5': 'Very satisfied'}, 'question_name': 'satisfaction_level'}, {'question_text': 'Would you recommend our service to others?', 'question_type': 'multiple_choice', 'question_options': ['Yes', 'No'], 'question_name': 'recommendation'}, {'question_text': 'What improvements would you like to see?', 'question_type': 'free_text', 'question_name': 'improvements'}, {'question_text': 'How often do you use our service?', 'question_type': 'multiple_choice', 'question_options': ['Daily', 'Weekly', 'Monthly', 'Rarely'], 'question_name': 'usage_frequency'}]

Creating a new EDSL survey

Now we can construct a new EDSL survey with the reformatted components of the original survey. This is done by creating Question objects with the question components, passing them to a new Survey, and then optionally designing and assigning AI agents to answer the survey. Here we select one of the model’s responses to use:

from edsl import Question

questions_list = results.filter("model.model == 'claude-3-5-sonnet-20240620'").select("q_name").to_list()[0]
questions_list

[{'question_text': 'How many times have you visited this store in the past month?',
  'question_type': 'multiple_choice',
  'question_options': [0, 1, 2, 3, '4 or more'],
  'question_name': 'visit_frequency'},
 {'question_text': 'How satisfied were you with your overall shopping experience?',
  'question_type': 'linear_scale',
  'question_options': [1, 2, 3, 4, 5],
  'option_labels': {'1': 'Very Dissatisfied', '5': 'Very Satisfied'},
  'question_name': 'overall_satisfaction'},
 {'question_text': 'How likely are you to recommend this store to a friend or family member?',
  'question_type': 'linear_scale',
  'question_options': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  'option_labels': {'0': 'Not at all likely', '10': 'Extremely likely'},
  'question_name': 'recommendation_likelihood'},
 {'question_text': 'How would you rate the cleanliness of the store?',
  'question_type': 'linear_scale',
  'question_options': [1, 2, 3, 4, 5],
  'question_name': 'store_cleanliness'},
 {'question_text': 'How would you rate the friendliness of our staff?',
  'question_type': 'linear_scale',
  'question_options': [1, 2, 3, 4, 5],
  'question_name': 'staff_friendliness'},
 {'question_text': 'How would you rate the variety of products available?',
  'question_type': 'linear_scale',
  'question_options': [1, 2, 3, 4, 5],
  'question_name': 'product_variety'},
 {'question_text': 'How would you rate the prices of our products?',
  'question_type': 'linear_scale',
  'question_options': [1, 2, 3, 4, 5],
  'question_name': 'product_prices'},
 {'question_text': 'What is your age range?',
  'question_type': 'multiple_choice',
  'question_options': ['Under 18',
   '18-24',
   '25-34',
   '35-44',
   '45-54',
   '55-64',
   '65 or older'],
  'question_name': 'age_range'},
 {'question_text': 'What is your gender?',
  'question_type': 'multiple_choice',
  'question_options': ['Male', 'Female', 'Non-binary', 'Prefer not to say'],
  'question_name': 'gender'},
 {'question_text': 'Do you have any additional comments or suggestions for improvement?',
  'question_type': 'free_text',
  'question_name': 'additional_comments'}]

edsl_questions = [Question(**q) for q in questions_list]
edsl_questions

[Question('multiple_choice', question_name = """visit_frequency""", question_text = """How many times have you visited this store in the past month?""", question_options = [0, 1, 2, 3, '4 or more']),
 Question('linear_scale', question_name = """overall_satisfaction""", question_text = """How satisfied were you with your overall shopping experience?""", question_options = [1, 2, 3, 4, 5], option_labels = {1: 'Very Dissatisfied', 5: 'Very Satisfied'}),
 Question('linear_scale', question_name = """recommendation_likelihood""", question_text = """How likely are you to recommend this store to a friend or family member?""", question_options = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], option_labels = {0: 'Not at all likely', 10: 'Extremely likely'}),
 Question('linear_scale', question_name = """store_cleanliness""", question_text = """How would you rate the cleanliness of the store?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
 Question('linear_scale', question_name = """staff_friendliness""", question_text = """How would you rate the friendliness of our staff?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
 Question('linear_scale', question_name = """product_variety""", question_text = """How would you rate the variety of products available?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
 Question('linear_scale', question_name = """product_prices""", question_text = """How would you rate the prices of our products?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
 Question('multiple_choice', question_name = """age_range""", question_text = """What is your age range?""", question_options = ['Under 18', '18-24', '25-34', '35-44', '45-54', '55-64', '65 or older']),
 Question('multiple_choice', question_name = """gender""", question_text = """What is your gender?""", question_options = ['Male', 'Female', 'Non-binary', 'Prefer not to say']),
 Question('free_text', question_name = """additional_comments""", question_text = """Do you have any additional comments or suggestions for improvement?""")]

new_survey = Survey(edsl_questions)

We can inspect the survey that has been created:

new_survey

Survey # questions: 10; question_name list: [‘visit_frequency’, ‘overall_satisfaction’, ‘recommendation_likelihood’, ‘store_cleanliness’, ‘staff_friendliness’, ‘product_variety’, ‘product_prices’, ‘age_range’, ‘gender’, ‘additional_comments’];

	question_type	question_name	question_options	question_text	option_labels
0	multiple_choice	visit_frequency	[0, 1, 2, 3, ‘4 or more’]	How many times have you visited this store in the past month?	nan
1	linear_scale	overall_satisfaction	[1, 2, 3, 4, 5]	How satisfied were you with your overall shopping experience?	`{1: 'Very Dissatisfied', 5: 'Very Satisfied'}`
2	linear_scale	recommendation_likelihood	[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]	How likely are you to recommend this store to a friend or family member?	`{0: 'Not at all likely', 10: 'Extremely likely'}`
3	linear_scale	store_cleanliness	[1, 2, 3, 4, 5]	How would you rate the cleanliness of the store?
4	linear_scale	staff_friendliness	[1, 2, 3, 4, 5]	How would you rate the friendliness of our staff?
5	linear_scale	product_variety	[1, 2, 3, 4, 5]	How would you rate the variety of products available?
6	linear_scale	product_prices	[1, 2, 3, 4, 5]	How would you rate the prices of our products?
7	multiple_choice	age_range	[‘Under 18’, ‘18-24’, ‘25-34’, ‘35-44’, ‘45-54’, ‘55-64’, ‘65 or older’]	What is your age range?	nan
8	multiple_choice	gender	[‘Male’, ‘Female’, ‘Non-binary’, ‘Prefer not to say’]	What is your gender?	nan
9	free_text	additional_comments	nan	Do you have any additional comments or suggestions for improvement?	nan

Designing AI agents

EDSL comes with methods for designing AI agent personas for language models to use in answering questions. An Agent is created by passing a dictionary of relevant traits. It can then be assigned to a survey using the by() method when the survey is run (the same as we do with scenarios and models). We can import existing data to create agents representing audiences of interest, or use EDSL to generate personas:

q_personas = QuestionList(
    question_name="personas",
    question_text="Draft 5 diverse personas for customers of a landscape business in New England capable of answering a feedback survey."
)

If we do not specify a model to use in running the question, the default model gpt-5.2 is used:

personas = q_personas.run().select("personas").to_list()[0]
personas

['Retired Couple in Suburbia',
 'Young Urban Professional',
 'Environmentally Conscious Family',
 'Small Business Owner',
 'Historic Home Enthusiast']

Note:Note that the personas can be (much) longer and include key/value pairs for any desired traits; we keep it simple here for demonstration purposes. Here we pass the personas to a list of agents and have them answer the survey:

from edsl import AgentList, Agent

agents = AgentList(
    Agent(
        traits = {"persona":p},
        instruction = """
        You are answering a customer feedback survey for a landscaping business that you have engaged in the past.
        Your answers are completely confidential.
        """
    )
    for p in personas
)

new_results = new_survey.by(agents).by(models).run()

(
    new_results
    .sort_by("model", "persona")
    .select("model", "persona", "answer.*")
)

	model.model	agent.persona	answer.additional_comments	answer.age_range	answer.product_variety	answer.recommendation_likelihood	answer.product_prices	answer.gender	answer.overall_satisfaction	answer.visit_frequency	answer.staff_friendliness	answer.store_cleanliness
0	claude-3-5-sonnet-20240620	Environmentally Conscious Family	As an environmentally conscious family, I really appreciated the eco-friendly approach of your landscaping services. However, I have a few suggestions that could further improve your offerings: 1. Native plant options: It would be great to see a wider selection of native plants that support local ecosystems and require less water and maintenance. 2. Organic pest control: I’d love more information on natural pest control methods you use or recommend, avoiding harmful chemicals. 3. Water conservation: Perhaps you could offer installation of rain barrels or smart irrigation systems to help reduce water usage. 4. Composting services: It would be fantastic if you could offer composting services or guidance on how to start a home compost system. 5. Sustainable materials: When possible, use recycled or sustainable materials for hardscaping projects. 6. Educational resources: Provide more information on sustainable gardening practices that we can implement ourselves. 7. Green waste disposal: Ensure all green waste is properly composted or recycled, not sent to landfills. Overall, we’ve been pleased with your services, but these additions would really align with our family’s environmental values and potentially attract more eco-conscious customers.	35-44	4	8	3	Female	4	1	4	4
1	claude-3-5-sonnet-20240620	Historic Home Enthusiast	As a historic home enthusiast, I really appreciate when landscaping companies understand the unique needs and aesthetics of period properties. I would suggest offering specialized services or packages tailored to historic homes, such as: 1. Heritage plant selections that are appropriate for different architectural eras 2. Expertise in maintaining and restoring historic garden layouts 3. Knowledge of traditional landscaping techniques and materials 4. Ability to work around delicate old structures and mature trees 5. Familiarity with local historic preservation guidelines It would also be great if the staff received some training on the history of landscape design to better serve clients with historic properties. Perhaps partnering with local historical societies or preservation groups could provide additional resources and credibility in this niche. Overall, I’ve been satisfied with the service, but these additions would really set the company apart for homeowners like myself who are passionate about maintaining the historical integrity of our properties.	55-64	4	8	4	Female	4	1	4	3
2	claude-3-5-sonnet-20240620	Retired Couple in Suburbia	As a retired couple living in suburbia, we’ve generally been pleased with the landscaping service. However, we do have a few suggestions that might improve things: 1. We’d appreciate if the crew could be a bit more mindful of noise levels, especially in the early morning hours. As retirees, we tend to enjoy our quiet mornings. 2. It would be helpful to have more flexibility in scheduling. Perhaps offering service time slots in the afternoon could be an option for those of us who aren’t on a typical work schedule. 3. We’ve noticed that some of the younger plants haven’t been thriving as well as we’d hoped. A bit more guidance on plant care or perhaps a follow-up visit to check on new plantings would be appreciated. 4. While we understand the importance of chemical treatments for lawn health, we’d be interested in exploring more eco-friendly options if available. 5. Lastly, it might be nice to offer some simple landscaping workshops for customers. As retirees, we have more time to tend to our garden and would love to learn more about maintaining it between your visits. Overall, we’re satisfied with the service, but these small improvements could really enhance our experience. Thank you for asking for our feedback!	65 or older	4	8	4	Female	4	1	5	3
3	claude-3-5-sonnet-20240620	Small Business Owner	As a small business owner myself, I appreciate the importance of customer feedback. Here are a few suggestions I’d offer: 1. Communication: Perhaps consider implementing a system for more regular updates on project progress, especially for longer jobs. This helps clients feel informed and engaged. 2. Flexibility: As a business owner, I know schedules can be tight. Having some flexibility in scheduling or offering early morning/late evening slots could be beneficial for clients with busy workdays. 3. Eco-friendly options: If not already offered, consider providing environmentally friendly landscaping choices. This could include native plant options or water-saving designs, which may appeal to environmentally conscious clients. 4. Follow-up service: A quick follow-up call or email a few weeks after job completion to ensure continued satisfaction could go a long way in building customer loyalty. 5. Referral program: If you don’t already have one, consider implementing a referral program. As a business owner, I’m always happy to recommend good services to my network, especially if there’s an incentive. Overall, I’ve been satisfied with the service, but these small tweaks could potentially enhance the customer experience and help grow your business further.	45-54	4	8	3	Male	4	1	4	3
4	claude-3-5-sonnet-20240620	Young Urban Professional	As a young urban professional, I might respond: I appreciate the quality of work your team does, but I think there’s room for improvement in a few areas: 1. Digital presence: It would be great if you had a more user-friendly website or app where I could easily schedule services, view my account, and make payments. 2. Eco-friendly options: I’d love to see more sustainable landscaping choices offered, like native plant installations or water-saving irrigation systems. 3. Flexible scheduling: As someone with a busy work schedule, it would be helpful to have more flexible appointment times, including early mornings, evenings, or weekends. 4. Modern design options: Consider offering more contemporary landscape design choices that appeal to younger homeowners and complement urban environments. 5. Communication: While your team does good work, sometimes I’m not sure exactly when they’ll arrive. A text notification system would be really convenient. Overall, I’m satisfied with your services, but implementing some of these suggestions could really set you apart and appeal more to clients like myself. Thanks for asking for feedback!	25-34	4	8	4	Female	4	1	4	4
5	gemini-1.5-flash	Environmentally Conscious Family	Yes, I do. Overall, we were very pleased with the results of the landscaping work. However, we’d love to see you expand your selection of native plants and drought-tolerant options. As an environmentally conscious family, we’re trying to reduce our water usage and support local ecosystems. Offering more choices in this area would be a huge plus and would align with the growing interest in sustainable landscaping practices. Perhaps you could offer a consultation service that helps clients choose the best native plants for their specific needs and soil conditions.	35-44	4	8	4	Prefer not to say	4	0	5	5
6	gemini-1.5-flash	Historic Home Enthusiast	While I was extremely pleased with the overall quality of the work, particularly the sensitive restoration of the original stonework around my historic home’s foundation, I do have a couple of suggestions. First, perhaps a more detailed pre-project consultation outlining the specific types of materials to be used, including sourcing information for historically accurate replacements, would be beneficial. Knowing the provenance of the materials used would have enhanced my appreciation of the craftsmanship. Secondly, a slightly more comprehensive post-project walkthrough highlighting maintenance recommendations specific to the materials used and the age of the property would be a welcome addition. This would ensure the longevity of the beautiful work you’ve completed.	45-54	4	8	4	Prefer not to say	4	0	5	5
7	gemini-1.5-flash	Retired Couple in Suburbia	While we were very pleased with the overall outcome of the landscaping project, a few minor suggestions might enhance the customer experience. Firstly, perhaps a more detailed, perhaps even illustrated, pre-project plan could be provided. We found ourselves needing to clarify a few points during the process, and a more comprehensive initial plan might have preempted those questions. Secondly, a slightly more proactive communication schedule – perhaps a brief check-in midway through the project – would have been reassuring. We understand you’re busy, but a quick update would have been appreciated. Finally, a small suggestion: offering a brief post-project care guide, perhaps outlining basic watering and maintenance tips for the newly installed plants, would be a thoughtful touch. Overall, we were very happy with the work, and these are just minor suggestions for potential improvements.	65 or older	4	8	3	Prefer not to say	4	0	5	5
8	gemini-1.5-flash	Small Business Owner	Overall, I was pleased with the work. However, as a small business owner myself, I’m always looking for ways to improve efficiency and communication. Perhaps offering a more detailed, itemized quote upfront, including potential material costs and timelines, would be beneficial. Sometimes, unexpected costs can arise, and having a clearer understanding from the outset would help manage expectations and budgeting. Also, a quick follow-up email after the job is complete, confirming satisfaction and perhaps including a link to online review platforms, would be a nice touch. Small things like that can make a big difference in customer experience.	35-44	4	8	3	Prefer not to say	4	1	5	5
9	gemini-1.5-flash	Young Urban Professional	Overall, I was happy with the work. My only suggestion would be to offer more detailed, perhaps even visual, options upfront for plant selection. I felt a little overwhelmed by the sheer number of choices and ended up relying heavily on your expertise, which was great, but a curated selection of popular options with photos would have made the initial consultation smoother and more efficient for both of us. Something like a digital portfolio or online catalog would be fantastic.	25-34	4	8	3	Prefer not to say	4	0	5	5
10	gpt-4o	Environmentally Conscious Family	We really appreciate the efforts your team makes to incorporate sustainable practices. It would be great if you could expand your selection of native plants and offer more organic lawn care options. Additionally, providing information on how your services contribute to local biodiversity could be a nice touch. Thank you for your commitment to environmentally friendly landscaping!	35-44	4	9	3	Prefer not to say	5	0	5	5
11	gpt-4o	Historic Home Enthusiast	I was really pleased with the landscaping work done on my historic home. The team was respectful of the property’s unique characteristics and managed to enhance its charm while maintaining its original feel. One suggestion would be to perhaps offer more guidance on plant selections that are historically appropriate for different architectural styles. This could be a great addition for those of us who want to stay true to our home’s period. Overall, I was very satisfied with the service!	45-54	4	8	4	Prefer not to say	5	0	5	5
12	gpt-4o	Retired Couple in Suburbia	We were really pleased with the landscaping service and how our yard looks now. The team was professional and timely, and they really understood our vision for a low-maintenance garden. One suggestion might be to offer a seasonal check-up service to help us keep everything looking its best throughout the year. It could be a great option for retirees like us who want to enjoy our garden without too much effort. Overall, we’re very satisfied and would definitely recommend your services to our neighbors.	65 or older	4	9	4	Prefer not to say	5	0	5	5
13	gpt-4o	Small Business Owner	I appreciate the quality of work and professionalism your team consistently demonstrates. One suggestion for improvement would be to offer more flexible scheduling options, as it can sometimes be challenging to coordinate with my business hours. Additionally, providing a detailed maintenance guide or tips for seasonal care could be beneficial for small business owners like myself who want to keep the landscape looking its best year-round. Overall, I’m very satisfied with your services.	35-44	4	8	4	Prefer not to say	5	0	5	5
14	gpt-4o	Young Urban Professional	Overall, I was quite satisfied with the service. The team was professional and efficient, and my outdoor space looks great. One suggestion might be to offer a digital platform where clients can easily schedule services, view past appointments, and perhaps even see design mockups or ideas. As someone who is always on the go, having everything accessible online would be incredibly convenient. Keep up the great work!	25-34	4	8	3	Prefer not to say	5	0	5	5

Posting to Expected Parrot

Expected Parrot is a platform for creating, storing and sharing LLM-based research. It is fully integrated with EDSL and accessible from your workspace or Expected Parrot account page. Learn more about creating an account and the platform. Here we demonstrate how to post this notebook:

from edsl import Notebook

nb = Notebook(path = "google_form_to_edsl.ipynb")

if refresh := False:
    nb.push(
        description = "Example code for using EDSL to convert a non-EDSL survey into EDSL",
        alias = "google-form-to-edsl-notebook",
        visibility = "public"
    )
else:
    nb.patch('815c1988-8656-4644-8ab8-70ce36b8d62d', value = nb)

Introduction

Getting Started

Core Concepts

Getting Data

Working with Results

Validating with Humans

No-code Apps

Platform

How-to Guides

Notebook Examples

Developers

Designing the task as an EDSL survey

Creating a meta-survey

Adding content to questions

Selecting language models

Running a survey

Creating a new EDSL survey

Designing AI agents

Posting to Expected Parrot

​Designing the task as an EDSL survey

​Creating a meta-survey

​Adding content to questions

​Selecting language models

​Running a survey

​Creating a new EDSL survey

​Designing AI agents

​Posting to Expected Parrot

Designing the task as an EDSL survey

Creating a meta-survey

Adding content to questions

Selecting language models

Running a survey

Creating a new EDSL survey

Designing AI agents

Posting to Expected Parrot