Google Form -> EDSL
This notebook provides example EDSL code for converting a non-EDSL survey into in an EDSL survey. This can be useful for accessing EDSL’s built-in methods for analyzing survey data, and extending it with responses simulated with AI agents and diverse large language models.
EDSL is an open-source library for simulating surveys, experiments and other research with AI agents and large language models. Before running the code below, please ensure that you have installed the EDSL library and either activated remote inference from your Coop account or stored API keys for the language models that you want to use with EDSL. Please also see our documentation page for tips and tutorials on getting started using EDSL.
Designing the task as an EDSL survey
We design the task as an EDSL survey about the survey to be converted: a series of questions prompting a language model to read and reformat the contents of a given survey. The formatted responses of the language model are readily usable components of a new EDSL survey that can be administered to AI agents and/or human audiences.
Creating a meta-survey
We start by selecting appropriate question types for reformatting the contents of a given survey. EDSL comes with many common question types that we can choose from based on the form of the response that we want to get back from a model: multiple choice, checkbox, free text, linear scale, etc.
Here we use QuestionList to return information about all the questions in the survey at once, as a list. We create a sequence of questions, using the response to one question as an input to the next question. This step-wise approach can improve performance by allowing a model to focus on distinct tasks, and also allow us to pinpoint modifications to instructions as needed (some models will perform better and need fewer instructions than others). Note that we use a {{ placeholder }} for
the text of the survey to be reformatted that we want to add to the initial question, which allows us to reuse it with other content (e.g., another survey):
[1]:
from edsl import QuestionList
First we ask the model to return just the questions from the survey:
[2]:
q1 = QuestionList(
question_name="q_text",
question_text="""
You are being asked to extract questions from the text of a survey.
Read the text and then return a list of all the questions in the
order that you find them. Return only the list of questions.
Survey: {{ scenario.text }}
"""
)
Next we ask the model to format the questions as dictionaries, and specify the question text and type:
[3]:
q2 = QuestionList(
question_name="q_type",
question_text="""
Now create a dictionary for each question, using keys 'question_text' and 'question_type'.
The value for 'question_text' is the question text you already identified.
The value for 'question_type' should be the most appropriate of the following types:
'multiple_choice', 'checkbox', 'linear_scale' or 'free_text'.
Return only the list of dictionaries you have created, with the 2 key/value pairs for each question.
"""
)
Next we ask the model to add the question options (if any):
[4]:
q3 = QuestionList(
question_name="q_options",
question_text="""
Now add a key 'question_options' to each dictionary for all questions that are not free text,
with a value that is a list of the answer options for the question.
Preserve any integer options as integers, not strings.
If there are labels for linear scale answer options then add another key 'option_labels'
with a value that is a dictionary: the keys are the relevant integers and the values are the labels.
Return only the list of dictionaries you have created with all relevant key/value pairs for each question.
"""
)
Finally, we ask the model to give each question a name:
[5]:
q4 = QuestionList(
question_name="q_name",
question_text="""
Now add a key 'question_name' to each dictionary.
The value should be a unique short pythonic string.
Return only the list of dictionaries that you have created,
with all the key/value pairs for each question.
"""
)
Next we combine the questions into a Survey in order to administer them together. We add a “memory” of each prior question in the survey so that the model will have the context and its answers on hand when answering each successive question:
[6]:
from edsl import Survey
[7]:
survey = Survey(questions = [q1, q2, q3, q4]).set_full_memory_mode()
Adding content to questions
Next we create a Scenario object for the contents of a (non-EDSL) survey to be inserted in the first question. This allows us to reuse the questions with other content. Learn more about using scenarios to scale data labeling and other tasks.
Here we create a scenario for a Google Form (a customer feedback survey) that we have stored as a publicly-accessible PDF at Coop.
Code for posting a PDF to the Coop (rerun with your own file):
[8]:
from edsl import FileStore
fs = FileStore("customer_feedback_survey.pdf") # file type is automatically inferred
info = fs.push(
description = "Example customer feedback survey",
alias = "mock-survey",
visibility = "public"
)
info
[8]:
{'description': 'Example customer feedback survey',
'object_type': 'scenario',
'url': 'https://www.expectedparrot.com/content/d8bdddd2-7af9-49b1-86b7-51ebed2f71e1',
'uuid': 'd8bdddd2-7af9-49b1-86b7-51ebed2f71e1',
'version': '0.1.47.dev1',
'visibility': 'public'}
Retrieving a file (replace with UUID of any desired object at Coop):
[9]:
pdf_file = FileStore.pull(info["uuid"])
Creating a scenario for the content:
[10]:
from edsl import Scenario
[11]:
s = Scenario({"text":pdf_file.to_tempfile()})
s
[11]:
| key | value | |
|---|---|---|
| 0 | text | /var/folders/j0/xq1nxxt51j7_1dgv8s116fmh0000gn/T/tmp5cqcqowt.pdf |
Selecting language models
EDSL works with many popular language models that we can select to use in generating survey responses. You can provide your own API keys for models or activate remote inference to run surveys at the Expected Parrot server with any available models. Learn more about working with language models and using remote inference.
[12]:
from edsl import ModelList, Model
A list of all available models can be viewed at the model pricing and performance page.
[13]:
# Model.available()
Here we select several models to compare their responses:
[14]:
models = ModelList(
Model(m) for m in ["gemini-1.5-flash", "gpt-4o", "claude-3-5-sonnet-20240620"]
)
Running a survey
Next we add the scenario and models to the survey and run it. This generates a dataset of Results that we can access with built-in methods for analysis. Learn more about working with results.
[15]:
results = survey.by(s).by(models).run()
▼
Job Status (2025-03-03 10:15:22)
| Job UUID | 3863fbb2-4bca-4480-be93-9972fbb13e48 |
| Progress Bar URL | https://www.expectedparrot.com/home/remote-job-progress/3863fbb2-4bca-4480-be93-9972fbb13e48 |
| Exceptions Report URL | None |
| Results UUID | 0d31b57d-be32-471a-a0ec-96a072eee19b |
| Results URL | https://www.expectedparrot.com/content/0d31b57d-be32-471a-a0ec-96a072eee19b |
✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/0d31b57d-be32-471a-a0ec-96a072eee19b
To see a list of all the components of the results that have been generated:
[16]:
results.columns
[16]:
| 0 | |
|---|---|
| 0 | agent.agent_index |
| 1 | agent.agent_instruction |
| 2 | agent.agent_name |
| 3 | answer.q_name |
| 4 | answer.q_options |
| 5 | answer.q_text |
| 6 | answer.q_type |
| 7 | cache_keys.q_name_cache_key |
| 8 | cache_keys.q_options_cache_key |
| 9 | cache_keys.q_text_cache_key |
| 10 | cache_keys.q_type_cache_key |
| 11 | cache_used.q_name_cache_used |
| 12 | cache_used.q_options_cache_used |
| 13 | cache_used.q_text_cache_used |
| 14 | cache_used.q_type_cache_used |
| 15 | comment.q_name_comment |
| 16 | comment.q_options_comment |
| 17 | comment.q_text_comment |
| 18 | comment.q_type_comment |
| 19 | generated_tokens.q_name_generated_tokens |
| 20 | generated_tokens.q_options_generated_tokens |
| 21 | generated_tokens.q_text_generated_tokens |
| 22 | generated_tokens.q_type_generated_tokens |
| 23 | iteration.iteration |
| 24 | model.frequency_penalty |
| 25 | model.inference_service |
| 26 | model.logprobs |
| 27 | model.maxOutputTokens |
| 28 | model.max_tokens |
| 29 | model.model |
| 30 | model.model_index |
| 31 | model.presence_penalty |
| 32 | model.stopSequences |
| 33 | model.temperature |
| 34 | model.topK |
| 35 | model.topP |
| 36 | model.top_logprobs |
| 37 | model.top_p |
| 38 | prompt.q_name_system_prompt |
| 39 | prompt.q_name_user_prompt |
| 40 | prompt.q_options_system_prompt |
| 41 | prompt.q_options_user_prompt |
| 42 | prompt.q_text_system_prompt |
| 43 | prompt.q_text_user_prompt |
| 44 | prompt.q_type_system_prompt |
| 45 | prompt.q_type_user_prompt |
| 46 | question_options.q_name_question_options |
| 47 | question_options.q_options_question_options |
| 48 | question_options.q_text_question_options |
| 49 | question_options.q_type_question_options |
| 50 | question_text.q_name_question_text |
| 51 | question_text.q_options_question_text |
| 52 | question_text.q_text_question_text |
| 53 | question_text.q_type_question_text |
| 54 | question_type.q_name_question_type |
| 55 | question_type.q_options_question_type |
| 56 | question_type.q_text_question_type |
| 57 | question_type.q_type_question_type |
| 58 | raw_model_response.q_name_cost |
| 59 | raw_model_response.q_name_one_usd_buys |
| 60 | raw_model_response.q_name_raw_model_response |
| 61 | raw_model_response.q_options_cost |
| 62 | raw_model_response.q_options_one_usd_buys |
| 63 | raw_model_response.q_options_raw_model_response |
| 64 | raw_model_response.q_text_cost |
| 65 | raw_model_response.q_text_one_usd_buys |
| 66 | raw_model_response.q_text_raw_model_response |
| 67 | raw_model_response.q_type_cost |
| 68 | raw_model_response.q_type_one_usd_buys |
| 69 | raw_model_response.q_type_raw_model_response |
| 70 | scenario.scenario_index |
| 71 | scenario.text |
We can filter, sort, select and print components in a table:
[17]:
(
results
.sort_by("model")
.select("model", "q_name") #"q_text", "q_type", "q_options", "q_name")
)
[17]:
| model.model | answer.q_name | |
|---|---|---|
| 0 | claude-3-5-sonnet-20240620 | [{'question_text': 'How many times have you visited this store in the past month?', 'question_type': 'multiple_choice', 'question_options': [0, 1, 2, 3, '4 or more'], 'question_name': 'visit_frequency'}, {'question_text': 'How satisfied were you with your overall shopping experience?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very Dissatisfied', '5': 'Very Satisfied'}, 'question_name': 'overall_satisfaction'}, {'question_text': 'How likely are you to recommend this store to a friend or family member?', 'question_type': 'linear_scale', 'question_options': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 'option_labels': {'0': 'Not at all likely', '10': 'Extremely likely'}, 'question_name': 'recommendation_likelihood'}, {'question_text': 'How would you rate the cleanliness of the store?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'store_cleanliness'}, {'question_text': 'How would you rate the friendliness of our staff?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'staff_friendliness'}, {'question_text': 'How would you rate the variety of products available?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'product_variety'}, {'question_text': 'How would you rate the prices of our products?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'question_name': 'product_prices'}, {'question_text': 'What is your age range?', 'question_type': 'multiple_choice', 'question_options': ['Under 18', '18-24', '25-34', '35-44', '45-54', '55-64', '65 or older'], 'question_name': 'age_range'}, {'question_text': 'What is your gender?', 'question_type': 'multiple_choice', 'question_options': ['Male', 'Female', 'Non-binary', 'Prefer not to say'], 'question_name': 'gender'}, {'question_text': 'Do you have any additional comments or suggestions for improvement?', 'question_type': 'free_text', 'question_name': 'additional_comments'}] |
| 1 | gemini-1.5-flash | [{'question_text': 'What is your age?', 'question_type': 'free_text', 'question_name': 'age'}, {'question_text': 'What is your gender?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'gender'}, {'question_text': 'What is your highest level of education?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'education'}, {'question_text': 'What is your annual household income?', 'question_type': 'free_text', 'question_name': 'income'}, {'question_text': 'How many people live in your household?', 'question_type': 'free_text', 'question_name': 'household_size'}, {'question_text': 'How many children live in your household?', 'question_type': 'free_text', 'question_name': 'children'}, {'question_text': 'What is your marital status?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'marital_status'}, {'question_text': 'What is your race/ethnicity?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'race'}, {'question_text': 'How would you rate your overall health?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Poor', '2': 'Fair', '3': 'Good', '4': 'Very Good', '5': 'Excellent'}, 'question_name': 'health'}, {'question_text': 'Do you have any chronic health conditions?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'chronic_conditions'}, {'question_text': 'If yes, please specify.', 'question_type': 'free_text', 'question_name': 'chronic_conditions_details'}, {'question_text': 'Do you have health insurance?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'health_insurance'}, {'question_text': 'If yes, what type of health insurance do you have?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'insurance_type'}, {'question_text': 'How often do you exercise?', 'question_type': 'multiple_choice', 'question_options': [], 'question_name': 'exercise'}, {'question_text': 'How many hours of sleep do you get per night?', 'question_type': 'free_text', 'question_name': 'sleep'}, {'question_text': 'How would you rate your stress level?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very Low', '2': 'Low', '3': 'Moderate', '4': 'High', '5': 'Very High'}, 'question_name': 'stress'}, {'question_text': 'Do you smoke?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'smoke'}, {'question_text': 'If yes, how many cigarettes do you smoke per day?', 'question_type': 'free_text', 'question_name': 'cigarettes_per_day'}, {'question_text': 'Do you drink alcohol?', 'question_type': 'multiple_choice', 'question_options': [True, False], 'question_name': 'alcohol'}, {'question_text': 'If yes, how many drinks do you consume per week?', 'question_type': 'free_text', 'question_name': 'drinks_per_week'}, {'question_text': 'How would you rate your overall satisfaction with life?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very Dissatisfied', '2': 'Dissatisfied', '3': 'Neutral', '4': 'Satisfied', '5': 'Very Satisfied'}, 'question_name': 'life_satisfaction'}] |
| 2 | gpt-4o | [{'question_text': 'What is your age?', 'question_type': 'free_text', 'question_name': 'age'}, {'question_text': 'How satisfied are you with our service?', 'question_type': 'linear_scale', 'question_options': [1, 2, 3, 4, 5], 'option_labels': {'1': 'Very dissatisfied', '5': 'Very satisfied'}, 'question_name': 'satisfaction_level'}, {'question_text': 'Would you recommend our service to others?', 'question_type': 'multiple_choice', 'question_options': ['Yes', 'No'], 'question_name': 'recommendation'}, {'question_text': 'What improvements would you like to see?', 'question_type': 'free_text', 'question_name': 'improvements'}, {'question_text': 'How often do you use our service?', 'question_type': 'multiple_choice', 'question_options': ['Daily', 'Weekly', 'Monthly', 'Rarely'], 'question_name': 'usage_frequency'}] |
Creating a new EDSL survey
Now we can construct a new EDSL survey with the reformatted components of the original survey. This is done by creating Question objects with the question components, passing them to a new Survey, and then optionally designing and assigning AI agents to answer the survey.
Here we select one of the model’s responses to use:
[18]:
from edsl import Question
[19]:
questions_list = results.filter("model.model == 'claude-3-5-sonnet-20240620'").select("q_name").to_list()[0]
questions_list
[19]:
[{'question_text': 'How many times have you visited this store in the past month?',
'question_type': 'multiple_choice',
'question_options': [0, 1, 2, 3, '4 or more'],
'question_name': 'visit_frequency'},
{'question_text': 'How satisfied were you with your overall shopping experience?',
'question_type': 'linear_scale',
'question_options': [1, 2, 3, 4, 5],
'option_labels': {'1': 'Very Dissatisfied', '5': 'Very Satisfied'},
'question_name': 'overall_satisfaction'},
{'question_text': 'How likely are you to recommend this store to a friend or family member?',
'question_type': 'linear_scale',
'question_options': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
'option_labels': {'0': 'Not at all likely', '10': 'Extremely likely'},
'question_name': 'recommendation_likelihood'},
{'question_text': 'How would you rate the cleanliness of the store?',
'question_type': 'linear_scale',
'question_options': [1, 2, 3, 4, 5],
'question_name': 'store_cleanliness'},
{'question_text': 'How would you rate the friendliness of our staff?',
'question_type': 'linear_scale',
'question_options': [1, 2, 3, 4, 5],
'question_name': 'staff_friendliness'},
{'question_text': 'How would you rate the variety of products available?',
'question_type': 'linear_scale',
'question_options': [1, 2, 3, 4, 5],
'question_name': 'product_variety'},
{'question_text': 'How would you rate the prices of our products?',
'question_type': 'linear_scale',
'question_options': [1, 2, 3, 4, 5],
'question_name': 'product_prices'},
{'question_text': 'What is your age range?',
'question_type': 'multiple_choice',
'question_options': ['Under 18',
'18-24',
'25-34',
'35-44',
'45-54',
'55-64',
'65 or older'],
'question_name': 'age_range'},
{'question_text': 'What is your gender?',
'question_type': 'multiple_choice',
'question_options': ['Male', 'Female', 'Non-binary', 'Prefer not to say'],
'question_name': 'gender'},
{'question_text': 'Do you have any additional comments or suggestions for improvement?',
'question_type': 'free_text',
'question_name': 'additional_comments'}]
[20]:
edsl_questions = [Question(**q) for q in questions_list]
edsl_questions
[20]:
[Question('multiple_choice', question_name = """visit_frequency""", question_text = """How many times have you visited this store in the past month?""", question_options = [0, 1, 2, 3, '4 or more']),
Question('linear_scale', question_name = """overall_satisfaction""", question_text = """How satisfied were you with your overall shopping experience?""", question_options = [1, 2, 3, 4, 5], option_labels = {1: 'Very Dissatisfied', 5: 'Very Satisfied'}),
Question('linear_scale', question_name = """recommendation_likelihood""", question_text = """How likely are you to recommend this store to a friend or family member?""", question_options = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10], option_labels = {0: 'Not at all likely', 10: 'Extremely likely'}),
Question('linear_scale', question_name = """store_cleanliness""", question_text = """How would you rate the cleanliness of the store?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
Question('linear_scale', question_name = """staff_friendliness""", question_text = """How would you rate the friendliness of our staff?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
Question('linear_scale', question_name = """product_variety""", question_text = """How would you rate the variety of products available?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
Question('linear_scale', question_name = """product_prices""", question_text = """How would you rate the prices of our products?""", question_options = [1, 2, 3, 4, 5], option_labels = {}),
Question('multiple_choice', question_name = """age_range""", question_text = """What is your age range?""", question_options = ['Under 18', '18-24', '25-34', '35-44', '45-54', '55-64', '65 or older']),
Question('multiple_choice', question_name = """gender""", question_text = """What is your gender?""", question_options = ['Male', 'Female', 'Non-binary', 'Prefer not to say']),
Question('free_text', question_name = """additional_comments""", question_text = """Do you have any additional comments or suggestions for improvement?""")]
[21]:
new_survey = Survey(edsl_questions)
We can inspect the survey that has been created:
[22]:
new_survey
[22]:
Survey # questions: 10; question_name list: ['visit_frequency', 'overall_satisfaction', 'recommendation_likelihood', 'store_cleanliness', 'staff_friendliness', 'product_variety', 'product_prices', 'age_range', 'gender', 'additional_comments'];
| question_type | question_name | question_options | question_text | option_labels | |
|---|---|---|---|---|---|
| 0 | multiple_choice | visit_frequency | [0, 1, 2, 3, '4 or more'] | How many times have you visited this store in the past month? | nan |
| 1 | linear_scale | overall_satisfaction | [1, 2, 3, 4, 5] | How satisfied were you with your overall shopping experience? | {1: 'Very Dissatisfied', 5: 'Very Satisfied'} |
| 2 | linear_scale | recommendation_likelihood | [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10] | How likely are you to recommend this store to a friend or family member? | {0: 'Not at all likely', 10: 'Extremely likely'} |
| 3 | linear_scale | store_cleanliness | [1, 2, 3, 4, 5] | How would you rate the cleanliness of the store? | {} |
| 4 | linear_scale | staff_friendliness | [1, 2, 3, 4, 5] | How would you rate the friendliness of our staff? | {} |
| 5 | linear_scale | product_variety | [1, 2, 3, 4, 5] | How would you rate the variety of products available? | {} |
| 6 | linear_scale | product_prices | [1, 2, 3, 4, 5] | How would you rate the prices of our products? | {} |
| 7 | multiple_choice | age_range | ['Under 18', '18-24', '25-34', '35-44', '45-54', '55-64', '65 or older'] | What is your age range? | nan |
| 8 | multiple_choice | gender | ['Male', 'Female', 'Non-binary', 'Prefer not to say'] | What is your gender? | nan |
| 9 | free_text | additional_comments | nan | Do you have any additional comments or suggestions for improvement? | nan |
Designing AI agents
EDSL comes with methods for designing AI agent personas for language models to use in answering questions. An Agent is created by passing a dictionary of relevant traits. It can then be assigned to a survey using the by() method when the survey is run (the same as we do with scenarios and models).
We can import existing data to create agents representing audiences of interest, or use EDSL to generate personas:
[23]:
q_personas = QuestionList(
question_name="personas",
question_text="Draft 5 diverse personas for customers of a landscape business in New England capable of answering a feedback survey."
)
If we do not specify a model to use in running the question, the default model GPT 4 preview is used:
[24]:
personas = q_personas.run().select("personas").to_list()[0]
personas
▼
Job Status (2025-03-03 10:15:36)
| Job UUID | 3e142b8c-3a74-4d47-afc4-a0af18d93468 |
| Progress Bar URL | https://www.expectedparrot.com/home/remote-job-progress/3e142b8c-3a74-4d47-afc4-a0af18d93468 |
| Exceptions Report URL | None |
| Results UUID | 74e2860f-4b01-48cc-b4a4-08de112c5918 |
| Results URL | https://www.expectedparrot.com/content/74e2860f-4b01-48cc-b4a4-08de112c5918 |
✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/74e2860f-4b01-48cc-b4a4-08de112c5918
[24]:
['Retired Couple in Suburbia',
'Young Urban Professional',
'Environmentally Conscious Family',
'Small Business Owner',
'Historic Home Enthusiast']
Note that the personas can be (much) longer and include key/value pairs for any desired traits; we keep it simple here for demonstration purposes. Here we pass the personas to a list of agents and have them answer the survey:
[25]:
from edsl import AgentList, Agent
[26]:
agents = AgentList(
Agent(
traits = {"persona":p},
instruction = """
You are answering a customer feedback survey for a landscaping business that you have engaged in the past.
Your answers are completely confidential.
"""
)
for p in personas
)
[27]:
new_results = new_survey.by(agents).by(models).run()
▼
Job Status (2025-03-03 10:16:16)
| Job UUID | 75e5c169-e906-401c-b421-89461a3afddc |
| Progress Bar URL | https://www.expectedparrot.com/home/remote-job-progress/75e5c169-e906-401c-b421-89461a3afddc |
| Exceptions Report URL | None |
| Results UUID | 6d8e728a-97e9-4dd5-ba95-1168535899da |
| Results URL | https://www.expectedparrot.com/content/6d8e728a-97e9-4dd5-ba95-1168535899da |
✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/6d8e728a-97e9-4dd5-ba95-1168535899da
[28]:
(
new_results
.sort_by("model", "persona")
.select("model", "persona", "answer.*")
)
[28]:
| model.model | agent.persona | answer.additional_comments | answer.age_range | answer.product_variety | answer.recommendation_likelihood | answer.product_prices | answer.gender | answer.overall_satisfaction | answer.visit_frequency | answer.staff_friendliness | answer.store_cleanliness | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | claude-3-5-sonnet-20240620 | Environmentally Conscious Family | As an environmentally conscious family, I really appreciated the eco-friendly approach of your landscaping services. However, I have a few suggestions that could further improve your offerings: 1. Native plant options: It would be great to see a wider selection of native plants that support local ecosystems and require less water and maintenance. 2. Organic pest control: I'd love more information on natural pest control methods you use or recommend, avoiding harmful chemicals. 3. Water conservation: Perhaps you could offer installation of rain barrels or smart irrigation systems to help reduce water usage. 4. Composting services: It would be fantastic if you could offer composting services or guidance on how to start a home compost system. 5. Sustainable materials: When possible, use recycled or sustainable materials for hardscaping projects. 6. Educational resources: Provide more information on sustainable gardening practices that we can implement ourselves. 7. Green waste disposal: Ensure all green waste is properly composted or recycled, not sent to landfills. Overall, we've been pleased with your services, but these additions would really align with our family's environmental values and potentially attract more eco-conscious customers. | 35-44 | 4 | 8 | 3 | Female | 4 | 1 | 4 | 4 |
| 1 | claude-3-5-sonnet-20240620 | Historic Home Enthusiast | As a historic home enthusiast, I really appreciate when landscaping companies understand the unique needs and aesthetics of period properties. I would suggest offering specialized services or packages tailored to historic homes, such as: 1. Heritage plant selections that are appropriate for different architectural eras 2. Expertise in maintaining and restoring historic garden layouts 3. Knowledge of traditional landscaping techniques and materials 4. Ability to work around delicate old structures and mature trees 5. Familiarity with local historic preservation guidelines It would also be great if the staff received some training on the history of landscape design to better serve clients with historic properties. Perhaps partnering with local historical societies or preservation groups could provide additional resources and credibility in this niche. Overall, I've been satisfied with the service, but these additions would really set the company apart for homeowners like myself who are passionate about maintaining the historical integrity of our properties. | 55-64 | 4 | 8 | 4 | Female | 4 | 1 | 4 | 3 |
| 2 | claude-3-5-sonnet-20240620 | Retired Couple in Suburbia | As a retired couple living in suburbia, we've generally been pleased with the landscaping service. However, we do have a few suggestions that might improve things: 1. We'd appreciate if the crew could be a bit more mindful of noise levels, especially in the early morning hours. As retirees, we tend to enjoy our quiet mornings. 2. It would be helpful to have more flexibility in scheduling. Perhaps offering service time slots in the afternoon could be an option for those of us who aren't on a typical work schedule. 3. We've noticed that some of the younger plants haven't been thriving as well as we'd hoped. A bit more guidance on plant care or perhaps a follow-up visit to check on new plantings would be appreciated. 4. While we understand the importance of chemical treatments for lawn health, we'd be interested in exploring more eco-friendly options if available. 5. Lastly, it might be nice to offer some simple landscaping workshops for customers. As retirees, we have more time to tend to our garden and would love to learn more about maintaining it between your visits. Overall, we're satisfied with the service, but these small improvements could really enhance our experience. Thank you for asking for our feedback! | 65 or older | 4 | 8 | 4 | Female | 4 | 1 | 5 | 3 |
| 3 | claude-3-5-sonnet-20240620 | Small Business Owner | As a small business owner myself, I appreciate the importance of customer feedback. Here are a few suggestions I'd offer: 1. Communication: Perhaps consider implementing a system for more regular updates on project progress, especially for longer jobs. This helps clients feel informed and engaged. 2. Flexibility: As a business owner, I know schedules can be tight. Having some flexibility in scheduling or offering early morning/late evening slots could be beneficial for clients with busy workdays. 3. Eco-friendly options: If not already offered, consider providing environmentally friendly landscaping choices. This could include native plant options or water-saving designs, which may appeal to environmentally conscious clients. 4. Follow-up service: A quick follow-up call or email a few weeks after job completion to ensure continued satisfaction could go a long way in building customer loyalty. 5. Referral program: If you don't already have one, consider implementing a referral program. As a business owner, I'm always happy to recommend good services to my network, especially if there's an incentive. Overall, I've been satisfied with the service, but these small tweaks could potentially enhance the customer experience and help grow your business further. | 45-54 | 4 | 8 | 3 | Male | 4 | 1 | 4 | 3 |
| 4 | claude-3-5-sonnet-20240620 | Young Urban Professional | As a young urban professional, I might respond: I appreciate the quality of work your team does, but I think there's room for improvement in a few areas: 1. Digital presence: It would be great if you had a more user-friendly website or app where I could easily schedule services, view my account, and make payments. 2. Eco-friendly options: I'd love to see more sustainable landscaping choices offered, like native plant installations or water-saving irrigation systems. 3. Flexible scheduling: As someone with a busy work schedule, it would be helpful to have more flexible appointment times, including early mornings, evenings, or weekends. 4. Modern design options: Consider offering more contemporary landscape design choices that appeal to younger homeowners and complement urban environments. 5. Communication: While your team does good work, sometimes I'm not sure exactly when they'll arrive. A text notification system would be really convenient. Overall, I'm satisfied with your services, but implementing some of these suggestions could really set you apart and appeal more to clients like myself. Thanks for asking for feedback! | 25-34 | 4 | 8 | 4 | Female | 4 | 1 | 4 | 4 |
| 5 | gemini-1.5-flash | Environmentally Conscious Family | Yes, I do. Overall, we were very pleased with the results of the landscaping work. However, we'd love to see you expand your selection of native plants and drought-tolerant options. As an environmentally conscious family, we're trying to reduce our water usage and support local ecosystems. Offering more choices in this area would be a huge plus and would align with the growing interest in sustainable landscaping practices. Perhaps you could offer a consultation service that helps clients choose the best native plants for their specific needs and soil conditions. | 35-44 | 4 | 8 | 4 | Prefer not to say | 4 | 0 | 5 | 5 |
| 6 | gemini-1.5-flash | Historic Home Enthusiast | While I was extremely pleased with the overall quality of the work, particularly the sensitive restoration of the original stonework around my historic home's foundation, I do have a couple of suggestions. First, perhaps a more detailed pre-project consultation outlining the specific types of materials to be used, including sourcing information for historically accurate replacements, would be beneficial. Knowing the provenance of the materials used would have enhanced my appreciation of the craftsmanship. Secondly, a slightly more comprehensive post-project walkthrough highlighting maintenance recommendations specific to the materials used and the age of the property would be a welcome addition. This would ensure the longevity of the beautiful work you've completed. | 45-54 | 4 | 8 | 4 | Prefer not to say | 4 | 0 | 5 | 5 |
| 7 | gemini-1.5-flash | Retired Couple in Suburbia | While we were very pleased with the overall outcome of the landscaping project, a few minor suggestions might enhance the customer experience. Firstly, perhaps a more detailed, perhaps even illustrated, pre-project plan could be provided. We found ourselves needing to clarify a few points during the process, and a more comprehensive initial plan might have preempted those questions. Secondly, a slightly more proactive communication schedule – perhaps a brief check-in midway through the project – would have been reassuring. We understand you're busy, but a quick update would have been appreciated. Finally, a small suggestion: offering a brief post-project care guide, perhaps outlining basic watering and maintenance tips for the newly installed plants, would be a thoughtful touch. Overall, we were very happy with the work, and these are just minor suggestions for potential improvements. | 65 or older | 4 | 8 | 3 | Prefer not to say | 4 | 0 | 5 | 5 |
| 8 | gemini-1.5-flash | Small Business Owner | Overall, I was pleased with the work. However, as a small business owner myself, I'm always looking for ways to improve efficiency and communication. Perhaps offering a more detailed, itemized quote upfront, including potential material costs and timelines, would be beneficial. Sometimes, unexpected costs can arise, and having a clearer understanding from the outset would help manage expectations and budgeting. Also, a quick follow-up email after the job is complete, confirming satisfaction and perhaps including a link to online review platforms, would be a nice touch. Small things like that can make a big difference in customer experience. | 35-44 | 4 | 8 | 3 | Prefer not to say | 4 | 1 | 5 | 5 |
| 9 | gemini-1.5-flash | Young Urban Professional | Overall, I was happy with the work. My only suggestion would be to offer more detailed, perhaps even visual, options upfront for plant selection. I felt a little overwhelmed by the sheer number of choices and ended up relying heavily on your expertise, which was great, but a curated selection of popular options with photos would have made the initial consultation smoother and more efficient for both of us. Something like a digital portfolio or online catalog would be fantastic. | 25-34 | 4 | 8 | 3 | Prefer not to say | 4 | 0 | 5 | 5 |
| 10 | gpt-4o | Environmentally Conscious Family | We really appreciate the efforts your team makes to incorporate sustainable practices. It would be great if you could expand your selection of native plants and offer more organic lawn care options. Additionally, providing information on how your services contribute to local biodiversity could be a nice touch. Thank you for your commitment to environmentally friendly landscaping! | 35-44 | 4 | 9 | 3 | Prefer not to say | 5 | 0 | 5 | 5 |
| 11 | gpt-4o | Historic Home Enthusiast | I was really pleased with the landscaping work done on my historic home. The team was respectful of the property's unique characteristics and managed to enhance its charm while maintaining its original feel. One suggestion would be to perhaps offer more guidance on plant selections that are historically appropriate for different architectural styles. This could be a great addition for those of us who want to stay true to our home's period. Overall, I was very satisfied with the service! | 45-54 | 4 | 8 | 4 | Prefer not to say | 5 | 0 | 5 | 5 |
| 12 | gpt-4o | Retired Couple in Suburbia | We were really pleased with the landscaping service and how our yard looks now. The team was professional and timely, and they really understood our vision for a low-maintenance garden. One suggestion might be to offer a seasonal check-up service to help us keep everything looking its best throughout the year. It could be a great option for retirees like us who want to enjoy our garden without too much effort. Overall, we're very satisfied and would definitely recommend your services to our neighbors. | 65 or older | 4 | 9 | 4 | Prefer not to say | 5 | 0 | 5 | 5 |
| 13 | gpt-4o | Small Business Owner | I appreciate the quality of work and professionalism your team consistently demonstrates. One suggestion for improvement would be to offer more flexible scheduling options, as it can sometimes be challenging to coordinate with my business hours. Additionally, providing a detailed maintenance guide or tips for seasonal care could be beneficial for small business owners like myself who want to keep the landscape looking its best year-round. Overall, I'm very satisfied with your services. | 35-44 | 4 | 8 | 4 | Prefer not to say | 5 | 0 | 5 | 5 |
| 14 | gpt-4o | Young Urban Professional | Overall, I was quite satisfied with the service. The team was professional and efficient, and my outdoor space looks great. One suggestion might be to offer a digital platform where clients can easily schedule services, view past appointments, and perhaps even see design mockups or ideas. As someone who is always on the go, having everything accessible online would be incredibly convenient. Keep up the great work! | 25-34 | 4 | 8 | 3 | Prefer not to say | 5 | 0 | 5 | 5 |
Posting to the Coop
Coop is a platform for creating, storing and sharing LLM-based research. It is fully integrated with EDSL and accessible from your workspace or Coop account page. Learn more about creating an account and using the Coop.
Here we demonstrate how to post this notebook:
[30]:
from edsl import Notebook
nb = Notebook(path = "google_form_to_edsl.ipynb")
if refresh := False:
nb.push(
description = "Example code for using EDSL to convert a non-EDSL survey into EDSL",
alias = "google-form-to-edsl-notebook",
visibility = "public"
)
else:
nb.patch('815c1988-8656-4644-8ab8-70ce36b8d62d', value = nb)