Using EDSL to sense check data
This notebook provides example code for sense checking survey data using EDSL, an open-source library for simulating surveys, experiments and market research with AI agents and large language models.
Contents
Using a set of responses to a survey about online marketplaces as an example, we demonstrate EDSL methods for:
Evaluating survey questions (e.g., for clarity and improvements)
Analyzing each respondent’s set of answers (e.g., to summarize or identify sentiment, themes, etc.)
Reviewing each answer individually (e.g., to evaluate its relevance or usefulness)
Coop
We also show how to post EDSL questions, surveys, results and notebooks (like this one) to the Coop: a new platform for creating and sharing LLM-based research.
How EDSL works
EDSL is a flexible library that can be used to perform a broad variety of research tasks. A typical workflow consists of the following steps:
Construct questions in EDSL
Add data to the questions (e.g., for data labeling tasks)
Use an AI agent to answer the questions
Select a language model to generate the answers
Analyze results in a formatted dataset
Technical setup
Before running the code below please ensure that you have completed setup:
Install the EDSL library.
Create a Coop account and activate remote inference OR store your own API Keys for language models that you want to use.
Our Starter Tutorial provides examples of EDSL basic components.
Example data
Our example data is a CSV consisting of several questions and a few rows of responses. We have stored it at the Coop and can re-import it:
[1]:
from edsl.scenarios.FileStore import CSVFileStore
[2]:
csv_file = CSVFileStore.pull('2e3c4292-8fdb-4d17-9eea-858dadf1e42d', expected_parrot_url='https://www.expectedparrot.com')
[3]:
# Code for uploading a CSV to the Coop:
# refresh = False
# if refresh:
# from edsl.scenarios.FileStore import CSVFileStore
# fs = CSVFileStore("marketplace_survey_results.csv")
# info = fs.push()
# print(info)
Creating questions about the data
There are many questions we might want to ask about the data, such as:
Does this survey question have any logical or syntactical problems? {{ question }}
What is the overall sentiment of this respondent’s answers? {{ responses }}
Is this answer responsive to the question that was asked? {{ question }} {{ answer }}
Question types
EDSL comes with many common question types that we can select from based on the form of the response that we want to get back from the model: multiple choice, checkbox, linear scale, free text, etc. Learn more about EDSL question types.
Here we construct Question objects for the questions that we want to ask about the data, using {{ placeholders }} for the information that we will add to the questions in the steps that follow:
[4]:
from edsl import QuestionFreeText, QuestionMultipleChoice, QuestionYesNo
[5]:
q_logic = QuestionFreeText(
question_name = "logic",
question_text = "Describe any logical or syntactical problems in the following survey question: {{ question }}"
)
[6]:
q_sentiment = QuestionMultipleChoice(
question_name = "sentiment",
question_text = "What is the overall sentiment of this respondent's survey answers? {{ responses }}",
question_options = ["Very unsatisfied", "Somewhat unsatisfied", "Somewhat satisfied", "Very satisfied"]
)
[7]:
q_responsive = QuestionYesNo(
question_name = "responsive",
question_text = "Is this answer responsive to the question that was asked? Question: {{ question }} Answer: {{ answer }}"
)
Adding survey data to the questions
Next we’ll add our data to our questions. This can be done efficiently by creating a ScenarioList representing the data. The individual Scenario objects in the list can be constructed in a variety of ways depending on the information that we want to include in a particular question.
We start by calling the from_csv() method to create a ScenarioList for the data in its original form. We can see that this generates a Scenario dictionary for each respondent’s set of answers with key/value pairs for the individual questions and answers:
[8]:
from edsl import ScenarioList
[9]:
sl = ScenarioList.from_csv(csv_file.to_tempfile()) # replace with CSV file name if importing a local file
sl
[9]:
{
"scenarios": [
{
"Respondent ID": "101",
"What do you like most about using our online marketplace?": "The wide variety of products and the ease of use.",
"What is one feature you would like to see added to improve your shopping experience?": "It would be great to have a personalized recommendation system based on my browsing history.",
"Can you describe a recent experience where you were dissatisfied with our service?": "I was disappointed when an item I ordered arrived damaged, but customer service quickly resolved it.",
"How do you feel about the current product search and filtering options?": "The search and filtering options are intuitive and work well for me.",
"Is there anything else you would like to share about your experience with us?": "No, keep up the great work!"
},
{
"Respondent ID": "102",
"What do you like most about using our online marketplace?": "I enjoy the simplicity of the interface.",
"What is one feature you would like to see added to improve your shopping experience?": "A feature that helps compare similar products side by side would be useful.",
"Can you describe a recent experience where you were dissatisfied with our service?": "No complaints here.",
"How do you feel about the current product search and filtering options?": "I find the product search to be pretty effective.",
"Is there anything else you would like to share about your experience with us?": "I think the sky is a beautiful shade of purple today."
},
{
"Respondent ID": "103",
"What do you like most about using our online marketplace?": "The platform is user-friendly and offers a vast selection of products.",
"What is one feature you would like to see added to improve your shopping experience?": "Would love to see an option to save and compare different products.",
"Can you describe a recent experience where you were dissatisfied with our service?": "My delivery was late by a few days, which was frustrating.",
"How do you feel about the current product search and filtering options?": "It\u2019s okay.",
"Is there anything else you would like to share about your experience with us?": "No."
}
]
}
Evaluating the questions
For our first question we want to create a Scenario for each survey question:
[10]:
from edsl import QuestionFreeText, Survey
q_logic = QuestionFreeText(
question_name = "logic",
question_text = "Describe any logical or syntactical problems in the following survey question: {{ question }}"
)
q_improved = QuestionFreeText(
question_name = "improved",
question_text = "Please draft an improved version of the survey question. Return only the revised question text."
)
survey = Survey([q_logic, q_improved]).add_targeted_memory(q_improved, q_logic)
The survey questions are the parameters of the ScenarioList created above:
[11]:
questions = list(sl.parameters)
questions
[11]:
['How do you feel about the current product search and filtering options?',
'What do you like most about using our online marketplace?',
'Can you describe a recent experience where you were dissatisfied with our service?',
'Is there anything else you would like to share about your experience with us?',
'What is one feature you would like to see added to improve your shopping experience?',
'Respondent ID']
We can pass them to the from_list() method to create a new ScenarioList, specifying that the key for each Scenario will be question in order to match the parameter of our logic question:
[12]:
sl_questions = ScenarioList.from_list("question", questions)
sl_questions
[12]:
{
"scenarios": [
{
"question": "How do you feel about the current product search and filtering options?"
},
{
"question": "What do you like most about using our online marketplace?"
},
{
"question": "Can you describe a recent experience where you were dissatisfied with our service?"
},
{
"question": "Is there anything else you would like to share about your experience with us?"
},
{
"question": "What is one feature you would like to see added to improve your shopping experience?"
},
{
"question": "Respondent ID"
}
]
}
We add the scenarios to the survey when we run it:
[13]:
results = survey.by(sl_questions).run()
This generates a dataset of Results that we can access with built-in methods for analysis:
[14]:
results.select("question", "logic", "improved").print(format="rich")
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ scenario ┃ answer ┃ answer ┃ ┃ .question ┃ .logic ┃ .improved ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ Respondent ID │ The survey question "Respondent ID" │ Please enter your Respondent ID as │ │ │ has several issues: │ provided in your survey invitation │ │ │ │ email. │ │ │ 1. **Lack of clarity**: The phrase │ │ │ │ "Respondent ID" is vague and does │ │ │ │ not clearly indicate what is being │ │ │ │ asked of the respondent. It is not │ │ │ │ framed as a question, so │ │ │ │ respondents might be confused about │ │ │ │ what they need to do. │ │ │ │ │ │ │ │ 2. **No context**: There is no │ │ │ │ explanation of what the "Respondent │ │ │ │ ID" is or where the respondent │ │ │ │ should find it. If the ID is │ │ │ │ something they need to provide, the │ │ │ │ survey should give clear │ │ │ │ instructions on how to locate it. │ │ │ │ │ │ │ │ 3. **Assumption of prior │ │ │ │ knowledge**: The question assumes │ │ │ │ that respondents know their ID │ │ │ │ without providing any guidance. If │ │ │ │ this is an internal survey where │ │ │ │ respondents are expected to know │ │ │ │ their IDs, it might be okay, but │ │ │ │ otherwise, it needs more context. │ │ ├─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┤ │ What is one feature you would like │ The survey question "What is one │ What is one specific improvement │ │ to see added to improve your │ feature you would like to see added │ you would like to see in our online │ │ shopping experience? │ to improve your shopping │ or in-store shopping experience? │ │ │ experience?" is generally │ │ │ │ well-constructed, but there are a │ │ │ │ few points that could be improved │ │ │ │ for clarity and precision: │ │ │ │ │ │ │ │ 1. **Specificity**: The question │ │ │ │ could be more specific about the │ │ │ │ context. For example, is it │ │ │ │ referring to online shopping, │ │ │ │ in-store shopping, or both? Adding │ │ │ │ context can help respondents │ │ │ │ provide more relevant and focused │ │ │ │ answers. │ │ │ │ │ │ │ │ 2. **Open-Ended Nature**: While │ │ │ │ open-ended questions can provide │ │ │ │ rich data, they can also lead to a │ │ │ │ wide variety of responses that │ │ │ │ might be difficult to categorize │ │ │ │ and analyze. Depending on the │ │ │ │ survey's goals, it might be useful │ │ │ │ to offer some categories or │ │ │ │ examples to guide respondents. │ │ │ │ │ │ │ │ 3. **Ambiguity in "Feature"**: The │ │ │ │ term "feature" is somewhat vague │ │ │ │ and can be interpreted in many │ │ │ │ ways. Clarifying what is meant by │ │ │ │ "feature" (e.g., a new service, a │ │ │ │ technological improvement, a │ │ │ │ product offering) can help │ │ │ │ respondents understand the question │ │ │ │ better. │ │ │ │ │ │ │ │ A revised version of the question │ │ │ │ might be: │ │ │ │ "What is one specific improvement │ │ │ │ you would like to see in our online │ │ │ │ shopping experience?" │ │ ├─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┤ │ What do you like most about using │ The survey question "What do you │ What aspects of our online │ │ our online marketplace? │ like most about using our online │ marketplace do you find most │ │ │ marketplace?" is generally clear │ appealing? If you have any negative │ │ │ and straightforward, but there are │ experiences, please describe them │ │ │ a couple of potential issues to │ as well. │ │ │ consider: │ │ │ │ │ │ │ │ 1. **Assumption of Positive │ │ │ │ Experience**: The question assumes │ │ │ │ that the respondent has something │ │ │ │ they like about the online │ │ │ │ marketplace. If a user has had a │ │ │ │ negative experience, they might │ │ │ │ find it difficult to answer this │ │ │ │ question accurately. To address │ │ │ │ this, you could add an option for │ │ │ │ users who may not have a positive │ │ │ │ experience, such as "If you don't │ │ │ │ like anything about our online │ │ │ │ marketplace, please explain why." │ │ │ │ │ │ │ │ 2. **Open-Ended Nature**: While │ │ │ │ open-ended questions can provide │ │ │ │ valuable qualitative data, they can │ │ │ │ also be more challenging to analyze │ │ │ │ systematically. Depending on the │ │ │ │ goals of your survey, you might │ │ │ │ want to include some │ │ │ │ multiple-choice options or │ │ │ │ follow-up questions to gather more │ │ │ │ specific feedback. │ │ │ │ │ │ │ │ Here’s a revised version that │ │ │ │ addresses these issues: │ │ │ │ "What aspects of our online │ │ │ │ marketplace do you find most and │ │ │ │ least appealing? Please provide │ │ │ │ specific examples." │ │ ├─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┤ │ How do you feel about the current │ The survey question "How do you │ On a scale of 1 to 5, how satisfied │ │ product search and filtering │ feel about the current product │ are you with the ease of use of the │ │ options? │ search and filtering options?" is │ current product search options? │ │ │ generally clear, but there could be │ │ │ │ a few improvements: │ │ │ │ │ │ │ │ 1. **Ambiguity**: The question is │ │ │ │ somewhat broad and may result in │ │ │ │ varied interpretations. Respondents │ │ │ │ might focus on different aspects of │ │ │ │ the search and filtering options, │ │ │ │ such as usability, effectiveness, │ │ │ │ speed, or interface design. │ │ │ │ │ │ │ │ 2. **Specificity**: It might be │ │ │ │ beneficial to break this question │ │ │ │ down into more specific components │ │ │ │ to get more targeted feedback. For │ │ │ │ example: │ │ │ │ - "How satisfied are you with │ │ │ │ the ease of use of the product │ │ │ │ search options?" │ │ │ │ - "How effective do you find the │ │ │ │ filtering options in narrowing down │ │ │ │ your search results?" │ │ │ │ │ │ │ │ 3. **Response Format**: The │ │ │ │ question does not specify how │ │ │ │ respondents should answer. │ │ │ │ Providing a scale or │ │ │ │ multiple-choice options could help │ │ │ │ in quantifying the feedback. For │ │ │ │ example: │ │ │ │ - "On a scale of 1 to 5, how │ │ │ │ satisfied are you with the current │ │ │ │ product search options?" │ │ │ │ - "Which of the following best │ │ │ │ describes your feelings about the │ │ │ │ current filtering options? (Very │ │ │ │ satisfied, Satisfied, Neutral, │ │ │ │ Dissatisfied, Very dissatisfied)" │ │ ├─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┤ │ Is there anything else you would │ The survey question "Is there │ Do you have any additional feedback │ │ like to share about your experience │ anything else you would like to │ about your experience with us? │ │ with us? │ share about your experience with │ │ │ │ us?" is clear and straightforward, │ │ │ │ but there are a few potential areas │ │ │ │ for improvement: │ │ │ │ │ │ │ │ 1. **Open-Ended Nature**: While │ │ │ │ open-ended questions can provide │ │ │ │ valuable insights, they can also be │ │ │ │ difficult to analyze │ │ │ │ quantitatively. Depending on the │ │ │ │ survey's goals, you might want to │ │ │ │ follow up with more specific │ │ │ │ questions or provide categories for │ │ │ │ feedback. │ │ │ │ │ │ │ │ 2. **Ambiguity**: The term │ │ │ │ "experience" is broad. If you want │ │ │ │ more targeted feedback, you could │ │ │ │ specify which aspect of the │ │ │ │ experience you're interested in │ │ │ │ (e.g., customer service, product │ │ │ │ quality, etc.). │ │ │ │ │ │ │ │ 3. **Politeness**: The question is │ │ │ │ polite, but adding a "please" could │ │ │ │ make it sound even more courteous. │ │ │ │ For example, "Is there anything │ │ │ │ else you would like to share about │ │ │ │ your experience with us, please?" │ │ │ │ │ │ │ │ 4. **Length and Complexity**: The │ │ │ │ question is already simple, but you │ │ │ │ could make it even more concise │ │ │ │ without losing meaning. For │ │ │ │ example, "Do you have any │ │ │ │ additional feedback about your │ │ │ │ experience with us?" │ │ ├─────────────────────────────────────┼─────────────────────────────────────┼─────────────────────────────────────┤ │ Can you describe a recent │ The survey question "Can you │ Could you please describe any │ │ experience where you were │ describe a recent experience where │ recent experiences, within the last │ │ dissatisfied with our service? │ you were dissatisfied with our │ month, where you were either │ │ │ service?" is generally clear and │ satisfied or dissatisfied with our │ │ │ direct, but there are a few │ service? │ │ │ potential issues to consider: │ │ │ │ │ │ │ │ 1. **Negative Bias**: The question │ │ │ │ assumes that the respondent had a │ │ │ │ recent experience that was │ │ │ │ dissatisfactory. This can introduce │ │ │ │ a negative bias and might not │ │ │ │ capture the full range of customer │ │ │ │ experiences. │ │ │ │ │ │ │ │ 2. **Open-Ended Nature**: While │ │ │ │ open-ended questions can provide │ │ │ │ detailed insights, they may also │ │ │ │ result in varied and lengthy │ │ │ │ responses that can be difficult to │ │ │ │ analyze systematically. │ │ │ │ │ │ │ │ 3. **Specificity**: The term │ │ │ │ "recent" is subjective and can vary │ │ │ │ from one respondent to another. It │ │ │ │ might be helpful to define what you │ │ │ │ mean by "recent" (e.g., within the │ │ │ │ last month). │ │ │ │ │ │ │ │ 4. **Leading Question**: The │ │ │ │ question might lead respondents to │ │ │ │ focus on negative experiences. A │ │ │ │ more balanced approach could be to │ │ │ │ ask about both positive and │ │ │ │ negative experiences. │ │ └─────────────────────────────────────┴─────────────────────────────────────┴─────────────────────────────────────┘
Evaluating respondents’ collective answers
Next we can create a ScenarioList for each respondent’s answers to use with our question about sentiment:
[15]:
sl_responses = ScenarioList.from_list("responses", sl['scenarios'])
sl_responses
[15]:
{
"scenarios": [
{
"responses": {
"Respondent ID": "101",
"What do you like most about using our online marketplace?": "The wide variety of products and the ease of use.",
"What is one feature you would like to see added to improve your shopping experience?": "It would be great to have a personalized recommendation system based on my browsing history.",
"Can you describe a recent experience where you were dissatisfied with our service?": "I was disappointed when an item I ordered arrived damaged, but customer service quickly resolved it.",
"How do you feel about the current product search and filtering options?": "The search and filtering options are intuitive and work well for me.",
"Is there anything else you would like to share about your experience with us?": "No, keep up the great work!",
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Scenario"
}
},
{
"responses": {
"Respondent ID": "102",
"What do you like most about using our online marketplace?": "I enjoy the simplicity of the interface.",
"What is one feature you would like to see added to improve your shopping experience?": "A feature that helps compare similar products side by side would be useful.",
"Can you describe a recent experience where you were dissatisfied with our service?": "No complaints here.",
"How do you feel about the current product search and filtering options?": "I find the product search to be pretty effective.",
"Is there anything else you would like to share about your experience with us?": "I think the sky is a beautiful shade of purple today.",
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Scenario"
}
},
{
"responses": {
"Respondent ID": "103",
"What do you like most about using our online marketplace?": "The platform is user-friendly and offers a vast selection of products.",
"What is one feature you would like to see added to improve your shopping experience?": "Would love to see an option to save and compare different products.",
"Can you describe a recent experience where you were dissatisfied with our service?": "My delivery was late by a few days, which was frustrating.",
"How do you feel about the current product search and filtering options?": "It\u2019s okay.",
"Is there anything else you would like to share about your experience with us?": "No.",
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Scenario"
}
}
]
}
Next we add these scenarios to our sentiment question (and any others we want to add) and run it:
[16]:
from edsl import QuestionMultipleChoice, QuestionLinearScale, Survey
q_sentiment = QuestionMultipleChoice(
question_name = "sentiment",
question_text = "What is the overall sentiment of this respondent's survey answers? {{ responses }}",
question_options = ["Very unsatisfied", "Somewhat unsatisfied", "Somewhat satisfied", "Very satisfied"]
)
q_recommend = QuestionLinearScale(
question_name = "recommend",
question_text = "On a scale from 1 to 5, how likely do you think this respondent is to recommend the company to a friend? {{ responses }}",
question_options = [1, 2, 3, 4, 5],
option_labels = {1:"Not at all likely", 5:"Very likely"}
)
survey = Survey([q_sentiment, q_recommend])
[17]:
results = survey.by(sl_responses).run()
[18]:
results.select("responses", "sentiment", "recommend").print(format="rich")
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ scenario ┃ answer ┃ answer ┃ ┃ .responses ┃ .sentiment ┃ .recommend ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ {'Respondent ID': '101', 'What do you like most about using our online │ Somewhat satisfied │ 4 │ │ marketplace?': 'The wide variety of products and the ease of use.', 'What is │ │ │ │ one feature you would like to see added to improve your shopping │ │ │ │ experience?': 'It would be great to have a personalized recommendation system │ │ │ │ based on my browsing history.', 'Can you describe a recent experience where │ │ │ │ you were dissatisfied with our service?': 'I was disappointed when an item I │ │ │ │ ordered arrived damaged, but customer service quickly resolved it.', 'How do │ │ │ │ you feel about the current product search and filtering options?': 'The │ │ │ │ search and filtering options are intuitive and work well for me.', 'Is there │ │ │ │ anything else you would like to share about your experience with us?': 'No, │ │ │ │ keep up the great work!', 'edsl_version': '0.1.33.dev1', 'edsl_class_name': │ │ │ │ 'Scenario'} │ │ │ ├───────────────────────────────────────────────────────────────────────────────┼────────────────────┼────────────┤ │ {'Respondent ID': '102', 'What do you like most about using our online │ Very satisfied │ 4 │ │ marketplace?': 'I enjoy the simplicity of the interface.', 'What is one │ │ │ │ feature you would like to see added to improve your shopping experience?': 'A │ │ │ │ feature that helps compare similar products side by side would be useful.', │ │ │ │ 'Can you describe a recent experience where you were dissatisfied with our │ │ │ │ service?': 'No complaints here.', 'How do you feel about the current product │ │ │ │ search and filtering options?': 'I find the product search to be pretty │ │ │ │ effective.', 'Is there anything else you would like to share about your │ │ │ │ experience with us?': 'I think the sky is a beautiful shade of purple │ │ │ │ today.', 'edsl_version': '0.1.33.dev1', 'edsl_class_name': 'Scenario'} │ │ │ ├───────────────────────────────────────────────────────────────────────────────┼────────────────────┼────────────┤ │ {'Respondent ID': '103', 'What do you like most about using our online │ Somewhat satisfied │ 3 │ │ marketplace?': 'The platform is user-friendly and offers a vast selection of │ │ │ │ products.', 'What is one feature you would like to see added to improve your │ │ │ │ shopping experience?': 'Would love to see an option to save and compare │ │ │ │ different products.', 'Can you describe a recent experience where you were │ │ │ │ dissatisfied with our service?': 'My delivery was late by a few days, which │ │ │ │ was frustrating.', 'How do you feel about the current product search and │ │ │ │ filtering options?': 'It’s okay.', 'Is there anything else you would like to │ │ │ │ share about your experience with us?': 'No.', 'edsl_version': '0.1.33.dev1', │ │ │ │ 'edsl_class_name': 'Scenario'} │ │ │ └───────────────────────────────────────────────────────────────────────────────┴────────────────────┴────────────┘
Evaluating individual answers
Next we create a ScenarioList for each individual question and answer to use with our question about the responsiveness of each answer. We can use the unpivot() method to expand the scenarios by desired identifiers (e.g., respondent ID):
[19]:
sl_qa = sl.unpivot(id_vars = ["Respondent ID"])
sl_qa
[19]:
{
"scenarios": [
{
"Respondent ID": "101",
"variable": "What do you like most about using our online marketplace?",
"value": "The wide variety of products and the ease of use."
},
{
"Respondent ID": "101",
"variable": "What is one feature you would like to see added to improve your shopping experience?",
"value": "It would be great to have a personalized recommendation system based on my browsing history."
},
{
"Respondent ID": "101",
"variable": "Can you describe a recent experience where you were dissatisfied with our service?",
"value": "I was disappointed when an item I ordered arrived damaged, but customer service quickly resolved it."
},
{
"Respondent ID": "101",
"variable": "How do you feel about the current product search and filtering options?",
"value": "The search and filtering options are intuitive and work well for me."
},
{
"Respondent ID": "101",
"variable": "Is there anything else you would like to share about your experience with us?",
"value": "No, keep up the great work!"
},
{
"Respondent ID": "102",
"variable": "What do you like most about using our online marketplace?",
"value": "I enjoy the simplicity of the interface."
},
{
"Respondent ID": "102",
"variable": "What is one feature you would like to see added to improve your shopping experience?",
"value": "A feature that helps compare similar products side by side would be useful."
},
{
"Respondent ID": "102",
"variable": "Can you describe a recent experience where you were dissatisfied with our service?",
"value": "No complaints here."
},
{
"Respondent ID": "102",
"variable": "How do you feel about the current product search and filtering options?",
"value": "I find the product search to be pretty effective."
},
{
"Respondent ID": "102",
"variable": "Is there anything else you would like to share about your experience with us?",
"value": "I think the sky is a beautiful shade of purple today."
},
{
"Respondent ID": "103",
"variable": "What do you like most about using our online marketplace?",
"value": "The platform is user-friendly and offers a vast selection of products."
},
{
"Respondent ID": "103",
"variable": "What is one feature you would like to see added to improve your shopping experience?",
"value": "Would love to see an option to save and compare different products."
},
{
"Respondent ID": "103",
"variable": "Can you describe a recent experience where you were dissatisfied with our service?",
"value": "My delivery was late by a few days, which was frustrating."
},
{
"Respondent ID": "103",
"variable": "How do you feel about the current product search and filtering options?",
"value": "It\u2019s okay."
},
{
"Respondent ID": "103",
"variable": "Is there anything else you would like to share about your experience with us?",
"value": "No."
}
]
}
We can call the rename() method to rename the keys as desired to match our question parameters syntax:
[20]:
sl_qa = sl_qa.rename({"Respondent ID": "id", "variable": "question", "value": "answer"})
sl_qa
[20]:
{
"scenarios": [
{
"id": "101",
"question": "What do you like most about using our online marketplace?",
"answer": "The wide variety of products and the ease of use."
},
{
"id": "101",
"question": "What is one feature you would like to see added to improve your shopping experience?",
"answer": "It would be great to have a personalized recommendation system based on my browsing history."
},
{
"id": "101",
"question": "Can you describe a recent experience where you were dissatisfied with our service?",
"answer": "I was disappointed when an item I ordered arrived damaged, but customer service quickly resolved it."
},
{
"id": "101",
"question": "How do you feel about the current product search and filtering options?",
"answer": "The search and filtering options are intuitive and work well for me."
},
{
"id": "101",
"question": "Is there anything else you would like to share about your experience with us?",
"answer": "No, keep up the great work!"
},
{
"id": "102",
"question": "What do you like most about using our online marketplace?",
"answer": "I enjoy the simplicity of the interface."
},
{
"id": "102",
"question": "What is one feature you would like to see added to improve your shopping experience?",
"answer": "A feature that helps compare similar products side by side would be useful."
},
{
"id": "102",
"question": "Can you describe a recent experience where you were dissatisfied with our service?",
"answer": "No complaints here."
},
{
"id": "102",
"question": "How do you feel about the current product search and filtering options?",
"answer": "I find the product search to be pretty effective."
},
{
"id": "102",
"question": "Is there anything else you would like to share about your experience with us?",
"answer": "I think the sky is a beautiful shade of purple today."
},
{
"id": "103",
"question": "What do you like most about using our online marketplace?",
"answer": "The platform is user-friendly and offers a vast selection of products."
},
{
"id": "103",
"question": "What is one feature you would like to see added to improve your shopping experience?",
"answer": "Would love to see an option to save and compare different products."
},
{
"id": "103",
"question": "Can you describe a recent experience where you were dissatisfied with our service?",
"answer": "My delivery was late by a few days, which was frustrating."
},
{
"id": "103",
"question": "How do you feel about the current product search and filtering options?",
"answer": "It\u2019s okay."
},
{
"id": "103",
"question": "Is there anything else you would like to share about your experience with us?",
"answer": "No."
}
]
}
[21]:
from edsl import QuestionYesNo
q_responsive = QuestionYesNo(
question_name = "responsive",
question_text = "Is this answer responsive to the question that was asked? Question: {{ question }} Answer: {{ answer }}"
)
[22]:
results = q_responsive.by(sl_qa).run()
[23]:
(results
.filter("responsive == 'No'")
.select("id", "question", "answer")
.print(format="rich")
)
┏━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ scenario ┃ scenario ┃ scenario ┃ ┃ .id ┃ .question ┃ .answer ┃ ┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │ 102 │ Is there anything else you would like to share │ I think the sky is a beautiful shade of purple │ │ │ about your experience with us? │ today. │ ├──────────┼───────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ │ 103 │ How do you feel about the current product search │ It’s okay. │ │ │ and filtering options? │ │ ├──────────┼───────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ │ 102 │ Can you describe a recent experience where you │ No complaints here. │ │ │ were dissatisfied with our service? │ │ ├──────────┼───────────────────────────────────────────────────┼──────────────────────────────────────────────────┤ │ 102 │ How do you feel about the current product search │ I find the product search to be pretty │ │ │ and filtering options? │ effective. │ └──────────┴───────────────────────────────────────────────────┴──────────────────────────────────────────────────┘
Uploading content to the Coop
Coop is a new platform for creating, storing and sharing LLM-based research. It is fully integrated with EDSL, and a convenient place to post and access surveys, agents, results and notebooks. Learn more about using the Coop.
Here we post the contents of this notebook:
[24]:
from edsl import Notebook
[25]:
n = Notebook(path = "scenariolist_unpivot.ipynb")
[26]:
n.push(description = "ScenarioList methods for sense checking survey data", visibility = "public")
[26]:
{'description': 'ScenarioList methods for sense checking survey data',
'object_type': 'notebook',
'url': 'https://www.expectedparrot.com/content/c4a75b92-1835-4345-b171-42450b876278',
'uuid': 'c4a75b92-1835-4345-b171-42450b876278',
'version': '0.1.33.dev1',
'visibility': 'public'}
To update an object at the Coop:
[27]:
n = Notebook(path = "scenariolist_unpivot.ipynb") # resave
[28]:
n.patch(uuid = "c4a75b92-1835-4345-b171-42450b876278", value = n)
[28]:
{'status': 'success'}