Data labeling

This notebook shows how to conduct data labeling and content analysis using EDSL, an open-source library for simulating surveys, experiments and other research with AI agents and large language models.

Using a dataset of mock customer service tickets as an example, we demonstrate how to:

  1. Import data into EDSL

  2. Create questions about the data

  3. Design an AI agent to answer the questions

  4. Select a language model to generate responses

  5. Analyze results as a formatted dataset

This workflow can be visualized as follows: general_survey.png

Technical setup

Before running the code below please ensure that you have completed setup:

Our Starter Tutorial also provides examples of EDSL basic components.

Selecting data for review

First we identify some data for review. Data can be created using the EDSL tools or imported from other sources. For purposes of this demo we import a set of hypothetical customer tickets for a transportation app:

[1]:
tickets = [
    "I just realized I left my phone in the car on my last ride. Can you help me get it back?",
    "I'm unhappy with my recent experience. The driver was very rude and unprofessional.",
    "I was charged more than the estimated fare for my trip yesterday. Can you explain why?",
    "The car seat provided was not properly installed, and I felt my child was at risk. Please ensure driver training.",
    "My driver took a longer route than necessary, resulting in a higher fare. I request a fare adjustment.",
    "I had a great experience with my driver today! Very friendly and efficient service.",
    "I'm concerned about the vehicle's cleanliness. It was not up to the standard I expect.",
    "The app keeps crashing every time I try to book a ride. Please fix this issue.",
    "My driver was exceptional - safe driving, polite, and the car was spotless. Kudos!",
    "I felt unsafe during my ride due to the driver's erratic behavior. This needs to be addressed immediately.",
    "The driver refused to follow my preferred route, which is shorter. I'm not satisfied with the service.",
    "Impressed with the quick response to my ride request and the driver's professionalism.",
    "I was charged for a ride I never took. Please refund me as soon as possible.",
    "The promo code I tried to use didn't work. Can you assist with this?",
    "There was a suspicious smell in the car, and I'm worried about hygiene standards.",
    "My driver was very considerate, especially helping me with my luggage. Appreciate the great service!",
    "The app's GPS seems inaccurate. It directed the driver to the wrong pick-up location.",
    "I want to compliment my driver's excellent navigation and time management during rush hour.",
    "The vehicle didn't match the description in the app. It was confusing and concerning.",
    "I faced an issue with payment processing after my last ride. Can you look into this?",
]

Constructing questions about the data

Next we create some questions about the data. EDSL provides a variety of question types that we can choose from based on the form of the response that we want to get back from the model (multiple choice, free text, checkbox, linear scale, etc.). Learn more about question types.

Note that we use a {{ placeholder }} in each question text in order to parameterize the questions with the individual ticket contents in the next step:

[2]:
from edsl import (
    QuestionMultipleChoice,
    QuestionCheckBox,
    QuestionFreeText,
    QuestionYesNo,
    QuestionLinearScale,
)
[3]:
question_issues = QuestionCheckBox(
    question_name="issues",
    question_text="Check all of the issues mentioned in this ticket: {{ scenario.ticket }}",
    question_options=[
        "safety",
        "cleanliness",
        "driver performance",
        "GPS/route",
        "lost item",
        "other",
    ],
)
[4]:
question_primary_issue = QuestionFreeText(
    question_name="primary_issue",
    question_text="What is the primary issue in this ticket? Ticket: {{ scenario.ticket }}",
)
[5]:
question_accident = QuestionMultipleChoice(
    question_name="accident",
    question_text="If the primary issue in this ticket is safety, was there an accident where someone was hurt? Ticket: {{ scenario.ticket }}",
    question_options=["Yes", "No", "Not applicable"],
)
[6]:
question_sentiment = QuestionMultipleChoice(
    question_name="sentiment",
    question_text="What is the sentiment of this ticket? Ticket: {{ scenario.ticket }}",
    question_options=[
        "Very positive",
        "Somewhat positive",
        "Neutral",
        "Somewhat negative",
        "Very negative",
    ],
)
[7]:
question_refund = QuestionYesNo(
    question_name="refund",
    question_text="Does the customer ask for a refund in this ticket? Ticket: {{ scenario.ticket }}",
)
[8]:
question_priority = QuestionLinearScale(
    question_name="priority",
    question_text="On a scale from 0 to 5, what is the priority level of this ticket? Ticket: {{ scenario.ticket }}",
    question_options=[0, 1, 2, 3, 4, 5],
    option_labels={0: "Lowest", 5: "Highest"},
)

Building a survey

We combine the questions into a survey in order to administer them together:

[9]:
from edsl import Survey

survey = Survey(
    questions=[
        question_issues,
        question_primary_issue,
        question_accident,
        question_sentiment,
        question_refund,
        question_priority,
    ]
)

Survey questions are administered asynchronously by default. Learn more about adding conditional logic and memory to your survey.

We can review our questions in a readable format, or export them as a survey to use with human respondents or at other survey platforms:

[10]:
survey
[10]:

Survey # questions: 6; question_name list: ['issues', 'primary_issue', 'accident', 'sentiment', 'refund', 'priority'];

  question_options option_labels question_type use_code question_name question_text
0 ['safety', 'cleanliness', 'driver performance', 'GPS/route', 'lost item', 'other'] nan checkbox True issues Check all of the issues mentioned in this ticket: {{ scenario.ticket }}
1 nan nan free_text nan primary_issue What is the primary issue in this ticket? Ticket: {{ scenario.ticket }}
2 ['Yes', 'No', 'Not applicable'] nan multiple_choice nan accident If the primary issue in this ticket is safety, was there an accident where someone was hurt? Ticket: {{ scenario.ticket }}
3 ['Very positive', 'Somewhat positive', 'Neutral', 'Somewhat negative', 'Very negative'] nan multiple_choice nan sentiment What is the sentiment of this ticket? Ticket: {{ scenario.ticket }}
4 ['No', 'Yes'] nan yes_no nan refund Does the customer ask for a refund in this ticket? Ticket: {{ scenario.ticket }}
5 [0, 1, 2, 3, 4, 5] {0: 'Lowest', 5: 'Highest'} linear_scale nan priority On a scale from 0 to 5, what is the priority level of this ticket? Ticket: {{ scenario.ticket }}

Designing AI agents

A key feature of EDSL is the ability to create personas for AI agents that the language models are prompted to use in generating responses to the questions. This is done by passing a dictionary of traits to Agent objects:

[11]:
from edsl import Agent

agent = Agent(
    traits={
        "persona": "You are an expert customer service agent.",
        "years_experience": 15,
    }
)
agent
[11]:

Agent

  key value
0 traits:persona You are an expert customer service agent.
1 traits:years_experience 15

Selecting language models

EDSL allows us to select the language models to use in generating results. See the `model pricing page <>`__ for pricing and performance information for available models.

Here we select gpt-4o (if no model is specified, the default model is used – run Model() to verify the current default model):

[12]:
from edsl import Model

model = Model("gpt-4o")
model
[12]:

LanguageModel

  key value
0 model gpt-4o
1 parameters:temperature 0.500000
2 parameters:max_tokens 1000
3 parameters:top_p 1
4 parameters:frequency_penalty 0
5 parameters:presence_penalty 0
6 parameters:logprobs False
7 parameters:top_logprobs 3
8 inference_service openai

Adding data to the questions

We add the contents of each ticket into each question as an independent “scenario” for review. This allows us to create versions of the questions for each job post and deliver them to the model all at once:

[13]:
from edsl import ScenarioList

scenarios = ScenarioList.from_list("ticket", tickets)
scenarios
[13]:

ScenarioList scenarios: 20; keys: ['ticket'];

  ticket
0 I just realized I left my phone in the car on my last ride. Can you help me get it back?
1 I'm unhappy with my recent experience. The driver was very rude and unprofessional.
2 I was charged more than the estimated fare for my trip yesterday. Can you explain why?
3 The car seat provided was not properly installed, and I felt my child was at risk. Please ensure driver training.
4 My driver took a longer route than necessary, resulting in a higher fare. I request a fare adjustment.
5 I had a great experience with my driver today! Very friendly and efficient service.
6 I'm concerned about the vehicle's cleanliness. It was not up to the standard I expect.
7 The app keeps crashing every time I try to book a ride. Please fix this issue.
8 My driver was exceptional - safe driving, polite, and the car was spotless. Kudos!
9 I felt unsafe during my ride due to the driver's erratic behavior. This needs to be addressed immediately.
10 The driver refused to follow my preferred route, which is shorter. I'm not satisfied with the service.
11 Impressed with the quick response to my ride request and the driver's professionalism.
12 I was charged for a ride I never took. Please refund me as soon as possible.
13 The promo code I tried to use didn't work. Can you assist with this?
14 There was a suspicious smell in the car, and I'm worried about hygiene standards.
15 My driver was very considerate, especially helping me with my luggage. Appreciate the great service!
16 The app's GPS seems inaccurate. It directed the driver to the wrong pick-up location.
17 I want to compliment my driver's excellent navigation and time management during rush hour.
18 The vehicle didn't match the description in the app. It was confusing and concerning.
19 I faced an issue with payment processing after my last ride. Can you look into this?

Running the survey

We run the survey by adding the scenarios, agent and model with the by() method and then calling the run() method:

[14]:
results = survey.by(scenarios).by(agent).by(model).run()
Job Status (2025-03-03 14:49:50)
Job UUID 27d021b9-dde4-4b01-a40c-9cb2a2950813
Progress Bar URL https://www.expectedparrot.com/home/remote-job-progress/27d021b9-dde4-4b01-a40c-9cb2a2950813
Exceptions Report URL None
Results UUID 274ceb4e-c98c-4e1a-aec5-ae3b4d50c126
Results URL https://www.expectedparrot.com/content/274ceb4e-c98c-4e1a-aec5-ae3b4d50c126
Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/274ceb4e-c98c-4e1a-aec5-ae3b4d50c126

This generates a formatted dataset of Results that includes information about all the components, including the prompts and responses. We can see a list of all the components:

[15]:
results.columns
[15]:
  0
0 agent.agent_index
1 agent.agent_instruction
2 agent.agent_name
3 agent.persona
4 agent.years_experience
5 answer.accident
6 answer.issues
7 answer.primary_issue
8 answer.priority
9 answer.refund
10 answer.sentiment
11 cache_keys.accident_cache_key
12 cache_keys.issues_cache_key
13 cache_keys.primary_issue_cache_key
14 cache_keys.priority_cache_key
15 cache_keys.refund_cache_key
16 cache_keys.sentiment_cache_key
17 cache_used.accident_cache_used
18 cache_used.issues_cache_used
19 cache_used.primary_issue_cache_used
20 cache_used.priority_cache_used
21 cache_used.refund_cache_used
22 cache_used.sentiment_cache_used
23 comment.accident_comment
24 comment.issues_comment
25 comment.primary_issue_comment
26 comment.priority_comment
27 comment.refund_comment
28 comment.sentiment_comment
29 generated_tokens.accident_generated_tokens
30 generated_tokens.issues_generated_tokens
31 generated_tokens.primary_issue_generated_tokens
32 generated_tokens.priority_generated_tokens
33 generated_tokens.refund_generated_tokens
34 generated_tokens.sentiment_generated_tokens
35 iteration.iteration
36 model.frequency_penalty
37 model.inference_service
38 model.logprobs
39 model.max_tokens
40 model.model
41 model.model_index
42 model.presence_penalty
43 model.temperature
44 model.top_logprobs
45 model.top_p
46 prompt.accident_system_prompt
47 prompt.accident_user_prompt
48 prompt.issues_system_prompt
49 prompt.issues_user_prompt
50 prompt.primary_issue_system_prompt
51 prompt.primary_issue_user_prompt
52 prompt.priority_system_prompt
53 prompt.priority_user_prompt
54 prompt.refund_system_prompt
55 prompt.refund_user_prompt
56 prompt.sentiment_system_prompt
57 prompt.sentiment_user_prompt
58 question_options.accident_question_options
59 question_options.issues_question_options
60 question_options.primary_issue_question_options
61 question_options.priority_question_options
62 question_options.refund_question_options
63 question_options.sentiment_question_options
64 question_text.accident_question_text
65 question_text.issues_question_text
66 question_text.primary_issue_question_text
67 question_text.priority_question_text
68 question_text.refund_question_text
69 question_text.sentiment_question_text
70 question_type.accident_question_type
71 question_type.issues_question_type
72 question_type.primary_issue_question_type
73 question_type.priority_question_type
74 question_type.refund_question_type
75 question_type.sentiment_question_type
76 raw_model_response.accident_cost
77 raw_model_response.accident_one_usd_buys
78 raw_model_response.accident_raw_model_response
79 raw_model_response.issues_cost
80 raw_model_response.issues_one_usd_buys
81 raw_model_response.issues_raw_model_response
82 raw_model_response.primary_issue_cost
83 raw_model_response.primary_issue_one_usd_buys
84 raw_model_response.primary_issue_raw_model_response
85 raw_model_response.priority_cost
86 raw_model_response.priority_one_usd_buys
87 raw_model_response.priority_raw_model_response
88 raw_model_response.refund_cost
89 raw_model_response.refund_one_usd_buys
90 raw_model_response.refund_raw_model_response
91 raw_model_response.sentiment_cost
92 raw_model_response.sentiment_one_usd_buys
93 raw_model_response.sentiment_raw_model_response
94 scenario.scenario_index
95 scenario.ticket

Analyzing results

EDSL comes with built-in methods for analyzing results. Here we filter, sort, select and print components in a table:

[16]:
(
    results
    .filter("priority in [4, 5]")
    .sort_by("issues", "sentiment")
    .select("ticket", "issues", "primary_issue", "accident", "sentiment", "refund", "priority")
)
[16]:
  scenario.ticket answer.issues answer.primary_issue answer.accident answer.sentiment answer.refund answer.priority
0 The app's GPS seems inaccurate. It directed the driver to the wrong pick-up location. ['GPS/route'] The primary issue in this ticket is that the app's GPS is inaccurate, leading to the driver being directed to the wrong pick-up location. This suggests there may be a problem with the app's location services or mapping functionality. No Somewhat negative No 4
1 I'm unhappy with my recent experience. The driver was very rude and unprofessional. ['driver performance'] The primary issue in this ticket is customer dissatisfaction due to the driver's rude and unprofessional behavior. It's important to address this concern by apologizing to the customer, gathering more details about the incident, and taking appropriate action to ensure such behavior is not repeated in the future. No Very negative No 4
2 I just realized I left my phone in the car on my last ride. Can you help me get it back? ['lost item'] The primary issue in this ticket is the customer's lost phone, which was left in the car during their last ride. They are seeking assistance to retrieve it. Not applicable Somewhat negative No 4
3 The app keeps crashing every time I try to book a ride. Please fix this issue. ['other'] The primary issue in this ticket is that the app crashes whenever the user attempts to book a ride. This indicates a potential bug or technical problem within the app that needs to be addressed by the development team to ensure the functionality is restored. Not applicable Somewhat negative No 4
4 The car seat provided was not properly installed, and I felt my child was at risk. Please ensure driver training. ['safety', 'driver performance'] The primary issue in this ticket is the improper installation of the car seat provided by the driver, which made the customer feel that their child's safety was at risk. The customer is requesting that drivers receive proper training to ensure car seats are installed correctly. No Very negative No 5
5 I felt unsafe during my ride due to the driver's erratic behavior. This needs to be addressed immediately. ['safety', 'driver performance'] The primary issue in this ticket is the customer's feeling of being unsafe due to the driver's erratic behavior. This is a serious concern that needs immediate attention to ensure the safety and comfort of passengers. No Very negative No 5

We can apply some lables to our table:

[17]:
(
    results.select(
        "ticket",
        "issues",
        "primary_issue",
        "accident",
        "sentiment",
        "refund",
        "priority",
    ).print(
        pretty_labels={
            "scenario.ticket": "Ticket",
            "answer.issues": "Issues",
            "answer.primary_issue": "Primary issue",
            "answer.accident": "Accident",
            "answer.sentiment": "Sentiment",
            "answer.refund": "Refund request",
            "answer.priority": "Priority",
        }
    )
)
[17]:
  Ticket Issues Primary issue Accident Sentiment Refund request Priority
0 I just realized I left my phone in the car on my last ride. Can you help me get it back? ['lost item'] The primary issue in this ticket is the customer's lost phone, which was left in the car during their last ride. They are seeking assistance to retrieve it. Not applicable Somewhat negative No 4
1 I'm unhappy with my recent experience. The driver was very rude and unprofessional. ['driver performance'] The primary issue in this ticket is customer dissatisfaction due to the driver's rude and unprofessional behavior. It's important to address this concern by apologizing to the customer, gathering more details about the incident, and taking appropriate action to ensure such behavior is not repeated in the future. No Very negative No 4
2 I was charged more than the estimated fare for my trip yesterday. Can you explain why? ['other'] The primary issue in this ticket is a discrepancy between the estimated fare and the actual charge for the customer's trip. The customer is seeking an explanation for why they were charged more than the estimated amount. To address this, you would typically review the trip details to identify any factors that could have contributed to the increased fare, such as changes in route, additional stops, traffic conditions, or any applicable surcharges. Not applicable Somewhat negative No 3
3 The car seat provided was not properly installed, and I felt my child was at risk. Please ensure driver training. ['safety', 'driver performance'] The primary issue in this ticket is the improper installation of the car seat provided by the driver, which made the customer feel that their child's safety was at risk. The customer is requesting that drivers receive proper training to ensure car seats are installed correctly. No Very negative No 5
4 My driver took a longer route than necessary, resulting in a higher fare. I request a fare adjustment. ['driver performance', 'GPS/route'] The primary issue in this ticket is that the driver took a longer route than necessary, which led to a higher fare for the customer. The customer is requesting a fare adjustment to address this discrepancy. No Somewhat negative No 2
5 I had a great experience with my driver today! Very friendly and efficient service. ['driver performance'] The ticket you've provided is actually a positive feedback rather than an issue. The customer is expressing their satisfaction with the driver's friendly and efficient service. It seems like everything went well in this particular interaction. If there's anything else you need help with, feel free to let me know! Not applicable Very positive No 0
6 I'm concerned about the vehicle's cleanliness. It was not up to the standard I expect. ['cleanliness'] The primary issue in this ticket is the customer's concern about the cleanliness of the vehicle, which did not meet their expected standards. No Somewhat negative No 2
7 The app keeps crashing every time I try to book a ride. Please fix this issue. ['other'] The primary issue in this ticket is that the app crashes whenever the user attempts to book a ride. This indicates a potential bug or technical problem within the app that needs to be addressed by the development team to ensure the functionality is restored. Not applicable Somewhat negative No 4
8 My driver was exceptional - safe driving, polite, and the car was spotless. Kudos! ['safety', 'cleanliness', 'driver performance'] The ticket you provided is actually not highlighting an issue. Instead, it's a positive feedback from a customer praising their driver for safe driving, politeness, and maintaining a clean vehicle. If you're looking to address this ticket, you might consider forwarding the commendation to the driver and recognizing their excellent service. No Very positive No 0
9 I felt unsafe during my ride due to the driver's erratic behavior. This needs to be addressed immediately. ['safety', 'driver performance'] The primary issue in this ticket is the customer's feeling of being unsafe due to the driver's erratic behavior. This is a serious concern that needs immediate attention to ensure the safety and comfort of passengers. No Very negative No 5
10 The driver refused to follow my preferred route, which is shorter. I'm not satisfied with the service. ['driver performance', 'GPS/route'] The primary issue in this ticket is the driver's refusal to follow the customer's preferred route, which the customer believes to be shorter. This has led to the customer being dissatisfied with the service provided. No Very negative No 2
11 Impressed with the quick response to my ride request and the driver's professionalism. ['driver performance'] The ticket you provided seems to be more of a positive feedback rather than an issue. The customer is expressing satisfaction with the quick response to their ride request and the professionalism of the driver. If there are any underlying concerns or questions that need addressing, please let me know! No Very positive No 0
12 I was charged for a ride I never took. Please refund me as soon as possible. ['other'] The primary issue in this ticket is that the customer was charged for a ride they did not take and is requesting a refund. Not applicable Somewhat negative Yes 3
13 The promo code I tried to use didn't work. Can you assist with this? ['other'] The primary issue in this ticket is that the customer is experiencing difficulties with a promo code that is not working. They are seeking assistance to resolve the problem. It would be important to verify the promo code's validity, check for any restrictions or expiration dates, and ensure the customer is entering it correctly. Not applicable Neutral No 2
14 There was a suspicious smell in the car, and I'm worried about hygiene standards. ['cleanliness', 'other'] The primary issue in this ticket is a concern about a suspicious smell in the car, which raises worries about the hygiene standards of the vehicle. This issue needs to be addressed to ensure customer satisfaction and safety. Not applicable Somewhat negative No 3
15 My driver was very considerate, especially helping me with my luggage. Appreciate the great service! ['driver performance'] The ticket you provided doesn't seem to indicate any issues. It actually contains positive feedback about the driver's considerate behavior and helpfulness with luggage. If you have any other concerns or need assistance with a different matter, feel free to let me know! No Very positive No 0
16 The app's GPS seems inaccurate. It directed the driver to the wrong pick-up location. ['GPS/route'] The primary issue in this ticket is that the app's GPS is inaccurate, leading to the driver being directed to the wrong pick-up location. This suggests there may be a problem with the app's location services or mapping functionality. No Somewhat negative No 4
17 I want to compliment my driver's excellent navigation and time management during rush hour. ['driver performance'] The primary issue in this ticket is not a problem or complaint but rather a compliment. The customer wants to praise their driver's excellent navigation and time management skills during rush hour. Not applicable Very positive No 0
18 The vehicle didn't match the description in the app. It was confusing and concerning. ['other'] The primary issue in this ticket is that the vehicle provided did not match the description given in the app, leading to confusion and concern for the customer. This discrepancy between the expected and actual vehicle could affect the customer's trust and satisfaction with the service. It's important to address this issue promptly to ensure the customer feels heard and to prevent similar incidents in the future. Not applicable Somewhat negative No 3
19 I faced an issue with payment processing after my last ride. Can you look into this? ['other'] The primary issue in this ticket is a problem with payment processing after the user's last ride. The customer is requesting assistance to investigate and resolve the payment issue. Not applicable Somewhat negative No 3

EDSL also comes with methods for accessing results as a dataframe or SQL table:

[18]:
df = (
    results
    .select(
        "issues",
        "primary_issue",
        "accident",
        "sentiment",
        "refund",
        "priority"
    )
    .to_pandas(remove_prefix=True)
)
df
[18]:
issues primary_issue accident sentiment refund priority
0 ['lost item'] The primary issue in this ticket is the custom... Not applicable Somewhat negative No 4
1 ['driver performance'] The primary issue in this ticket is customer d... No Very negative No 4
2 ['other'] The primary issue in this ticket is a discrepa... Not applicable Somewhat negative No 3
3 ['safety', 'driver performance'] The primary issue in this ticket is the improp... No Very negative No 5
4 ['driver performance', 'GPS/route'] The primary issue in this ticket is that the d... No Somewhat negative No 2
5 ['driver performance'] The ticket you've provided is actually a posit... Not applicable Very positive No 0
6 ['cleanliness'] The primary issue in this ticket is the custom... No Somewhat negative No 2
7 ['other'] The primary issue in this ticket is that the a... Not applicable Somewhat negative No 4
8 ['safety', 'cleanliness', 'driver performance'] The ticket you provided is actually not highli... No Very positive No 0
9 ['safety', 'driver performance'] The primary issue in this ticket is the custom... No Very negative No 5
10 ['driver performance', 'GPS/route'] The primary issue in this ticket is the driver... No Very negative No 2
11 ['driver performance'] The ticket you provided seems to be more of a ... No Very positive No 0
12 ['other'] The primary issue in this ticket is that the c... Not applicable Somewhat negative Yes 3
13 ['other'] The primary issue in this ticket is that the c... Not applicable Neutral No 2
14 ['cleanliness', 'other'] The primary issue in this ticket is a concern ... Not applicable Somewhat negative No 3
15 ['driver performance'] The ticket you provided doesn't seem to indica... No Very positive No 0
16 ['GPS/route'] The primary issue in this ticket is that the a... No Somewhat negative No 4
17 ['driver performance'] The primary issue in this ticket is not a prob... Not applicable Very positive No 0
18 ['other'] The primary issue in this ticket is that the v... Not applicable Somewhat negative No 3
19 ['other'] The primary issue in this ticket is a problem ... Not applicable Somewhat negative No 3

We can also access results as a SQL table:

[19]:
results.sql("""
select ticket, issues, primary_issue, accident, sentiment, refund, priority
from self
""")
[19]:
  ticket issues primary_issue accident sentiment refund priority
0 I just realized I left my phone in the car on my last ride. Can you help me get it back? ['lost item'] The primary issue in this ticket is the customer's lost phone, which was left in the car during their last ride. They are seeking assistance to retrieve it. Not applicable Somewhat negative No 4
1 I'm unhappy with my recent experience. The driver was very rude and unprofessional. ['driver performance'] The primary issue in this ticket is customer dissatisfaction due to the driver's rude and unprofessional behavior. It's important to address this concern by apologizing to the customer, gathering more details about the incident, and taking appropriate action to ensure such behavior is not repeated in the future. No Very negative No 4
2 I was charged more than the estimated fare for my trip yesterday. Can you explain why? ['other'] The primary issue in this ticket is a discrepancy between the estimated fare and the actual charge for the customer's trip. The customer is seeking an explanation for why they were charged more than the estimated amount. To address this, you would typically review the trip details to identify any factors that could have contributed to the increased fare, such as changes in route, additional stops, traffic conditions, or any applicable surcharges. Not applicable Somewhat negative No 3
3 The car seat provided was not properly installed, and I felt my child was at risk. Please ensure driver training. ['safety', 'driver performance'] The primary issue in this ticket is the improper installation of the car seat provided by the driver, which made the customer feel that their child's safety was at risk. The customer is requesting that drivers receive proper training to ensure car seats are installed correctly. No Very negative No 5
4 My driver took a longer route than necessary, resulting in a higher fare. I request a fare adjustment. ['driver performance', 'GPS/route'] The primary issue in this ticket is that the driver took a longer route than necessary, which led to a higher fare for the customer. The customer is requesting a fare adjustment to address this discrepancy. No Somewhat negative No 2
5 I had a great experience with my driver today! Very friendly and efficient service. ['driver performance'] The ticket you've provided is actually a positive feedback rather than an issue. The customer is expressing their satisfaction with the driver's friendly and efficient service. It seems like everything went well in this particular interaction. If there's anything else you need help with, feel free to let me know! Not applicable Very positive No 0
6 I'm concerned about the vehicle's cleanliness. It was not up to the standard I expect. ['cleanliness'] The primary issue in this ticket is the customer's concern about the cleanliness of the vehicle, which did not meet their expected standards. No Somewhat negative No 2
7 The app keeps crashing every time I try to book a ride. Please fix this issue. ['other'] The primary issue in this ticket is that the app crashes whenever the user attempts to book a ride. This indicates a potential bug or technical problem within the app that needs to be addressed by the development team to ensure the functionality is restored. Not applicable Somewhat negative No 4
8 My driver was exceptional - safe driving, polite, and the car was spotless. Kudos! ['safety', 'cleanliness', 'driver performance'] The ticket you provided is actually not highlighting an issue. Instead, it's a positive feedback from a customer praising their driver for safe driving, politeness, and maintaining a clean vehicle. If you're looking to address this ticket, you might consider forwarding the commendation to the driver and recognizing their excellent service. No Very positive No 0
9 I felt unsafe during my ride due to the driver's erratic behavior. This needs to be addressed immediately. ['safety', 'driver performance'] The primary issue in this ticket is the customer's feeling of being unsafe due to the driver's erratic behavior. This is a serious concern that needs immediate attention to ensure the safety and comfort of passengers. No Very negative No 5
10 The driver refused to follow my preferred route, which is shorter. I'm not satisfied with the service. ['driver performance', 'GPS/route'] The primary issue in this ticket is the driver's refusal to follow the customer's preferred route, which the customer believes to be shorter. This has led to the customer being dissatisfied with the service provided. No Very negative No 2
11 Impressed with the quick response to my ride request and the driver's professionalism. ['driver performance'] The ticket you provided seems to be more of a positive feedback rather than an issue. The customer is expressing satisfaction with the quick response to their ride request and the professionalism of the driver. If there are any underlying concerns or questions that need addressing, please let me know! No Very positive No 0
12 I was charged for a ride I never took. Please refund me as soon as possible. ['other'] The primary issue in this ticket is that the customer was charged for a ride they did not take and is requesting a refund. Not applicable Somewhat negative Yes 3
13 The promo code I tried to use didn't work. Can you assist with this? ['other'] The primary issue in this ticket is that the customer is experiencing difficulties with a promo code that is not working. They are seeking assistance to resolve the problem. It would be important to verify the promo code's validity, check for any restrictions or expiration dates, and ensure the customer is entering it correctly. Not applicable Neutral No 2
14 There was a suspicious smell in the car, and I'm worried about hygiene standards. ['cleanliness', 'other'] The primary issue in this ticket is a concern about a suspicious smell in the car, which raises worries about the hygiene standards of the vehicle. This issue needs to be addressed to ensure customer satisfaction and safety. Not applicable Somewhat negative No 3
15 My driver was very considerate, especially helping me with my luggage. Appreciate the great service! ['driver performance'] The ticket you provided doesn't seem to indicate any issues. It actually contains positive feedback about the driver's considerate behavior and helpfulness with luggage. If you have any other concerns or need assistance with a different matter, feel free to let me know! No Very positive No 0
16 The app's GPS seems inaccurate. It directed the driver to the wrong pick-up location. ['GPS/route'] The primary issue in this ticket is that the app's GPS is inaccurate, leading to the driver being directed to the wrong pick-up location. This suggests there may be a problem with the app's location services or mapping functionality. No Somewhat negative No 4
17 I want to compliment my driver's excellent navigation and time management during rush hour. ['driver performance'] The primary issue in this ticket is not a problem or complaint but rather a compliment. The customer wants to praise their driver's excellent navigation and time management skills during rush hour. Not applicable Very positive No 0
18 The vehicle didn't match the description in the app. It was confusing and concerning. ['other'] The primary issue in this ticket is that the vehicle provided did not match the description given in the app, leading to confusion and concern for the customer. This discrepancy between the expected and actual vehicle could affect the customer's trust and satisfaction with the service. It's important to address this issue promptly to ensure the customer feels heard and to prevent similar incidents in the future. Not applicable Somewhat negative No 3
19 I faced an issue with payment processing after my last ride. Can you look into this? ['other'] The primary issue in this ticket is a problem with payment processing after the user's last ride. The customer is requesting assistance to investigate and resolve the payment issue. Not applicable Somewhat negative No 3

To export results to a CSV file:

[20]:
results.to_csv("data_labeling_example.csv")
File written to data_labeling_example.csv

Posting content to the Coop

We can post any objects to the Coop, including this notebook. Objects can be updated or modified at your Coop account, and shared with others or stored privately (default visibility is unlisted):

[21]:
# results.push(
#     description = "Customer service tickets data labeling example",
#     alias = "customer-service-tickets-results-example",
#     visibility="public"
# )
[22]:
# survey.push(
#     description = "Customer service tickets data labeling example survey",
#     alias = "customer-service-tickets-survey-example",
#     visibility="public"
# )

We can post this notebook as well:

[23]:
from edsl import Notebook

nb = Notebook("data_labeling_example.ipynb")

if refresh := False:
    nb.push(
        description = "Data labeling example",
        alias = "data-labeling-example-notebook",
        visibility = "public"
    )
else:
    nb.patch("https://www.expectedparrot.com/content/RobinHorton/data-labeling-example-notebook", value = nb)