Overview
Using a dataset of mock customer service tickets as an example, we demonstrate how to:1
Import data into EDSL as scenarios
2
Create questions about the data
3
Design an AI agent to answer the questions
4
Select a language model to generate responses
5
Analyze results as a formatted dataset

Selecting data for review
First we identify some data for review. Data can be created in EDSL or imported from other sources (CSV, PDF, PNG, MP4, DOC, tables, lists, dicts, etc.). For purposes of demonstration we import a set of hypothetical customer tickets for a transportation app:Constructing questions about the data
Next we create some questions about the data. EDSL provides a variety of question types that we can choose from based on the form of the response that we want to get back from the model (multiple choice, free text, checkbox, linear scale, etc.). Learn more about question types.Note:Note that we use a
{{ placeholder }}
in each question text in order to parameterize the questions with the individual ticket contents in the next step:Building a survey
We combine the questions into a survey in order to administer them together: from edsl import Survey survey = Survey( questions=[ q_issues, q_primary_issue, q_accident, q_sentiment, q_refund, q_priority, ] ) Survey questions are administered asynchronously by default. Learn more about adding conditional logic and memory to your survey. Here we inspect them:question_text | question_options | question_type | question_name | option_labels | |
---|---|---|---|---|---|
0 | Check all of the issues mentioned in this ticket: | [‘safety’, ‘cleanliness’, ‘driver performance’, ‘GPS/route’, ‘lost item’, ‘other’] | checkbox | issues | nan |
1 | What is the primary issue in this ticket? Ticket: | nan | free_text | primary_issue | nan |
2 | If the primary issue in this ticket is safety, was there an accident where someone was hurt? Ticket: | [‘Yes’, ‘No’, ‘Not applicable’] | multiple_choice | accident | nan |
3 | What is the sentiment of this ticket? Ticket: | [‘Very positive’, ‘Somewhat positive’, ‘Neutral’, ‘Somewhat negative’, ‘Very negative’] | multiple_choice | sentiment | nan |
4 | Does the customer ask for a refund in this ticket? Ticket: | [‘No’, ‘Yes’] | yes_no | refund | nan |
5 | On a scale from 0 to 5, what is the priority level of this ticket? Ticket: | [0, 1, 2, 3, 4, 5] | linear_scale | priority | {0: 'Lowest', 5: 'Highest'} |
Designing AI agents
A key feature of EDSL is the ability to create personas for AI agents that the language models are prompted to use in generating responses to the questions. This is done by passing a dictionary of traits to Agent objects:key | value | |
---|---|---|
0 | traits:persona | You are an expert customer service agent. |
1 | traits:years_experience | 15 |
Selecting language models
EDSL allows us to select the language models to use in generating results. See themodel pricing page <>
__ for pricing and performance information for available models.
Here we select gpt-4o (if no model is specified, the default model is used – run Model()
to verify the current default model):
key | value | |
---|---|---|
0 | model | gpt-4o |
1 | parameters:temperature | 0.500000 |
2 | parameters:max_tokens | 1000 |
3 | parameters:top_p | 1 |
4 | parameters:frequency_penalty | 0 |
5 | parameters:presence_penalty | 0 |
6 | parameters:logprobs | False |
7 | parameters:top_logprobs | 3 |
8 | inference_service | openai |
Adding data to the questions
We add the contents of each ticket into each question as an independent “scenario” for review. This allows us to create versions of the questions for each job post and deliver them to the model all at once:ticket | |
---|---|
0 | I just realized I left my phone in the car on my last ride. Can you help me get it back? |
1 | I’m unhappy with my recent experience. The driver was very rude and unprofessional. |
2 | I was charged more than the estimated fare for my trip yesterday. Can you explain why? |
3 | The car seat provided was not properly installed, and I felt my child was at risk. Please ensure driver training. |
4 | My driver took a longer route than necessary, resulting in a higher fare. I request a fare adjustment. |
5 | I had a great experience with my driver today! Very friendly and efficient service. |
6 | I’m concerned about the vehicle’s cleanliness. It was not up to the standard I expect. |
7 | The app keeps crashing every time I try to book a ride. Please fix this issue. |
8 | My driver was exceptional - safe driving, polite, and the car was spotless. Kudos! |
9 | I felt unsafe during my ride due to the driver’s erratic behavior. This needs to be addressed immediately. |
10 | The driver refused to follow my preferred route, which is shorter. I’m not satisfied with the service. |
11 | Impressed with the quick response to my ride request and the driver’s professionalism. |
12 | I was charged for a ride I never took. Please refund me as soon as possible. |
13 | The promo code I tried to use didn’t work. Can you assist with this? |
14 | There was a suspicious smell in the car, and I’m worried about hygiene standards. |
15 | My driver was very considerate, especially helping me with my luggage. Appreciate the great service! |
16 | The app’s GPS seems inaccurate. It directed the driver to the wrong pick-up location. |
17 | I want to compliment my driver’s excellent navigation and time management during rush hour. |
18 | The vehicle didn’t match the description in the app. It was confusing and concerning. |
19 | I faced an issue with payment processing after my last ride. Can you look into this? |
Running the survey
We run the survey by adding the scenarios, agent and model with theby()
method and then calling the run()
method:
Results
that includes information about all the components, including the prompts and responses. We can see a list of all the components: