Data labeling agents
This notebook shows how to conduct data labeling tasks using EDSL, an open-source library for simulating surveys, experiments and other research with AI agents and large language models. This workflow consists of the following steps:
Import data into EDSL
Create questions about the data
Design an AI agent to answer the questions
Select a language model to generate responses
Analyze results as a formatted dataset
This workflow can be visualized as follows:
Conducting agent-specific tasks
We can add a layer of complexity to this generalized flow by creating different AI agents for subsets of the data to be reviewed. For example, we can design agents with specific “expertise” to review only the data that is relevant to that expertise. This can be useful if our data is sorted (or sortable) in some way that is important to our task. We can also use EDSL to prompt a language model to sort the data as needed.
This modified workflow can be visualized as follows:
Example task: Evaluating job posts
Using a dataset of job posts as an example, in the steps below we create AI agents with expertise in the relevant job categories and then prompt them to evaluate relevant job posts in a variety of ways. The steps are:
Import a dataset of job categories and job posts.
Construct questions about the job posts and combine them in a survey.
Design AI agents with job category expertise.
Administer the survey to each agent with job posts for the relevant category.
Inspect the results using built-in methods for analysis.
Technical setup
Before running the code below please ensure that you have completed setup:
Create a Coop account and activate remote inference OR store your own API Keys for language models that you want to use.
Our Starter Tutorial provides examples of EDSL basic components. An introductory data labeling example notebook may also be useful to you.
Import the tools
We start by selecting question types and survey components that we will use. Please see the EDSL Docs for examples of all question types and details on these basic components.
[1]:
from edsl import (
QuestionMultipleChoice, QuestionFreeText, QuestionLinearScale, QuestionList, QuestionNumerical,
Survey, ScenarioList, Scenario, AgentList, Agent, ModelList, Model
)
Import data
Next we import a dataset for review, using Scenario objects to represent the individual data that will be added to each of our data labeling questions.
[2]:
# job_category,job_title,job_post
# Content Writing,Blog Post Writing,"Looking for a skilled writer to produce 5 blog posts on digital marketing topics. Each post should be 800-1000 words, well-researched, and SEO-optimized."
# Content Writing,Product Description Writing,"We need a writer to craft compelling product descriptions for our online store. Each description should highlight the key features and benefits of the product."
# Content Writing,Technical Writing for Software Documentation,"Seeking an experienced technical writer to create user manuals and API documentation for our software product. Must have a background in tech writing and be familiar with software development terminology."
# Content Writing,Website Copywriting,"Looking for a copywriter to create persuasive content for our company’s website. The content should be clear, concise, and align with our brand voice."
# Content Writing,Press Release Writing,"We need a writer to draft a press release for our upcoming product launch. The release should be attention-grabbing and follow industry standards."
# Digital Marketing,Social Media Management,"We are looking for a social media manager to handle our Instagram and Twitter accounts. Responsibilities include content creation, scheduling posts, and engaging with followers."
# Digital Marketing,SEO Optimization,"Need an SEO expert to optimize our website for search engines. The project includes keyword research, on-page optimization, and link-building strategies."
# Digital Marketing,Google Ads Campaign Management,"Looking for a PPC specialist to manage our Google Ads campaigns. The goal is to increase traffic and conversions for our online store."
# Digital Marketing,Email Marketing Campaign,"Seeking an email marketing expert to design and execute a series of email campaigns for our new product launch. Experience with Mailchimp is preferred."
# Digital Marketing,Content Marketing Strategy,"Seeking a content marketing strategist to develop a comprehensive plan to increase our online visibility. The strategy should include content creation, distribution, and performance tracking."
# Graphic Design,Logo Design for New Startup,"We are a new tech startup looking for a creative designer to create a unique logo for our brand. The logo should be modern and represent innovation. Please provide portfolio examples."
# Graphic Design,Brochure Design,"Looking for an experienced designer to create a professional brochure for our real estate company. The brochure should highlight our services and properties. Must be delivered in print-ready format."
# Graphic Design,Social Media Graphics,"Need a designer to create eye-catching social media graphics for our upcoming campaign. We need a set of 10 images optimized for Instagram and Facebook."
# Graphic Design,Website Banner Design,"Seeking a skilled designer to create a series of banners for our e-commerce website. Banners should be consistent with our brand’s aesthetic. Please include examples of previous work."
# Graphic Design,Infographic Design,"We need a designer to create a visually appealing infographic based on our provided data. The infographic should be easy to understand and shareable on social media."
# Web Development,WordPress Website Setup,"We need a developer to set up a WordPress site for our small business. The site should be responsive and include a contact form, blog, and e-commerce functionality. Experience with WooCommerce is a plus."
# Web Development,Custom Web Application Development,"Looking for a full-stack developer to build a custom web application for managing employee schedules. The app should include a login system, user roles, and reporting features."
# Web Development,Shopify Store Customization,"Seeking a Shopify expert to customize our online store. We need theme adjustments, product page enhancements, and integration with third-party tools."
# Web Development,API Integration,"Need a developer to integrate our existing CRM system with an external API. The integration should sync customer data in real-time. Previous experience with similar projects required."
# Web Development,Landing Page Development,"Looking for a developer to create a high-converting landing page for our marketing campaign. The page should be optimized for mobile and desktop users."
We have stored the file at the Coop and can re-import it:
[3]:
from edsl.scenarios.FileStore import CSVFileStore
[4]:
csv_file = CSVFileStore.pull('97ca56a7-4916-4fc4-9bc7-521f134add2b', expected_parrot_url='https://www.expectedparrot.com')
[5]:
scenarios = ScenarioList.from_csv(csv_file.to_tempfile())
[6]:
# Code for uploading a CSV to the Coop:
# refresh = False
# if refresh:
# from edsl.scenarios.FileStore import CSVFileStore
# fs = CSVFileStore("job_posts.csv")
# info = fs.push()
# print(info)
[7]:
# Alternatively importing a local file:
# scenarios = ScenarioList.from_csv("job_posts.csv")
We can inspect the scenarios that have been created and edit them as desired. Learn more about working with scenarios.
[8]:
scenarios
[8]:
{
"scenarios": [
{
"job_category": "Content Writing",
"job_title": "Blog Post Writing",
"job_post": "Looking for a skilled writer to produce 5 blog posts on digital marketing topics. Each post should be 800-1000 words, well-researched, and SEO-optimized."
},
{
"job_category": "Content Writing",
"job_title": "Product Description Writing",
"job_post": "We need a writer to craft compelling product descriptions for our online store. Each description should highlight the key features and benefits of the product."
},
{
"job_category": "Content Writing",
"job_title": "Technical Writing for Software Documentation",
"job_post": "Seeking an experienced technical writer to create user manuals and API documentation for our software product. Must have a background in tech writing and be familiar with software development terminology."
},
{
"job_category": "Content Writing",
"job_title": "Website Copywriting",
"job_post": "Looking for a copywriter to create persuasive content for our company\u2019s website. The content should be clear, concise, and align with our brand voice."
},
{
"job_category": "Content Writing",
"job_title": "Press Release Writing",
"job_post": "We need a writer to draft a press release for our upcoming product launch. The release should be attention-grabbing and follow industry standards."
},
{
"job_category": "Digital Marketing",
"job_title": "Social Media Management",
"job_post": "We are looking for a social media manager to handle our Instagram and Twitter accounts. Responsibilities include content creation, scheduling posts, and engaging with followers."
},
{
"job_category": "Digital Marketing",
"job_title": "SEO Optimization",
"job_post": "Need an SEO expert to optimize our website for search engines. The project includes keyword research, on-page optimization, and link-building strategies."
},
{
"job_category": "Digital Marketing",
"job_title": "Google Ads Campaign Management",
"job_post": "Looking for a PPC specialist to manage our Google Ads campaigns. The goal is to increase traffic and conversions for our online store."
},
{
"job_category": "Digital Marketing",
"job_title": "Email Marketing Campaign",
"job_post": "Seeking an email marketing expert to design and execute a series of email campaigns for our new product launch. Experience with Mailchimp is preferred."
},
{
"job_category": "Digital Marketing",
"job_title": "Content Marketing Strategy",
"job_post": "Seeking a content marketing strategist to develop a comprehensive plan to increase our online visibility. The strategy should include content creation, distribution, and performance tracking."
},
{
"job_category": "Graphic Design",
"job_title": "Logo Design for New Startup",
"job_post": "We are a new tech startup looking for a creative designer to create a unique logo for our brand. The logo should be modern and represent innovation. Please provide portfolio examples."
},
{
"job_category": "Graphic Design",
"job_title": "Brochure Design",
"job_post": "Looking for an experienced designer to create a professional brochure for our real estate company. The brochure should highlight our services and properties. Must be delivered in print-ready format."
},
{
"job_category": "Graphic Design",
"job_title": "Social Media Graphics",
"job_post": "Need a designer to create eye-catching social media graphics for our upcoming campaign. We need a set of 10 images optimized for Instagram and Facebook."
},
{
"job_category": "Graphic Design",
"job_title": "Website Banner Design",
"job_post": "Seeking a skilled designer to create a series of banners for our e-commerce website. Banners should be consistent with our brand\u2019s aesthetic. Please include examples of previous work."
},
{
"job_category": "Graphic Design",
"job_title": "Infographic Design",
"job_post": "We need a designer to create a visually appealing infographic based on our provided data. The infographic should be easy to understand and shareable on social media."
},
{
"job_category": "Web Development",
"job_title": "WordPress Website Setup",
"job_post": "We need a developer to set up a WordPress site for our small business. The site should be responsive and include a contact form, blog, and e-commerce functionality. Experience with WooCommerce is a plus."
},
{
"job_category": "Web Development",
"job_title": "Custom Web Application Development",
"job_post": "Looking for a full-stack developer to build a custom web application for managing employee schedules. The app should include a login system, user roles, and reporting features."
},
{
"job_category": "Web Development",
"job_title": "Shopify Store Customization",
"job_post": "Seeking a Shopify expert to customize our online store. We need theme adjustments, product page enhancements, and integration with third-party tools."
},
{
"job_category": "Web Development",
"job_title": "API Integration",
"job_post": "Need a developer to integrate our existing CRM system with an external API. The integration should sync customer data in real-time. Previous experience with similar projects required."
},
{
"job_category": "Web Development",
"job_title": "Landing Page Development",
"job_post": "Looking for a developer to create a high-converting landing page for our marketing campaign. The page should be optimized for mobile and desktop users."
}
]
}
Construct questions about the data
Next we construct questions to ask about the job posts, selecting question types based on the form of the response that we want to get back from the language model (multiple choice, linear scale, free text, numerical, etc.–see examples of all question types). We include a {{ placeholder }} for the scenario keys in order to parameterize each question with each job post and category when we run the survey:
[9]:
q_skills = QuestionList(
question_name="skills",
question_text="""
Consider the following job category and job post at an online labor marketplace.
Job category: {{ job_category }}
Job post: {{ job_post }}
What are some key skills required for this job?
""",
)
q_experience = QuestionMultipleChoice(
question_name="experience",
question_text="""
Consider the following job category and job post at an online labor marketplace.
Job category: {{ job_category }}
Job post: {{ job_post }}
What level of experience is required for this job?
""",
question_options=["Entry-level", "Mid-level", "Senior-level"],
)
q_days = QuestionNumerical(
question_name="days",
question_text="""
Consider the following job category and job post at an online labor marketplace.
Job category: {{ job_category }}
Job post: {{ job_post }}
Estimate the number of days until this job post is fulfilled.
""",
)
Combining questions into a Survey
Next we combine our questions into a survey that will be administered to the AI agents. By default, the questions will be administered asynchronously. If desired, we can also specify survey rules (skip/stop logic) and within-survey memories of prior questions and responses. See the EDSL Docs for details on methods for applying survey rules.
[10]:
survey = Survey(questions=[q_skills, q_experience, q_days])
Creating personas for Agents
Next we draft personas for AI agents that will answer the questions. For each job category we construct an AI agent that is an expert in the category. Agents are constructed by passing a dictionary of traits to an Agent object. Learn more about designing AI agents to answer surveys.
To get the set of job categories from the scenarios:
[11]:
job_categories = list(set(scenarios.select("job_category").to_list()))
job_categories
[11]:
['Content Writing', 'Digital Marketing', 'Web Development', 'Graphic Design']
Next we use them to create an agent for each job category:
[12]:
agents = AgentList(
Agent(
traits = {
"persona": "You are an experienced freelancer on online labor marketplaces.",
"job_category": job_category,
"expertise": f"You regularly perform jobs in the following category: {job_category}."
}
) for job_category in job_categories
)
agents
[12]:
[
{
"traits": {
"persona": "You are an experienced freelancer on online labor marketplaces.",
"job_category": "Content Writing",
"expertise": "You regularly perform jobs in the following category: Content Writing."
},
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Agent"
},
{
"traits": {
"persona": "You are an experienced freelancer on online labor marketplaces.",
"job_category": "Digital Marketing",
"expertise": "You regularly perform jobs in the following category: Digital Marketing."
},
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Agent"
},
{
"traits": {
"persona": "You are an experienced freelancer on online labor marketplaces.",
"job_category": "Web Development",
"expertise": "You regularly perform jobs in the following category: Web Development."
},
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Agent"
},
{
"traits": {
"persona": "You are an experienced freelancer on online labor marketplaces.",
"job_category": "Graphic Design",
"expertise": "You regularly perform jobs in the following category: Graphic Design."
},
"edsl_version": "0.1.33.dev1",
"edsl_class_name": "Agent"
}
]
Selecting language models
EDSL works with many popular language models that we can select to generate the agents’ responses to the survey. We can check a current list of available models:
[13]:
from edsl import Model
# Model.available()
If we do not specify a model to use, GPT 4 preview is used by default. Here we create a Model for GPT 4o that we will add to the survey when we run it:
[14]:
model = Model("gpt-4o")
Running the survey
We administer a survey by appending the components with the by() method and then calling run() method. In the simplest case where we want a single agent or list of agents to answer all questions with the same scenarios, this takes the following form:
results = survey.by(scenarios).by(agents).by(models).run()
Here we have individual agents answer the questions only for category-specific job posts:
[15]:
results = {}
for job_category in job_categories:
# Create an agent for the job category
a = agents.filter(f"job_category == '{job_category}'")
# Filter the relevant scenarios
s = scenarios.filter(f"job_category == '{job_category}'")
# Run the survey with the agent and scenarios
job_category_results = survey.by(s).by(a).run()
# Store the results
results[job_category] = job_category_results
Accessing Results
In the previous step we created independent Results objects for our individual agents’ survey results. In the next steps we show how to access results with built-in print and analytical methods.
We can identify the column names to select the fields that we want to inspect:
[16]:
results.keys()
[16]:
dict_keys(['Content Writing', 'Digital Marketing', 'Web Development', 'Graphic Design'])
[17]:
results["Graphic Design"].columns
[17]:
['agent.agent_instruction',
'agent.agent_name',
'agent.expertise',
'agent.persona',
'answer.days',
'answer.experience',
'answer.skills',
'comment.days_comment',
'comment.experience_comment',
'comment.skills_comment',
'generated_tokens.days_generated_tokens',
'generated_tokens.experience_generated_tokens',
'generated_tokens.skills_generated_tokens',
'iteration.iteration',
'model.frequency_penalty',
'model.logprobs',
'model.max_tokens',
'model.model',
'model.presence_penalty',
'model.temperature',
'model.top_logprobs',
'model.top_p',
'prompt.days_system_prompt',
'prompt.days_user_prompt',
'prompt.experience_system_prompt',
'prompt.experience_user_prompt',
'prompt.skills_system_prompt',
'prompt.skills_user_prompt',
'question_options.days_question_options',
'question_options.experience_question_options',
'question_options.skills_question_options',
'question_text.days_question_text',
'question_text.experience_question_text',
'question_text.skills_question_text',
'question_type.days_question_type',
'question_type.experience_question_type',
'question_type.skills_question_type',
'raw_model_response.days_cost',
'raw_model_response.days_one_usd_buys',
'raw_model_response.days_raw_model_response',
'raw_model_response.experience_cost',
'raw_model_response.experience_one_usd_buys',
'raw_model_response.experience_raw_model_response',
'raw_model_response.skills_cost',
'raw_model_response.skills_one_usd_buys',
'raw_model_response.skills_raw_model_response',
'scenario.job_category',
'scenario.job_post',
'scenario.job_title']
We can select individual fields in a variety of ways:
[18]:
(
results["Graphic Design"]
.select("job_post", "skills", "experience", "days")
.print(format="rich")
)
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━┓ ┃ scenario ┃ answer ┃ answer ┃ answer ┃ ┃ .job_post ┃ .skills ┃ .experience ┃ .days ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━┩ │ We are a new tech startup looking for a │ ['Creativity', 'Logo Design', 'Branding', │ Mid-level │ 3.5 │ │ creative designer to create a unique logo │ 'Adobe Illustrator', 'Portfolio │ │ │ │ for our brand. The logo should be modern │ Presentation'] │ │ │ │ and represent innovation. Please provide │ │ │ │ │ portfolio examples. │ │ │ │ ├────────────────────────────────────────────┼─────────────────────────────────────────────┼─────────────┼────────┤ │ Looking for an experienced designer to │ ['Adobe InDesign', 'Adobe Photoshop', │ Mid-level │ 5 │ │ create a professional brochure for our │ 'Adobe Illustrator', 'Typography', 'Layout │ │ │ │ real estate company. The brochure should │ Design', 'Print Design'] │ │ │ │ highlight our services and properties. │ │ │ │ │ Must be delivered in print-ready format. │ │ │ │ ├────────────────────────────────────────────┼─────────────────────────────────────────────┼─────────────┼────────┤ │ Need a designer to create eye-catching │ ['Adobe Photoshop', 'Adobe Illustrator', │ Mid-level │ 5 │ │ social media graphics for our upcoming │ 'Social Media Marketing', 'Creativity', │ │ │ │ campaign. We need a set of 10 images │ 'Attention to Detail', 'Typography'] │ │ │ │ optimized for Instagram and Facebook. │ │ │ │ ├────────────────────────────────────────────┼─────────────────────────────────────────────┼─────────────┼────────┤ │ Seeking a skilled designer to create a │ ['Graphic design', 'Brand consistency', │ Mid-level │ 3 │ │ series of banners for our e-commerce │ 'Banner design', 'Adobe Creative Suite │ │ │ │ website. Banners should be consistent with │ proficiency', 'Attention to detail', │ │ │ │ our brand’s aesthetic. Please include │ 'Portfolio of previous work'] │ │ │ │ examples of previous work. │ │ │ │ ├────────────────────────────────────────────┼─────────────────────────────────────────────┼─────────────┼────────┤ │ We need a designer to create a visually │ ['Data Visualization', 'Adobe Illustrator', │ Mid-level │ 3 │ │ appealing infographic based on our │ 'Creativity', 'Attention to Detail', │ │ │ │ provided data. The infographic should be │ 'Social Media Design'] │ │ │ │ easy to understand and shareable on social │ │ │ │ │ media. │ │ │ │ └────────────────────────────────────────────┴─────────────────────────────────────────────┴─────────────┴────────┘
We can apply some labels to our table for readability. Note that each question field also automatically includes a <question>_comment field for any commentary by the LLM on the question:
[19]:
(
results["Graphic Design"]
.select("job_post", "experience", "experience_comment")
.print(
pretty_labels={
"scenario.job_post": "Job post description",
"answer.experience": "Experience level",
"answer.experience_comment": "Comment",
}
)
)
| Job post description | Experience level | comment.experience_comment |
|---|---|---|
| We are a new tech startup looking for a creative designer to create a unique logo for our brand. The logo should be modern and represent innovation. Please provide portfolio examples. | Mid-level | The job post requires a creative designer to create a unique and modern logo that represents innovation, which typically requires a certain level of experience and expertise. Additionally, the request for portfolio examples suggests that the client is looking for someone with a proven track record, which is more commonly associated with mid-level experience. |
| Looking for an experienced designer to create a professional brochure for our real estate company. The brochure should highlight our services and properties. Must be delivered in print-ready format. | Mid-level | Given the requirement for a professional brochure that highlights services and properties in a print-ready format, this job likely requires someone with a solid understanding of design principles and experience with print production. This suggests a mid-level designer who has worked on similar projects before. |
| Need a designer to create eye-catching social media graphics for our upcoming campaign. We need a set of 10 images optimized for Instagram and Facebook. | Mid-level | This job requires creating eye-catching and optimized graphics for social media, which suggests a need for a good understanding of design principles, social media trends, and platform-specific requirements. These skills typically align with a mid-level designer's experience. |
| Seeking a skilled designer to create a series of banners for our e-commerce website. Banners should be consistent with our brand’s aesthetic. Please include examples of previous work. | Mid-level | This job requires the designer to create a series of banners that are consistent with the brand’s aesthetic, which implies a need for a good understanding of branding and design principles. Additionally, the request for examples of previous work suggests that the client expects a certain level of experience and expertise. |
| We need a designer to create a visually appealing infographic based on our provided data. The infographic should be easy to understand and shareable on social media. | Mid-level | This job requires a designer who can interpret data and create a visually appealing, easy-to-understand infographic suitable for social media. This requires a good understanding of both design principles and data visualization, which typically comes with some experience. |
We can also access results as a SQL table (called self) with the .sql() method, choosing between a “wide” horizontal view of all fields and a “long” vertical view, and optionally removing the column name prefixes ‘agent’, ‘model’, ‘prompt’, etc.:
[20]:
results["Graphic Design"].sql("select * from self", shape="long")
[20]:
| id | data_type | key | value | |
|---|---|---|---|---|
| 0 | 0 | agent | persona | You are an experienced freelancer on online la... |
| 1 | 0 | agent | job_category | Graphic Design |
| 2 | 0 | agent | expertise | You regularly perform jobs in the following ca... |
| 3 | 0 | agent | agent_name | Agent_0 |
| 4 | 0 | agent | agent_instruction | You are answering questions as if you were a h... |
| ... | ... | ... | ... | ... |
| 245 | 4 | comment | experience_comment | This job requires a designer who can interpret... |
| 246 | 4 | comment | days_comment | // Based on my experience, a straightforward i... |
| 247 | 4 | generated_tokens | skills_generated_tokens | ["Data Visualization", "Adobe Illustrator", "C... |
| 248 | 4 | generated_tokens | experience_generated_tokens | Mid-level\n\nThis job requires a designer who ... |
| 249 | 4 | generated_tokens | days_generated_tokens | 3\n// Based on my experience, a straightforwar... |
250 rows × 4 columns
Posting content at the Coop
We can post any EDSL objects to the Coop, including this notebook:
[21]:
agents.push(description = "Agents for job posts data labeling task", visibility = "public")
[21]:
{'description': 'Agents for job posts data labeling task',
'object_type': 'agent_list',
'url': 'https://www.expectedparrot.com/content/90131f79-74db-42d0-b2e0-a9c938366e07',
'uuid': '90131f79-74db-42d0-b2e0-a9c938366e07',
'version': '0.1.33.dev1',
'visibility': 'public'}
[22]:
survey.push(description = "Survey for job posts data labeling task", visibility = "public")
[22]:
{'description': 'Survey for job posts data labeling task',
'object_type': 'survey',
'url': 'https://www.expectedparrot.com/content/89fd522c-33f4-43c5-abdf-548f5c0d6cef',
'uuid': '89fd522c-33f4-43c5-abdf-548f5c0d6cef',
'version': '0.1.33.dev1',
'visibility': 'public'}
[23]:
for job_category in job_categories:
results[job_category].push(description = f"Results for job posts data labeling task: {job_category}", visibility = "public")
[24]:
from edsl import Notebook
[25]:
n = Notebook(path = "data_labeling_agent.ipynb")
[26]:
n.push(description = "Example code for data labeling using agents", visibility = "public")
[26]:
{'description': 'Example code for data labeling using agents',
'object_type': 'notebook',
'url': 'https://www.expectedparrot.com/content/d6574805-6d17-4768-ad96-25d3419ae42b',
'uuid': 'd6574805-6d17-4768-ad96-25d3419ae42b',
'version': '0.1.33.dev1',
'visibility': 'public'}
To update an object:
[27]:
n = Notebook(path = "data_labeling_agent.ipynb") # resave
[28]:
n.patch(uuid = "d6574805-6d17-4768-ad96-25d3419ae42b", value = n)
[28]:
{'status': 'success'}
Learn more about using the Coop to conduct LLM-based research.