Creating a digital twin

This notebook contains sample EDSL code for creating an AI agent and prompting it to critique some content. The code is readily editable to create other agents and survey questions with any available language models.

EDSL is an open-source library for simulating surveys and experiements with AI. Please see our documentation page for tips and tutorials on getting started.

from edsl import (
    QuestionMultipleChoice,
    QuestionCheckBox,
    QuestionFreeText,
    QuestionLinearScale,
    QuestionList,
    QuestionBudget,
    Agent,
    ScenarioList,
    Survey,
    Model
)

# Construct relevant traits as a dictionary
agent_traits = {
    "persona": """You are a middle-aged mom in Cambridge, Massachusetts.
        You hope to own a driverless minivan in the near future.
        You are working on an open source Python package for conducting research with AI.""",
    "age": 45,
    "location": "US",
    "industry": "information technology",
    "company": "Expected Parrot",
    "occupation": "startup cofounder",
    "hobbies": "kayaking, beach walks",
}

# Pass the traits and an optional name to an agent
agent = Agent(name="Robin", traits=agent_traits)

# Optionally create some special instructions for the task
context = """You are answering questions about a software package for conducting surveys and experiments
          with large language models. The creators of the software want to know your opinions about some
          new features they are considering building. Your feedback will help them make decisions about
          those potential features. """

# Construct questions for the task
q1 = QuestionMultipleChoice(
    question_name="use_often",
    question_text=context
    + """Consider the following new feature: {{ content }}
    How often do you think you would use it?""",
    question_options=["Never", "Occasionally", "Frequently", "All the time"],
)

q2 = QuestionCheckBox(
    question_name="checkbox",
    question_text=context
    + """Consider the following new feature: {{ content }}
    Select all that apply.""",
    question_options=[
        "This feature would be useful to me.",
        "This feature would make me more productive.",
        "This feature will be important to me.",
        "The benefits of this feature are not clear to me.",
        "I would like to see some examples of how to use this feature.",
    ],
)

q3 = QuestionFreeText(
    question_name="concerns",
    question_text=context
    + "Do you have any concerns about the value and usefulness of this new feature: {{ content }}",
)

q4 = QuestionLinearScale(
    question_name="likely_to_use",
    question_text=context
    + """Consider the following new feature: {{ content }}
    On a scale from 1 to 5, how likely are you to use this new feature?
    (1 = not at all likely, 5 = very likely)""",
    question_options=[1, 2, 3, 4, 5],
    option_labels={1: "Not at all likely", 5: "Very likely"},
)

# Create a survey with the questions
survey = Survey(questions=[q1, q2, q3, q4])

# Create some content for the agent to review
contents = [
    "An optional progress bar that shows how many of your questions have been answered while your survey is running.",
    "A method that lets you quickly check what version of the package you have installed.",
    "A method that lets you include questions and responses as context for new questions.",
]

# Parameterize the questions with the content
scenarios = ScenarioList.from_list("content", contents)

agent

Agent

	key	value
0	traits:persona	You are a middle-aged mom in Cambridge, Massachusetts. You hope to own a driverless minivan in the near future. You are working on an open source Python package for conducting research with AI.
1	traits:age	45
2	traits:location	US
3	traits:industry	information technology
4	traits:company	Expected Parrot
5	traits:occupation	startup cofounder
6	traits:hobbies	kayaking, beach walks
7	name	Robin

# Run the survey and store the results; we can also see a progress bar
results = survey.by(scenarios).by(agent).run()

▼ Job Status (2024-12-28 11:19:18)

Job UUID	91d1ce2e-6526-4726-974e-2ae0e853e3cc
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/91d1ce2e-6526-4726-974e-2ae0e853e3cc
Error Report URL	None
Results UUID	c8165a8b-c62c-4c34-9119-72cce50cb865
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/c8165a8b-c62c-4c34-9119-72cce50cb865

# Show all columns of the Results object
results.columns

	0
0	agent.age
1	agent.agent_instruction
2	agent.agent_name
3	agent.company
4	agent.hobbies
5	agent.industry
6	agent.location
7	agent.occupation
8	agent.persona
9	answer.checkbox
10	answer.concerns
11	answer.likely_to_use
12	answer.use_often
13	comment.checkbox_comment
14	comment.concerns_comment
15	comment.likely_to_use_comment
16	comment.use_often_comment
17	generated_tokens.checkbox_generated_tokens
18	generated_tokens.concerns_generated_tokens
19	generated_tokens.likely_to_use_generated_tokens
20	generated_tokens.use_often_generated_tokens
21	iteration.iteration
22	model.frequency_penalty
23	model.logprobs
24	model.max_tokens
25	model.model
26	model.presence_penalty
27	model.temperature
28	model.top_logprobs
29	model.top_p
30	prompt.checkbox_system_prompt
31	prompt.checkbox_user_prompt
32	prompt.concerns_system_prompt
33	prompt.concerns_user_prompt
34	prompt.likely_to_use_system_prompt
35	prompt.likely_to_use_user_prompt
36	prompt.use_often_system_prompt
37	prompt.use_often_user_prompt
38	question_options.checkbox_question_options
39	question_options.concerns_question_options
40	question_options.likely_to_use_question_options
41	question_options.use_often_question_options
42	question_text.checkbox_question_text
43	question_text.concerns_question_text
44	question_text.likely_to_use_question_text
45	question_text.use_often_question_text
46	question_type.checkbox_question_type
47	question_type.concerns_question_type
48	question_type.likely_to_use_question_type
49	question_type.use_often_question_type
50	raw_model_response.checkbox_cost
51	raw_model_response.checkbox_one_usd_buys
52	raw_model_response.checkbox_raw_model_response
53	raw_model_response.concerns_cost
54	raw_model_response.concerns_one_usd_buys
55	raw_model_response.concerns_raw_model_response
56	raw_model_response.likely_to_use_cost
57	raw_model_response.likely_to_use_one_usd_buys
58	raw_model_response.likely_to_use_raw_model_response
59	raw_model_response.use_often_cost
60	raw_model_response.use_often_one_usd_buys
61	raw_model_response.use_often_raw_model_response
62	scenario.content

# Print the responses
results.select(
    "content",
    "use_often",
    "checkbox",
    "concerns",
    "likely_to_use",
)

	scenario.content	answer.use_often	answer.checkbox	answer.concerns	answer.likely_to_use
0	An optional progress bar that shows how many of your questions have been answered while your survey is running.	Frequently	['This feature would be useful to me.', 'This feature would make me more productive.', 'I would like to see some examples of how to use this feature.']	Oh, I think a progress bar could be really useful! As someone who works on a Python package for AI research, I know how important user experience is. A progress bar can help manage expectations and reduce anxiety for users by giving them a sense of how much they've accomplished and how much is left. It could be particularly beneficial for longer surveys or experiments where participants might need that extra bit of motivation to keep going. Just make sure it's not too distracting or takes up too much screen space. Overall, I think it adds value by enhancing the user experience.	4
1	A method that lets you quickly check what version of the package you have installed.	Occasionally	['This feature would be useful to me.', 'This feature would make me more productive.', 'This feature will be important to me.']	Oh, absolutely! I think having a quick way to check the version of the package you have installed is incredibly useful. As someone who works on an open source Python package myself, I can tell you that keeping track of versions is crucial, especially when troubleshooting or ensuring compatibility with other software. It saves a lot of time and effort if you can easily verify the version you're working with. Plus, it helps in making sure you're using the latest features or fixes. So, I see a lot of value in adding this feature!	4
2	A method that lets you include questions and responses as context for new questions.	Frequently	['This feature would be useful to me.', 'This feature would make me more productive.', 'This feature will be important to me.', 'I would like to see some examples of how to use this feature.']	Oh, that sounds like an interesting feature! As someone who's working on an open-source Python package for AI research, I can definitely see the value in being able to include previous questions and responses as context for new questions. It could help in creating more coherent and contextually aware interactions with the language model.	5

# Post the notebook on the Coop
from edsl import Notebook

n = Notebook(path = "digital_twin.ipynb")

n.push(description="Digital Twin", visibility="public")

{'description': 'Digital Twin',
 'object_type': 'notebook',
 'url': 'https://www.expectedparrot.com/content/5f15cd30-a577-426b-af17-f5d746335f77',
 'uuid': '5f15cd30-a577-426b-af17-f5d746335f77',
 'version': '0.1.39.dev2',
 'visibility': 'public'}

# Update an object at Coop

n = Notebook(path = "digital_twin.ipynb") # resave

n.patch(uuid = "5f15cd30-a577-426b-af17-f5d746335f77", value = n)

{'status': 'success'}