Simulating randomness

This notebook provides example EDSL code for exploring ways of prompting AI agents to return “random” numbers. We do this by passing agents different instructions for picking a number at random, and by including and excluding memories of prior selections.

We also show how to use the QuestionFunctional question type to compare the responses using a function instead of calling a model. Learn more about this question type.

Creating questions to generate “random” numbers

We start by creating some questions prompting an agent to return a single random number and a list of random numbers. We then combine the questions in a survey and add “memories” of prior questions to some of the questions in order to compare responses generated with and without information about other responses. We also investigate how agents handle instructions to ignore a prior response that is nevertheless included, and to an instruction to return a list of random numbers all at once. This code is readily editable for further exploration, such as adding agent traits and personas to compare their additional impact on responses.

[1]:

from edsl import QuestionNumerical, QuestionList, Survey, Agent, AgentList, Model, ModelList

m = ModelList([
    Model("gemini-1.5-flash", service_name = "google"),
    Model("gpt-4o", service_name = "openai"),
    Model("claude-sonnet-4-20250514", service_name = "anthropic")
])

q_random = QuestionNumerical(
    question_name="random",
    question_text="Pick a random number between 1 and 50."
)

# We will administer this question with no memory of the prior question
q_random_no_memory = QuestionNumerical(
    question_name="random_no_memory",
    question_text="Pick a random number between 1 and 50.",
)

# We will administer this question with a memory of the first question
q_random_memory = QuestionNumerical(
    question_name="random_memory",
    question_text="Pick a random number between 1 and 50.",
)

# We will administer this question with a memory of the first question but instruct the agent to ignore it
q_random_ignore = QuestionNumerical(
    question_name="random_ignore",
    question_text="""Pick a random number between 1 and 50.
    Be sure to completely disregard the other random number that you picked.""",
)

# We will administer this question with no memory of prior questions
q_random_list = QuestionList(
    question_name="random_list",
    question_text="Pick 5 random numbers between 1 and 50. Return a list of integers.",
    max_list_items=5,
)

# Combine the questions in a survey
survey = Survey(questions =
    [q_random, q_random_no_memory, q_random_memory, q_random_ignore, q_random_list]
)

# Adding targeted question memories - the prior question and answer become part of the new question prompt
survey = (
    survey
        .add_targeted_memory(q_random_memory, q_random)
        .add_targeted_memory(q_random_ignore, q_random)
)

# Create a variety of instructions for the agents
instructions = [
    "Respond as if you are a real human trying to pick randomly.",
    "Respond as if you are simulating actual random events.",
    "Respond as if you are playing a game.",
    "Respond as if you are participating in a lottery.",
    "Respond as if you are testing software.",
    "Respond as if you are choosing among indistinguishable options.",
    "Respond spontaneously.",
    "Respond erratically.",
    "Respond chaotically.",
]

# Just pass the instructions - we could also explore variations in agent traits, eg personas or knowledge
a = AgentList(
    Agent(traits={}, instruction=i) for i in instructions
)

results = survey.by(a).by(m).run(fresh=True)

(
    results
        .sort_by("random")
        .select(
            "model",
            "agent_instruction",
            "random",
            "random_no_memory",
            "random_memory",
            "random_ignore",
            "random_list"
        )
)

⌃ Job Status 🦜

Completed (27 completed, 0 failed)

Job Links

Results

Progress Report

Content

Remote Jobs

Remote Cache

Identifiers

Results UUID:

6abc6473...3cee

Use Results.pull(uuid) to fetch results.

Job UUID:

ee5df465...f691

Use Jobs.pull(uuid) to fetch job.