Jobs

A Jobs object combines a survey with agents, scenarios, and models. Calling by() constructs the execution specification; calling run() executes it. Think of a job as the complete set of interviews you want to collect. Each interview runs the survey with one agent, one scenario, and one model and produces one result record containing its answers. EDSL runs the combinations for you, so you can compare conditions without writing a separate loop for each one.

Construct a job

This example crosses four respondent profiles with two message conditions. It constructs eight interviews per model and iteration without making model calls.

from edsl import Agent, AgentList, QuestionMultipleChoice, Scenario, ScenarioList

agents = AgentList([
    Agent(traits={"experience": experience, "role": role},
          instruction="Answer using only the supplied profile and message.")
    for experience in ["new user", "experienced user"]
    for role in ["individual contributor", "team lead"]
])
messages = ScenarioList([
    Scenario({"condition": "short", "message": "Export your report."}),
    Scenario({"condition": "explicit", "message": "Download your report as a CSV file."}),
])
question = QuestionMultipleChoice(
    question_name="clarity",
    question_text="How clear is this instruction: {{ scenario.message }}?",
    question_options=["Clear", "Unclear", "Unsure"],
)
job = question.by(agents).by(messages)
# Four profiles crossed with two messages gives eight interviews per model.

Before execution, choose a model, inspect the prompts with job.by(model).prompts(), and estimate costs. A ModelList expands the job across the selected configurations.

Execute and inspect

After configuring credentials and model access:

from edsl import Model

results = job.by(Model()).run()
results.select("scenario.condition", "agent.experience", "answer.clarity")

This step makes model calls and can incur costs. Use disable_remote_inference=True for local orchestration; provider calls may still be remote. Use a test model for an offline smoke test as shown in the Run your first survey tutorial. With one model and iteration, this example returns eight rows. Each row contains a profile’s clarity rating for one message. The selected columns retain the condition and experience level needed to compare those ratings. See Results for filtering, grouping, and exporting them.

Background runs and persistence

Save a job as an .ep package to retain the full run specification. Use Background jobs for long remote work and the CLI Guide to submit, poll, and retrieve results from scripts or coding agents.

Pair inputs or specify assignments

By default, by() crosses agents, scenarios, and models. Use zip_assign() when each agent should receive the scenario at the same position:

from edsl import Agent, AgentList, Model, QuestionFreeText, ScenarioList

agents = AgentList([Agent(traits={"role": "editor"}), Agent(traits={"role": "reader"})])
messages = ScenarioList.from_list("message", ["First draft", "Second draft"])
question = QuestionFreeText(
    question_name="feedback", question_text="Review {{ scenario.message }}."
)
paired_job = question.by(agents).by(messages).by(Model("test"))
paired_job.zip_assign(over=("agents", "scenarios"))
len(paired_job.interviews())  # Two paired interviews, instead of four crossed interviews.

The paired collections must have equal lengths by default (strict=True). Each pair is still crossed with the remaining models. job.assign([{"agent": 0, "scenario": 1, "model": 0}, ...]) instead specifies exact combinations using zero-based indices into the job’s component lists. For a condition over combinations, use job.include_when("{{ agent.role == scenario.required_role }}"). The named fields must exist in your inputs. Calling assign() or zip_assign() replaces an inclusion expression; calling include_when() replaces an assignment plan. Inspect job.interviews() after choosing the design and before running it.

Reuse a cache or run asynchronously

job.using(cache) or job.using_cache(cache) attaches a Cache to the job; passing cache=cache to run() is the explicit per-run alternative. survey.using(cache) and question.using(cache) construct Jobs with that cache attached. In an asynchronous function or a notebook that supports top-level await, use await job.run_async(...). It accepts run options such as cache, n, disable_remote_inference, and disable_remote_cache. Background submission is a different workflow: see Background jobs.

Common methods

Call	Arguments and result
`survey.by(agents).by(scenarios).by(models)`	Build a Jobs object from study components.
`job.interviews()`	Return the planned interview objects so you can inspect the combinations before execution.
`job.prompts(iterations=1)`	Return a Dataset of rendered prompts for the requested number of iterations.
`job.estimate_job_cost(iterations=1)`	Return estimated costs and token counts; see Costs.
`job.estimate_remote_job_cost(...)`	Return a detailed estimate with question and model breakdowns; see Detailed estimates.
`job.run(...)`	Execute the configured interviews and return Results for a completed blocking run.
`job.save("job.ep")` / `Jobs.load("job.ep")`	Save and restore the full job specification locally.

Execution options

Pass these keywords directly to job.run(). A real model run can incur costs; review model access and costs first.

Keyword	Use
`n=1`	Number of iterations for each combination of agents, scenarios, and models.
`cache=cache`	Use a specific local Cache object for response reuse.
`fresh=True`	Request fresh responses from hosted execution. For direct local runs, use an empty Cache and disable remote caching to avoid existing entries.
`disable_remote_inference=True`	Run orchestration locally with direct model access. Provider calls can still be remote.
`offload_execution=True`	Request hosted execution on Expected Parrot.
`disable_remote_cache=True`	Disable access to the hosted response cache.
`max_concurrency=10`	Bound concurrent interviews in locally orchestrated execution; choose a value appropriate to the workload and provider quotas.
`progress_bar=True`	Show execution progress.
`stop_on_exception=True`	Stop on an interview exception.

For a small local run using the eight combinations constructed above:

from edsl import Cache, Model

results = job.by(Model("test", canned_response="Clear")).run(
    n=2,
    cache=Cache(),
    disable_remote_inference=True,
    disable_remote_cache=True,
    progress_bar=False,
)
results.select("scenario.condition", "agent.experience", "answer.clarity")
# In a script, append .print() to display the table.

n=2 repeats each of the eight combinations twice, producing 16 interview records with fixed test answers here. It does not create new profiles or conditions. Select iteration alongside the answers when comparing repetitions. Use background jobs for submission and later retrieval, and remote execution for hosted execution settings.

Package a reusable workflow

A Macro combines an input survey, a job that uses those inputs as scenario fields, and named output formats. Use one when other users should supply a few parameters to a repeatable EDSL workflow.

from edsl import Model, QuestionFreeText, Survey
from edsl.macros import Macro, OutputFormatter

inputs = Survey([
    QuestionFreeText(question_name="text", question_text="Which text should be reviewed?"),
])
review_job = QuestionFreeText(
    question_name="feedback",
    question_text="Suggest one improvement to: {{ scenario.text }}",
).by(Model("test", canned_response="Use a more specific heading."))

macro = Macro(
    application_name="text_review",
    display_name="Text review",
    short_description="Suggest an improvement to a text.",
    long_description="Review supplied text and return feedback as a table.",
    initial_survey=inputs,
    jobs_object=review_job,
    output_formatters={
        "answers": OutputFormatter(output_type="Dataset").select("answer.feedback"),
    },
    default_formatter_name="answers",
)

Construction does not execute the job. The test model demonstrates the structure; replace it with your selected provider model for actual feedback. Then supply values keyed by the input survey’s question names:

output = macro.output(
    params={"text": "Our tool improves work."},
    disable_remote_inference=True,
)
output.answers
# output.results gives the underlying Results object.

macro.output() returns a MacroRunOutput; accessing a named format applies the corresponding OutputFormatter to its Results. Provider execution requires credentials and may incur costs. macro.by({"text": "..."}).run() and macro.run(text="...") are parameter-setting shortcuts using the default execution settings; use output() when specifying execution options.

macro.with_output_formatter("rows", OutputFormatter(output_type="ScenarioList").select("answer.feedback").to_scenario_list())

returns a macro with another named output. A macro instance can reuse previously generated results for the same prepared job, so repeated calls are not automatically independent model draws. For sharing, macro.push(visibility="private") uploads the workflow; Macro.pull(identifier) retrieves a full macro when you have access. Macro("owner/alias") creates a client for a deployed macro and executes it remotely when called. Access to a deployed macro and its underlying definition can differ.

Introduction

Setup and tutorials

EDSL objects

How-to guides

Research examples

Human research

Execution and costs

Saving and sharing

Research and teaching

Expected Parrot platform

Contributing

Construct a job

Execute and inspect

Background runs and persistence

Pair inputs or specify assignments

Reuse a cache or run asynchronously

Common methods

Execution options

Package a reusable workflow

​Construct a job

​Execute and inspect

​Background runs and persistence

​Pair inputs or specify assignments

​Reuse a cache or run asynchronously

​Common methods

​Execution options

​Package a reusable workflow

Construct a job

Execute and inspect

Background runs and persistence

Pair inputs or specify assignments

Reuse a cache or run asynchronously

Common methods

Execution options

Package a reusable workflow