Estimating and inspecting costs

EDSL provides methods to estimate a job before running it and inspect recorded costs afterward. Cost estimation can retrieve pricing information or request an estimate from Expected Parrot; it does not run the survey or generate model responses.

Estimate a prepared job

Combine a survey with the agents, scenarios, and models you plan to use. For example:

from edsl import Agent, Model, QuestionFreeText, Survey

survey = Survey([
    QuestionFreeText(
        question_name="favorite_flower",
        question_text="What is your favorite flower?",
    ),
    QuestionFreeText(
        question_name="flower_color",
        question_text="What color is {{ favorite_flower.answer }}?",
    ),
])
agent = Agent(traits={"occupation": "botanist"})
model = Model()  # Use your configured default, or specify a model from the catalog.
job = survey.by(agent).by(model)
# Alternatively: from edsl import Jobs; job = Jobs.load("jobs.ep")

Call estimate_job_cost() to get a dictionary of estimated costs and token counts:

estimate = job.estimate_job_cost(iterations=3)
estimate
# Omit iterations for one run; use iterations=3 to estimate a run with n=3.

Returned field	Meaning
`estimated_total_cost_usd`	Estimated cost in USD for all requested iterations.
`total_credits_hold`	Corresponding estimated credits.
`estimated_total_input_tokens`	Estimated input tokens across the job.
`estimated_total_output_tokens`	Estimated output tokens across the job.
`detailed_costs`	Breakdown by inference service, model, and input/output token type, including token counts, prices, and costs.

To inspect the breakdown as a table:

from edsl import ScenarioList

ScenarioList.from_list_of_dicts(estimate["detailed_costs"])

This method estimates tokens from rendered prompt lengths and applies the pricing table. Its text approximation uses four characters per input token and output tokens equal to 75% of input tokens, rounded up. Prompt text containing unresolved Jinja placeholders receives a multiplier of two before token estimation. These are estimation assumptions; no answers are generated. For question-specific estimates and adjustable assumptions, see Detailed estimates.

Estimate hosted execution in credits and USD

With Expected Parrot authentication configured, request the server’s estimate for the same job:

from edsl import Coop

coop = Coop()
remote_estimate = coop.remote_inference_cost(job, iterations=3)
remote_estimate

The returned dictionary includes usd and credits_hold, along with estimate metadata such as estimate_kind, assumptions, and warnings when available. remote_inference_cost() also accepts a Survey; pass a Jobs object when you want the estimate to use your selected agents, scenarios, and models. The CLI equivalent is ep jobs cost jobs.ep --iterations 3. See Credits for account balances and usage.

Inspect prompts and their estimated costs

job.prompts() returns the rendered prompts and an estimated cost for each question in each planned interview:

job.prompts().select("question_name", "model", "estimated_cost", "user_prompt", "system_prompt")
# To print only the prompts: job.show_prompts()
# To print all prompt-table columns: job.show_prompts(all=True)

Each estimated_cost is in USD for one question in one interview, before multiplying by iterations. The preview includes scenario values and agent instructions; answers to earlier questions become available when the survey runs. See Prompts and instructions for inspecting the prompts recorded with completed responses.

Detailed estimates

For a breakdown that accounts for question types, file inputs, and survey memory, use estimate_remote_job_cost():

detailed = job.estimate_remote_job_cost()
detailed.detail.select("question_name", "model", "total_input_tokens", "total_output_tokens", "cost_usd")

Despite its name, this method calculates the estimate locally, using retrieved prices unless you supply a price_lookup. It returns a JobCostEstimate for one iteration; it does not submit a remote job.

Attribute or method	What it provides
`detailed.total_cost_usd`, `detailed.total_credits`	Total estimated cost.
`detailed.total_input_tokens`, `detailed.total_output_tokens`	Total estimated token counts.
`detailed.detail`	Dataset with a row for each question in each planned interview.
`detailed.summary_by_question()`	List of dictionaries aggregating costs by question.
`detailed.summary_by_model()`	List of dictionaries aggregating costs by model and service.
`detailed.warnings`	Notes about approximations and unsupported inputs.
`detailed.to_markdown("cost-estimate.md")`	Save a report of costs, assumptions, and warnings; omit the path to return the Markdown without writing a file.

You can supply your own expected response lengths. For example, estimate a 100-token answer to the favorite_flower question:

from edsl.jobs.cost_estimation import TokenOverride

adjusted = job.estimate_remote_job_cost(
    token_overrides={"favorite_flower": TokenOverride(answer_tokens=100)},
)
adjusted

TokenOverride also accepts comment_tokens and thinking_tokens; omitted fields retain their estimates. Optional service and model fields restrict an override to a particular model. These values change the estimate, not the model’s output limits. For surveys with branching, branch_weights={("from_question", "to_question"): probability} supplies expected transition probabilities. Without branch weights, the estimator assumes every question is reached. chars_per_token adjusts the text-to-token approximation.

Reuse a pricing table

Coop.fetch_prices() returns a pricing dictionary keyed by (inference_service, model). You can reuse that table across estimates:

prices = coop.fetch_prices()
job.estimate_job_cost_from_external_prices(price_lookup=prices, iterations=3)
# The detailed estimator accepts the same table:
# job.estimate_remote_job_cost(price_lookup=prices)

The table contains input and output price entries. If a model is missing, the estimator falls back to available service prices or default rates; inspect the prices used in the breakdown.

Inspect recorded costs

Load completed results and compute the recorded job cost:

from edsl import Results

results = Results.load("results.ep")
results.compute_job_cost()
# Include recorded costs for cached responses as well:
# results.compute_job_cost(include_cached_responses_in_cost=True)

By default this excludes cached responses. include_cached_responses_in_cost=True includes their recorded costs for comparison; those costs are not necessarily new charges. Missing provider usage or pricing information can limit the estimate. Use your account or provider’s billing record to reconcile actual charges.

Keep a cost ledger

The CLI can record an estimate or the cost from a saved result package:

ep costs log --output costs.jsonl --estimated 1.25 --note "Budget estimate for pilot"
ep costs log --output costs.jsonl --actual-from results.ep --note "Completed pilot"

The first amount is an illustrative manual estimate. Recording it does not reserve credits or enforce a limit. See Credits for Expected Parrot account usage, Token limits for execution constraints, and Caching for response reuse.

Introduction

Setup and tutorials

EDSL objects

How-to guides

Research examples

Human research

Execution and costs

Saving and sharing

Research and teaching

Expected Parrot platform

Contributing

Estimate a prepared job

Estimate hosted execution in credits and USD

Inspect prompts and their estimated costs

Detailed estimates

Reuse a pricing table

Inspect recorded costs

Keep a cost ledger

​Estimate a prepared job

​Estimate hosted execution in credits and USD

​Inspect prompts and their estimated costs

​Detailed estimates

​Reuse a pricing table

​Inspect recorded costs

​Keep a cost ledger

Estimate a prepared job

Estimate hosted execution in credits and USD

Inspect prompts and their estimated costs

Detailed estimates

Reuse a pricing table

Inspect recorded costs

Keep a cost ledger