Remote Inference

Remote inference allows you to run surveys at the Expected Parrot server instead of locally on your own machine, and to use Remote Caching to store survey results and logs at your Coop account.

Note: You must have a Coop account in order to use remote inference and caching. By using remote inference you agree to any terms of use of service providers, which Expected Parrot may accept on your behalf and enforce in accordance with our terms of use.

How it works

When remote inference is activated, calling the run() method on a survey will send it to the Expected Parrot server. Survey results and job details (history, costs, etc.) are automatically stored at the server and accessible from your workspace or at the Jobs page of your account.

By default, a remote cache is used to retrieve responses to any questions that have already been run. You can choose whether to use it or generate fresh responses to questions. See the Remote Caching section for more details.

Activating remote inference

Log in to your Coop account and navigate to your Settings page. Toggle on the remote inference setting:

Managing keys

An Expected Parrot key is required to use remote inference and to interact with Coop. Your key can be viewed (and reset) at the Settings page of your account (where you activate remote inference; see above).

It is automatically stored at your Keys page where you can select options for adding keys, sharing them with other users and prioritizing them for use with your surveys:

See the Managing Keys section for more details on methods for storing and managing keys.

Credits

Running surveys with your Expected Parrot API key requires credits to cover API calls to service providers. Your account comes with free credits for getting started; you can check your balance and purchase additional credits at the Credits page of your account. Running surveys with your own keys does not consume credits. Learn more about purchasing credits and calculating costs at the credits section.

Using remote inference

When remote inference is activated, calling the run() method will send a survey to the Expected Parrot server. You can access results and all information about the job (history, costs, etc.) from your workspace or your Jobs page.

For example, here we run a simple survey with remote inference activated and inspect the job information that is automatically posted. We optionally pass description and visibility parameters (these can be edited at any time):

from edsl import Model, QuestionFreeText, Survey

m = Model("gemini-1.5-flash")

q = QuestionFreeText(
  question_name = "prime",
  question_text = "Is 2 a prime number?"
)

survey = Survey(questions = [q])

results = survey.by(m).run(
  remote_inference_description = "Example survey", # optional
  remote_inference_visibility = "public" # optional
)

Output (details will be unique to your job):

✓ Current Status: Job completed and Results stored on Coop: http://www.expectedparrot.com/content/cfc51a12-63fe-41cf-b441-66d78ba47fb0

When the job has finished, it will appear with a status of Completed:

Remote inference page on the Coop web app. There is one job shown, and it has a status of "Completed."

We can view the results of the job:

Remote inference results page on the Coop web app. There is one result shown.

Job details and costs

When you run a job using your Expected Parrot API key you are charged credits based on the number of tokens used. (When you run a job using your own keys you are charged directly by service providers based on the terms of your accounts.)

Before running a job, you can estimate the cost of the job by calling the estimate_job_cost() method on the Job object (a survey combined with a model). This will return information about the estimated total cost, input tokens, output tokens and per-model costs:

For example, here we estimate the cost of running a simple survey with a model:

from edsl import Model, QuestionFreeText, Survey

m = Model("gemini-1.5-flash")

q = QuestionFreeText(
  question_name = "prime",
  question_text = "Is 2 a prime number?"
)

survey = Survey(questions = [q])

job = survey.by(m)

estimated_job_cost = job.estimate_job_cost()
estimated_job_cost

Output:

{'estimated_total_cost_usd': 1.575e-06,
'estimated_total_input_tokens': 5,
'estimated_total_output_tokens': 4,
'model_costs': [{'inference_service': 'google',
  'model': 'gemini-1.5-flash',
  'estimated_cost_usd': 1.575e-06,
  'estimated_input_tokens': 5,
  'estimated_output_tokens': 4}]}

We can also estimate the cost in credits to run the job remotely by passing the job to the remote_inference_cost() method of a Coop client object:

from edsl import Coop

coop = Coop()

estimated_remote_inference_cost = coop.remote_inference_cost(job) # using the job object from above
estimated_remote_inference_cost

Output:

{'credits': 0.01, 'usd': 1.575e-06}

Details on these methods can be found in the credits section.

After running a job, you can view the actual cost in your job history or by calling the remote_inference_cost() method and passing it the job UUID (this is distinct from the results UUID, and can be found in your job history page).

You can also check the details of a job using the remote_inference_get() method as pass it the job UUID.

Note: When you run a job using your own keys, the cost estimates are based on the prices listed in the model pricing page. Your actual charges from service providers may vary based on the terms of your accounts with service providers.

Job history

You can click on any job to view its history. When a job fails, the job history logs will describe the error that caused the failure. The job history also shows which key was used to run each job (your own key, a key that has been share with you or your Expected Parrot API key):

Remote inference methods

Coop class

class edsl.coop.coop.Coop(api_key: str | None = None, url: str | None = None)[source]

Bases: CoopFunctionsMixin

Client for the Expected Parrot API.

remote_inference_cost(input: Jobs | Survey, iterations: int = 1) → int[source]

Get the cost of a remote inference job.

Parameters:: input – The EDSL job to send to the server.

>>> job = Jobs.example()
>>> coop.remote_inference_cost(input=job)
{'credits': 0.77, 'usd': 0.0076950000000000005}

remote_inference_create(job: Jobs, description: str | None = None, status: Literal['queued', 'running', 'completed', 'failed'] = 'queued', visibility: Literal['private', 'public', 'unlisted'] | None = 'unlisted', initial_results_visibility: Literal['private', 'public', 'unlisted'] | None = 'unlisted', iterations: int | None = 1) → RemoteInferenceCreationInfo[source]

Send a remote inference job to the server.

Parameters:

job – The EDSL job to send to the server.
description (optional) – A description for this entry in the remote cache.
status – The status of the job. Should be ‘queued’, unless you are debugging.
visibility – The visibility of the cache entry.
iterations – The number of times to run each interview.

>>> job = Jobs.example()
>>> coop.remote_inference_create(job=job, description="My job")
{'uuid': '9f8484ee-b407-40e4-9652-4133a7236c9c', 'description': 'My job', 'status': 'queued', 'iterations': None, 'visibility': 'unlisted', 'version': '0.1.38.dev1'}

remote_inference_get(job_uuid: str | None = None, results_uuid: str | None = None) → RemoteInferenceResponse[source]

Get the details of a remote inference job. You can pass either the job uuid or the results uuid as a parameter. If you pass both, the job uuid will be prioritized.

Parameters:

job_uuid – The UUID of the EDSL job.
results_uuid – The UUID of the results associated with the EDSL job.

>>> coop.remote_inference_get("9f8484ee-b407-40e4-9652-4133a7236c9c")
{'job_uuid': '9f8484ee-b407-40e4-9652-4133a7236c9c', 'results_uuid': 'dd708234-31bf-4fe1-8747-6e232625e026', 'results_url': 'https://www.expectedparrot.com/content/dd708234-31bf-4fe1-8747-6e232625e026', 'latest_error_report_uuid': None, 'latest_error_report_url': None, 'status': 'completed', 'reason': None, 'credits_consumed': 0.35, 'version': '0.1.38.dev1'}