Using images in a survey

This notebook provides sample code for using images with an EDSL survey.

EDSL is an open-source library for simulating surveys, experiments and other research with AI agents and large language models. Before running the code below, please ensure that you have installed the EDSL library and either activated remote inference from your Coop account or stored API keys for the language models that you want to use with EDSL. Please also see our documentation page for tips and tutorials on getting started using EDSL.

Scenarios

A Scenario is a dictionary containing a key/value pair that is used to add data or content to questions in an EDSL survey. Scenarios allow you create variations and versions of questions efficiently, and with data or content from different sources.

EDSL provides a variety of methods for automatically generating scenarios from PDFs, CSVs, docs, tables, lists, dicts – and images. In the steps below we demonstrate how to create a scenario for an image and use it in a survey.

Learn more about working with scenarios.

Creating a scenario

We start by creating a Scenario for an image. For purposes of demonstration, we post a PNG image to the Coop, and then retrieve it as a scenario (this can be done by any user with a Coop account).

Code for posting a PNG file to the Coop (uncomment and replace with your own file):

# from edsl.scenarios.FileStore import PNGFileStore
# fs = PNGFileStore("parrot_logo.png")
# info = fs.push()
# print(info)

Retrieving a Coop file to use as a scenario (replace with the UUID of a desired object):

from edsl.scenarios.FileStore import PNGFileStore

png_file = PNGFileStore.pull("e9edda74-d493-42df-92bc-34bf49c6d1bf", expected_parrot_url="https://www.expectedparrot.com")

Converting the retrieved file into a scenario:

from edsl import Scenario

s = Scenario.from_image(png_file.to_tempfile(), "parrot_logo")

Alternative method for creating a scenario from a local file:

# s = Scenario.from_image("parrot_logo.png", "parrot_logo")

Verify the scenario keys to use in questions:

s.keys()

['parrot_logo']

Creating questions using the image

Next we construct questions with the image scenario. Note that we use a {{ placeholder }} for the scenario key. We also demonstrate how to pipe an answer into a follow-on question:

from edsl import QuestionYesNo, QuestionFreeText, QuestionList, Survey

q1 = QuestionYesNo(
    question_name = "animal",
    question_text = "Is there an animal in this image? {{ parrot_logo }}"
)

q2 = QuestionFreeText(
    question_name = "identify",
    question_text = "Identify the animal in this image: {{ parrot_logo }}"
)

q3 = QuestionList(
    question_name = "colors",
    question_text = "What color(s) is the animal?",
)

survey = Survey(questions = [q1, q2, q3])

Adding logic and rules for adminitering the questions:

survey = (
    survey
    .add_stop_rule(q1, "animal == 'No'")
    .add_targeted_memory(q3, q2)
)

We use the scenario by adding it to the survey when we run it, as with any other scenarios:

results = survey.by(s).run()

results.select("animal", "identify", "colors", "colors_comment").print(format="rich")

┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ answer  ┃ answer                           ┃ answer                           ┃ comment                         ┃
┃ .animal ┃ .identify                        ┃ .colors                          ┃ .colors_comment                 ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Yes     │ The animal in the image is a     │ ['green', 'red', 'blue',         │ Comment: Parrots are often      │
│         │ parrot.                          │ 'yellow']                        │ brightly colored, and the most  │
│         │                                  │                                  │ common colors for many parrot   │
│         │                                  │                                  │ species include green, red,     │
│         │                                  │                                  │ blue, and yellow.               │
└─────────┴──────────────────────────────────┴──────────────────────────────────┴─────────────────────────────────┘

from edsl import Notebook

n = Notebook(path = "image_scenario_example.ipynb")

n.push(description = "Using an image scenario", visibility = "public")

{'description': 'Using an image scenario',
 'object_type': 'notebook',
 'url': 'https://www.expectedparrot.com/content/50a000c8-d48c-49d6-a11b-2e13ad2e5c69',
 'uuid': '50a000c8-d48c-49d6-a11b-2e13ad2e5c69',
 'version': '0.1.33.dev1',
 'visibility': 'public'}

n = Notebook(path = "image_scenario_example.ipynb")

n.patch(uuid = "50a000c8-d48c-49d6-a11b-2e13ad2e5c69", value = n)

{'status': 'success'}