Questions

A question defines what is asked and the expected answer format. Choose the format before writing the prompt so responses can be compared and analyzed consistently. For data labeling, a question’s answer becomes a label: “What topic does this passage discuss?” can return a category, while “Extract the dates and organizations” can return named fields. Put the passages or documents in scenarios to reuse the same questions across your material. Label data with survey questions shows how to use an existing codebook or develop labels through exploratory questions first.

Choose a question type

Type	Use when you need	Answer shape and constraints
`free_text`	An explanation or quotation	String; correctness still needs evaluation.
`multiple_choice`, `yes_no`	One category or decision	One permitted option.
`checkbox`, `top_k`	Several categories	List of permitted options; set selection bounds.
`linear_scale`, `likert_five`	An ordered rating	One configured scale value.
`numerical`	A count or measurement	Number; optionally set minimum and maximum.
`rank`	An ordering	Ordered list of options with a configured selection count.
`list`	Open-ended items	List; optionally limit its length.
`extract`	Fields extracted from supplied material	Dictionary following an answer template.
`dict`	Named fields with declared types	Dictionary with configured keys and value types.
`matrix`	Ratings for several items	Mapping from each item to an allowed option.
`interview`	A guided interview	Structured transcript; see Human interviews.

These are common choices. The type details below explain their parameters and include examples. Use ep schema list and ep schema show --question_type TYPE to inspect required parameters.

Construct questions

The example asks for three complementary measures of a message: a clarity category, an explanation, and a numerical score. It constructs the questions and survey without generating answers. A scenario supplies the message text when the survey runs.

from edsl import QuestionFreeText, QuestionMultipleChoice, QuestionNumerical, Survey

clarity = QuestionMultipleChoice(
    question_name="clarity",
    question_text="How clear is this message: {{ scenario.message }}?",
    question_options=["Clear", "Unclear", "Unsure"],
)
reason = QuestionFreeText(
    question_name="reason",
    question_text="Explain your assessment of the message.",
)
score = QuestionNumerical(
    question_name="score",
    question_text="Rate the clarity of this message from 0 to 10: {{ scenario.message }}",
    min_value=0,
    max_value=10,
)
survey = Survey([clarity, reason, score]).add_targeted_memory(reason, clarity)

The answers will be a category, a string, and a number from 0 to 10. Validation checks response structure and configured constraints; it does not establish whether an answer is true or a classification is correct. Invalid model answers can lead to retries or interview errors, which should be inspected with the results.

Names, inputs, and instructions

Use unique question_name values within a survey; a question named clarity produces the column answer.clarity. The name identifies the variable in your analysis, while question_text contains the wording to answer. Put varying source material in Scenarios, respondent traits in Agents, and answer dependencies in Surveys. Use question_text for the research question and answering_instructions for how to answer it. Inspect the generated Prompts before changing formatting instructions supplied by the question type.

Run a question

Questions can run individually or as part of a survey. Follow the Run your first survey tutorial for an executable local example and model setup.

Common methods

Call	Arguments and result
`Question(type_name, question_name=..., question_text=..., ...)`	Construct a question of the named type with its specific parameters. You can also use classes such as `QuestionFreeText` directly.
`question.to_survey()`	Return a one-question Survey.
`question.by(scenarios).by(model)`	Return a Jobs object configured to ask the question for those inputs.
`question.loop(scenarios)`	Return separate question objects with scenario values substituted and names adjusted.
`question.to_dict()`	Return the serialized question definition, including its type and parameters.
`question.from_dict(data)`	Reconstruct a question from its serialized dictionary.
`Question.list_question_types()`	List the types supported by the installed EDSL version.
`Question.example(type_name)`	Construct a local example of the named type without running it.

For example, inspect a multiple-choice question’s definition:

from edsl import Question

question = Question(
    "multiple_choice",
    question_name="clarity",
    question_text="How clear is this message?",
    question_options=["Clear", "Unclear", "Unsure"],
)
question.to_dict()
# In a script: print(question.to_dict())

Preview and vary a question

Use render() to substitute known values without calling a model. It returns a new question:

from edsl import QuestionFreeText

template = QuestionFreeText(
    question_name="feedback",
    question_text="What would improve {{ scenario.product }}?",
)
preview = template.render({"product": "the search tool"})
preview.question_text
# For the assembled model instructions: template.prompt_preview(scenario={"product": "the search tool"})

question.copy() makes an independent copy. For questions with a fixed list of options, question.draw(random_seed=42) returns a copy with shuffled options; pin_options=["Other"] keeps those options in their original positions. question.option_permutations() returns every ordering as separately named questions, so its size grows factorially with the number of options. Use question.add_question(other_question) to build a Survey, or question.to_jobs() to prepare a job directly. question.run(...) is a one-question shortcut for execution; await question.run_async(...) is available in asynchronous Python. Both accept run options described under Jobs.

Question type details

Most types share question_name and question_text. The table lists additional parameters for common response formats. Constructing these questions makes no model calls.

Type	Parameters to set	Response
`free_text`	No additional required fields.	A string.
`multiple_choice`	`question_options`: allowed choices.	One choice.
`yes_no`	Built-in Yes/No options.	One of the built-in choices.
`checkbox`	`question_options`; optional `min_selections` and `max_selections`.	A list of choices within the bounds.
`top_k`	`question_options`; set `min_selections` and `max_selections` to the same positive count.	A list with exactly that many choices.
`linear_scale`	`question_options`: consecutive integers; optional `option_labels`.	One scale value.
`likert_five`	Built-in five-point agreement scale.	One agreement label.
`numerical`	Optional `min_value` and `max_value`.	A number within any specified bounds.
`rank`	`question_options`; optional `num_selections`.	An ordered list of choices.
`list`	Optional `max_list_items`.	A list of open-ended items.
`extract`	`answer_template`: keys and example values.	A dictionary with the requested fields.
`dict`	`answer_keys`, with optional `value_types` and `value_descriptions`.	A dictionary with named, typed values.
`matrix`	`question_items`, `question_options`, and optional `option_labels`.	A mapping from each item to a rating.
`distribution`	`question_options` and optional `tolerance`.	A probability for each option, summing to one within the tolerance.

For option-based types that support use_code, setting it to True asks for option indices; setting it to False asks for option values. The examples below use values explicitly where needed.

Select several options or rank them

from edsl import Question

features = ["Search", "Export", "Sharing"]
selected = Question(
    "checkbox", question_name="features",
    question_text="Which features do you use?",
    question_options=features, min_selections=1, max_selections=3,
    use_code=False,
)
priorities = Question(
    "rank", question_name="priorities",
    question_text="Rank your two highest priorities, most important first.",
    question_options=features, num_selections=2, use_code=False,
)

checkbox records a selection; rank preserves the requested ordering. Use top_k when you need a fixed selection count without ranking.

Collect ratings or measurements

from edsl import Question

rating = Question(
    "linear_scale", question_name="ease",
    question_text="How easy was this task?",
    question_options=[1, 2, 3, 4, 5],
    option_labels={1: "Very difficult", 5: "Very easy"},
)
minutes = Question(
    "numerical", question_name="minutes",
    question_text="How many minutes did the task take?",
    min_value=0, max_value=120,
)

The rating must be one of the configured scale values. The measurement can be any number within its bounds.

Extract structured fields

from edsl import Question

details = Question(
    "extract", question_name="details",
    question_text="Extract the product and issue from: {{ scenario.feedback }}",
    answer_template={"product": "product name", "issue": "reported issue"},
)
summary = Question(
    "dict", question_name="summary",
    question_text="Summarize: {{ scenario.feedback }}",
    answer_keys=["issue", "urgent"],
    value_types=["str", "bool"],
    value_descriptions=["The reported issue", "Whether it requires urgent attention"],
)

Supply feedback through a Scenario when running these questions. Use dict when you need to declare value types explicitly. Neither format establishes the factual accuracy of an extracted value.

Rate several items together

from edsl import Question

ratings = Question(
    "matrix", question_name="feature_ratings",
    question_text="Rate your satisfaction with each feature.",
    question_items=["Search", "Export", "Sharing"],
    question_options=[1, 2, 3, 4, 5],
    option_labels={1: "Very dissatisfied", 5: "Very satisfied"},
)

The response contains one configured rating per item. Use separate questions if the items need different scales or survey logic.

Additional types

EDSL also includes specialized types for Other responses (multiple_choice_with_other, checkbox_with_other), long choice lists (dropdown), structured validation (pydantic), and formats such as markdown and url. Inspect an example and its fields before adapting it:

from edsl import Question

example = Question.example("multiple_choice_with_other")
example.to_dict()
# In a script: print(example.to_dict())
# Question.list_question_types() lists all types in your installed version.

For a machine-readable schema, use ep schema show --question_type multiple_choice_with_other. Choose a specialized type for the behavior you need:

Type	Purpose and configuration
`multiple_choice_with_other`, `checkbox_with_other`	Allow supplied choices and custom Other responses. Set `question_options` and optionally `other_option_text`; the checkbox type also supports selection bounds.
`dropdown`	Search a large option set. Supply `question_options`, optionally `question_options_details`, and `max_options_shown` / `top_k` to bound the displayed or retrieved choices.
`pydantic`	Validate a structured answer against a Pydantic class supplied as `pydantic_model`.
`markdown`, `url`	Collect Markdown text or a valid absolute URL. URL validation does not establish whether the destination exists or supports a claim.
`compute`	Render a template and record its value without an LLM call; useful for deterministic calculated fields.
`functional`	Run a supplied Python `func(scenario, agent_traits)` to produce the answer. Preserve the function with the study.
`random`	Generate a pseudo-random number between 0 and 1 without asking a model.
`thinking`	Send question text directly to a specified `model` and optional `system_prompt`, and record the response.
`demand`	Collect quantities demanded at the supplied `prices`.
`edsl_object`	Collect an EDSL object representation with a specified `expected_object_type`, such as `"ScenarioList"`. Useful for workflows accepting EDSL objects as inputs.
`file_upload`	Collect a respondent’s file; see file upload responses for retrieving FileStore objects.
`image_generation`	Return a generated image FileStore. Set its image `model`, `service_name`, and `generation_parameters`; these are separate from the language model used by other questions. See image generation.
`diagram`	Generate a Graphviz DOT diagram and return a FileStore. Set `output_format` and `engine`; rendering requires Graphviz.
`interview`	Conduct a guided human interview; see Human interviews.
`group_discussion`	Simulate discussion among configured `participants`, with `max_rounds`, `turn_order`, and optional decision settings. Several participant turns can incur model costs.
`survey_message`	Display a routable message without asking for an answer; see Survey messages.

For example, define a structured answer and a deterministic calculation:

from pydantic import BaseModel
from edsl import Question

class FeedbackCode(BaseModel):
    issue: str
    urgent: bool

coding = Question(
    "pydantic", question_name="coding",
    question_text="Code this feedback: {{ scenario.feedback }}",
    pydantic_model=FeedbackCode,
)
double_score = Question(
    "compute", question_name="double_score",
    question_text="{{ scenario.score * 2 }}",
)

Construction makes no model calls. coding requires model execution to generate an answer; double_score computes its answer from a scenario’s score when run. Validate specialized workflows with representative inputs before using them in a study.

Introduction

Setup and tutorials

EDSL objects

How-to guides

Research examples

Human research

Execution and costs

Saving and sharing

Research and teaching

Expected Parrot platform

Contributing

Choose a question type

Construct questions

Names, inputs, and instructions

Run a question

Common methods

Preview and vary a question

Question type details

Additional types

​Choose a question type

​Construct questions

​Names, inputs, and instructions

​Run a question

​Common methods

​Preview and vary a question

​Question type details

​Additional types

Choose a question type

Construct questions

Names, inputs, and instructions

Run a question

Common methods

Preview and vary a question

Question type details

Additional types