Jobs
The Jobs class is a collection of agents, scenarios and models and one survey. It is used to run a collection of interviews.
Base Question
Abstract base class for all question types in EDSL.
QuestionBase defines the core interface and behavior that all question types must implement. It provides the foundation for asking questions to agents, validating responses, generating prompts, and integrating with the rest of the EDSL framework.
The class inherits from multiple mixins to provide different capabilities: - PersistenceMixin: Serialization and deserialization - RepresentationMixin: String representation - SimpleAskMixin: Basic asking functionality - QuestionBasePromptsMixin: Template-based prompt generation - QuestionBaseGenMixin: Generate responses with language models - AnswerValidatorMixin: Response validation
It also uses the RegisterQuestionsMeta metaclass to enforce constraints on child classes and automatically register them for serialization and runtime use.
- Class attributes:
question_name (str): Name of the question, used as an identifier question_text (str): The actual text of the question to be asked
- Required attributes in derived classes:
question_type (str): String identifier for the question type _response_model (Type): Pydantic model class for validating responses response_validator_class (Type): Validator class for responses
- Key Methods:
by(model): Connect this question to a language model for answering run(): Execute the question with the connected language model duplicate(): Create an exact copy of this question is_valid_question_name(): Verify the question_name is valid
- Lifecycle:
Instantiation: A question is created with specific parameters
Connection: The question is connected to a language model via by()
Execution: The question is run to generate a response
Validation: The response is validated based on the question type
Result: The validated response is returned for analysis
- Template System:
Questions use Jinja2 templates for generating prompts. Each question type has associated template files: - answering_instructions.jinja: Instructions for how the model should answer - question_presentation.jinja: Format for how the question is presented Templates support variable substitution using scenario variables.
- Response Validation:
Each question type has a dedicated response validator that: - Enforces the expected response structure - Ensures the response is valid for the question type - Attempts to fix invalid responses when possible - Uses Pydantic models for schema validation
- Example:
Derived classes must define the required attributes:
```python class FreeTextQuestion(QuestionBase):
question_type = “free_text” _response_model = FreeTextResponse response_validator_class = FreeTextResponseValidator
- def __init__(self, question_name, question_text, **kwargs):
self.question_name = question_name self.question_text = question_text # Additional initialization as needed
Using a question:
```python # Create a question question = FreeTextQuestion(
question_name=”opinion”, question_text=”What do you think about AI?”
)
# Connect to a language model and run from edsl.language_models import Model model = Model() result = question.by(model).run()
# Access the answer answer = result.select(“answer.opinion”).to_list()[0] print(f”The model’s opinion: {answer}”) ```
- Notes:
QuestionBase is abstract and cannot be instantiated directly
Child classes must implement required methods and attributes
The RegisterQuestionsMeta metaclass handles registration of question types
Questions can be serialized to and from dictionaries for storage
Questions can be used independently or as part of surveys
Jobs class
A collection of agents, scenarios, models, and a survey that orchestrates interviews.
The Jobs class is the central component for running large-scale experiments or simulations in EDSL. It manages the execution of interviews where agents interact with surveys through language models, possibly in different scenarios.
Key responsibilities: 1. Managing collections of agents, scenarios, and models 2. Configuring execution parameters (caching, API keys, etc.) 3. Managing parallel execution of interviews 4. Handling remote cache and inference capabilities 5. Collecting and organizing results
A typical workflow involves: 1. Creating a survey with questions 2. Creating a Jobs instance with that survey 3. Adding agents, scenarios, and models using the by() method 4. Running the job with run() or run_async() 5. Analyzing the results
Jobs implements a fluent interface pattern, where methods return self to allow method chaining for concise, readable configuration.