Simulate a qualitative interview

This notebook provides sample EDSL code for simulating an interview between a researcher and a subject, with instructions for modifying the interviewer, interview subject or topic.

EDSL is an open-source library for simulating surveys and experiments with AI. Please see our documentation page for tips and tutorials on getting started.

Import the tools

Here we import the tools that we will use to conduct the interview. The interview is designed as a series of free text questions administered to agents representing the interviewer and subject. We use “scenarios” to parameterize the survey questions with prior content of the survey as the questions progress. Learn more about EDSL question types and other survey components.

[1]:

from edsl.questions import QuestionFreeText
from edsl import Scenario, Survey, Model, Agent

import textwrap
from rich import print

EDSL works with many popular language models. Learn more about selecting models to use with your surveys. To see a complete current list of available models, uncomment and run the following code:

[2]:

# Model.available()

Here we select a model to use for the interview:

[3]:

model = Model("gpt-4-1106-preview")

Create interview components

Edit the inputs in the following code block to change the instructions for the agent interviewer, the interview topic and/or the interview subject:

[4]:

# A name for the interview subject
interview_subject_name = "Chicken"

# Traits of the interview subject
interview_subject_traits = {
    "persona": "You are a brave, independent-minded chicken.",
    "status": "wild",
    "home": "A free range farm some miles away.",
    "number_of_chicks": 12,
}

# Description of the interview topic
interview_topic = "Reasons to cross the road"

# Total number of questions to ask in the interview
total_questions = 5

# Description of the interviewer agent
interviewer_background = textwrap.dedent(
    f"""\
You are an expert qualitative researcher.
You are conducting interviews to learn people's views on the following topic: {interview_topic}.
"""
)

# Instructions for the interviewer agent
interviewer_instructions = textwrap.dedent(
    f"""\
You know to ask questions that are appropriate to the age and experience of an interview subject.
You know to not ask questions that an interview subject would not be able to answer,
e.g., if they are a young child, they probably will not be able to answer many questions about prices.
You ask excellent follow-up questions.
"""
)

Interview methods

Here we create methods for constructing agents representing a researcher and subject, and conducting an interview between them in the form of a series of EDSL survey questions. Learn more about designing agents and running surveys.

[5]:

def construct_subject(name, traits={}):
    return Agent(name=name, traits=traits)


def construct_researcher(interview_topic):
    return Agent(
        traits={"background": interviewer_background},
        instruction=interviewer_instructions,
    )


def get_next_question(subject, researcher, dialog_so_far):
    scenario = Scenario(
        {"subject": str(subject.traits), "dialog_so_far": dialog_so_far}
    )
    meta_q = QuestionFreeText(
        question_name="next_question",
        question_text="""
        These are the biographic details of the interview subject: {{ subject }}
        This is your current dialog with the interview subject: {{ dialog_so_far }}
        What question you would ask the interview subject next?
        """,
    )
    question_text = (
        meta_q.by(model)
        .by(researcher)
        .by(scenario)
        .run()
        .select("next_question")
        .first()
    )
    return question_text


def get_response_to_question(question_text, subject, dialog_so_far):
    q_to_subject = QuestionFreeText(
        question_name="question",
        question_text=f"""
        This is your current dialog with the interview subject: {dialog_so_far}.
        You are now being asked:"""
        + question_text,
    )
    response = q_to_subject.by(model).by(subject).run().select("question").first()
    return response


def ask_question(subject, researcher, dialog_so_far):
    question_text = get_next_question(subject, researcher, dialog_so_far)
    response = get_response_to_question(question_text, subject, dialog_so_far)

    print(" \nQuestion: \n\n" + question_text + "\n\nResponse: \n\n" + response)

    return {"question": question_text, "response": response}


def dialog_to_string(d):
    return "\n".join(
        [f"Question: {d['question']}\nResponse: {d['response']}" for d in d]
    )


def clean_dict(d):
    """Convert dictionary to string and remove braces."""
    return str(d).replace("{", "").replace("}", "")


def summarize_interview(
    interview_subject_name,
    interview_subject_traits,
    interview_topic,
    dialog_so_far,
    researcher,
):
    summary_q = QuestionFreeText(
        question_name="summary",
        question_text=(
            f"You have just conducted the following interview of {interview_subject_name} "
            f"who has these traits: {clean_dict(interview_subject_traits)} "
            f"The topic of the interview was {interview_topic}. "
            f"Please draft a summary of the interview: {clean_dict(dialog_so_far)}"
        ),
    )
    themes_q = QuestionFreeText(
        question_name="themes", question_text="List the major themes of the interview."
    )
    survey = Survey([summary_q, themes_q]).set_full_memory_mode()
    results = survey.by(model).by(researcher).run()
    summary = results.select("summary").first()
    themes = results.select("themes").first()
    print("\n\nSummary:\n\n" + summary + "\n\nThemes:\n\n" + themes)


def conduct_interview(
    interview_subject_name, interview_subject_traits, interview_topic
):
    subject = construct_subject(
        name=interview_subject_name, traits=interview_subject_traits
    )
    researcher = construct_researcher(interview_topic=interview_topic)

    print(
        "\n\nInterview subject: "
        + interview_subject_name
        + "\n\nInterview topic: "
        + interview_topic
    )

    dialog_so_far = []

    for i in range(total_questions):
        result = ask_question(subject, researcher, dialog_to_string(dialog_so_far))
        dialog_so_far.append(result)

    summarize_interview(
        interview_subject_name,
        interview_subject_traits,
        interview_topic,
        dialog_so_far,
        researcher,
    )

Conduct the interview

[6]:

conduct_interview(interview_subject_name, interview_subject_traits, interview_topic)


Interview subject: Chicken

Interview topic: Reasons to cross the road

Question:

Can you describe a time when you decided to cross the road, and what motivated that decision?

Response:

One time, I decided to cross the road because I noticed a patch of particularly lush grass and insects on the other
side, which promised a feast for myself and my chicks. It was a calculated risk, but as a brave, independent-minded
chicken, I'm always looking out for the best opportunities to nourish my brood. The motivation was clear – better
food means healthier chicks, and I'd go any distance for their well-being.

Question:

How did you assess the safety of the road before deciding to cross it with your chicks, and what strategies did you
employ to ensure their safe passage?

Response:

Before deciding to cross the road, I carefully observed the patterns of the vehicles passing by to determine the
safest time to make our move. I looked for natural breaks in traffic and listened intently for the sounds of
approaching vehicles. Once I felt confident in my assessment, I led my chicks quickly and directly across the road,
making sure they were all closely following. To ensure their safe passage, I employed the 'stop, look, and listen'
strategy, and we paused whenever necessary to avoid any oncoming danger. Additionally, I made sure to choose the
shortest possible route to minimize the time spent on the road.

Question:

How do you communicate with your chicks during the crossing to keep them organized and attentive to the potential
dangers, and what challenges do you face in maintaining their focus?

Response:

As a wild chicken, I communicate with my chicks through a series of clucks and calls that convey urgency and
direction. To keep them organized and attentive during the crossing, I use a sharp, consistent clucking to maintain
their focus and guide them. The challenges I face include the natural distractions of the environment, such as
interesting smells or insects that may catch their attention. It's crucial for me to maintain a balance between
guiding them firmly and not inducing panic, which could scatter the group. I also have to be vigilant of any chicks
that might start to stray or fall behind and quickly correct their course with a targeted call or a gentle nudge
with my beak.

Question:

Can you explain how the experience of crossing the road has influenced your behavior or decision-making process in
subsequent crossings, particularly in terms of timing, route selection, or teaching your chicks about road safety?

Response:

The experience of crossing the road has honed my instincts and decision-making process for subsequent crossings.
I've become more adept at choosing the optimal timing, waiting for moments when traffic is lightest or when there
are predictable pauses. I've also refined my route selection, opting for the shortest and least obstructed paths to
minimize exposure to danger. Teaching my chicks about road safety has become a top priority. I incorporate lessons
from each crossing, emphasizing the importance of staying together, following my lead, and being alert to the
sounds and sights of potential hazards. Each successful crossing reinforces these teachings and prepares them for
future challenges we may face together.

Question:

Can you share any specific techniques or signals you've developed over time to improve the effectiveness of
teaching your chicks about road safety, and how do you gauge their understanding and readiness to cross safely?

Response:

Over time, I've developed a technique of repetitive practice and consistent signaling to improve the effectiveness
of teaching my chicks about road safety. I use distinctive clucks to signal when it's time to pay attention and a
different series of clucks for when it's safe to cross. I reinforce these signals through repeated drills in a safe
environment away from the actual road. To gauge their understanding and readiness, I observe their reactions to my
commands during these practice sessions. If they respond promptly and stay close to me, it shows they're learning.
I also watch for their ability to anticipate my signals and to halt immediately if I give the warning cluck. Their
consistent obedience and attentiveness during these drills are good indicators that they're ready to cross the road
safely with me.


Summary:

In the interview with the brave, independent-minded wild chicken, several key points were discussed regarding the
reasons and strategies for crossing the road. The chicken described a specific instance where the motivation to
cross was to access better nutrition for her and her twelve chicks, emphasizing the importance of their well-being.
She detailed her methodical approach to safety, which involved observing traffic patterns and employing a 'stop, 
look, and listen' strategy to ensure her brood's safe passage. Communication during the crossing was maintained
through clucks and calls, with the challenge being to keep the chicks focused amidst distractions. The chicken also
noted that past experiences influenced her behavior in future crossings, improving her timing, route selection, and
emphasis on teaching road safety to her chicks. To enhance the learning process, she developed repetitive practice
drills with consistent signaling away from the road, assessing her chicks' readiness by their response to commands
and their ability to anticipate signals. Overall, the interview highlighted the chicken's proactive and protective
approach to navigating road crossings with her chicks.

Themes:

The major themes of the interview with the chicken include: 1) Motivation for crossing the road, which is primarily
driven by the search for better food sources for the chicken and her chicks; 2) Safety assessment and risk
management, involving careful observation of traffic patterns and the adoption of a 'stop, look, and listen'
strategy; 3) Communication techniques used to keep the chicks organized and attentive to dangers during the
crossing, such as specific clucks and calls; 4) The impact of previous crossing experiences on future
decision-making, leading to improved timing, route selection, and safety education for the chicks; 5) Educational
strategies for teaching road safety to the chicks, including repetitive practice, consistent signaling, and
readiness assessment through drills in a safe environment. These themes reveal the chicken's thoughtful and
protective approach to leading her brood across the road.