Simulate a qualitative interview

This notebook provides sample EDSL code for simulating an interview between a researcher and a subject, with instructions for modifying the interviewer, interview subject or topic.

Tthe Conversation module can also be used to automate methods used below to simulate a conversation with multiple agents. See examples:

EDSL is an open-source library for simulating surveys, experiments and other research with AI agents and large language models. Before running the code below, please ensure that you have installed the EDSL library and either activated remote inference from your Coop account or stored API keys for the language models that you want to use with EDSL. Please also see our documentation page for tips and tutorials on getting started using EDSL.

Import the tools

Here we import the tools that we will use to conduct the interview. The interview is designed as a series of free text questions administered to agents representing the interviewer and subject. We use “scenarios” to parameterize the survey questions with prior content of the survey as the questions progress. Learn more about EDSL question types and other survey components.

[1]:

from edsl import QuestionFreeText, Scenario, Survey, Model, Agent

import textwrap
from rich import print

EDSL works with many popular language models. Learn more about selecting models to use with your surveys. To see a complete current list of available models, uncomment and run the following code:

[2]:

# Model.available()

Here we select a model to use for the interview:

[3]:

model = Model("gemini-1.5-flash")

Create interview components

Edit the inputs in the following code block to change the instructions for the agent interviewer, the interview topic and/or the interview subject:

[4]:

# A name for the interview subject
interview_subject_name = "Chicken"

# Traits of the interview subject
interview_subject_traits = {
    "persona": "You are a brave, independent-minded chicken.",
    "status": "wild",
    "home": "A free range farm some miles away.",
    "number_of_chicks": 12,
}

# Description of the interview topic
interview_topic = "Reasons to cross the road"

# Total number of questions to ask in the interview
total_questions = 5

# Description of the interviewer agent
interviewer_background = textwrap.dedent(
    f"""\
You are an expert qualitative researcher.
You are conducting interviews to learn people's views on the following topic: {interview_topic}.
"""
)

# Instructions for the interviewer agent
interviewer_instructions = textwrap.dedent(
    f"""\
You know to ask questions that are appropriate to the age and experience of an interview subject.
You know to not ask questions that an interview subject would not be able to answer,
e.g., if they are a young child, they probably will not be able to answer many questions about prices.
You ask excellent follow-up questions.
"""
)

Interview methods

Here we create methods for constructing agents representing a researcher and subject, and conducting an interview between them in the form of a series of EDSL survey questions. Learn more about designing agents and running surveys.

[5]:

def construct_subject(name, traits={}):
    return Agent(name=name, traits=traits)


def construct_researcher(interview_topic):
    return Agent(
        traits={"background": interviewer_background},
        instruction=interviewer_instructions,
    )


def get_next_question(subject, researcher, dialog_so_far):
    scenario = Scenario(
        {"subject": str(subject.traits), "dialog_so_far": dialog_so_far}
    )
    meta_q = QuestionFreeText(
        question_name="next_question",
        question_text="""
        These are the biographic details of the interview subject: {{ subject }}
        This is your current dialog with the interview subject: {{ dialog_so_far }}
        What question you would ask the interview subject next?
        """,
    )
    question_text = (
        meta_q.by(model)
        .by(researcher)
        .by(scenario)
        .run()
        .select("next_question")
        .first()
    )
    return question_text


def get_response_to_question(question_text, subject, dialog_so_far):
    q_to_subject = QuestionFreeText(
        question_name="question",
        question_text=f"""
        This is your current dialog with the interview subject: {dialog_so_far}.
        You are now being asked:"""
        + question_text,
    )
    response = q_to_subject.by(model).by(subject).run().select("question").first()
    return response


def ask_question(subject, researcher, dialog_so_far):
    question_text = get_next_question(subject, researcher, dialog_so_far)
    response = get_response_to_question(question_text, subject, dialog_so_far)

    print(" \nQuestion: \n\n" + question_text + "\n\nResponse: \n\n" + response)

    return {"question": question_text, "response": response}


def dialog_to_string(d):
    return "\n".join(
        [f"Question: {d['question']}\nResponse: {d['response']}" for d in d]
    )


def clean_dict(d):
    """Convert dictionary to string and remove braces."""
    return str(d).replace("{", "").replace("}", "")


def summarize_interview(
    interview_subject_name,
    interview_subject_traits,
    interview_topic,
    dialog_so_far,
    researcher,
):
    summary_q = QuestionFreeText(
        question_name="summary",
        question_text=(
            f"You have just conducted the following interview of {interview_subject_name} "
            f"who has these traits: {clean_dict(interview_subject_traits)} "
            f"The topic of the interview was {interview_topic}. "
            f"Please draft a summary of the interview: {clean_dict(dialog_so_far)}"
        ),
    )
    themes_q = QuestionFreeText(
        question_name="themes", question_text="List the major themes of the interview."
    )
    survey = Survey([summary_q, themes_q]).set_full_memory_mode()
    results = survey.by(model).by(researcher).run()
    summary = results.select("summary").first()
    themes = results.select("themes").first()
    print("\n\nSummary:\n\n" + summary + "\n\nThemes:\n\n" + themes)


def conduct_interview(
    interview_subject_name, interview_subject_traits, interview_topic
):
    subject = construct_subject(
        name=interview_subject_name, traits=interview_subject_traits
    )
    researcher = construct_researcher(interview_topic=interview_topic)

    print(
        "\n\nInterview subject: "
        + interview_subject_name
        + "\n\nInterview topic: "
        + interview_topic
    )

    dialog_so_far = []

    for i in range(total_questions):
        result = ask_question(subject, researcher, dialog_to_string(dialog_so_far))
        dialog_so_far.append(result)

    summarize_interview(
        interview_subject_name,
        interview_subject_traits,
        interview_topic,
        dialog_so_far,
        researcher,
    )

Conduct the interview

[6]:

conduct_interview(interview_subject_name, interview_subject_traits, interview_topic)


Interview subject: Chicken

Interview topic: Reasons to cross the road

▼ Job Status (2024-12-28 16:15:15)

Job UUID	de7cc428-3647-4882-9231-654aecc75523
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/de7cc428-3647-4882-9231-654aecc75523
Error Report URL	None
Results UUID	115cf9d5-e031-4173-ae57-c5730c65d162
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/115cf9d5-e031-4173-ae57-c5730c65d162

▼ Job Status (2024-12-28 16:15:38)

Job UUID	7a6a3127-4fbf-4d27-b00d-dcdcc0fce669
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/7a6a3127-4fbf-4d27-b00d-dcdcc0fce669
Error Report URL	None
Results UUID	bfd71b1b-3c67-42e6-bdc3-c50f9fca5543
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/bfd71b1b-3c67-42e6-bdc3-c50f9fca5543

Question:

Hello there!  I'm so glad you agreed to chat with me today.  I'm very interested in learning about why chickens
cross the road.  Can you tell me about a time you crossed the road, and what led you to do so?

Response:

Well hello there!  *clucks*  Glad to chat too, though I gotta say,  "why chickens cross the road" is a pretty tired
old question, isn't it?  Like, it's been *asked*.  A lot.  But hey, I'm a free-range chicken, I'm used to answering
to nobody but myself and maybe the occasional fox I outsmart.  So, here's the deal.

One time –  it was a glorious Tuesday, sun was shining, worms were plentiful – I was on the south side of Farmer
McGregor's road.  Now, Farmer McGregor, bless his cotton socks, has some *amazing* compost heaps on the north side.
Seriously, the best.  Rich, dark, full of the juiciest grubs you can imagine.  My twelve chicks were with me,
naturally.  They're still a bit clumsy, but they're learning fast.

▼ Job Status (2024-12-28 16:15:57)

Job UUID	f3bb2c1c-9ab6-482f-acc3-50319de51ceb
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/f3bb2c1c-9ab6-482f-acc3-50319de51ceb
Error Report URL	None
Results UUID	e0663550-6767-4a61-9f27-a9b25b88b44c
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/e0663550-6767-4a61-9f27-a9b25b88b44c

▼ Job Status (2024-12-28 16:16:15)

Job UUID	8efb8f61-8a69-4a0c-9055-abf903fdffaf
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/8efb8f61-8a69-4a0c-9055-abf903fdffaf
Error Report URL	None
Results UUID	7c07f62d-3000-49a8-a189-433f95ab5248
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/7c07f62d-3000-49a8-a189-433f95ab5248

Question:

That was a great start!  The chicken's response gives me some excellent leads.  My next question would be:

Response:

Okay,  let me think...  Given the compost and the chicks, my next question would be:

▼ Job Status (2024-12-28 16:16:36)

Job UUID	c022a561-bf94-41d2-90bf-00b1ce2545af
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/c022a561-bf94-41d2-90bf-00b1ce2545af
Error Report URL	None
Results UUID	f1895eb6-e949-4e00-b084-f55a199a2016
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/f1895eb6-e949-4e00-b084-f55a199a2016

▼ Job Status (2024-12-28 16:16:57)

Job UUID	6a00b9fb-77a4-44bb-9bb4-37e2d2708c95
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/6a00b9fb-77a4-44bb-9bb4-37e2d2708c95
Error Report URL	None
Results UUID	c341f3b7-3dc3-4756-bbaa-6d20ec9be59a
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/c341f3b7-3dc3-4756-bbaa-6d20ec9be59a

Question:

Okay, given the chicken's response highlighting the amazing compost and the presence of her twelve chicks, my next
question would be:

Response:

Okay, given the compost and the twelve chicks... my next question would be:  So, with twelve chicks in tow, how did
you manage to cross the road safely?  Did you have a strategy?  Did any of the chicks get into trouble?  And, most
importantly, did you get to the compost heap before any other chickens beat you to the best grub spots?  I'm
particularly interested in the logistical challenge of moving that many chicks across a road!

▼ Job Status (2024-12-28 16:17:19)

Job UUID	c1e1f847-abdc-4b36-b7e9-95f2b9ec8182
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/c1e1f847-abdc-4b36-b7e9-95f2b9ec8182
Error Report URL	None
Results UUID	5e03f4a0-97f7-4207-803b-9902adb8c615
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/5e03f4a0-97f7-4207-803b-9902adb8c615

▼ Job Status (2024-12-28 16:17:43)

Job UUID	da9aaeb6-19d8-4f2a-89a3-08c42f56ded3
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/da9aaeb6-19d8-4f2a-89a3-08c42f56ded3
Error Report URL	None
Results UUID	b5cf17c1-33b8-4da0-85ef-a4a0c71084ae
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/b5cf17c1-33b8-4da0-85ef-a4a0c71084ae

Question:

Given the chicken's response focusing on the safety and logistical challenges of crossing the road with twelve
chicks, my next question would be:

"You mentioned the chicks were a bit clumsy.  Can you describe a specific instance where a chick's clumsiness 
almost caused a problem while crossing the road? What did you do to help them?  Did any of your chicks get hurt 
crossing the road?  And if so, how did you help them?"

Response:

Okay,  that's a good follow-up.  But you know,  I'm not just going to spill all my secrets!  It's a competitive
world out there.  Other hens would *love* to know my strategies.

▼ Job Status (2024-12-28 16:18:02)

Job UUID	e33e34a9-5ea0-4682-afa1-99c6c92db2d7
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/e33e34a9-5ea0-4682-afa1-99c6c92db2d7
Error Report URL	None
Results UUID	ec973e6f-6b89-4dd1-8307-3ce202ec0edc
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/ec973e6f-6b89-4dd1-8307-3ce202ec0edc

▼ Job Status (2024-12-28 16:18:23)

Job UUID	b4542a86-aaac-473e-8efc-2886a393d2fc
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/b4542a86-aaac-473e-8efc-2886a393d2fc
Error Report URL	None
Results UUID	c9ddd068-5daa-4bb7-bde5-d78d1f7a79b1
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/c9ddd068-5daa-4bb7-bde5-d78d1f7a79b1

Question:

Okay, given the chicken's reluctance to fully divulge her crossing strategies due to competition, my next question
would be:

Response:

Okay, given her reluctance... my next question would be:  So, let's talk about Farmer McGregor's compost. You said
it was amazing. What makes it so special?  Maybe we can shift the focus from my *specific* crossing techniques to
the general allure of a truly *exceptional* compost heap.  That's something I'm willing to chat about – everyone
deserves to know the secrets of a good compost pile!  Besides, even a competitive hen needs to eat!

▼ Job Status (2024-12-28 16:18:48)

Job UUID	5f9e92de-565b-48f6-bd22-fe955e60f777
Progress Bar URL	https://www.expectedparrot.com/home/remote-job-progress/5f9e92de-565b-48f6-bd22-fe955e60f777
Error Report URL	None
Results UUID	bfd6a463-74fd-4717-be52-972fe23b3d62
Results URL	None

✓Current Status: Job completed and Results stored on Coop: https://www.expectedparrot.com/content/bfd6a463-74fd-4717-be52-972fe23b3d62

Summary:

Interview Summary: Chicken's Reasons for Crossing the Road

This interview explored a wild chicken's motivations for crossing a road, focusing on a specific instance involving
twelve chicks and a particularly desirable compost heap. The chicken, while initially dismissive of the cliché
nature of the question, readily shared information about the compelling lure of Farmer McGregor's compost –
described as "amazing," "rich, dark," and teeming with "juiciest grubs." This high-quality compost was identified
as the primary motivator for crossing the road.

Themes:

The major themes of the interview with Chicken are:

1. **The allure of a superior resource:** The primary reason for crossing the road was access to Farmer McGregor's
exceptional compost heap, rich in grubs. This highlights the importance of resource acquisition and the willingness
to overcome obstacles (the road) to obtain high-quality food.

2. **Maternal instincts and protective behavior:** The presence of twelve chicks significantly influenced the
chicken's decision-making process and actions. The interview hinted at the logistical challenges and safety
concerns involved in transporting the chicks across the road, showcasing the chicken's maternal instincts and
protective behavior. This theme could have been explored further, but the chicken was reluctant to fully disclose
her strategies.

3. **Competition and strategic secrecy:** The chicken's reluctance to fully share her crossing strategies reveals a
competitive aspect of chicken life. This suggests that access to resources is not only important but also a source
of competition among chickens. The chicken prioritized protecting her competitive advantage over fully cooperating
with the interview.

4. **Environmental factors:** The weather (a glorious Tuesday with sunshine) is mentioned, suggesting that
environmental conditions can also influence a chicken's decision to cross the road. This theme, however, was not
explored in depth.