Simulate a qualitative interview
This notebook provides sample EDSL code for simulating an interview between a researcher and a subject, with instructions for modifying the interviewer, interview subject or topic.
EDSL is an open-source library for simulating surveys and experiments with AI. Please see our documentation page for tips and tutorials on getting started.
Import the tools
Here we import the tools that we will use to conduct the interview. The interview is designed as a series of free text questions administered to agents representing the interviewer and subject. We use “scenarios” to parameterize the survey questions with prior content of the survey as the questions progress. Learn more about EDSL question types and other survey components.
[1]:
from edsl.questions import QuestionFreeText
from edsl import Scenario, Survey, Model, Agent
import textwrap
from rich import print
EDSL works with many popular language models. Learn more about selecting models to use with your surveys. To see a complete current list of available models, uncomment and run the following code:
[2]:
# Model.available()
Here we select a model to use for the interview:
[3]:
model = Model("gpt-4-1106-preview")
Create interview components
Edit the inputs in the following code block to change the instructions for the agent interviewer, the interview topic and/or the interview subject:
[4]:
# A name for the interview subject
interview_subject_name = "Chicken"
# Traits of the interview subject
interview_subject_traits = {
"persona": "You are a brave, independent-minded chicken.",
"status": "wild",
"home": "A free range farm some miles away.",
"number_of_chicks": 12,
}
# Description of the interview topic
interview_topic = "Reasons to cross the road"
# Total number of questions to ask in the interview
total_questions = 5
# Description of the interviewer agent
interviewer_background = textwrap.dedent(
f"""\
You are an expert qualitative researcher.
You are conducting interviews to learn people's views on the following topic: {interview_topic}.
"""
)
# Instructions for the interviewer agent
interviewer_instructions = textwrap.dedent(
f"""\
You know to ask questions that are appropriate to the age and experience of an interview subject.
You know to not ask questions that an interview subject would not be able to answer,
e.g., if they are a young child, they probably will not be able to answer many questions about prices.
You ask excellent follow-up questions.
"""
)
Interview methods
Here we create methods for constructing agents representing a researcher and subject, and conducting an interview between them in the form of a series of EDSL survey questions. Learn more about designing agents and running surveys.
[5]:
def construct_subject(name, traits={}):
return Agent(name=name, traits=traits)
def construct_researcher(interview_topic):
return Agent(
traits={"background": interviewer_background},
instruction=interviewer_instructions,
)
def get_next_question(subject, researcher, dialog_so_far):
scenario = Scenario(
{"subject": str(subject.traits), "dialog_so_far": dialog_so_far}
)
meta_q = QuestionFreeText(
question_name="next_question",
question_text="""
These are the biographic details of the interview subject: {{ subject }}
This is your current dialog with the interview subject: {{ dialog_so_far }}
What question you would ask the interview subject next?
""",
)
question_text = (
meta_q.by(model)
.by(researcher)
.by(scenario)
.run()
.select("next_question")
.first()
)
return question_text
def get_response_to_question(question_text, subject, dialog_so_far):
q_to_subject = QuestionFreeText(
question_name="question",
question_text=f"""
This is your current dialog with the interview subject: {dialog_so_far}.
You are now being asked:"""
+ question_text,
)
response = q_to_subject.by(model).by(subject).run().select("question").first()
return response
def ask_question(subject, researcher, dialog_so_far):
question_text = get_next_question(subject, researcher, dialog_so_far)
response = get_response_to_question(question_text, subject, dialog_so_far)
print(" \nQuestion: \n\n" + question_text + "\n\nResponse: \n\n" + response)
return {"question": question_text, "response": response}
def dialog_to_string(d):
return "\n".join(
[f"Question: {d['question']}\nResponse: {d['response']}" for d in d]
)
def clean_dict(d):
"""Convert dictionary to string and remove braces."""
return str(d).replace("{", "").replace("}", "")
def summarize_interview(
interview_subject_name,
interview_subject_traits,
interview_topic,
dialog_so_far,
researcher,
):
summary_q = QuestionFreeText(
question_name="summary",
question_text=(
f"You have just conducted the following interview of {interview_subject_name} "
f"who has these traits: {clean_dict(interview_subject_traits)} "
f"The topic of the interview was {interview_topic}. "
f"Please draft a summary of the interview: {clean_dict(dialog_so_far)}"
),
)
themes_q = QuestionFreeText(
question_name="themes", question_text="List the major themes of the interview."
)
survey = Survey([summary_q, themes_q]).set_full_memory_mode()
results = survey.by(model).by(researcher).run()
summary = results.select("summary").first()
themes = results.select("themes").first()
print("\n\nSummary:\n\n" + summary + "\n\nThemes:\n\n" + themes)
def conduct_interview(
interview_subject_name, interview_subject_traits, interview_topic
):
subject = construct_subject(
name=interview_subject_name, traits=interview_subject_traits
)
researcher = construct_researcher(interview_topic=interview_topic)
print(
"\n\nInterview subject: "
+ interview_subject_name
+ "\n\nInterview topic: "
+ interview_topic
)
dialog_so_far = []
for i in range(total_questions):
result = ask_question(subject, researcher, dialog_to_string(dialog_so_far))
dialog_so_far.append(result)
summarize_interview(
interview_subject_name,
interview_subject_traits,
interview_topic,
dialog_so_far,
researcher,
)
Conduct the interview
[6]:
conduct_interview(interview_subject_name, interview_subject_traits, interview_topic)
Interview subject: Chicken Interview topic: Reasons to cross the road
Question: Can you describe a time when you decided to cross the road, and what motivated that decision? Response: One time, I decided to cross the road because I noticed a patch of particularly lush grass and insects on the other side, which promised a feast for myself and my chicks. It was a calculated risk, but as a brave, independent-minded chicken, I'm always looking out for the best opportunities to nourish my brood. The motivation was clear – better food means healthier chicks, and I'd go any distance for their well-being.
Question:
How did you assess the safety of the road before deciding to cross it with your chicks, and what strategies did you
employ to ensure their safe passage?
Response:
Before deciding to cross the road, I carefully observed the patterns of the vehicles passing by to determine the
safest time to make our move. I looked for natural breaks in traffic and listened intently for the sounds of
approaching vehicles. Once I felt confident in my assessment, I led my chicks quickly and directly across the road,
making sure they were all closely following. To ensure their safe passage, I employed the 'stop, look, and listen'
strategy, and we paused whenever necessary to avoid any oncoming danger. Additionally, I made sure to choose the
shortest possible route to minimize the time spent on the road.
Question: How do you communicate with your chicks during the crossing to keep them organized and attentive to the potential dangers, and what challenges do you face in maintaining their focus? Response: As a wild chicken, I communicate with my chicks through a series of clucks and calls that convey urgency and direction. To keep them organized and attentive during the crossing, I use a sharp, consistent clucking to maintain their focus and guide them. The challenges I face include the natural distractions of the environment, such as interesting smells or insects that may catch their attention. It's crucial for me to maintain a balance between guiding them firmly and not inducing panic, which could scatter the group. I also have to be vigilant of any chicks that might start to stray or fall behind and quickly correct their course with a targeted call or a gentle nudge with my beak.
Question: Can you explain how the experience of crossing the road has influenced your behavior or decision-making process in subsequent crossings, particularly in terms of timing, route selection, or teaching your chicks about road safety? Response: The experience of crossing the road has honed my instincts and decision-making process for subsequent crossings. I've become more adept at choosing the optimal timing, waiting for moments when traffic is lightest or when there are predictable pauses. I've also refined my route selection, opting for the shortest and least obstructed paths to minimize exposure to danger. Teaching my chicks about road safety has become a top priority. I incorporate lessons from each crossing, emphasizing the importance of staying together, following my lead, and being alert to the sounds and sights of potential hazards. Each successful crossing reinforces these teachings and prepares them for future challenges we may face together.
Question: Can you share any specific techniques or signals you've developed over time to improve the effectiveness of teaching your chicks about road safety, and how do you gauge their understanding and readiness to cross safely? Response: Over time, I've developed a technique of repetitive practice and consistent signaling to improve the effectiveness of teaching my chicks about road safety. I use distinctive clucks to signal when it's time to pay attention and a different series of clucks for when it's safe to cross. I reinforce these signals through repeated drills in a safe environment away from the actual road. To gauge their understanding and readiness, I observe their reactions to my commands during these practice sessions. If they respond promptly and stay close to me, it shows they're learning. I also watch for their ability to anticipate my signals and to halt immediately if I give the warning cluck. Their consistent obedience and attentiveness during these drills are good indicators that they're ready to cross the road safely with me.
Summary: In the interview with the brave, independent-minded wild chicken, several key points were discussed regarding the reasons and strategies for crossing the road. The chicken described a specific instance where the motivation to cross was to access better nutrition for her and her twelve chicks, emphasizing the importance of their well-being. She detailed her methodical approach to safety, which involved observing traffic patterns and employing a 'stop, look, and listen' strategy to ensure her brood's safe passage. Communication during the crossing was maintained through clucks and calls, with the challenge being to keep the chicks focused amidst distractions. The chicken also noted that past experiences influenced her behavior in future crossings, improving her timing, route selection, and emphasis on teaching road safety to her chicks. To enhance the learning process, she developed repetitive practice drills with consistent signaling away from the road, assessing her chicks' readiness by their response to commands and their ability to anticipate signals. Overall, the interview highlighted the chicken's proactive and protective approach to navigating road crossings with her chicks. Themes: The major themes of the interview with the chicken include: 1) Motivation for crossing the road, which is primarily driven by the search for better food sources for the chicken and her chicks; 2) Safety assessment and risk management, involving careful observation of traffic patterns and the adoption of a 'stop, look, and listen' strategy; 3) Communication techniques used to keep the chicks organized and attentive to dangers during the crossing, such as specific clucks and calls; 4) The impact of previous crossing experiences on future decision-making, leading to improved timing, route selection, and safety education for the chicks; 5) Educational strategies for teaching road safety to the chicks, including repetitive practice, consistent signaling, and readiness assessment through drills in a safe environment. These themes reveal the chicken's thoughtful and protective approach to leading her brood across the road.