Speeding Along…
“Research is formalized curiosity. It is poking and prying with a purpose.”
- Zora Neale Hurston
Our second sprint was all about interviewing, pretotyping, and more interviewing. Operating in an incredibly complex service-scape with a large varying user base, we decided to focus on conducting lean user research that would allow us to learn as much as possible as quickly as possible.
Our goal was to get a holistic picture of the interaction between different actors within the end-to-end service, major touchpoints along the way, and potential opportunity areas for a conversational agent to create the most value. While we hit a few speed bumps along the way we were able to quickly improve our foundational knowledge of the space and get ourselves up to speed.
Lean but, purposeful research
To align on our primary research goals, we conducted a self-adapted game design method led by Zach called MDA-Lite. The process involved generating ideas for three sections; the ideal state (what we hoped to have a better understanding of after our primary research), the data points we needed to capture to help us get there, and the specific questions we needed to ask to address those data points. By doing this activity we were able to gain a better understanding of our priorities for our primary research. We found that our ideas of an ideal future state aligned in many ways and it helped us ground our goals for what we wanted to learn from our primary research.
Getting out there and talking to people
Talking to experts
We talked to CMU professors and faculty in relevant areas working on conversational agents, conversation design, and service design to help us understand important considerations and opportunity areas for designing context-aware conversational assistants within the context of digital services.
Talking to people researching cars
The first group we targeted for our primary research were people currently in the research process of finding a car. We conducted a remote contextual inquiry with participants and had them share their screen while navigating us through the different tools they used/are using in order to learn more about the options available to them.
Talking to CarMax customers
With the help of CarMax, we were able to recruit participants who had recently visited a CarMax lot to help us understand more about the current on-lot experience, the different kinds of individuals visiting CarMax lots, the high and low points of their experience, and potential opportunity areas that would be fruitful for experimentation with a conversational agent.
Talking to CarMax sales consultants
To understand the CarMax service more holistically we decided to talk to sales consultants to understand more about their daily routine, how they interact with customers, where they may have pain points in the current process, and how they could potentially be aided by automation.
Pretotyping a service
Why pretotype?
Since a core part of our project is to experiment with the different ways a conversational agent can provide value within a service we decided to pretotype a service. A pretotype is meant to be a prototype of a prototype and provide a way to quickly test ideas in a low stakes environment. Our goal for this pretotype was to understand what potential role a conversational agent could play in the end-to-end car buying experience, how people expected to interact with this agent in different contexts, and whether interacting with this agent could help them move forward in the car buying process.
The pretotype service
We designed a pretotype that involved a three part scenario driven by a make-shift voice agent. Each part of the scenario included an open-ended prompt that gave people enough information to have a conversation with the agent without being too prescriptive so we could really see how people would approach this interaction. We used cars, laptops, mobile phones, zoom, some python, and text to speech to simulate a fully conversational voice agent that a person could talk to in various settings and contexts.
The scenario
You are currently in the market for a new car with your old one having recently broken down. You are interested in looking at different used car options and want to explore the different options you have. You decide to use a new voice-enabled chat agent to explore different options available from your local dealer.
You have now chosen a car and arrive at the lot to check it out for yourself. The car has a voice enabled agent inside that is able to answer any of your questions. Feel free to walk around or go inside the car and ask any questions you normally might have before you deicide on whether you’d like to purchase the car. [For this part we also allowed some of our participants to take the car for a test drive and ask our agent questions as they drove.]
You have been a proud car owner for a few months now and are engaged by the built in agent about a routine service that is coming up. Interact with the agent to determine the best plan forward.
Adapting on the fly
Our goal for this method was to creatively find a way to identify the potential value of using a conversational agent and as such we adapted both during and after each study by adding and removing elements within each scenario as we learned new information to get the most value out of our testing. For example, one of our test drivers asked us to play music during their ride and give them directions. While it wasn’t something we intended to do initially we adapted and allowed them and future participants to listen to music during their test drive and had the researcher in the back communicate directions to our control center so that we could address this new need we identified on the fly.
Continuing the drive
As our hectic first round of primary research comes to a close we look forward to seeing how our analysis and synthesis identify new routes for us to explore. Will the new routes be the fastest and most scenic ones? Stay tuned to our road trip to find out!
