Context of Use Analysis Reloaded
Introduction
From the user's point of view, product innovation is mostly about operating functions ("features") that solve problems that users only become aware of when the product solves these problems. This is the natural source of wow effects. The statement "wow" articulates "this is something I did not expect, but it is great". But how do you produce innovative solutions systematically, say predictably, as a real product team instead of relying on geniuses? It feels like panning for gold, where the prospector must first identify the relevant physical environment that contains the gold nuggets and then is faced with hard work to finding as many of the gold nuggets as possible.
Various approaches to identifying such unmet or underserved user needs have been published, most notably the "jobs to be done" theory popularized by Anthony Ulwick and "contextual design" introduced by Karen Holtzblatt. User requirements engineering, as operationalized by the International Usability and User Experience Qualification Board (UXQB), is also a learnable methodology for identifying user needs in a given context of use that is increasingly being adopted by practitioners. This methodology is based on published ISO standards for usability and user experience.
User Research versus Context-of-use analysis
User research is an umbrella term for all activities during design, development, and evaluation of interactive systems (software, hardware, and combinations of both) that involve direct interaction with users. The term UX research is also used, but if you think about it, UX research is more focused on evaluating products with users for the level of achieved user experience.
User research activities include
- Context-of-use analysis (field observations of users and contextual interviews)
- Collaborative design with users
- User testing (i.e., testing interactive systems with users)
- Post-market user surveys about user experience and user satisfaction with a product in the field
The purpose of context-of-use analysis
Context-of-use analysis is one important aspect of user research. Context of use is defined as the “combination of users, goals and tasks, resources, and environment” (ISO 9241-11). The more an organization knows about the context in which their products are being used, the better it can identify unmet or underserved user needs and derive user requirements to be met by a product.
A context-of-use analysis has two goals:
- Find out what people actually do (tasks) and what the intended outcomes are (goals) with their given equipment and information (resources) in the environment they are in (physical, social, technical).
- Identify unmet user needs based on the current context of use and derive user requirements for the product / interactive system
The purpose of the context-of-use analysis is to become aware of the user requirements for an interactive system under development so that the product team knows what tasks the system must support and what the system must allow the user to do in performing each task.
Figure 1 illustrates what the context of use is comprised of and how it informs the design of the interactive system during its entire life cycle. All four components of the context of use, i.e.
- the users with their profile,
- their tasks and the task-related goals,
- the resources they utilize and,
- the environment they are in.
The user requirements for the interactive system must be explicitly derived from the context of use information. This step is essential according to ISO 9241-210 ("Human-centered design for interactive systems"). Team consensus on user requirements ensures a shared understanding of what needs to be enabled from the user's perspective and speeds up design decisions.
Upfront context-of-use analysis requires effort
Although it seems obvious that upfront context-of-use analysis is the ideal approach to getting the user requirements for an interactive system right as early as possible in the project, the hurdle of simply performing the analysis is high. Prospective users must be recruited, interviewed, and/or observed, and the information gathered must be captured in the right format to produce design input that truly informs the product team. Identifying user needs and deriving user requirements is demanding work. It also requires competence to express user requirements in a consistent style and keeping the statements at the appropriate level of precision.
Upfront context-of-use analysis saves money
Literature such as the book "Cost-justifying usability" by Randolph Bias, Lifetime Achievement Award recipient of the International User Experience Professionals Organization (UXPA), has shown that initial user research dramatically reduces the cost of implementing change requests in later stages of product development. But nevertheless, many product teams struggle to make the right investments upfront due to budget constraints and logistical challenges to get access to the right user profiles.
Using AI to cost-effectively analyze a product's context of use
Given those malpractices, secondary research which sources proxy information from subject matter experts in workshops or scanning the web and other documents for relevant information became a widely used alternative approach. No matter what context of use information you are looking for, there is a high probability that all of it is available on the web. Large language models such as Chat GPT prove this. So why not use it?
Of course, you can chat with an AI chatbot all day long to find out what you are looking for, but then the information needs to be captured, structured, and analyzed for implicit user needs and resulting user requirements. But the essence of a context of use description is to document the relationships between users, their goals, their tasks, and to know what pain points cause the users to fail to achieve the intended outcomes, so that the product designers and product owners can make the right design decisions. This information is difficult to derive from the fragmented text output of an LLM system.
At ProContext Inc, we have used our 30+ years of experience, knowledge, and wisdom to develop an AI-Agent solution that makes this discovery process of generating a context of use analysis more efficient. Our Product Context Analyzer interacts directly with OpenAI's LLM until it has fully captured the context of use for a given user profile or activity that your product should support. From this, it automatically generates industry-standard requirement models, desired outcomes, and user requirements that you can share with the product team. This process takes minutes and results in highly detailed and plausible descriptions of individual tasks, personas, or entire journeys. All you need to input into the system is a short statement about the activity you want to support (e.g., submitting travel expenses as an employee) or the target audience in form of a 'persona' you want to support (e.g., travel agent in a large company).
Figure 2 shows the Product Context Analyzer input form.
The Product Context Analyzer distinguishes between three types of activities, depending on the complexity/scope of the activity. The Product Context Analyzer automatically detects different types of activities based on your input:
- Task: An activity that is typically performed in one stretch.
Example: Submitting travel expenses as an employee - Responsibility: A set of tasks typically performed as part of a job role.
Example: Managing a travel services team in a large corporation - Journey: A multi-stage process that includes multiple tasks at each stage and involves multiple stakeholders
Example: Relocating an employee to another country
You can also enter just a user role (the Product Context Analyzer calls it a "persona") to find out all about the tasks that a person in that role performs.
- Persona: A job role or consumer role you want to learn all about.
Example: Travel Agent in a large corporation
If you already have conducted contextual interviews with users, you can also upload the interview documentation to the Product Context Analyzer which will then perform the analysis of your own interview transscript instead of using the collected knowledge of the LLM.
Product Context Analyzer Output
Each contextual analysis performed with the Product Context Analyzer takes several minitues because it generates the entire analysis in one run. This analysis is saved for future use, including all the details identified about the particular context of use. Figure 3 shows an example of a list of completed contextual analyses based on the above examples.
Analysis Results for an identified task
Depending on the type of context (task, responsibility, journey, persona), the Product Context Analyzer automatically generates the following information: For each identified task, the Product Context Analyzer produces
- Primary user and involved stakeholders
- Task factsheet
- All sub tasks
- Stakeholders and their involvement
- Who performs the task?
- Who contributes to the task?
- Who authorizes the task?
- Who is interested in the task?
- Who is affected by the task?
- Pain points experienced during each subtask
- Resources utilized for each subtask
- Task diagram giving an overview on task structure and resources used for each subtask
- Task object for the overall task and task objects for each subtask
- User requirements by subtasks
- Decisions to be made for each task
- User story map including all user stories by subtasks of the task
- Affinity diagram illustrating key problem areas that product design should focus on
- Affinity diagram illustrating Key outcomes that product design should focus on
- Job map according to the Jobs-to-be-done Framework
Figure 4 shows an excerpt of a task diagram that has been automatically produced by the Product Context Analyzer. The task diagram allows to quickly get the “big picture” of a task, e.g., for briefing a product team. The hyperlinks on the subtasks take you directly to the pain points that users experience during each subtask and the task object that the user creates or modifies during each subtask.
“Submit travel expenses as an employee"
Below Figure 5 shows an excerpt of a user story map for the task “Submit travel expenses as an employee”. The user story map contains all subtasks of the task (red post its) and for each subtasks the user stories (blue post its). User stories are statements of a piece of functionality to be implemented as part of a sprint within the SCRUM framework. The Product Context Analyzer automatically creates the user story map as part of the automated context-of-use analysis.
“Submit travel expenses as an employee"
Analysis results for an identified responsibility or journey
On top of what is being produced for each task, the Product Context Analyzer produces additional information specific to responsibilities and journeys:
- All subtasks of a responsibility
- Journey map with all stakeholders, tasks, outcomes, and pain points for each phase
Figure 6 shows an excerpt of the journey map for the journey “Relocate an employee to a foreign country” as automatically produced by the Product Context Analyzer.
“Relocate an employee to a foreign country"
Conclusions
AI-Agents will have a significant impact on the way user research, product discovery and user requirements engineering are currently practiced. AI-Agents that have algorithms to produce derived information (e.g., user requirements) based on factual information contained in an LLM (e.g. tasks and associated pain points) have a high potential to automate work performed by highly skilled experts in the field. It refocuses their efforts on design execution and quality assurance with reference to the information generated by AI-Agents such as the Product Context Analyzer. The Product Context Analyzer automatically ensures that the level of detail and wording is consistent with the common industry formats for usability-related information, ISO 25063 ("Context of use description") and ISO 25065 ("User requirements specification").