This month was busy! I also contributed an article, Beyond User Research: Giving Voice of Customer Another Chance, to the UXPA’s User Experience magazine.. It’s an argument that while, on the one hand, there’s often inappropriate pressure for enterprise UX people to do what customers say, we nonetheless need to make sure we’re not losing touch with customers entirely. They’re important from a relationship perspective, but more importantly, they can provide context for understanding what our users need to accomplish.
IXDA gave me an opportunity to chat about applying product management skills to UX, in front of a very engaging and talkative audience. Slides are here!
I keep running into this confusion. An account manager refers to me as “that nice VOC lady.” (He’s partly right.) Another product manager wants to know why we have to have “so many people” participate in a research study (“you could just get a guy and ask him what he thinks!”) So it seems worth talking about the difference between the two, and why they’re both important. Because as much as I might hate to admit it, getting that punchlist from your Big Fricking Order customer is as critical to your business as ensuring a highly usable software UI.
I put together this chart of differences for a quick rundown of the differences. The biggest one to my mind is that VOC measures desires, while user research measures behavior. . The people (or personas) who participate are different, too: VOC is voice of the customer, but not necessarily of the user, whereas user research doesn’t necessarily include anything about the organization’s goals, or why the product is purchased.
I made a PDF, too.
Here’s what I talk about when I talk about ROI.
Periodically I need to justify the business value of user experience or customer experience activities in an enterprise environment. In an e-commerce B2C environment, the idea that usability testing will yield improved returns is common wisdom nowadays, but not all enterprise software companies have institutionalized user experience practices. Furthermore, even organizations with a strong UX presence contain pockets of entrenched doubters. While I believe that the value of an investment in user experience isn’t fully expressed in numbers, I usually try to address these concerns by talking about return on investment (ROI), ideally with a series of memorable killshots (pew pew pew). When I’m in this situation, I want to prove that UX is more than just a cost center — that it’s directly connected to revenue and other metrics that my audience cares about.
Enterprise-relevant proof points have been hard to find, however, which isn’t surprising. Many of the studies that have been done are twenty years out of date, and others aren’t especially relevant for an enterprise software product (rather than an e-commerce experience or a consumer product). In an enterprise environment the revenue outcomes of activities can be harder to track, as well. So I’ve pulled together a few of the key bullet points I’ve used over the past year or two. You’re welcome to use them, but make sure you credit the actual author, as I have below.
There’s a caveat here: No published numbers will be as compelling as internal and timely information from within your own organization. Even if they’re preliminary numbers, or half-baked anecdotes, describing the time we tested the prototype at EventCon and how the insight led to a 25% uptick in sales will probably pack a bigger punch. But in the absence of airtight success stories, the below points can help bolster a business case.
User-centered Design (UCD) Drives Customer Satisfaction and Revenue
- User-centered design can raise customer satisfaction 40% or more over other methods. (1992 Gartner Group study, cited in Bias & Mayhew).
- System Usability Score (SUS) metric significantly correlated with Net Promoter Score (NPS), a generally accepted measure of customers’ willingness to recommend a product or service. (Predicting Net Promoter Scores From System Usability Scale Scores)
UCD Reduces Development Cost
- Design changes made in an up-front user-centered design phase cost only 1% as much as those changes after product release. (Bias & Mayhew, 1994)
UCD Reduces Operational Cost
- Susan Weinschenk estimates that a well-designed internal application saved 300 help desk calls (against the average) in the first month after release — with a benchmark of $15/call, that’s a total savings of $75k in the first month. (ROI of User Experience)
An Appeal to Authority
- 10% of the Fortune 125 have executive- or C-level design talent. These include not just Apple, but companies with significant enterprise business such as General Electric, IBM, General Motors, Ford, 3M…In other words, it’s not just for the guys with the cool glasses. (Design in Tech 2015.)
Bias & Mayhew refers to Bias, Randolph G. and Mayhew, Deborah J., eds. (1994) “Cost-Justifying Usability.” Harcourt Brace & Co., Boston.
From Anthropology to Empathy in Enterprise Software Design
A unique challenge of designing enterprise software is the difficulty of developing a specific empathy for your users. You’re making tools for someone’s “work self” — probably a less social and less emotive persona than a consumer in private life. Learning about the users of an enterprise resource planning (ERP) system is not the same as talking to teenagers about the duckface selfies they send to their friends, or the same as health insurance customers navigating a complex, often infuriating bureaucracy. Your job is designing for someone doing their job — someone whose job is different from yours (unless by chance you’re designing software for people who design software to use when they design software for people who design software, in which case it’s turtles all the way down.) How do you do design for that other worker successfully? Walking a mile a user’s metaphorical shoes is harder when they’re white nurse’s clogs or the steel-toe work shoes that belong on a plant floor, instead of your own familiar Sauconys. It’s harder to imagine how those shoes feel when you’ve never worn them yourself.
We stand in the rain in a long line
waiting at Ford Highland Park. For work.
You know what work is — if you’re
old enough to read this you know what
work is, although you may not do it.
— Philip Levine, “What Work Is.”
In Philip Levine’s poem, work is both a way — often too scarce — to make a living, and the effort it takes to be present and aware. Just like in our lives, it’s a practical means to survival and a cognitive keystone as we struggle to understand ourselves in relationship to others, and to our place in the world. As we design for workers, we can create empathy for our users by using anthropology’s tools to understand their work and culture.
Understanding What Work Is
The distance between your job and someone else’s job — the very thing that makes it hard to empathize with enterprise users — makes it easier to design well for them. In my first introductions to user-centered design, at the beginning of graduate school, the idea that “you are not your user” was drilled in as a central tenet of practice. In a similar vein, Pragmatic Marketing used to sell a mug that said “Your opinion, although interesting, is irrelevant,” so that you’d remember that distance every time you took another sip of coffee. These reminders are valuable to us precisely because the lure of confusing ourselves with our users is so seductive, and more so when the commonalities seem obvious. But when you’re designing explicitly for a worker whose job is profoundly different from yours, it’s less tempting to give into that impulse. Meeting the challenge of enterprise design saves us from confusing ourselves with our users — which allows us to design more rigorously, and better.
(In talking about empathy, I like Seung Chan Lim‘s practical definition: “an explanatory principle for our ability to experience a phenomenon of feeling as if we are embodying or understanding someone else’s experience and the related meanings from the context and vantage point of that ‘other.’” The physicality of embodying someone else’s experience sounds about right to me, especially when we’re talking about something as location-specific as work. Additionally, the individuality of “someone else’s experience” rings true, since it separates the general goodwill we sometimes call empathy from a highly granular, specific interrogation.)
Since we know that designing for yourself is a dangerous path to tread, how can we actually develop the kind of empathy we need in order to come up with possible solutions before we can get feedback from users themselves? Anthropology and its subfields offer some practical techniques and frameworks for coming to understand the people who will use your software, where, and why. They can help you understand what work is for someone else.
Being an Ethnographer
Ethnography is a flavor of cultural anthropology, the practice of studying the culture or lifeways of a group from among its members, focusing on specific themes, in the context in which the group is gathered — where they live, work, worship or all of these. Material collected can include observations, recordings, as well as artifacts. These objects or experiences illuminate the culture of the group and increase the ethnographer’s understanding of the group dynamics as a whole. The ethnography itself— a written end product — is based on the observations, interactions and artifacts. While an essay or longer work of non-fiction may not be viable, reports on study results as well as personas that are influenced by the ethnographic work are significant outputs as well.
Every workplace has a culture, and anthropology offers us a lot of tools to use for developing an empathetic understanding of that culture. For example, each workplace has rules, spoken and unspoken. It has a native costume for different occasions or activities; in my workplace culture, the traditional outfits range from jeans and a sweater to a skirt suit with heels, depending on the occasion, but others include scrubs, canvas coveralls, or pleated khakis with a golf shirt. Explicit rules and policies exist, and at the same time, observed practice around those rules can vary (observe smokers outside, for example, clustered under a non-smoking sign.) Power structures can be complex and may not be visible to the outsider, or may be as explicit as an org chart or as predictable a hierarchy as a band of gorillas (are product managers the only ones who call seasoned veterans “silverbacks”?). Participant-observation, being “in the world but not of the world,” can yield these insights over time.
Spending time among your users is one of the most valuable, as well as the most expensive ways to learn about the invisible structures of their daily lives, their motivations and struggles—but just spending time with your users doesn’t necessarily build empathy. Struggling to take notes, to time activities or to document workflows with precision when your user is on their turf and on their time is unlikely — speaking purely from experience — to build empathy; it’s at least as likely to frustrate you (if she would just slow down I could get this in my notes!) as it is to create an intuitive appreciation for the experience of the user. Additionally, it may alienate the user, creating the unnerving impression of gimlet-eyed attention. Instead, patiently observing, asking gentle open-ended questions, and staying out of the way is much more likely to yield that appreciation. You want to be a fly on the wall, not an impassively staring wildcat.
There are different names for the process of learning about your users by spending time among them, and different flavors: contextual design; ethnographic research; in-depth interviews; participant-observation. They’re all influenced by anthropology, areas in which the classic approach to field research was to treat all observations as though they were objectively reported and inherently true, and in which the more contemporary methodology assumes that the observer is as much a biased, specific individual as the subjects of the research. Practices like asking open-ended questions are an attempt to shield our subjects from our own bias, in case it leads them to answer in a certain way.
Fieldwork has Practical Limitations
Field research done right, has a number of limitations in the corporate, for-profit environment that it doesn’t necessarily face as a tool for academic study. First is the cost — both the cost of gathering the information (since each individual can only gather useful data at a limited rate) and the opportunity cost of having one or multiple people engaged at a customer or user site rather than completing other work. It can be very difficult to make the case for investing in such a “slow” activity; in my own experience, it’s easiest to make this case when management (or whoever controls resource allocation) is bought in on the value of user research, and gives you the latitude to determine the value of your activities. I have a hard time making the case to myself to take days out of my schedule and spend time with customers — and I’m pretty thoroughly bought into the value of doing so.
Second, assuming finite resources, the number of data points available are limited, and there’s a non-zero risk of stereotyping or drawing inaccurate conclusions about the group from a too-small sample. Customers may not be willing to put up with your presence indefinitely, even if you can be unavailable to your regular activities that long. In the imperfect corporate environment, the best way to mitigate this limitation is to embrace it — when shallower, broader research is available (for example, survey results, or a sufficient number of in-depth interviews), pairing the results of your “field work” will produce a more robust framework for understanding the users’ work environment.
Third, the problem of access. So, it turns out that the problem with customers is that you have a commercial relationship with them. (Spoiler: this is also the really great thing about customers, since they pay you for goods and services) You’re probably not the person who’s responsible for maintaining that commercial relationship; in most software companies, there’s a sales or customer success team who’s responsible for managing accounts. Depending on the account manager and the customer, getting you some face time might be a slam dunk, especially if the customer’s happy. But since you learn at least as much from unhappy customers, it’s great if you can spend time with them as well. It can be difficult to convince a skeptical account manager that they should ask an unhappy customer for their time and resources so that you or your team can learn from their experience.
Finally, these practices won’t generate useful quantitative data; within a business organization, quantitative data has both theoretical and political value, not least because it’s easier to communicate concisely and to absorb quickly than more nuanced qualitative information. Therefore the value of immersion in the user group may be harder to sell up the chain of command — the results of the research will be more difficult to repurpose or pass on. Again, aligning the results of this research with different work on the same group will produce more value, to “numbers people” and “story people” alike.
Future posts will address ways around these obstacles.
More reading on anthropology and user research
Tales of the Field is a classic primer in ethnographic writing.
Philip Levine is an amazing poet on Detroit and the nature of work.