Introduction
Business success relies on strategy. Strategy comes from data. Research produces data. So why are some companies still not prioritizing user experience (UX) research? Until recent years, executives saw UX research as a niche field that existed only at leading technology companies like Apple. Now, the discipline holds an increasingly important role within Fortune 500 companies and emerging startups alike.
This trend originates from the world’s growing dependence on digital products, along with heightened competition in the software space. To stay ahead of the curve, executives need the best possible information to help make the right strategic decisions — and that’s where research comes into play. Jonathan Horowitz, the Head of User Research at TheoremOne states “Strategy is a living, breathing entity. It’s about choices made every step of the way.”
Today, large enterprises build their success on digital platforms: both the consumer-facing websites, apps, and services that generate revenue and the internal tools that support operations and workflow. These products are the lifeblood of any modern organization. Companies live, compete, prevail, or fail based on their effectiveness.
Strategy is a living, breathing thing. It’s about choices made every step of the way
<quote-author>Jonath Horowitz – Head of User Research, TheoremOne<quote-author>
With such high stakes, UX research offers critical guidance for leadership teams. It shapes the best-in-class software that allows companies to dominate their industries. This white paper explains how to incorporate user experience research into strategic planning for the primary milestones – from early idea generation through developed product. Combined with learnings from our real-world case studies, executives will discover the essential elements of a successful UX research program.
Executive Summary
What does the word “research” mean to you?Two executives rarely give the same answer, and that’s part of the problem. There are a variety of research practices, from market to product to technical research and beyond. Thus, every executive has past experiences that come to mind when they hear ‘research.’ Although the UX field continues to grow rapidly, it is less common for executive leadership to have hands-on experience with user research, however — causing initial resistance to emphasizing this critical function.
Executives that embrace UX research are best equipped to lead teams that build high-performance software. Organizations cannot successfully operate off of internal assumptions about what people want and need. Competition is too fierce, and user expectations are too high. “Change needs to happen from the top down,” says Horowitz.
Middle management cannot be held responsible for building resource-intensive digital products due to strategic misfires from above. It lies in the hands of the leadership team to establish a culture that values UX research — fully integrating the function into the development cycle. To effectively leverage user research, Horowitz describes three major milestones:
3 User research Milestones
- Validate intent to pursue a business idea
- Find product-market fit to reach profitability
- Design the best solution to support high-performance software
In the sections that follow, we discuss each of these milestones in depth, using relevant examples and outlining related research techniques.
By implementing the advice in this guide, executives can improve the process for strategic decision-making, increase profits by building better digital products, and avoid costly investments in the wrong development roadmap. UX research has the potential to unlock new revenue streams and massive efficiencies — all it takes is a little patience and well-crafted approach.
Milestone 1
Validate Intent to Pursue a Business Idea
Founders and executives typically come to TheoremOne for one of two reasons: either they have a fresh idea, or they want to revamp an existing platform that’s fundamental to their business.
In both cases, data supporting the project is a must. Whether to secure funding or convince the board, research will provide the required evidence to make their next move. And, if wise, this initial inquiry will inform whether or not the venture is worth undertaking in the first place.
During these seminal stages, UX research focuses on finding the right audience and target segmentation. It defines the specific problem the product solves, who faces that issue, and the size of the potential population of users. If the audience in question is minuscule, many choose to end the project. But if the opportunity appears sizable, the quest continues.
Key Steps for Validating a New Idea or Revamp
Defining your target audience
Who is the product for? The answer to this question must be crystal clear. Perhaps it’s an existing customer segment with unaddressed needs, or internal staff that’s stuck using a clunky but important tool. Either way, you need to know your exact users. Trying to build a product ‘for everyone’ leads to serving no one.
Once you formulate a hypothesis about your audience, go talk to those people. Do they actually have the problem that your idea intends to solve? This qualitative feedback often yields surprising results — you may discover that the right customer is a different, tangential group to your initial hypothesis.
Interviewing real potential users
When early-stage growth at AirBnB stagnated, co-founder Joe Gebbia flew to New York City to interview the company’s best customers. By speaking face-to-face with real users, Gebbia learned something that traditional metrics had failed to tell him: photos were absolutely essential to the future of the business. He and his team took professional pictures and updated listings. The result? Revenue doubled within a week.
Companies sometimes forget that users are real people — they look only at business metrics, and qualitative feedback falls to the wayside. At the beginning of any new project or major software revamp, a semi-structured, open conversation with customers is essential. You need to leave all assumptions behind and listen.
“Ask them about past examples. Run through how the product would make their lives easier. We are not looking for the person we interview to create the solution — it’s about understanding how their situation could change in the future. You want to see passion and enthusiasm for their problem to be solved,” Horowitz explains.
We are not looking for the person we interview to create the solution — it’s more about understanding how their situation could change in the future.
Surveying to validate at scale
After talking to your target audience, you may want to corroborate learnings with a larger sample size. It’s not possible to speak with every potential customer in-person. But, emailing a survey to a sizable subset is usually within reach.
A user research professional will help craft the takeaways from customer interviews into survey questions and distribute it to an appropriate panel. The results will eliminate potential discrepancies between the limited one-on-one discussions and the broader audience. “If you are investing a lot of capital into the development of a new program, this is a relatively low-cost, low-time activity that grounds your qualitative research findings,” says Horowitz.
Real-World Example: A New Idea
A major telecommunications company approached TheoremOne with a product idea. The leadership team wanted to expand into a new market and had already set aside a predetermined budget for the initiative. The project involved building a marketplace of developer tools that could be packaged and sold with connectivity. We expected “makers” to be the primary audience for this product, so the TheoremOne team went to hackathons with over 800 attendees with backgrounds in business, development, product management, design, and more. We brought prototypes and showed them to these groups for feedback.
After speaking with potential customers, we discovered that demand did exist for the platform. However, rather than serving makers broadly, we found that the real audience was a specific subset of users: developers who work at service providers. The ability to choose an Internet of Things platform and then add a connectivity bundle that fits their projects served a genuine need.
The research validated the idea and allowed the telecom company to successfully launch the product by crafting it for this unique audience. In this case, we did not need to pivot because the hackathon visits confirmed demand for the product. Instead, we built upon the concept to make it even better.
Real-World Example: A major revamp
Another client needed to identify the breadth of a critical technology problem and create a future vision for the company. Upper management wanted evidence to help choose the correct direction. They were grappling with old, inefficient software and asked us, “Should we build a new product, or fix what we already have?” We had 10 weeks to propose a solution.
Leadership at this company was not on the same page with the group that managed the software. We interviewed 60 people in the organization that used the product and summarized the existing pain points into high-level themes. Then, we sent a survey to over 2200 employees with the same job functions to test the original findings at scale. The survey found that 50% of the employees didn’t trust the reliability of the “automation” function, and up to 90% agreed that automation was a problem — despite being a core part of the current software’s purpose. Close to 40% had this issue on all of their job files, while up to 80% said it happens on half of their files.
This evidence gave leadership a clear answer: it was time to rebuild, rather than place another bandaid on the tool. We presented a vision of a future scenario in comparison with the current situation to help demonstrate the tradeoffs and ROI. After many years of wrestling with this question, they were able to move forward unified and confident in the new direction.
50% of the employees didn’t trust the reliability of the “automation” function, and up to 90% agreed that automation was a problem — despite being a core part of the current software’s purpose.
Advice for Executive
- Organizations throw good money after ideas that fail all the time. Don’t invest resources without first clearly defining the problem and the audience with user research.
- Validate at scale for large organizational initiatives. When the investment is significant, it’s well worth the effort to verify learnings through surveys before solidifying a direction.
- Pivot if needed. Often you find that your original hypothesis is flawed, and the best time to course-correct is at the very beginning.
Milestone 2
Find Product-Market Fit to Reach Profitability
To address product-market fit (PMF), you first need to define it. Descriptions of PMF will vary, but the easiest way to understand it is this: right audience, right problem, right solution.
Although it may sound similar to concepts in the previous section, PMF takes things to the next level in a major way. You have an actual product that solves the problem, and you may even have a customer base already paying for it. The grey area relates to whether
the product will become profitable or not. It doesn’t need to be generating profits today to indicate PMF, but there should be strong signals of future growth, beyond just projections in a spreadsheet.
In our experience, the design component is essential to PMF. You can validate a great idea, but effective execution is the only way to actually solve the problem for users. Initial research leads you down the right path, yet the validated ideas need further attention as your assumptions creep back into the process — unintentionally nudging the project off course. For this reason, UX research must continue throughout the design process. It ensures that your team keeps working towards PMF.
Key Steps for Finding Product-Market Fi
Concept testing
A primary UX research method at this stage is concept testing. With this technique,
you present early designs to a small number of users to gather feedback. Through the interviews previously conducted, we develop a mental model for the target audience that captures how they think, what they need, and how they behave. We then apply this model to the design concepts.
At TheoremOne, we usually meet with three to four users, show our latest design, get their input, iterate, and repeat. These sessions are very talkative and engaging. During this time, we also create an interactive prototype of the concept. Prototyping gives users something tangible to try out. It brings the concept to life and ensures that we are headed in the right direction.
Through concept testing, you learn whether or not the original goal for the product has been achieved. Does it serve its intended purpose? As the concept matures through iteration, we gain a deeper understanding of the user’s mental model. The interface begins to to reflect the solutions for how they think and how they need the problem solved.
Task-based tests with real users
Once a final concept is delivered, you move into usability testing. The researcher gives the user a task to complete with the working prototype, then allows them to complete it without any help or commentary. Unlike concept testing, the facilitator remains silent to simulate a real-life scenario.
If the tool falls short, you see exactly when and how during the task. The more thorough the concept testing, the fewer surprises that emerge in usability testing. Ideally, with these tests, you are looking for minor tweaks, but sometimes larger modifications are needed.
The more unique the product, the more unknowns you must explore.
Trial runs
When dealing with highly interactive tools like digital products, it is essential to vet all functionality and UI elements through user feedback. The more unique the product, the more unknowns you must explore. Solving the “job to be done” is not simple, and it is common for the actual execution to surface themes that the product and engineering teams did not consider.
This is where trial runs come in. A trial run takes the product out of the lab and allows users to put it to work in a real-life setting, sort of like a test drive before purchasing a car.
However, in this case, we are testing a product that is a work in progress. The goal is for your team to uncover any big challenges or opportunities previously missed. This feedback prevents any major oversights prior to a full launch.
Ideally, users go through all their top tasks numerous times during trial runs. If you have multiple user roles on the platform, they interact with each other to simulate real-life activity. The exercise gives the platform the ability to prove its value. If it is not doing something right or how it works feels off to people, you will surely hear about it. When it comes to product-market fit, this is the last mile test to ensure the product is ready for prime time.
Real-World Example: Taming the Wild Wild West
A large financial institution approached us with an idea for a new business venture. We helped validate the idea and then took it a step further to test for PMF. In the client’s part of the finance industry, deals include many different businesses. Moreover, there are buyers, sellers, attorneys, and many other parties involved. The process is chaotic. It’s like the Wild West, but everyone is just used to it.
Our client wanted to organize this crazy process — storing documents in a single place instead of disparate email inboxes. They wished to create order, streamline the complex workflow, increase the security of documentation, and save time. The need was clear but required a simple solution.
Designers were tasked with solving this puzzle. It is important to have UX/UI designers that understand how to create an interface tool using learnings from research if you want to build a high-functioning product. For this project, our design team zeroed in on a single user type — lawyers — and focused on their workflow first.
When your product is released, adoption is fundamental. If users do not grasp the value of the product in about three seconds, then it may be dead in the water. You want the target user to see the product and then tell you what it is and how it solves their problem. You want them jumping out of their seat with excitement while this is happening. It takes iteration on your designs and hypothesis to achieve this goal.
We showed lawyers the interface to see if they understood what it was, then asked them to explain what it could do for them. Next, we gave them basic tasks to run. We were going through this typical research exercise with one lawyer using our latest iteration of the interface. The client saw the interface and described the product perfectly. She was over the moon excited and exclaimed, “I need this now. When can I start using this?!”
Still, a couple of steps remained before launch. As a trial run, TheoremOne created “mock” deals and ran tests with real users to streamline the process. As these trial runs matured, law firms began using the software on actual deals, free of charge. Our client needed this experience to say, “We ran X number of deals on this platform and have testimonials to support how well it works.” We took them across this finish line, and now the product and all its success belongs 100% to them.
Advice for Executives
- A great idea is nothing without great execution. Emphasize design and make sure that your designers work very closely with the user research team, collaborating on every iteration.
- Keep users involved throughout the development process. It is not enough to collect feedback at the start. Conduct concept tests and trial runs until users jump out of their seats with excitement.
- Be consistent and diligent at every step. Research takes a little patience, but it pays off in spades when you launch a market-leading product.
Milestone 3
Design the best solution to support high-performance software
Many user interfaces exist to serve the same purpose. Look at all the online banking solutions and various online news applications. The New York Times app, Flipboard, and Apple News all offer access to news, providing that service to interested users. The interface and its content are what differentiate each business, allowing itto stand out in a crowded marketplace. Some apps you choose to use. At TheoremOne, we often help with software that you have to use, i.e., enterprise applications. People rely on these platforms as part of their everyday jobs, but they were not involved in the selection or building of the software.
If the interface is tough to use, it impedes workflow and hurts the business. It may even affect morale and employee retention. Operational tools must be high-performance machines for organizations to rise above the competition.interference — conclusive. When the findings are understood, well-interpreted, and acted upon, A/B tests are powerful drivers of high-performance software development.
For instance, imagine working on a home improvement project. Your tas krequires a screwdriver, but the handle keeps coming off and you’re incredibly frustrated. It takes 10 times longer than the job would with an electric drill. Office jobs are the same — an employee needs a high-performance tool to do a high-performance job instead of wasting many hours simply trying to make a system perform basic functions.
Early movers can launch a deeply flawed interface and still succeed in the short term. In the long-term, however, better solutions will take over if they don’t evolve the design through user feedback. This section focuses on optimization, which never ends. The world changes, technology changes, business changes, and user needs evolve too.
A/B testing
Along with continued qualitative feedback, A/B — or even A/B/C testing — provides quantitative data for ongoing optimization. These tests may pull from a wide variety of metrics. A user research professional determines the correct metrics for measuring success, depending on the goals of the project. Then, you launch one to three variants within the product to see which one wins. Once statistically significant, the results of an A/B test generally decide which version of an experience or feature is rolled out. The data is objective and — without spurious interference — conclusive. When the findings are understood, well interpreted, and acted upon, A/B tests are powerful drivers of high performance software development.
Insiders
Not all product updates lend themselves well to A/B tests. When launching a significant overhaul or feature, “insiders” testing provides a subset of current users with access to the revised experience. You can gather a wealth of both qualitative and quantitative data based on real user interactions. Insider groups are different from trial runs. The main purpose of the trial run is to test how the overall workflow holds together and see if any big holes exist. Also, with trials, anyone that screens in as a potential user can participate. The insider group involves the actual people that use the software. They participate in multiple feedback cycles with the primary goal of refining features based on real-world tasks and data. This feedback is vital for the continued maturation of an interface. It definitively shows which features matter the most and where to focus efforts.
Insiders help with adoption as well. These users can provide new ideas for features, giving the product team a user-driven set of options for short and long-term roadmaps. As one of the final tests before a release, insiders contribute feedback in the most realistic circumstances possible. The findings lead to precise optimizations that allow a digital product to evolve and flourish.
Real world example: a vital search
A client in the real estate industry needed to build an internal search tool for a new platform that manages their insurance deals. Internal employees are the primary users, working from several offices in different regions across the U.S. For this product, we knew the search function was vital, but what the actual search solution needed to offer was unclear. In this instance, the company did not have an existing tool — we were building from scratch.
We created an insiders group of internal employees that agreed to support our research initiatives, with representation across every region and job function. These individuals helped us shape the tool from the beginning to the end of development. After going through the interviewing and concept testing process, we learned exactly what users were searching for: existing open or closed deals, client names, and internal employee names to find help with day-to-day operational tasks. When looking for deals, they often had only partial information to use in a search.
When trying to locate an internal employee, they usually didn’t know the person’s name but knew the person worked in a certain office and might have worked on a past deal on which the user needs information. The tool had to help staff members make these connections. Once we understood the user’s mental model, we introduced variations of the design.
We developed one design with a text entry tool like a Google search field, and another that offered categories as a mechanism for input into searching. We found that an open-ended search was the right model for how the employees think and how they want to take action. The category-based option didn’t win, but it turned out to be valuable to the process — giving us ideas on how to filter results. Insiders are leveraged through beta launch — testing new designs in production environments. This framework gives our clients and us the confidence that we are on track. It also provides certainty around adoption, which the beta testers prove before a release. For our real estate client, the search tool completely revolutionized employee workflow for the better.
Advice for Executives
- Create a representative group of the end users that can provide continuous feedback. These individuals provide the best possible real-world testing environment before a release.
- Avoid unnecessary features that waste time and money. Sometimes, a feature sounds like a good idea in theory, but no one ends up using it. You can eliminate these pitfalls in advance by putting updates through the appropriate UX research tests.
- End users will ultimately choose what works for them. Include them at every phase to crush the competition. Leave them out of the development cycle and watch users abandon the product for a competitor that provides a superior solution.
What’s next?
The field of user research is evolving rapidly. Tools and techniques are constantly improving, and it will be hard for organizations to keep up without dedicated support. As more and more executives see the business impact of research, demand for these insights will grow — giving UX researchers a proverbial “seat at the table.” Leadership teams that value this data will find decision-making much easier, thanks to the objective guidance provided by actual users. After all, the core job of a UX researcher is to surface actionable data.
We recommend integrating UX research into operations post-haste to stay ahead of the curve. As mentioned, a little research goes a long way. No matter the size or resources of your organization, you can start reaping the strategic rewards by applying the learnings from this white paper. TheoremOne offers user research support to a wide variety of industries. To learn more about how we can help your organization, send an email to info@theoremone.co.
