The Maturity of UX Organizations

What makes a UX practice “mature?,” how do we measure UX maturity, and does maturity really matter?

In an earlier article, we discussed the history and challenges of assessing the UX maturity of companies and departments within large organizations.

Existing models of maturity generally consist of different stages, with maturity progressing from unrecognized or ad hoc to institutionalized (for example, Nielsen’s steps). The models also identify common components to assess, including the importance of budgets, methods, processes, organization structures, and the attitudes toward UX.

While the models are helpful, they do lack empirical validation to ensure we’re assessing the right things. To improve the rigor of the UX maturity assessment process, we created an industry-wide survey to benchmark UX practices across different companies. With the results, we identified patterns that differentiate mature from less mature companies and then used the differences to create a model of UX maturity. Here are the main findings from that survey, which we also presented at CHI 2017 . The full report is also available for purchase.

Sample Characteristics

We used an international non-probability snowball sampling method,  along with social media and distribution lists, to invite UX professionals to take the survey. Keep this in mind as you assess the generalizability of the findings.

We received 150 completed responses between Dec 2016 and June 2017. The survey took participants an average of 18 minutes to complete.

Note: To preserve anonymity we did not ask for company names; therefore, it is probable that some respondents are from the same organization.

Respondents represented a range of employment levels with over 90% being in a mid- or senior-level position. 38% of respondents were UX researchers, 30% identified as UX generalists, and 19% were managers/directors.

Most of the organizations were for-profit (86%), with 4% non-profit and 8% in the government or military (given the high percentage of non-governmental response, we use the term organization and company interchangeably). The median company size consisted of 1,350 employees with around half (45%) of respondents’ companies having between 1,000 and 50,000 employees.

Fifteen industries were represented, with most respondents in software (27%), finance (13%), retail (11%), and government/public sector (8%).

More than half of respondents’ companies work on consumer products/services (65%) and B2B products (57%). Most respondents’ companies also work on mobile (69%) and desktop software (48%).

Most respondents indicated being involved in user research (86%) and usability testing (86%), which is consistent with the salary survey results providing a good point of corroboration.

Maturity

Respondents were asked to rate the maturity of their organizations on a six-stage UX scale:

1. Unrecognized: UX is not an issue; UI designed mostly by developers
2. Ad hoc: UX defined as an issue; methods are inconsistent
3. Considered: Dedicated staff hired; not applied to all projects
4. Managed: UX process well defined; iterative evaluation
5. Integrated: UX processes consistently integrated into the product development life cycle
6. Institutionalized: UX is corporate business strategy and all human-facing products follow UX design process.

Most respondents (52%) selected the “Considered” stage to describe the company they work for (see the figure below). The relatively symmetrical distribution of responses across the stages, with similar percentages at the extremes, also suggests this six-stage scale does a good job of describing the distribution of UX maturity. Organizations that choose to use this scale now have a third-party industry benchmark to compare their scores (and individual department scores) against, making their designing more meaningful. For example, if a department in a company is rated as “integrated,” then its maturity is higher than 80% of companies.

 

Differences between Mature and Less Mature Organizations

To understand the major differences between mature and less mature organizations, we created a new binary variable to separate them (also seen in the figure above). Mature organizations were from respondents who identified their company as managed, integrated, or institutionalized. A company was labeled as less mature when unrecognized, ad hoc, or considered was selected. Using this segmentation, 36% of respondents described their companies having a high level of UX maturity.

Across the dozens of variables we collected from respondents, we found many statistically significant differences. We’ve called out the biggest differences below. Results of all variables are available in the downloadable report.

1. Mature companies are nearly three times as likely to have a dedicated budget. Less mature companies are 8 times more likely to have NO budget.
   
2. Mature organizations are more likely to have distributed in-house teams.
   

3. While company size did not significantly differ by maturity, mature companies have significantly more employees who work full time in UX (69 versus 10). We suspect UX maturity and department/company size are confounded and future research can investigate how to separate company size from maturity.
4. Employees at mature organizations tend to be distributed across products. Employees from less mature organizations tend to be distributed across departments or locations.



5. End users are involved at each stage at higher rates among mature companies, which is consistent with one of the fundamental concepts in usability: test with users early and often.



6. Mature organizations assess UX at more stages, particularly in early phases of product development. Not surprisingly, less mature companies reported having more challenges in testing designs with end users (28% difference), securing UX budget/resources (27%), and getting buy-in from executives (27%).



7. Mature UX organizations are more likely to include all roles, particularly manager/director, researcher, and visual designer.
8. Mature companies are ~50% more likely to use success metrics and 50% more likely to use any method. Mature companies use on average 20 methods (out of 30); less mature use 14. In particular, we saw mature companies were more than 3 times as likely to use eye tracking and 2 times as likely to use card sorting, strategy consulting, and benchmarking. Note: Using a method like eye tracking will not cause your UX practice to become more mature (correlation is to causation). These are simply the symptoms of UX maturity and not the cause.



9. Respondents from mature companies have a significantly higher perceived value of UX (a 29% higher mean rating of perceived value).
10. More respondents from mature companies report having funding and time for continued education and perceive this funding as sufficient.

Summary

Data from 150 UX industry professionals provided a first look at how UX organizations differ with respect to self-reported UX maturity. The six-stage scale does a good job of describing the distribution of UX maturity and can be used by other organizations to gauge how well their self-reported maturity stage compares to the industry.

We found that mature and not mature organizations differ on what we’d expect: budget, org structure, metrics, and methods. Understanding differences helps identify which variables matter when assessing UX maturity. This study relied on self-reported data, which means it’s susceptible to biased judgment. A full UX maturity assessment would involve both a self-assessment and an objective (and observational) audit from a third party.

It’s likely that company size is confounded with UX maturity and future research may better differentiate between company size and UX maturity. The next and more aspirational goal of this research is to determine if and how UX maturity is related to company and product success to better prioritize what matters in a UX organization.