AI may be rapidly increasing the speed at which we can generate software and find answers, but where there’s an interface for a human, interactions aren’t always smooth. As AI providers compete for users, they add features. Although features add capabilities, they can also add complexity. Simple chat interfaces are giving way to tabbed experiences with more complicated terminology and navigation structures. It’s time to see how that may impact the user experience.

To follow-up our 2025 benchmark of AI-based chat software, we conducted another retrospective study of four AI-based chat software products: ChatGPT, Claude, Gemini, and Grok. In this article, we present key UX findings from our 2026 investigation with some comparisons to those 2025 findings. For more details, see the full report.

AI-Based Chat Software Benchmark Study

In May 2026, we conducted a retrospective study of four AI-based chat software products with 420 U.S.-based panel participants. This study included the metrics we typically collect in our standard UX and NPS study of consumer software.

There was a roughly equal gender split (52% female, 47% male). Respondents tended to be younger, with 65% under the age of 40. Participants were asked to reflect on their most recent experiences with the software and complete several questionnaires, including the NPS, SUS, UX-Lite^®, and TAC-10™. The AI-based chat products and sample sizes were:

• ChatGPT: 113
• Claude: 103
• Gemini: 101
• Grok: 103

The sample sizes are modest but adequate to establish baselines and identify medium-sized differences relative to each other and other software products we measure (e.g., ±10% for binary metrics; ±5 for 0–100-point rating scales).

UX Metrics Results

Ease of use and usefulness affect whether people use and recommend products. In this section, we review the UX-Lite, SUS, and NPS results.

Perceived Usefulness and Ease (UX-Lite)

The UX-Lite has emerged as an industry standard for assessing the two constructs that matter most in technology adoption: ease of use and usefulness. These aren’t arbitrary choices. Research on the Technology Acceptance Model (TAM) has repeatedly shown, as far back as the mid-1980s, that ease and usefulness are key drivers of intention to use a product, which is in turn a significant driver of actual use.

The UX-Lite captures both constructs with just two items: one rating perceived ease of use (“This product is easy to use”) and one rating usefulness (“This product’s features meet my needs”). Together, they give UX researchers a compact but validated measure of acceptance—or more broadly, satisfaction and product quality.

Figure 1 shows the ease and usefulness scores for 2025 (blue) and 2026 (green). The dashed red lines indicate the overall means.

Figure 1: Scatterplot of the two UX-Lite subscales for the AI-based chat products in 2025 and 2026 (means across products indicated by red dashed lines; Grok collected in 2026 only).

The mean UX-Lite scores for 2026 ranged by just 5.5 points, from 77.8 for Grok to 83.3 for ChatGPT. There was no significant difference for the main effect of Product (F(3, 416) = 1.7, p = .16).

The most notable mover was Gemini, which dropped from its 2025 position in the upper right quadrant to the lower left in 2026. This was the largest year-over-year shift of any product. ChatGPT remained the most stable, with nearly identical scores in both years. Claude improved meaningfully in perceived usefulness but still lags in ease, keeping it left of center. Grok, measured for the first time, has room to grow as it sits slightly below the group average on both dimensions.

Perceived Usability (SUS)

While the UX-Lite is a compact way to measure ease, many organizations still use the System Usability Scale (SUS) for historical comparability. SUS is a ten-item questionnaire with possible scores ranging from 0 to 100. The average SUS score from over 500 products (including websites and business software) is 68 (a grade of C on the Sauro-Lewis curved grading scale).

Our 2026 finding was that ChatGPT led with the highest SUS score, but the range for the products was just 3.1 points (78.4 to 81.5, no statistically significant difference, F(3,416) = .97, p = .41). All products had above-average perceived usability (at least a grade of B+).

As shown in Figure 2, the SUS scores were reasonably stable from 2025 to 2026. For the products measured in both years, the main effects were not statistically significant (Product: F(2, 464) = .63, p = .63; Year: F(2, 464) = .58, p = .45), but there was some indication of interaction likely due to the 5.3-point increase for ChatGPT (F(2, 464) = 2.3, p = .10).

Figure 2: SUS with 95% confidence intervals from data collected in 2025 and 2026.

Recommendation Intention (NPS)

AI software usage has grown dramatically from word of mouth as friends and colleagues describe the latest thing they did with AI. The Net Promoter Score provides a good gauge of word-of-mouth recommendations that can portend rapid growth of products. NPS is calculated using an 11-point (0 to 10) likelihood-to-recommend (LTR) question, computed by subtracting the percentage of detractors (0–6) from promoters (9–10).

Figure 3 shows the NPS we obtained for these AI-based chat products compared to our 2025 findings. General guidelines for the interpretation of the NPS are that anything above 0 is good (more promoters than detractors), above 20 is favorable, and above 50 is excellent.

Figure 3: NPS with 95% confidence intervals from data collected in 2025 and 2026.

NPS declined for all the products measured in 2025. Although the number of ChatGPT users hasn’t changed, its percentage of market share declined in 2026 as competitors’ share has increased, likely contributing to the observed drop in likelihood to recommend. Most notably, Claude now has the relatively highest NPS, reflecting its recent dominant growth. Gemini’s user base has increased, though much of this growth is not from user choice but due to integration in the Google ecosystem, possibly diluting recommendation intensity. The confidence intervals show that an NPS of 0% for 2026 is plausible for ChatGPT and Gemini, but not for Claude or Grok.

Analysis of Verbatim Comments

To dig into the “why” behind the current numbers, we asked participants to name one thing they disliked about the product they rated. Table 1 shows the top three issues for each product (with sample participant quotes). Some clear themes emerged.

Accuracy and reliability were the most common complaints across all four products, with ChatGPT, Gemini, and Grok all drawing criticism for inaccurate or inconsistent responses (Grok in particular for hallucination loops). Claude stood apart, with users citing usage limits and prompt misinterpretation rather than accuracy, while slow performance was a recurring frustration for Gemini and Grok users.

Table 1: Top issues for AI-based chat software products.

Do Users of These Products Differ in Tech Savviness?

We don’t want differences in metrics to just be a result of differences in participant tech-savviness. It could be that less tech-savvy users gravitate toward ChatGPT, and more tech-savvy users prefer Claude or Grok. To differentiate between differences in participant ability and differences in the user experience, we collected tech-savviness scores using our TAC-10 measure for each of the three products.

The TAC-10 (Technical Activity Checklist with ten items) is a reliable (consistent) and valid (predictive) measure of tech savviness. The TAC-10 score for a person is the number of items selected from its checklist.

In 2026, the TAC-10 scores for the four AI products ranged from 6.3 to 7.0 (see Figure 4), a statistically significant main effect (F(3, 416) = 2.8, p = .04), due to the difference between ChatGPT and Claude (a pattern similar to what we saw in 2025 when the difference was 2.3 points, but less pronounced in 2026). All mean scores were in the lower part of the high range for two-group classification and medium range for three-group classification except for ChatGPT in 2025, which just missed the cut-off of 5 to be in those groups.

Figure 4: TAC-10 scores by product and year (with 95% confidence intervals).

Summary and Discussion

Results of our AI-based chat benchmarks based on 420 participants revealed:

ChatGPT was the leader in perceived usability, but just barely. ChatGPT led in SUS scores in 2026, enjoying a 5.3-point increase from 2025. The range of SUS scores was, however, just 3.1 points (not statistically significant). On the UX-Lite, ChatGPT also led in perceived ease of use and usefulness, though differences among products were not statistically significant.

For Net Promoter Scores, Claude was the leader, and ChatGPT was the laggard. In 2025, the NPS for Claude was only 4 points higher than ChatGPT, but in 2026 the difference was 21 points in favor of Claude (28% vs. 7% for ChatGPT), with Gemini (12%) and Grok (17%) in between. This reflects Claude’s recent surge in growth.

Claude showed the biggest gain in perceived usefulness. Of the products measured in both years, Claude had the largest year-over-year improvement on the UX-Lite usefulness dimension, though it still lags behind ChatGPT on perceived ease of use.

Frequently reported issues included inaccurate responses and limited capabilities. Respondents reported accuracy issues with ChatGPT, Gemini, and Grok, and criticized Claude for limited capabilities and frustrating usage limits. Other reported problems were slow response time and not responding as prompted.

ChatGPT users had slightly lower tech savviness than Claude users. The mean TAC-10 scores were all in the medium range of tech savviness. Within that range, there was a statistically significant difference (6.3 for ChatGPT vs. 7.0 for Claude), though the 0.7-point gap on a 10-point scale suggests it may not be practically significant (compared to the 2.3-point difference in 2025).

For more details on these products, see the full report.