We Had an AI Read Thousands of Support Tickets — and It Reversed a 14-Month Decline

There’s a kind of decline no alert ever catches: the company works hard, the product ships improvements constantly, everyone is busy — and the numbers fall, month after month. At this study’s client, a SaaS with two people on support (one full-time, one part-time), the slide had already lasted fourteen months. The harder they worked, the worse it seemed to get.

The frustration started inside the development team itself: they kept working on the platform’s “problem areas,” yet traction never came. The uncomfortable question: are we even working on the things that matter?

A roadmap fed on impressions

As in many companies, product direction fed on impressions. Support messages were read occasionally, and the conclusions reached the team pre-filtered, as instructions — almost never as “here’s what customers are saying, how do we solve it?”. A trend analysis of support had never been done. Nobody knew, with data, what customers actually complained about.

The apparent situation seemed clear: a few very loud customers opened a lot of tickets, therefore “the service has quality problems.” And the team pointed its effort wherever the noise was.

What we built

We attacked it in two layers:

1. We brought the data home. We imported the entire support-conversation history — thousands of conversations spanning more than two years — into our own database, then automated a daily import, so the analysis would be a film, not a snapshot.
2. We had an AI read all of it. An automation grouped each week’s conversations, cut them into chunks of at most 50, and extracted the main topics through the OpenAI API, tracking how themes evolved week over week. The output: a weekly email report.

As usual, the final version wasn’t the first. We first tried feeding the model the whole history at once, then a full week at a time — both hit limits. The winning shape: 50-conversation chunks, with one prompt trick that changed everything — each chunk received the running tally of themes already identified that week (“trend: count”) and only added on top of it. That let chunks consolidate naturally into a single report, and later we stabilised the taxonomy by supplying the main categories ourselves, so weeks stayed comparable.

The shock: four identical reports

The surprise didn’t come from one report. It came when we laid four weekly reports side by side: they were identical. Same themes, same proportions — while the team had been shipping improvements the entire time. The work wasn’t moving the needle on anything that actually hurt.

And what actually hurt overturned every assumption:

• Topic #1, by far: elementary questions about subscription configuration. Not exotic bugs, not sophisticated requests — people failing to set up their service.
• Topic #2: services that didn’t match the needs of part of the customer base — requests the existing offer simply didn’t cover.

Cross-referencing with customer lifetime value, on random samples, delivered the finishing blow: the loud customers shaping perception — and, indirectly, the roadmap — were largely the least valuable ones. The company was optimising its product for its complainers, while the needs of its valuable customers stayed quiet and unaddressed. (We’ve written separately about what wrong customers cost: The Wrong Customers Are More Expensive Than No Customers.)

And one more discovery, perhaps the most valuable: many of the people stuck on configuration were, in fact, a different customer avatar — home users with simple needs, on a platform designed for power users. Which also exposed a dissonance in the marketing: it attracted an audience the product didn’t speak to.

What changed

With the data on the table, the conversation moved from impressions to evidence — and the decisions cascaded:

• The panel was completely rethought and massively simplified;
• The subscription-configuration page got direct links to usage guides;
• The configurator was rebuilt from scratch, on a new format, with defaults set to the most-used options;
• New service packages were introduced to directly address the customers who previously couldn’t use the platform;
• Marketing was retargeted at the real avatar.

The result

The continuous, roughly 14-month decline reversed. Not through one spectacular move, but through something almost mundane: the company finally started working on what its customers were saying — instead of what was said loudest.

The lesson

We didn’t turn this into a template process. What changed permanently is the weighting: customer frustrations now get real priority — solved fast and for good, not through endless firefighting. And for any company steering “by feel”: impressions are data too, just unfair data — they amplify the loudest voice, not the biggest need. An AI that reads everything, week after week, costs a fraction of the first wrong decision made on impressions.