Pylon's 𝗺𝗼𝘀𝘁 𝗵𝗮𝘁𝗲𝗱 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 is our Analytics.

That ends today. 

We're shipping our third iteration of Analytics & Reporting.

Here's what we tried, what we screwed up, and what's coming.

𝗩𝗲𝗿𝘀𝗶𝗼𝗻 𝟭, The Basics (Nov 2023)

Our first attempt at analytics was quite loved by customers.

At the time our customers were mostly small startups with simple needs. We built an out-of-the-box set of dashboards that covered the common use cases of support analytics (SLA-tracking, CSAT, TTFR, TTR, basic filtering...).

As we moved upmarket...

1. Everyone was requesting custom metrics

2. Queries were becoming inefficient and slow

We needed an upgrade.

𝗩𝗲𝗿𝘀𝗶𝗼𝗻 𝟮, Advanced Reporting (June 2024)

We knew custom reporting was going to be a blackhole of work that long-term led to fully customer-customizable dashboards. 

We had four choices: 

1. Do nothing for now

2. Do custom work per customer 

3. Build full custom reporting in-house 

4. Use an embeddable analytics vendor

At the time Pylon was under 10 people total and we had no capacity to do the frontend work so we chose Option 4 (use a vendor). This was the first time we chose to not build a core feature like this in-house as we ultimately want full control of the end-user experience.

We built out the new reporting with the chosen vendor over ~3 weeks.

On the surface the new reporting looked really good (not visually, but in terms of functionality). You could add custom charts of any type, create custom formulas, label the Y and X axis, and effectively build most of what you would want. It was really great for demos.

But in practice it was incredibly hard to use, lacked core capabilities (like the ability to filter off of dynamic custom fields), and visually looked not stylized to the rest of the product.

We started to discover some of these issues during the implementation, but it still felt like there was more upside than downside so we released it.

Feedback was not great but we hoped our vendor would fix changes quickly. Unfortunately they weren't fast enough and we lost confidence that they would be a good long-term solution.

As a stop-gap we also built out a data warehouse integration so customers could export their data back to Snowflake or BigQuery to use with their own BI tools.

Finally, a few months ago the vendor told us they were being acquired. That was the final straw. We needed to move off ASAP.

𝗩𝗲𝗿𝘀𝗶𝗼𝗻 𝟯, New Reporting (Today)

Today's release is back to being built entirely in-house. It's been rolled out in beta to all customers with an option to flip back to old analytics until we plug some custom reporting gaps.

This time we have the capacity to do it right between Wendy (prev product design at Amplitude), Matt, and Tom.

We've managed to greatly improve:

1. Desired filter options (custom field support)

2. Performance

3. Setup UX

4. Style (looks native)

Early feedback has been really positive so far and as we bring it out of beta we're thinking about how to make the best natively-offered reporting of any support platform. 

𝗩𝗲𝗿𝘀𝗶𝗼𝗻 𝟰 (What's coming soon)

To get to first-class reporting, we need to study not only our learnings, but also what the incumbents have screwed up as well.

Funny enough, Zendesk's analytics have similar complaints to our v2, and for the exact same reason as we did: they integrated an external tool. In 2015 they bought a company called BIME Analytics which they became Zendesk Explore.

The complaints they have to this day are similar to our v2:

• Steep learning curve

• Advanced, yet still not enough flexibility

• Random feature gaps

• Data accuracy and reliability concerns

• Performance issues

• Complicated UX

v4 will follow three core principals:

1. Offer a simple default setup. We want to continue being startup friendly and we'll feature gate custom reporting and data exports by tier in the product.

2. Offer maximal configuration, with AI-assisted setup. As we go upmarket, customers will want to Explore (pun intended) data in every single direction. We need to allow them to do that. For those more complicated use cases we think AI will be the Ultimate (also pun intended) way to reduce setup friction. 

3. Build it all in-house. Although using a 3rd party embedded analytics provider didn't work for us, we don't think that is the case for everyone. It's just in customer support, reporting is REALLY important. They are probably some of the highest-complexity reporting of most SaaS vendors (maybe second to marketing products). So... we have to do it right. And since this is end-user facing, we have to own every detail of it.

If you got this far, thank you for reading. 

See our new Analytics at app.usepylon.com/analytics