Scaling Rye’s Dashboard to serve $10B+ enterprise non-technical teams

Overview

Rye lets developers sell any product from any store in their app

Rye powers shopping experiences for $10B+ consumer fintechs, AI shopping assistants and marketplaces. Rye orchestrates a fleet of agents that allow users on customer’s apps to order anything from any store on the internet.

In Rye’s early days, a barebones developer dashboard that just showed API keys and tracked the status of API calls was sufficient. With small teams (e.g. 10-people start ups) and low order volumes, they were able to check on orders directly through the API.

As we grew to serve scaleups and enterprise customers, the company saw three challenges in customer success and retention:

1. Leadership had no visibility into how Rye is driving impact for their business. Our dashboard was for managing API keys.
2. Non-technical users had no easy way of doing their jobs (e.g. order refunds, cancellations, and status checks). Everything was done through API calls up until that point.
3. We started patch-fixing these issues by building ad-hoc tabs. This approach quickly made the dashboard challenging to navigate and information was buried, ultimately adding burden to our CS and engineers.

I therefore led this redesign of the dashboard to make sure we can serve the unique needs of enterprise customers in a scalable and reliable way, while saving time for our CS and engineers.

The redesign

I based the redesign on three key principles to address the challenges:

1. Business insights first. Leadership and decision makers need to see Rye’s impact upfront, which directly drove retention and increased LTV of these customers.
2. Order management second. Customer support teams need to self-serve reliably so Rye can scale without adding burden to our CS and engineers.
3. Maintain developer experience. Developers still need an easy way to onboard and debug any issues.

For leadership

Communicating Rye’s value instantly

The home page is designed to help leadership answer two questions:

1. How much money am I making with Rye?
2. If so, what are the opportunities to make more?

For support

Empowering non-technical support teams

The goal was to help non-technical users:

1. Identify the status of an order - was it successful or not, and why?
2. Make manual changes to the order e.g. handling cancellations, refunds, or returns

Making API states human-friendly and useful

One of the key challenges was to figure out a way to communicate Rye’s 37 API response states simply to the user and in a way that clearly indicates what action needs to be taken to resolve it if needed.

Of those 37 states, 23 are failures. I sorted all the failures by who can fix them: support, developers, or Rye.

Support7

A customer is waiting. There’s something to say or do.

• Payment declinedpayment_failed
• Card expiredpayment_cvc_expired
• Sold outproduct_out_of_stock
• Not enough stockinsufficient_stock
• Address not acceptedform_validation_error
• Price over limitconstraint_total_price_exceeded
• Shipping over limitconstraint_shipping_cost_exceeded

Developers9

The integration sent something a store can’t use.

• Size or color missingvariant_selections_required
• Size or color invalidvariant_selections_invalid
• Request was invalidinvalid_input
• Product page not foundproduct_not_found
• Store needs an accountunsupported_store_no_guest_checkout
• + 4 more

Rye7

Ours to fix. We’re alerted before anyone asks.

• Couldn’t get a priceoffer_retrieval_failed
• Merchant didn’t confirmorder_placement_failed
• Store blocked checkoutbot_protection_blocked
• Store asked for a captchacaptcha_blocked
• Totals didn’t matchincorrect_cost_breakdown
• + 2 more

Every order that needs attention names who has to act, so no one has to guess whose problem it is.

Details to make managing orders easier

For developers

The API logs are always one keystroke away

The inspector opens over any screen with ` so developers can access relevant API logs anytime to help debug specific orders.

The dashboard is redesigned to have orders as the primary atomic unit, instead of by API calls. This way one dashboard can cater to customer support, enterprise engineers, and indie-developers' workflows.

Impact

One dashboard, powering $50M in GMV from startups and enterprises

We shipped the dashboard over a few weeks. Two months after launch:

Most of the customer questions stopped. The ones that still reached us in Slack were questions only Rye could answer.

Within a year the same dashboard served 45 customers, from two-person startups to a $10B+ credit card fintech, placing millions of dollars of orders a month at 98.9% checkout success, with their developers, customer success teams and leadership all working in it.

Reflection

What I learned

1. Be crystal clear who we’re building for. We initially redesigned the dashboard with only our customers' CS staff in mind since this was the process bottleneck that risked our customers from churning. However, we realized that the primary stakeholder we should build for was leadership because they were the ultimate decision maker on whether they continued with Rye or not. This led me to add the business metrics dashboard and placed it front and centre.
2. Pre-empt issues before customers hit the walls. Knowing how much the dashboard ended up impacting retention for customers of all sizes, I’d have prioritized it as soon as we entered enterprise. Instead we started as enterprise customers were rolling out and running into limits.
3. Listen to everyone who uses the product. I spent most of my time talking with leadership at customers that I overlooked customer success teams, the people in the dashboard every day. They drove whether a product was adopted as intended or not which could lead to churn. Now I keep a close pulse on the feedback and experience of everyone who touches what I build.