Streamlining reviews moderation
Stamped
2024
Web
Summary
Role
Design Lead
Duration
1 month
Team
Natalie (Director of Product), Javi (Data Analyst), Anna (BE Developer), Andy (BE Developer), Vanessa (FE Developer), Alfonso (FE Developer)
I led the redesign of Stamped’s reviews moderation tool, making it easier for merchants to manage hundreds to thousands of reviews. By introducing smarter filters and reducing interface clutter, we transformed a frustrating workflow into a streamlined, flexible experience. In the first month of the general release, we saw an 82% adoption rate.
Problem
Merchants were growing impatient with the reviews moderation tool as it became impossibly difficult to navigate.
For most of its history, Stamped was a reviews platform. It streamlined the process of not only collecting reviews, but also moderating them. The moderation process was lacking in functionality as the only way to narrow down reviews was by status via the side navigation bar. Merchants had to scroll and paginate in order to search for specific reviews.
As some merchants collected thousands of reviews, this became highly problematic and was compromising the core value proposition of the company. This was reflected in NPS surveys which scored a 4.8, with a poor moderation experience being cited as one of the main reasons for the score in 41% of the responses.
Research
After watching session replays and conducting twelve user interviews, we uncovered points of frustration and revealed common use cases to account for.
Datadog allowed me to watch session replays of merchants navigating the Reviews page. More importantly, it highlighted where merchants were showing the most frustration by identifying where rage clicks were occurring.
User interviews with twelve merchants helped build additional context. More importantly, I was able to discuss what problems would be solved by being able to filter, and which ones in particular they would need. This uncovered a variety of use cases such as finding reviews to repurpose as testimonials, finding negative reviews to respond to, or finding reviews with photo or video to collect user-generated content.
Discovery
Early conversations revealed complexity. This meant being strategic with our approach.
To kick off the discovery phase, I spent time generating different HMWs and corresponding hypotheses we could test with the goal of figuring out how we might improve the moderation experience. The goal was to go wide and then work with the team to narrow the focus. Here were a few examples:
HMW help users quickly find reviews that serve their specific goals?
If we allow filtering by sentiment (positive/negative), then merchants will be able to locate relevant reviews faster
If we introduce keyword search within reviews, then merchants will spend less time scrolling through irrelevant content
If we allow custom tagging of reviews, then merchants will create personalized systems for managing content at scale
HMW make it easier for users to repurpose positive reviews as testimonials?
If we allow filtering by rating (e.g., 4–5 stars), then merchants will more easily surface testimonial-worthy content
If we introduce a “highlight/save” function, then merchants will be more likely to collect and reuse reviews in their marketing
HMW enable merchants to identify negative reviews that need a response?
If we enable filtering by 1–2 star ratings, then merchants will locate dissatisfied customers faster
If we create a “needs attention” filter, then merchants will be more consistent in addressing unhappy customers
In the kickoff call, developers raised concerns about technical constraints I hadn’t been aware of. The outdated, messy codebase meant many of my ideas weren’t feasible within the timeline. To move forward, I partnered with the product director to reduce scope and plan incremental releases. By focusing on the left panel and staying close to the existing layout, we could deliver value quickly and demonstrate progress to merchants.
Solution & Testing
Exposed filters outperformed the multi-filter dropdown, enabling merchants to complete filtering tasks nearly 2x faster
Given both time and technical constraints, we kept efforts focused on the left panel. Leveraging pre-existing filter components, it would allow us to launch quicker. We ultimately narrowed down to two concepts.
We wrestled with the idea of having filters nested within a single dropdown or having them all exposed. My concern was that having the filters exposed would create too much cognitive load, leading to a poor moderation experience. However, testing revealed that merchants appreciated the larger working area and the immediate visibility it provided.
Refine
Having filters exposed performed better but came with a tradeoff. The vertical space it used up made it difficult to browse through reviews.
Going back to the drawing board, I continued to iterate and brainstorm how we could maintain the filters without compromising overall functionality of the page. I added a simple expand/collapse function so that merchants can tuck the filters away when not in use. If merchants wanted to hide the applied filter tags in view, they could easily do so with the checkbox.
Outcome
Achieved an 82% filter adoption rate among active merchants in the first month after general release.
When the final designs were presented to the twelve merchants, they were all ecstatic. Engineers built out the new experience over a couple of weeks and then enabled the beta version for 20% of merchants under a feature flag. After all the bugs were fixed, the moderation experience was released to all merchants where we continue to collect feedback.
One month post-general release, we witnessed an 82% filter adoption rate among active merchants, meaning those merchants have applied at least one new filter.
Distribution of filters applied in the first month
Reflection
There’s always going to be tradeoffs. Was the solution exactly what I envisioned? Not yet, but it had an immediate, and meaningful impact on the business.
After the moderation experience launched, here are some of the things I've been thinking about:
Micro-interactions for first-time visitors → After continuing to watch session replays, some merchants don't notice that the filter area can be collapsed. A simple affordance has the potential to improve the user experience dramatically.
Moving fast, but calculated → Giving a feature a full refresh can be exciting but sometimes it’s not feasible given the time. I had to find a balance between innovating while keeping the experience similar in some aspects so that we could deploy quicker. This tradeoff allowed us to avoid extending the timeline, which would have increased the risk of merchants churning.
Intuition is important but not 100% reliable → I initially assumed the dropdown version would perform better because it appeared to reduce cognitive load by keeping the interface clean and minimal. However, usability testing proved otherwise. Users completed tasks almost twice as fast with the exposed filters. This highlighted an important lesson: what feels simpler to a designer doesn’t always translate to a better user experience. Testing not only validated the design direction but also revealed user behaviours and needs that weren’t immediately obvious, ensuring the final solution was grounded in real evidence rather than assumptions.
Case Study Title
How an ultimatum sparked innovation
Stamped
May 1993
Mobile & Web
Streamlining reviews moderation
Stamped
2024
Web
Summary
Role
Design Lead
Duration
1 month
Team
Natalie (Director of Product), Javi (Data Analyst), Anna (BE Developer), Andy (BE Developer), Vanessa (FE Developer), Alfonso (FE Developer)
I led the redesign of Stamped’s reviews moderation tool, making it easier for merchants to manage hundreds to thousands of reviews. By introducing smarter filters and reducing interface clutter, we transformed a frustrating workflow into a streamlined, flexible experience. In the first month of the general release, we saw an 82% adoption rate.
Problem
Merchants were growing impatient with the reviews moderation tool as it became impossibly difficult to navigate.
For most of its history, Stamped was a reviews platform. It streamlined the process of not only collecting reviews, but also moderating them. The moderation process was lacking in functionality as the only way to narrow down reviews was by status via the side navigation bar. Merchants had to scroll and paginate in order to search for specific reviews.
As some merchants collected thousands of reviews, this became highly problematic and was compromising the core value proposition of the company. This was reflected in NPS surveys which scored a 4.8, with a poor moderation experience being cited as one of the main reasons for the score in 41% of the responses.
Research
After watching session replays and conducting twelve user interviews, we uncovered points of frustration and revealed common use cases to account for.
Datadog allowed me to watch session replays of merchants navigating the Reviews page. More importantly, it highlighted where merchants were showing the most frustration by identifying where rage clicks were occurring.
User interviews with twelve merchants helped build additional context. More importantly, I was able to discuss what problems would be solved by being able to filter, and which ones in particular they would need. This uncovered a variety of use cases such as finding reviews to repurpose as testimonials, finding negative reviews to respond to, or finding reviews with photo or video to collect user-generated content.
Discovery
Early conversations revealed complexity. This meant being strategic with our approach.
To kick off the discovery phase, I spent time generating different HMWs and corresponding hypotheses we could test with the goal of figuring out how we might improve the moderation experience. The goal was to go wide and then work with the team to narrow the focus. Here were a few examples:
HMW help users quickly find reviews that serve their specific goals?
If we allow filtering by sentiment (positive/negative), then merchants will be able to locate relevant reviews faster
If we introduce keyword search within reviews, then merchants will spend less time scrolling through irrelevant content
If we allow custom tagging of reviews, then merchants will create personalized systems for managing content at scale
HMW make it easier for users to repurpose positive reviews as testimonials?
If we allow filtering by rating (e.g., 4–5 stars), then merchants will more easily surface testimonial-worthy content
If we introduce a “highlight/save” function, then merchants will be more likely to collect and reuse reviews in their marketing
HMW enable merchants to identify negative reviews that need a response?
If we enable filtering by 1–2 star ratings, then merchants will locate dissatisfied customers faster
If we create a “needs attention” filter, then merchants will be more consistent in addressing unhappy customers
In the kickoff call, developers raised concerns about technical constraints I hadn’t been aware of. The outdated, messy codebase meant many of my ideas weren’t feasible within the timeline. To move forward, I partnered with the product director to reduce scope and plan incremental releases. By focusing on the left panel and staying close to the existing layout, we could deliver value quickly and demonstrate progress to merchants.
Solution & Testing
Exposed filters outperformed the multi-filter dropdown, enabling merchants to complete filtering tasks nearly 2x faster
Given both time and technical constraints, we kept efforts focused on the left panel. Leveraging pre-existing filter components, it would allow us to launch quicker. We ultimately narrowed down to two concepts.
We wrestled with the idea of having filters nested within a single dropdown or having them all exposed. My concern was that having the filters exposed would create too much cognitive load, leading to a poor moderation experience. However, testing revealed that merchants appreciated the larger working area and the immediate visibility it provided.
Refine
Having filters exposed performed better but came with a tradeoff. The vertical space it used up made it difficult to browse through reviews.
Going back to the drawing board, I continued to iterate and brainstorm how we could maintain the filters without compromising overall functionality of the page. I added a simple expand/collapse function so that merchants can tuck the filters away when not in use. If merchants wanted to hide the applied filter tags in view, they could easily do so with the checkbox.
Outcome
Achieved an 82% filter adoption rate among active merchants in the first month after general release.
When the final designs were presented to the twelve merchants, they were all ecstatic. Engineers built out the new experience over a couple of weeks and then enabled the beta version for 20% of merchants under a feature flag. After all the bugs were fixed, the moderation experience was released to all merchants where we continue to collect feedback.
One month post-general release, we witnessed an 82% filter adoption rate among active merchants, meaning those merchants have applied at least one new filter.
Distribution of filters applied in the first month
Reflection
There’s always going to be tradeoffs. Was the solution exactly what I envisioned? Not yet, but it had an immediate, and meaningful impact on the business.
After the moderation experience launched, here are some of the things I've been thinking about:
Micro-interactions for first-time visitors → After continuing to watch session replays, some merchants don't notice that the filter area can be collapsed. A simple affordance has the potential to improve the user experience dramatically.
Moving fast, but calculated → Giving a feature a full refresh can be exciting but sometimes it’s not feasible given the time. I had to find a balance between innovating while keeping the experience similar in some aspects so that we could deploy quicker. This tradeoff allowed us to avoid extending the timeline, which would have increased the risk of merchants churning.
Intuition is important but not 100% reliable → I initially assumed the dropdown version would perform better because it appeared to reduce cognitive load by keeping the interface clean and minimal. However, usability testing proved otherwise. Users completed tasks almost twice as fast with the exposed filters. This highlighted an important lesson: what feels simpler to a designer doesn’t always translate to a better user experience. Testing not only validated the design direction but also revealed user behaviours and needs that weren’t immediately obvious, ensuring the final solution was grounded in real evidence rather than assumptions.
Summary
Role
Design Lead
Duration
1 month
Team
Natalie (Director of Product), Javi (Data Analyst), Anna (BE Developer), Andy (BE Developer), Vanessa (FE Developer), Alfonso (FE Developer)
I led the redesign of Stamped’s reviews moderation tool, making it easier for merchants to manage hundreds to thousands of reviews. By introducing smarter filters and reducing interface clutter, we transformed a frustrating workflow into a streamlined, flexible experience. In the first month of the general release, we saw an 82% adoption rate.
Problem
Merchants were growing impatient with the reviews moderation tool as it became impossibly difficult to navigate.
For most of its history, Stamped was a reviews platform. It streamlined the process of not only collecting reviews, but also moderating them. The moderation process was lacking in functionality as the only way to narrow down reviews was by status via the side navigation bar. Merchants had to scroll and paginate in order to search for specific reviews.
As some merchants collected thousands of reviews, this became highly problematic and was compromising the core value proposition of the company. This was reflected in NPS surveys which scored a 4.8, with a poor moderation experience being cited as one of the main reasons for the score in 41% of the responses.
Research
After watching session replays and conducting twelve user interviews, we uncovered points of frustration and revealed common use cases to account for.
Datadog allowed me to watch session replays of merchants navigating the Reviews page. More importantly, it highlighted where merchants were showing the most frustration by identifying where rage clicks were occurring.
User interviews with twelve merchants helped build additional context. More importantly, I was able to discuss what problems would be solved by being able to filter, and which ones in particular they would need. This uncovered a variety of use cases such as finding reviews to repurpose as testimonials, finding negative reviews to respond to, or finding reviews with photo or video to collect user-generated content.
Discovery
Early conversations revealed complexity. This meant being strategic with our approach.
To kick off the discovery phase, I spent time generating different HMWs and corresponding hypotheses we could test with the goal of figuring out how we might improve the moderation experience. The goal was to go wide and then work with the team to narrow the focus. Here were a few examples:
HMW help users quickly find reviews that serve their specific goals?
If we allow filtering by sentiment (positive/negative), then merchants will be able to locate relevant reviews faster
If we introduce keyword search within reviews, then merchants will spend less time scrolling through irrelevant content
If we allow custom tagging of reviews, then merchants will create personalized systems for managing content at scale
HMW make it easier for users to repurpose positive reviews as testimonials?
If we allow filtering by rating (e.g., 4–5 stars), then merchants will more easily surface testimonial-worthy content
If we introduce a “highlight/save” function, then merchants will be more likely to collect and reuse reviews in their marketing
HMW enable merchants to identify negative reviews that need a response?
If we enable filtering by 1–2 star ratings, then merchants will locate dissatisfied customers faster
If we create a “needs attention” filter, then merchants will be more consistent in addressing unhappy customers
In the kickoff call, developers raised concerns about technical constraints I hadn’t been aware of. The outdated, messy codebase meant many of my ideas weren’t feasible within the timeline. To move forward, I partnered with the product director to reduce scope and plan incremental releases. By focusing on the left panel and staying close to the existing layout, we could deliver value quickly and demonstrate progress to merchants.
Solution & Testing
Exposed filters outperformed the multi-filter dropdown, enabling merchants to complete filtering tasks nearly 2x faster
Given both time and technical constraints, we kept efforts focused on the left panel. Leveraging pre-existing filter components, it would allow us to launch quicker. We ultimately narrowed down to two concepts.
We wrestled with the idea of having filters nested within a single dropdown or having them all exposed. My concern was that having the filters exposed would create too much cognitive load, leading to a poor moderation experience. However, testing revealed that merchants appreciated the larger working area and the immediate visibility it provided.
Refine
Having filters exposed performed better but came with a tradeoff. The vertical space it used up made it difficult to browse through reviews.
Going back to the drawing board, I continued to iterate and brainstorm how we could maintain the filters without compromising overall functionality of the page. I added a simple expand/collapse function so that merchants can tuck the filters away when not in use. If merchants wanted to hide the applied filter tags in view, they could easily do so with the checkbox.
Outcome
Achieved an 82% filter adoption rate among active merchants in the first month after general release.
When the final designs were presented to the twelve merchants, they were all ecstatic. Engineers built out the new experience over a couple of weeks and then enabled the beta version for 20% of merchants under a feature flag. After all the bugs were fixed, the moderation experience was released to all merchants where we continue to collect feedback.
One month post-general release, we witnessed an 82% filter adoption rate among active merchants, meaning those merchants have applied at least one new filter.
Distribution of filters applied in the first month
Reflection
There’s always going to be tradeoffs. Was the solution exactly what I envisioned? Not yet, but it had an immediate, and meaningful impact on the business.
After the moderation experience launched, here are some of the things I've been thinking about:
Micro-interactions for first-time visitors → After continuing to watch session replays, some merchants don't notice that the filter area can be collapsed. A simple affordance has the potential to improve the user experience dramatically.
Moving fast, but calculated → Giving a feature a full refresh can be exciting but sometimes it’s not feasible given the time. I had to find a balance between innovating while keeping the experience similar in some aspects so that we could deploy quicker. This tradeoff allowed us to avoid extending the timeline, which would have increased the risk of merchants churning.
Intuition is important but not 100% reliable → I initially assumed the dropdown version would perform better because it appeared to reduce cognitive load by keeping the interface clean and minimal. However, usability testing proved otherwise. Users completed tasks almost twice as fast with the exposed filters. This highlighted an important lesson: what feels simpler to a designer doesn’t always translate to a better user experience. Testing not only validated the design direction but also revealed user behaviours and needs that weren’t immediately obvious, ensuring the final solution was grounded in real evidence rather than assumptions.