Beni: Refuel helps Beni improve accuracy from 46% to 87% on product catalog of 200M+ items, driving partner revenue increase

Executive Summary

In the highly competitive $245B resale market, Beni, a secondhand shopping tool, partnered with Refuel to tackle its data normalization challenge across a 200M+ item catalog. Leveraging Refuel's custom LLMs, Beni dramatically enhanced the accuracy of normalized size values from 46% to 87% with only one day of team effort, surpassing their target of 80%. This improvement led to notable operational efficiencies, including up to a 99% reduction in data quality issues for some reseller partners and a significant 245% increase in Gross Merchandise Value (GMV) for a major partner, highlighting the transformative impact of AI-driven solutions in large-scale retail data management.

‍

About Beni

Beni is a secondhand shopping tool that helps users find the best resale deals from across 40+ resale sites in a matter of seconds. Beni was started by an impressive leadership team, has raised more than $5M to date, and is going after the $245B resale market.

‍

Challenges and objectives

One of the key challenges for Beni, is that data across their partner sites is messy and not normalized. In their words:

“We often find differences in the way that products are categorized amongst all of our resale partners, especially as it relates to product category, gender, color, brand, and size. If we are unable to appropriately classify products into the right bins, it makes it almost impossible to surface these listings to you in the right situations.”

This is an exceptionally tough problem to solve since the number of product listings in their database is greater than 200 million. A key product attribute is “size”, and the Beni team had identified that size not being normalized was leading to a sub-par search experience for their users.

Their existing solution was an in-house ML model that was trained on data labeled by their team, with a combination of heuristics and manual effort. The goal was to improve accuracy to more than 80% in order to deliver the desired experience.

‍

How Refuel helped

• Beni shared their product catalog data which contained 1.5M distinct size values across kids, women’s and men’s clothing and footwear.
• Beni provided simple natural language guidelines for how the  data should be normalized into ~50 size values, and used Refuel’s platform to train custom LLMs for size normalization.
• The Beni team only spent 1 day of their team gathering data, writing instructions, using Refuel and analyzing results, compared to the many weeks it would have taken to manually go through this large dataset.

‍

Results

• The accuracy of normalized size values went from 46% to 87% using the custom LLM from Refuel. This exceeded the goals on data quality that the Beni team was trying  to hit.
• For some of the reseller partners, the Beni team saw significant reduction in data quality issues — as high as a 99% reduction in fields marked as empty for one of their major partners.
• This was accompanied by an increase in GMV for their partners. As an example, the Beni team recorded a 245% increase in GMV for one of their largest partners ($10B+ public company) over a 2-week period after introducing the normalized data from Refuel to power their search and recommendations system.

‍

“Beni has a product catalog of 200M+ items and ensuring clean, structured data is an ongoing challenge. Using Refuel’s customized LLMs, we were able to label millions of items and improve accuracy on a key attribute from 46% to 87%. What would have taken us months, only took a few days with Refuel.”

Celine Lightfoot
CTO – Beni

‍

Future Plans

The Beni team is continuing to integrate AI and LLMs into their stack to improve data quality and search experiences for their users.

If you’re interested in learning more about their secondhand shopping tool, you can find more information here: https://www.joinbeni.com/

If you’re interested in how LLMs can help your marketplace operations and improve data quality, sign up here to get a demo of Refuel: https://www.refuel.ai/get-started

‍