Build versus Buy - SPAM Filtering Solution

Introduction

Maintaining the credibility of a user-generated content platform is paramount, especially when it comes to reviews. At GNP, users frequently encountered fake reviews, damaging the platform’s credibility and user trust. Despite having a bespoke content moderation pipeline, the spam reviews persisted, frustrating both users and businesses. The situation required a swift and effective solution to restore trust and integrity.

Situation

GNP’s platform was plagued with fake reviews, despite the existence of a content moderation pipeline. This not only frustrated users and businesses but also overburdened the sole moderator, who manually filtered flagged reviews. Additionally, GNP was in discussions with potential buyers for their ratings and review data, who demanded high-quality, authentic data. A robust spam filtering solution was needed to ensure the platform’s reviews remained trustworthy.

Task

As the Product and Engineering Manager, I was tasked with implementing an effective spam filtering solution. The challenge was to decide whether to build a custom solution in-house or purchase an existing one. The solution had to be quick to implement, capable of handling 15 years of historical data, and provide near-real-time filtering.

Action

Assessment and Decision Making:

1. Conducting a Thorough Assessment:

   • Evaluated the requirements for the spam filtering solution, considering accuracy, scalability, integration complexity, and maintenance.

   • Engaged with engineers to discuss the feasibility of updating the existing solution and understanding why it was initially selected.

2. Researching Third-Party Solutions:

   • Researched several third-party spam filtering solutions, evaluating their effectiveness, ease of integration, cost, and customer reviews.

The Decision to Buy:

1. Advising Leadership:

   • Based on the assessment, I recommended purchasing a proven third-party spam filtering technology. This decision was influenced by team capacity and the need for a quick, reliable solution that wouldn’t divert significant engineering resources from other critical projects.

2. Choosing the Right Vendor:

   • Selected a vendor offering a highly accurate and scalable spam detection system with machine learning capabilities to adapt to new spam patterns over time. The vendor also provided an NPO discount.

Implementation:

1. Scoping Out Engineering Work:

   • Set up APIs, configured the system to our specific needs, and ensured seamless integration with our existing review submission and moderation workflows.

Rollout and Monitoring:

1. Running on Historic Data:

   • Ran the spam filtering system on historical reviews, monitored its effectiveness, and analyzed its accuracy.

2. Launching on New Reviews:

   • Rolled out the spam filtering system for new reviews, ensuring continuous monitoring and adjustments as needed.

Result

Improved Accuracy: The system effectively detected and filtered out spam reviews, significantly reducing the volume of fake reviews on the platform.

Enhanced User Trust: Users reported increased trust in the platform’s reviews, which helped restore the platform’s credibility.

Operational Efficiency: The workload on the moderation team was reduced, allowing them to focus on other important tasks and improving overall operational efficiency.

Quick Implementation: The decision to buy rather than build allowed us to implement a robust solution quickly, addressing the spam problem in a timely manner.

Scalability and Adaptability: The machine learning capabilities of the purchased solution ensured that the spam filter adapted to new spam patterns, maintaining its effectiveness over time.

Key Takeaways

1. Thorough Assessment: Conducting a detailed assessment of internal capabilities and external solutions is crucial in making informed decisions.

2. Strategic Decision Making: Choosing to buy rather than build can be more efficient, especially when time and resources are limited.

3. Stakeholder Engagement: Engaging with stakeholders and understanding their perspectives helps in selecting the most appropriate solution.

4. Continuous Monitoring: Implementing a system with machine learning capabilities ensures that the solution remains effective over time.

5. Resource Optimization: Allocating resources effectively allows the team to focus on high-impact activities, improving overall operational efficiency.

By sharing this case study, I hope to provide insights and strategies that other product managers can apply in their own decision-making processes. Remember, thorough evaluation and strategic alignment are key to successful product management.