How FoodChéri Syncs MongoDB to BigQuery for Quicker Insights, with Less Engineering

Main Challenge

FoodChéri wanted to access its operational data to gain valuable insights into its processes, operations, and customer behavior. However, as the operational data was located in business-critical systems, accessing it presented a significant challenge from an engineering standpoint. It was therefore crucial to find a way to access the data that would minimize disruptions to production systems and reduce engineering complexity.

Key Outcomes

• 1:1 replication of data between databases—from their production database (MongoDB) to a data warehouse (BigQuery) for analysis, without disrupting the production system.
• Transferred large volumes of data (100s GBs - 1+ TBs daily)
• Reduced infrastructure bill by 25-50% via CDC-based replication by transferring only the data that changed instead of performing full replicas.
  
  

Solution

• Dataddo performs CDC-based data replication between FoodChéri's operational database and analytics database.
• Near real-time replication of production data creates a safe environment for data analysts to perform complex analytics queries and asymmetric workloads.
• Dataddo's out-of-the-box database connectors enabled rapid deployment and guarantee auto scalability of data pipelines.
• Embedded monitoring, alerting, and anomaly detection ensure that all delivered data is high-quality.

“We chose Dataddo for its user-friendliness, automatic transformations, clear pricing policy, and the quality of the human relationship that was established from our first exchanges. In the end, we understood we could bring a lot to each other.”

- Laurent Partouche, CPO, FoodChéri & Seazon

Full Case Study

French company FoodChéri specializes in delivery of healthy, ready-made employee lunches to businesses in the greater metropolitan area of Paris. Their sister brand, Seazon, offers flexible, subscription-based meal plans for busy individuals who want better nutrition without the hassle, and delivers weekly across all of France.

The brands share one menu, which features the kind of variety you might expect from the French. But they conduct operations with a highly sustainable twist—they have an entire team dedicated to reducing ecological impact, a network of partners to help distribute uneaten meals, and carefully source seasonal, local ingredients.

When FoodChéri was founded in 2015, the market for ready-made meal delivery was completely new. And like the players on any innovative market, they had to adjust quickly to the expectations of customers.

The Problem: Too Much Unstructured Data, Not Enough Insight

By 2020, FoodChéri had established a solid foothold in the meal delivery sector. But they realized that they could strengthen this foothold even further using data. Over the years, they had collected a mountain of it from both internal and customer-facing tools; to leverage it better, they would need to overhaul their data infrastructure.

They were already using MySQL and MongoDB to store unstructured data, but unstructured data is raw and unsuitable for analysis in a dashboarding app. So, they began searching for a flexible integration tool that would replicate data from their existing databases and sync it to a new warehouse capable of storing structured data. There, it could be enriched by the company’s data team, then sent directly to dashboards for analysis.

BigQuery was an obvious choice of warehouse given that FoodChéri was already using other Google Cloud solutions. But what about the data integration tool?

To find their winner, they tested two of the few tools on the market that can replicate unstructured data: Fivetran and Dataddo.

The Solution: Automatic Transformations, Comprehensive Support

Google BigQuery only accepts structured and semi-structured data, so in order to move fully unstructured data from MongoDB and MySQL to BigQuery, some sort of data transformation (or structuring) must first take place.

FoodChéri’s data team could of course do this themselves in the databases, then use either Fivetran or Dataddo to copy the data to BigQuery. But this is a lot of extra busywork. It would be much easier if the integration tool automatically transformed the data before sending it to BigQuery.

Fivetran doesn’t do this, but Dataddo does. But Fivetran had a MongoDB connector and Dataddo didn’t. 

In ten days, Dataddo built a MongoDB connector for FoodChéri, so that they could properly test both tools. Dataddo emerged as the winner.

“We chose Dataddo for its user-friendliness, automatic transformations, clear pricing policy, and the quality of the human relationship that was established from our first exchanges. In the end, we understood we could bring a lot to each other.”

- Laurent Partouche, CPO, FoodChéri & Seazon

After Dataddo sends the data to BigQuery, FoodChéri’s data team performs additional transformations and synchronizes it to their BI tool, Metabase, where all company stakeholders can view dashboards for insights.

The Outcomes: Less Engineering, More Data Analysis

Thanks to the MongoDB connector, FoodChéri’s engineers saved weeks of labor when overhauling their data infrastructure. And since Dataddo pre-cleans the data it extracts from MongoDB and MySQL, the data is always easier for them to work with once it hits BigQuery.

Even more importantly, the automatic daily syncing between the two warehouses ensures that stakeholders get the freshest data daily.

“Dataddo lets us focus on the essentials: transforming data in BigQuery and then analyzing it.” - Laurent Partouche, CPO, FoodChéri & Seazon

FoodChéri and Dataddo: Moving Forward Together

FoodChéri is much more than just a kitchen with a few trucks; it’s an ever-evolving service committed to the delivery of responsible food. As such, its data pipelines are anything but static.

As the company grows, so does its need to sync different datasets between warehouses, as well as use new connectors. Most recently, FoodChéri started using Dataddo to sync data from its helpdesk app, Gorgias, directly to BigQuery.

Wherever FoodChéri decides to move their data next, Dataddo will be moving with them.