Onda Digital

Scaling a High-Volume Data Platform – Onda Digital
Case Study

Scaling a High-Volume Data Platform

How a global meteorological data exchange platform moved from a tightly coupled architecture to an event-driven microservice model to keep pace with growth.

01

Problem

A large-scale data platform handling geospatial and operational data distribution faced mounting challenges. The existing architecture struggled to maintain performance, observability, and delivery efficiency while scaling up.

02

Approach

The team conducted a comprehensive review of platform architecture, data flows, and service boundaries to pinpoint bottlenecks and improvement opportunities. They prioritized scalability, operational complexity management, and future growth planning for a global meteorological data exchange platform.

Before — Monolith
flowchart TB
    subgraph M["Monolith"]
        direction LR
        Ingest --> Process --> Store --> Retrieve --> Serve --> Switch
    end
    M --> DS1
    subgraph DS1["Data Stores"]
        direction TB
        RDB[("Relational DB")]
        NFS[("NFS")]
    end
03

Solution

The platform transitioned from a tightly coupled architecture to an event-driven microservice model. New data pipelines and processing workflows enhanced system resilience, increased data throughput capacity, and improved observability across the system.

After — Event-Driven Microservices
flowchart TB
    subgraph M2["Monolith"]
        direction LR
        Ingest2["Ingest"] --> Store2["Store"] --> Switch2["Switch"]
    end
    M2 --> MQ[["Message Queue"]]
    MQ --> Proc
    subgraph Proc["Processing"]
        direction TB
        LB{{"Load Balancer (API)"}}
        JD{{"Job Distributor (SQS)"}}
        LB --> Catalog["Catalog Service"]
        LB --> Indexing["Indexing Service"]
        LB --> Serving["Serving Service"]
        JD --> Catalog
        JD --> Indexing
        JD --> Serving
    end
    subgraph DS["Data Stores"]
        direction TB
        Cache[("Cache")]
        ObjStore[("Object Store")]
        RDB2[("Relational DB")]
        NFS2[("NFS")]
    end
    Catalog --> DS
    Indexing --> DS
    Serving --> DS
    Proc -.-> OBS
    subgraph OBS["O11Y"]
        direction TB
        Observability["Observability"]
        Metrics["Metrics"]
    end
04

Outcome

The redesigned architecture delivered improved scalability and operational visibility. Enabled the company to execute effectively on a 2 year long backlog of features within 6 months with a smaller development team.
Event-driven architecture Microservices Observability Geospatial data Scalability

Get Started

Ready to streamline your operations? Let’s build smarter workflows together.

Go