6/17/2023 0 Comments Database sharding![]() We can attach a shop_id to all shop-owned tables and have it serve as a sharding key. Shopify’s data model lends itself nicely to this topology as a shop is the identifying entity for most data models. The podded application runtime is supported by a sharded database topology: each pod consists of its own shard. Pod 42 contains all the data required to serve traffic for Cool Cucumbers. Here, a sample request to is processed by a load balancer and forwarded to Pod 42. Shopify’s application runtime is split into multiple pods. Web requests for shops are processed by a load balancer that consults a routing table and forwards the request to the correct pod based on the shop. Every pod houses a unique subset of shops on the platform. A pod is an isolated instance of Shopify consisting of an individual MySQL database shard, along with other datastores like Redis and Memcached. Shopify’s application runtime is currently podded: the infrastructure is composed of many pods (not to be confused with Kubernetes Pods). To fully understand shard balancing, it helps to briefly review Shopify’s architecture. A Brief Overview of Shopify’s Current Architecture This post explains how we’re able to balance our MySQL shards by migrating shops across shards-entirely online and with virtually zero consumer-facing downtime. It’s important to ensure the shards remain well-balanced to mitigate risk of database failure, improve productivity of the wider infrastructure, and ultimately guarantee buyers can always access their favorite shops. Certain database shards become unbalanced in their database utilization, shop traffic, and load. As traffic patterns change and new merchants onboard onto the platform, it’s possible that resource intensive shops end up living in the same shards. A key component of the current infrastructure is the underlying fleet of MySQL database shards that together persist every shop’s critical data. Shopify’s infrastructure supports millions of merchants during their entrepreneurship journey.
0 Comments
Leave a Reply. |