-
Continue reading →: What are Sloppy Quorums and Hinted Handoffs?Why optimised quorums useful? Databases with appropriately configured quorums can tolerate the failure of individual nodes, this can allow systems to: These characteristics make leaderless replication appealing for use cases: “Achilles heel” of Quorums! Quorums can provide some reliable outcomes for end users, they are not fault tolerant… Network interruptions…
-
Continue reading →: How to Monitor StalenessWhat is Staleness? Describing staleness in an operational and monitoring perspective, this means understanding whether your databases are returning up to date results. The how to Manage Staleness? Even if your application can tolerate stale reads, you need to: Leader based replication The database typically exposes metrics for the replication…
-
Continue reading →: What are the Limitations of Quorum Consistency?Quorum Variables (before diving into this subject) (W + R) > N If you have N replicas and you choose W and R such as (W+R) > N you will generally expect every read to return the most recent value written. Pros Cons (W + R) < N But Quorums…
-
Continue reading →: How do Reading and Writing Quorums work?Illustrate the Quorum Variables Summary of Quorum Reads and writes that obey this R 📖 and W ✍️ values are called Quorum reads and writes. In a Dynamo style databases: The Quorum Condition (W+R) > N The (W+R) > N allows us to tolerate theses nodes with these types of…
-
Continue reading →: Leaderless Replication – Writing to a database when a node is downIf you need a refresher on leaderless replication please read my previous blog post on: What is Leaderless replication? Or read Martin Kleppman book on ‘Designing Data-Intensive Applications’. Scenario of a node going down Conceptualise a situation where you have a database with 3 replicas and 1 of the replicas…
-
Continue reading →: What is Leaderless Replication?Quickie on multi leader and single leader replication Typically a single or multi leader replication approach is adopted, this is based on the concept: Although.. some data storage systems can take another approach… The leaderless replication set up As the name entails this means abandoning the concept of a leader.…
-
Continue reading →: The Multi-Leader Replication TopologiesWhat is a replication topology? This outlines the communication paths, where writes are propagated to one node or the other. You only have two leaders? There is only one suitable topology… With more that two leaders various topologies are possible The most general topology is all to all, every leader…
-
Continue reading →: What is a Conflict in Replication?The simple question on what is a conflict? Pronounced types of conflicts There are some types conflicts that are obvious. For instance, the wiki example: Understated types of conflicts Whilst there are clear conflict we can understand, there are types of conflicts that we may not consider. For example, a…
-
Continue reading →: How to Resolve Conflicts with Automatic Conflict ResolutionConflict resolution rules can become complicated and custom code can be prone to errors. Effects of conflict resolution handler 🛒 Example: Amazon has had some interesting effects when utilising a conflict resolution handler. It has been documented that for some time the conflict resolution logic on the shopping cart preserves…
-
Continue reading →: How to Resolve Conflicts with Custom Conflict Resolution?In some scenarios, the most suitable way to resolve a conflict resolution may be dependent on the type of application being built. 👉 Most multi-leader replication tools allow you write custom conflict resolution logic using application code. ✋ To note, that conflict resolution is only applied on the individual row…
-
Continue reading →: How to Resolve Conflicts with Converging Conflicts?In this post of resolving conflicts, there are few strategies to achieve conflict resolution methodically. There is an importance to understand a diverse range of solutions as this can aid with applying the correct strategy for the type of optimisations that the requirements imply. The the solutions that will be…
-
Continue reading →: How to Resolve Conflicts with Avoiding Conflicts?One of the largest problems with multi-leader replication is handling conflicts. This is why conflict resolution is required. 🤔 How can a write conflict occur? For instance, lets considers a wiki page being edited simultaneously between two users. 👨💼👨💼 ❌ A scenario can occur when both users are editing the…
-
Continue reading →: How do we handle Collaborative Editing?What do we mean by collaboration editing? Real-time collaborative editing apps How do we resolve collaborative editing? Strangely enough collaborative editing has a lot in common with the replication problem in an offline editing use case. ⚙️ Collaboration model equivalent to single leader replicaton with transaction on the leader: 🤔…
-
Continue reading →: How to Handle Clients with Offline Operations?Ever wondered how some applications can perform even without an internet connection? This post will detail how some applications handle this challenge. In handling offline operations between devices, the Multi-Leader Replication architecture is suitable to achieve this. For example, consider a calendar application on your laptop, mobile phone and other…
-
Continue reading →: What is Multi-Leader Replication?In replication there are specific scenarios where replication patterns are more suitable than others, depending on an applications use case. Subsequently, multi-leader replication is an alternative from single leader replication, as previously mentioned in my initial post on what replication is. Why Multi-Leader Replication Over Single Leader Replication? Firstly, I…
-
Continue reading →: The Five Design Principles: SOLIDTo start, in object-oriented programming, there is a helpful acronym used within software development named SOLID. Whereby, this was created to define five simple design principles. The goals of the principles, are to achieve more understandable, maintainable and extendable software where designs are understandable. In addition, to give credit the…
-
Continue reading →: What is Replication?Conceptually, replication facilitates a distributed architecture. This is achieved by using two fundamental types of nodes, that handle application(s) reads and writes to the database(s). These are: The leader has the current data and the followers will update accordingly. Asynchronous vs Synchronous Replication Synchronous Replication Asynchronous Replication Setting up Failed…
-
Continue reading →: Sharding: What it is and why many database architectures rely on itIn this post I will breakdown how sharding works with it advantages and trade offs. In addition, I also aim to assist you with why you may what to consider this powerful database partitioning technique. As of this writing, sharding has been notably broadcasted to the mainstream media, that’s if…
-
Continue reading →: Horizontal Scaling vs Vertical Scaling?Vertical or horizontal? That is the question… this post will outline the differences between these two types of scalability strategies, when considering the architecture of your application. These nuggets of information were extracted from Martin Kleppmann book on “Designing Data Intensive Applications”, this is a highly recommend read for those…
-
Continue reading →: Schema (SQL) vs Schema-less (NoSQL) DatabasesThe dilemma of choosing schema or schema-less databases. This post aims to simplify this key design decision. Origins of Schema and Schema-less The schema based databases has stood the test of time, in fact Relational Database Management Systems (RDBMS), have been a pioneer in this area. Since the 1980s, and…
Hello,
I’m Sean
Welcome to the Scalable Human blog. Just a software engineer writing about algo trading, AI, and books. I learn in public, use AI tools extensively, and share what works. Educational purposes only – not financial advice.
Let’s connect
Join the mailing list
Stay updated with latest posts.
Scalable Human Blog 