-
Continue reading →: Understanding the “Happens-Before” Relationship in Distributed SystemsIn this post, we will delve into an algorithm that can tell whether two operations are concurrent or whether one happened before another. Scenario Let’s begin with a database with only one replica. (so we can simplify this and thereafter generalise the approach to a leaderless database with multiple replicas)…
-
Continue reading →: How to Define a Concurrent Operation?Let’s begin with understanding, on how do we decide whether two operations are concurrent or not? Consider this scenario: The two writes are not concurrent… On the other hand: How to define concurrency? Whether one operation happens before another operation is the key to defining what concurrency means… In fact…
-
Continue reading →: What to consider with Replication and Multi-Data Centre Operations?Leaderless replication is suitable for multi-data centre operations, due its ability to tolerate conflicts, such as: For instance both Cassandra and Voldemort implement there multi data centre support within the normal leaderless model. Detecting concurrent writes Dynamo styled databases allow for several clients to concurrently write to the same key.…
-
Continue reading →: What are Sloppy Quorums and Hinted Handoffs?Why optimised quorums useful? Databases with appropriately configured quorums can tolerate the failure of individual nodes, this can allow systems to: These characteristics make leaderless replication appealing for use cases: “Achilles heel” of Quorums! Quorums can provide some reliable outcomes for end users, they are not fault tolerant… Network interruptions…
-
Continue reading →: How to Monitor StalenessWhat is Staleness? Describing staleness in an operational and monitoring perspective, this means understanding whether your databases are returning up to date results. The how to Manage Staleness? Even if your application can tolerate stale reads, you need to: Leader based replication The database typically exposes metrics for the replication…
-
Continue reading →: What are the Limitations of Quorum Consistency?Quorum Variables (before diving into this subject) (W + R) > N If you have N replicas and you choose W and R such as (W+R) > N you will generally expect every read to return the most recent value written. Pros Cons (W + R) < N But Quorums…
-
Continue reading →: How do Reading and Writing Quorums work?Illustrate the Quorum Variables Summary of Quorum Reads and writes that obey this R 📖 and W ✍️ values are called Quorum reads and writes. In a Dynamo style databases: The Quorum Condition (W+R) > N The (W+R) > N allows us to tolerate theses nodes with these types of…
-
Continue reading →: Leaderless Replication – Writing to a database when a node is downIf you need a refresher on leaderless replication please read my previous blog post on: What is Leaderless replication? Or read Martin Kleppman book on ‘Designing Data-Intensive Applications’. Scenario of a node going down Conceptualise a situation where you have a database with 3 replicas and 1 of the replicas…
-
Continue reading →: What is Leaderless Replication?Quickie on multi leader and single leader replication Typically a single or multi leader replication approach is adopted, this is based on the concept: Although.. some data storage systems can take another approach… The leaderless replication set up As the name entails this means abandoning the concept of a leader.…
-
Continue reading →: The Multi-Leader Replication TopologiesWhat is a replication topology? This outlines the communication paths, where writes are propagated to one node or the other. You only have two leaders? There is only one suitable topology… With more that two leaders various topologies are possible The most general topology is all to all, every leader…
-
Continue reading →: What is a Conflict in Replication?The simple question on what is a conflict? Pronounced types of conflicts There are some types conflicts that are obvious. For instance, the wiki example: Understated types of conflicts Whilst there are clear conflict we can understand, there are types of conflicts that we may not consider. For example, a…
-
Continue reading →: How to Resolve Conflicts with Automatic Conflict ResolutionConflict resolution rules can become complicated and custom code can be prone to errors. Effects of conflict resolution handler 🛒 Example: Amazon has had some interesting effects when utilising a conflict resolution handler. It has been documented that for some time the conflict resolution logic on the shopping cart preserves…
-
Continue reading →: How to Resolve Conflicts with Custom Conflict Resolution?In some scenarios, the most suitable way to resolve a conflict resolution may be dependent on the type of application being built. 👉 Most multi-leader replication tools allow you write custom conflict resolution logic using application code. ✋ To note, that conflict resolution is only applied on the individual row…
-
Continue reading →: How to Resolve Conflicts with Converging Conflicts?In this post of resolving conflicts, there are few strategies to achieve conflict resolution methodically. There is an importance to understand a diverse range of solutions as this can aid with applying the correct strategy for the type of optimisations that the requirements imply. The the solutions that will be…
-
Continue reading →: How to Resolve Conflicts with Avoiding Conflicts?One of the largest problems with multi-leader replication is handling conflicts. This is why conflict resolution is required. 🤔 How can a write conflict occur? For instance, lets considers a wiki page being edited simultaneously between two users. 👨💼👨💼 ❌ A scenario can occur when both users are editing the…
-
Continue reading →: How do we handle Collaborative Editing?What do we mean by collaboration editing? Real-time collaborative editing apps How do we resolve collaborative editing? Strangely enough collaborative editing has a lot in common with the replication problem in an offline editing use case. ⚙️ Collaboration model equivalent to single leader replicaton with transaction on the leader: 🤔…
-
Continue reading →: How to Handle Clients with Offline Operations?Ever wondered how some applications can perform even without an internet connection? This post will detail how some applications handle this challenge. In handling offline operations between devices, the Multi-Leader Replication architecture is suitable to achieve this. For example, consider a calendar application on your laptop, mobile phone and other…
-
Continue reading →: What is Multi-Leader Replication?In replication there are specific scenarios where replication patterns are more suitable than others, depending on an applications use case. Subsequently, multi-leader replication is an alternative from single leader replication, as previously mentioned in my initial post on what replication is. Why Multi-Leader Replication Over Single Leader Replication? Firstly, I…
-
Continue reading →: The Five Design Principles: SOLIDTo start, in object-oriented programming, there is a helpful acronym used within software development named SOLID. Whereby, this was created to define five simple design principles. The goals of the principles, are to achieve more understandable, maintainable and extendable software where designs are understandable. In addition, to give credit the…
-
Continue reading →: What is Replication?Conceptually, replication facilitates a distributed architecture. This is achieved by using two fundamental types of nodes, that handle application(s) reads and writes to the database(s). These are: The leader has the current data and the followers will update accordingly. Asynchronous vs Synchronous Replication Synchronous Replication Asynchronous Replication Setting up Failed…
Hello,
I’m Sean
Welcome to the Scalable Human blog. Just a software engineer writing about algo trading, AI, and books. I learn in public, use AI tools extensively, and share what works. Educational purposes only – not financial advice.
Let’s connect
Join the mailing list
Stay updated with latest posts.
Scalable Human Blog 