About Blockchain

Looking at Blockchain through a distributed systems architecture lens.

"Blockchain" is one of the software industry's hottest topics. As such, it's an automatic discussion subject between distributed systems engineers from across the industry, and especially amongst those who build cloud-scale (or planet-scale) platforms. While most platform folks I've talked to about it are perfectly happy to help with frameworks that help selling their respective product or services to customers who are committed using blockchains, and some of those engineers even rode the crypto wave to make some hay, I have a hard time finding people in the distributed systems platform community who believe that blockchain as a universally applicable data structure is even remotely as significant as the hype wants to make us believe.

Blockchain in an architectural nutshell

Up front, allow a brief summary of blockchain untainted by the tribal lingo of the cryptocurrency community. A blockchain is an append-only log structure where each new log record (block) refers to the prior record as well as to its own contents with a "hash".

A "hash" is a cryptographic function that can compute a "summary" number from a large block of data such that if anything in that large block of data changes, the "summary" number will be different. With a hash, you can therefore verify that data hasn't changed by computing the "summary" number again over the data block using the same algorithm. A special quality of a cryptographic hash algorithm is that you can't guess the content of the large data block from just looking at the hash.

Any blockchain block is signed with a hash that is computed over the prior block's hash and the contents of the current block. Including the prior block's hash ties the current block to that prior block, and, in consequence, to all blocks before that, which is how the chain is formed. Since the chain is shared, everyone can compute the hashes along the chain and can trivially detect blocks that were tampered with. No data can be removed from the chain because that would invalidate the entire chain. Therefore, if a fact is recorded on a chain, the recorded fact can't ever be later denied, because the chain itself protects that fact from tampering. This is called "non-repudiation".

For a blockchain to progress, new records are appended. For many uses of the data structure, there's competition on who gets to append the next record. Bitcoin, for instance, maintains a singular ledger (one global log) that tracks Bitcoin ownership by ways of recording transactions. Transactions are recorded in the log by third parties. Those get paid for doing so.

Some infrastructures even allow constrained code execution in the context of a transaction to enforce certain policies ("smart contracts").

The process by which consensus is formed about who gets to process and record a (block of) transaction(s) is called "mining". The currently most common mining model is to solve a cryptographic puzzle that involves guessing the value of an element of the current block (the "nonce") such that the resulting hash falls into a certain numeric range.

Since the hash algorithm is cryptographic, it is expressedly designed to make prediction of that value impossible, meaning that a random guess is as good as any: it's a lottery. Whoever comes up with a fitting solution first and gets a majority of the network to proceed with their block as the latest log entry "wins" the associated reward. This is called "proof-of-work" (PoW); the miner proves to have actively participated in the lottery by doing hash computations.

The consensus about who gets to append the next log record can also be formed in many others ways, but for the Bitcoin use-case, the PoW model is architecturally a very clever choice for getting the log to converge to a single global structure without any central decision making or trust relationships.

The downside is that it's tremendously wasteful, because solving the cryptographic puzzle is very hard and requires many (think: many trillion) of random guesses and computations. An emerging alternate model is "proof-of-stake" (PoS), where consensus is formed by a deterministic algorithm that picks the next block appender from a set of candidates, and the candidates nominate themselves by putting a "stake" (funds) into a pool and get punished by having their stake forfeited if they're caught cheating. The reality of that theory is a bit tricky, because this consensus concept is itself based on having to settle transactions or it requires at least partial trust.

Many characteristics of the blockchain model explained above are obviously quite attractive, and many people who first come across concepts like a replicable append-only log, cryptographic non-repudation, and multi-writer consensus may believe that these are unique blockchain characteristics. They are not.

Distributed Systems View

Software engineers at the center of the distributed systems platform community understand the qualities of append-only logs, understand signatures and non-repudiation, and understand consensus protocols. They understand that those are a few building blocks from a large toolbox and that there are choices for each.

Append-only logs, to pick one of these building blocks, have become extremely popular over the last decade, in the form of event-ingestion infrastructure and architectural patterns like "event sourcing". The great advantage is that all records in an append-only log are .... [Click to go to my blog to keep reading]