Picture the kind of hyper-simplified scenario economists love. Ten people buy and sell various kinds of widgets to/from each other using hard currency. Ringo sells Edith one of his widgets for a coin, Prakash buys two of Sameera's widgets for three coins, and so on. When everybody's done trading for the day, there's no question of where they stand financially: they can just count their coins.
Things are a bit more complicated if the market operates on credit. Edith owes Ringo a coin, Prakash owes Sameera three, but no physical currency changes hands. In this scenario, any self-interested, sufficiently motivated party would note their transactions in order to keep track of what they owe and what others owe them. But unless every party is perfectly conscientious and implausibly scrupulous, never forgetting to carry a 1 or giving into the temptation to fudge, disagreements will arise.
To ensure that everything is fair, the group needs a central ledger. That way when there is a discrepancy, a final arbiter is there to resolve it. But who controls the ledger? Does the group hold an election? An arm-wrestling contest? How often does the position rotate? Should two people share the responsibility? Perhaps two ledgers should be kept simultaneously, or maybe that causes more problems than it solves. Most importantly, how does the group keep whoever it chooses from asserting the Divine Right of Ledger-Keepers and extracting rents from the masses?
Ideally, everyone would keep the ledger. In other words, each would have equal input and oversight when it comes to the central ledger, rather than each keeping their own. That is a daunting technical challenge, but fortunately, it is one that distributed ledgers, also known as blockchains, have largely overcome.
Trustlessness and Proof of Work
Distributed ledgers have (nearly) solved a problem that has long vexed cryptographers: trust. In any of the scenarios above, people have to rely on each other's decency. Personal, family and community ties can massage the temptation to cheat the disempowered and inattentive, but those mitigating forces break down on a larger scale.
Distributed ledgers overcome the trust issue by allowing every party in a network – the most famous is bitcoin, but there are many others – to verify the accuracy of the central ledger using their own copies. Aside from the internet, without which such a network would clearly not be possible, the crucial mechanism that eliminates trust in distributed ledgers is proof of work.
The concept of proof of work predates bitcoin. It is meant to prevent capricious of malicious use of computing power, such as email spam and denial of service attacks, by requiring a non-trivial but manageable amount of work. In 2004, Hal Finney applied this idea to money. His b-money proposal remained a white paper, but the marriage of money and proof of work became the crucial underpinning of bitcoin, which an unidentified person or people calling themselves Satoshi Nakamoto introduced five years later. (See also, Risks and Rewards of Investing in Bitcoin.)
If bitcoin's ledger were something as simple as a spreadsheet, it would be easy for someone with control over it to go back as far as they wanted, tweak a transaction or two and make themselves millionaires. Alternatively, they could spend money, award it back to themselves, and spend it again – a significant problem with currencies that don't exist in the physical world.
Proof of work makes it so that adding a new transaction is tricky, changing a recent transaction requires some serious gumption, and changing a transaction from last year is all but impossible. Imagine laying a new block on top of the Great Pyramid of Giza. It's not easy, but it's nothing compared to pulling a brick from the center of the structure's bottom layer: that would require reversing all the work that had been done to build it. Then, in order to fool anyone, you'd have to rebuild the thing too. And fast.
Into the Weeds: Blocks, Hashes and Mining
At some point, metaphors fail to convey what is actually going on in distributed ledgers. At its most basic, such a ledger is made up of "blocks" arranged sequentially – a blockchain. Each block is a record of all of the transactions that took place (note that in reality, some get passed over) during a set given period of time. In the bitcoin blockchain, a new block is added around every ten minutes. (See also, Video: What is the Blockchain?)
The "distributed" aspect of the ledger corresponds to the network's "nodes," computers running the relevant program and storing a record of the entire blockchain as it stands. These do not necessarily correspond to "miners," computers that perform work in order to add new blocks to the blockchain, but in practice there is a great deal of overlap. (See also, Top 6 Books to Learn About Bitcoin.)
Mining is where things begin to get technical. Each block records the transactions that occurred in the time since the last block was mined – simple enough. Each block also records every previous transaction, however, through a string of characters known as a "hash." Hashes are generated by subjecting a set of data – in this case transaction data – to cryptographic transformations. A given set of data will only generate one hash, and changing even one iota of the original data, due to a phenomenon known as the "avalanche effect," will cause it to spit out an obviously different hash. Nor can the hash be used to obtain the original data, only to check that the set of data in front of you matches the data that generated the hash.
In addition to its own transaction data, each block incorporates the hash of the previous block. Trying to sneak a fraudulent transaction into a long-forgotten block, then, will throw off every subsequent hash, and the network's other participants will reject that version of the blockchain.
The only way to alter the blockchain is to control a majority of the network's computing power. An entity with such control could execute a so-called 51% attack, preventing transactions from being confirmed and cancelling transactions that occurred while they controlled the network; in other words, they could spend their cryptocurrency twice, which would undermine its value just as quickly as if CNN reported that someone was printing millions of perfect dollar counterfeits a day, in every denomination. (See also, India: Migration, Remittance and Bitcoin.)
Even if a 51% attack occurred, however, the hackers would probably not be able to reverse or alter past transactions. The reason is the immense amount of work that has gone into recording those transactions, which requires and equally immense amount of work to undo (as in the pyramid metaphor above). Generating a hash is trivial for a modern computer, so the bitcoin network sets an artificial hurdle, known as "difficulty."
Making mining difficult involves setting a maximum value for a valid hash: the lower the maximum, the more work required. Block #0's hash is 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f. That string of 10 zeros at the beginning indicates a very low value, but that's easy to come up with compared to the 17 zeros at the beginning of block #431233's hash. For block #0, the difficulty is 1; for #431233, it is a little shy of 226 billion. (See also, Top 3 Books to Learn About Blockchain.)
The more difficult it is to find a valid hash, the more guesses it takes. Of course, a given set of data can only produce one hash, which is extremely unlikely to be below the target. In order to have more than one go at it, miners add a "nonce," an integer at the end of the transaction data which causes it to generate a totally different hash. There is no known way to determine a correct nonce besides trial and error: block #431223's nonce is 410871698, suggesting that the successful miner had to take more than a few cracks at it.
The network calibrates difficulty so that, on average, someone finds a valid nonce and mines a new block every ten minutes. Having more miners makes the network more robust, but it also makes it more likely that someone comes across the "golden ticket," so the network lowers the target and increases the difficulty. Mining is a competitive process, but it is not so much a race as a lottery. Anyone can hit on the lucky number. Miners pool together to increase their collective chance of success, as mining a new block comes with transaction fees as well as a reward of newly created bitcoins (the reward has decreased from 50 BTC to 12.5 at the time of writing, and will eventually peter out).
Mining consolidation has prodded fears of a 51% attack, but the only pool to gain a majority of the bitcoin network's computing power – ghash.io in July 2014 – voluntarily reduced its size to deflect suspicion. Ironically, the bitcoin community still occasionally depends on trust. (See also, Can Bitcoin Hard-Fork?)
Summing Up Distributed Ledgers
It's not the simplest solution for the ten friends in our though experiment to adopt, but it works pretty well. Each installs the bitcoin client (or that of another cryptocurrency), sets up a quick-and-dirty mining rig, and goes to work maintaining a single, public ledger that each of them has equal control over. After a few minutes of widget-trading, each of their computers begins running through nonces in order to find a valid hash. When one of these – it's anyone's guess whose – succeeds, it broadcasts the new hash to the network, and everyone updates their blockchain. Making sure that no one's tampered with prior transactions is as simple as making sure that two numbers match, a process that is of course automated.
If five of the friends were to gang up, they could potentially bring the network to a halt and cast crippling doubt on any new transaction, but they couldn't go back and revise the transactions' history. To do that, someone would need to control basically the entire network's computing power; of course, that would leave them with a convoluted, private money simulation, but nothing of value to anyone else. As with fiat, cryptocurrencies' value evaporates without trust, which is ironic, given that distributed ledgers exist to eliminate trust in centralized institutions. (See also, What Advisors Should Know About Cryptocurrencies.)
Bitcoin's distributed ledger has proved resilient and even relatively popular, and the technology has begun to pique the interest of banks, governments and others. Banks see a way to preempt expensive frauds and do away with the expensive audits needed to deter and detect them. Governments have something similar in mind: Honduras is working on committing its chaotic and abuse-prone land registry to a blockchain. (See also, Microsoft, Bank of America Team Up on Blockchain Technology.)
There is a litany of other potential applications, as any information – not just exchanges of value – can be committed to a blockchain. Email, marriage certificates, ballots, wills, you name it. "Smart contracts" promise to automate large parts, if not all, of some businesses by coding self-executing contracts into a blockchain.
Some of this enthusiasm may need to be tempered: the DAO, the first attempt at operating a "decentralized autonomous organization" though distributed ledger technology, fell victim to a spectacular hack in its infancy. The tension between distributed ledgers' need to be distributed – public and sans final arbiter – clashes with banks' desire to maintain control over their own systems. Critics wonder if the banks are spending a lot of money developing what, on final inspection, will turn out to be a conventional, private ledger. (See also, McKinsey: Traditional Banking Doomed, Blockchain Needed.)
Try telling that to the blockchain evangelists, though. For them, failing to see that this technology will change everything puts you in league with those who poopooed the internal combustion engine and the internet. And who knows, maybe they're right.