Blockchain is the technological breakthrough that powers almost all cryptocurrencies we’ve heard of today. But how is it architected? In this post, I break down the architecture of all the blockchains, identify all the standard components, break them down, and point out what’s missing.
Blockchain Motivation
There is usually a motivation for building most software, especially those meant to be used commercially and blockchain is no exception. It was built to satisfy technical requirements that would enable decentralized digital money. Before blockchain technology, the only way to represent money digitally was by a centralized approach where one organization was solely in charge of operating the ledger software. A bunch of academics worked hard to provide a decentralized approach to representing digital money and that led to the invention of blockchain with Bitcoin being the pioneer.
Now the blockchain technology that was birthed as a result of this process can handle many other use cases aside from decentralized digital money. Some of these include finance, games, science, supply chain, voting, etc … In general, blockchains have the following properties;
- Decentralized
- Transparent
- Immutable
- Secure
In simple terms, before blockchain, there wasn’t any technology that could satisfy all these properties at once. This makes it very well suited to certain use cases that were just not possible in the computing world. Blockchain is therefore a technology that isn’t going anywhere, it’s here to stay.
Blockchain Data Structure
I will first do a brief recap of the type of data and data structures used by the blockchain.
- Accounts
- Transactions
- Blocks
- Blockchain
Accounts
On the blockchain, accounts are created using public key cryptography. Every account has the following;
- Private Key
- Public Key
- Address
The private key as the name suggests is supposed to be private since it is used to sign transactions and spend funds from the account. The public key is generated from the private key and is used to publicly represent the account. The address is the hash of the public key. This is done to give all addresses on the blockchain a similar pattern and is used instead of the public key to represent accounts. Addresses are much cuter compared to the raw public key although some blockchains just use the public key instead of the address.
Transactions
A transaction is an action that can be performed on the blockchain by an account. For example, to spend funds from an account, a transaction must be created, signed, and submitted to the blockchain network to be recorded.
Blocks
A block is a collection of transactions that are created periodically on the blockchain, verified, and submitted to the blockchain network. This happens a periodic time intervals and is different for every blockchain network. For example, in Bitcoin, the interval is 10 minutes, in Ethereum, the interval is 15 seconds.
Blockchain
The blockchain is a collection of blocks that are sequentially and cryptographically linked together. Anytime a new block is submitted to the network, a special type of algorithm known as the consensus algorithm is used to determine if the block should be added to the blockchain or not.
Blockchain Components
Note that my focus is on public permissionless blockchains like Ethereum and Bitcoin, which you might have heard about already. The components are the various pieces of software that come together to give the end users the ability to experience the technology.
- Wallets
- Nodes
- Miners / Validators
- Indexers
- Block Explorers
Wallets
A wallet is a user-facing application that allows everyday users to interact with the blockchain. With a wallet application, the user can create, sign transactions, and interact with every application on the blockchain. In a permissionless blockchain, anyone can build their wallet application and interact with the blockchain. However, there are so many commercial wallets in the market that users can download and use. There are different types of wallets also, and I will create another post to discuss them.
Nodes
A blockchain node is a precise piece of software that implements a blockchain protocol. Nodes are deployed on the internet and communicate with each other to ensure that transactions and blocks are passed around the whole network. The node stores its copy of the blockchain data and it communicates with other nodes in the network to share new data.
In a permissionless blockchain, anyone can set up their node and immediately join the blockchain network. It is computer software, so setting it up involves running the software either locally, in the cloud, or on your servers. Ideally, in the spirit of decentralization, every user should at least have a running copy of the blockchain node. The electrum wallet, which was the first bitcoin wallet followed this principle. When you run the wallet on your computer it also spins up a local node which it then interacts with to verify and broadcast transactions.
This is however impractical for the most class of users and devices, notably mobile devices. Modern wallets therefore just resort to using blockchain nodes being hosted online in the cloud somewhere. The problem however is that most wallets don’t run their nodes and resort to using third-party provided nodes. This effectively means that the whole network ends up relying on fewer nodes and the purpose of decentralization is defeated. In my view, companies building wallet applications should at least run their blockchain nodes.
Miners / Validators
These are special types of blockchain nodes that are responsible for verifying transactions, and blocks and appending new blocks to the blockchain. They follow special algorithms known as the consensus protocol to achieve this feat. Every blockchain has its consensus protocol which simply means how the miner/validator software interacts to accept new blocks into the blockchain. In a permissionless blockchain, any entity can join the network as a miner/validator simply by running the software. The miners/validators determine what blocks and transactions should be accepted into the blockchain. This component is a very critical aspect of blockchain and determines if the blockchain is decentralized or not. A blockchain network with few miners/validators is seen as largely centralized. The ideal situation is to have as many miners/validators as possible, with different entities forming a little fraction of the miners/validators in the network. If a single entity forms at least 51% of the miners/validators in the network, then the blockchain is at risk of centralization and is no longer secure. This is already an issue in most existing blockchains since the requirements to run a miner/validator node are so high most entities cannot get into it.
Indexers
An indexer is an ETL (extract, transform, load) process that takes the data in a blockchain and stores them in another database in such a way that querying for certain data is fast and reliable. The original blockchain design and data structure cannot get the full transaction history for an account. I explain in detail in my recent post here. In summary, if you wanted to get the transaction history of an account you would have to loop through blocks and take transactions that belong to the account. Doing this in real-time is not feasible as users won’t wait for that long. Even for utxo-based blockchains like bitcoin, to get the balance of an account, you first need to get the full transaction history and then sum things up. Indexers were invented to solve this issue.
The problem is that, unlike blockchain nodes, indexers are not very standardized and little attention is given to them. For most blockchains, you will be lucky to find a well-maintained open-source indexer. In my opinion, this needs to change and blockchain core teams need to include indexers as part of the core software instead of just leaving it for a generous open-source contributor to probably pick it up. When I started my blockchain career, one of the shocks I took was that there was actually only one paid third-party provider which has APIs to get transaction history in the whole Ethereum ecosystem.
Indexers need to be taken seriously by the blockchain ecosystem. The problem compounds as more and more standardized smart contracts are being added. Wallets are forced to depend on third-party providers and this effectively centralizes the system. A great example is NFTs, where almost all wallets depend on APIs provided by NFT marketplaces to be able to get ownership and transaction details. I’m not happy with depending on third-party resources when it comes to building blockchain applications. I do understand the amount of work that needs to go into this but I strongly believe it's for the long-term benefit of blockchains and to bring out the full glory of decentralization.
Block Explorers
A block explorer is a website that allows users to view blockchain data. In permissionless blockchains, the data is open and available for the viewing and scrutiny of everyone. If you run your blockchain node, you have a full copy of all the transactions available on the blockchain. Alternatively, you could also use the API of a hosted blockchain node to view the data. All of these require you to have some technical skills. For normal users, there are several block explorers from which they can view blockchain data. I use block explorers all the time when I’m working on blockchain projects. Anyone could also build their block explorer for any use case, even just for fun. Using a block explorer means you trust that the developer is showing you accurate data from the blockchain. If you are someone with trust issues then you should probably build yours.
Blockchain Protocols
The blockchain ecosystem is riddled with so many algorithms that dictate the behavior of all components of the system.
- Peer-to-Peer Network Protocol
- Consensus Protocol
- State Transition Protocol
- Block / Transaction Validation Protocol
- Replication Protocol
- Smart Contracts
These are just a few that I will touch upon. However, there are so many of them and they keep piling up in the ecosystem. This is due to the nature of blockchain software and how everything must be carefully considered before implementing and pushing to production.
Peer-to-Peer Network Protocol
This defines how the nodes within the blockchain network communicate with each other to exchange information and create new blocks. This is usually just simple peer-to-peer network communication for most permissionless blockchains where any node can talk to any other node without restriction. The protocol is usually over HTTP APIs or RPCs.
Consensus Protocol
This protocol defines how the miners/validators within a blockchain collaborate to agree on new blocks to be added to the blockchain. There are so many different types of consensus protocols for various use cases, but the two most popular ones for most permissionless blockchains are proof of work and proof of stake.
State Transition Protocol
There are two types of blockchains when it comes to the state. Stateless and Stateful blockchain. A stateless blockchain does not store the state of accounts, it only stores transactions. Typical examples are utxo-based blockchains like bitcoin and its variants. A stateful blockchain stores the state of the accounts like their balance, transaction count, smart contract data, etc … After every new block, a new state is generated and stored. So basically blocks act as state transition inputs to a state transition protocol that determines the next state by combining the current state with the new block. Blockchains like Ethereum, Tezos, Near, Solana, Tron, etc … are stateful blockchains. One characteristic of stateful blockchains is that the nodes have an API to query the balance of accounts. This feature is however missing from stateless blockchains and the only way to compensate for that is to use indexers.
Block / Transaction Validation Protocol
As transactions are submitted blocks are created. Nodes need to be able to validate the blocks and the transactions within them. Usually, when a transaction is submitted, the nodes try to do some sort of validation on it even before trying to broadcast it to other nodes. An example of a simple check is if the sender has enough balance to cater to the amount they are trying to send and the transaction fees. This is a simple check done in most blockchains. In stateless blockchains with uxto-model, the node will also check if the input transactions are valid or even exists in the first place.
Replication Protocol
In most blockchains, data is replicated amongst the nodes implicitly since the nodes are always sharing data. However, some blockchains need special replication protocols to cater to their high-speed environments or different architecture. Solana blockchain has a special proof of replication protocol due to its high-speed environment. Near blockchain also has a different replication mechanism due to its multi-shard approach.
Smart Contracts
Smart contracts allow users to add their protocols to the blockchain. NFTs are protocols, and so are stablecoins DAOs, and more. These are all possible because of the smart contract protocol of the blockchain. Most blockchains can run smart contracts and for this, they define the programming language, the virtual environment for smart contract execution, and the objects that are available within that virtual environment. The most popular smart contract virtual environment is the EVM (Ethereum Virtual Environment) and with it also comes the programing language Solidity. Almost every modern blockchain already supports EVM or aims to support it.
Open Architectural Issues In Blockchain
Blockchain technologies have been running for more than a decade now. This is very commendable considering the amount of work that has gone into it. The journey has not been all rosy and the ecosystem is currently facing many interesting architectural issues.
- The blockchain trilemma.
- Custody and secret recovery.
- Centralization.
- Inter blockchain communication.
The Blockchain Trilemma.
Traditionally blockchains are secure and decentralized but are also slow. Slow in the sense that they can process a few transactions at a time. Bitcoin can process about 3 to 4 transactions per second, however, this is not enough to meet the high number of transaction requests placed by users. As a result, there are a lot of transactions that take a long time to be processed, even days. This also drives up fee prices since users need to pay higher fees if they want their transactions to go through fast. This is also the same problem with Ethereum.
One might then think that an obvious solution to the problem is to just increase the speed of the blockchain. However, this leads to other problems. The creator of Ethereum, Vitalik Buterin, popularized the term “blockchain trilemma” to explain this problem. If the speed is increased, either the security or decentralization of the blockchain is affected. In summary, you can have one of the following 3 combinations but not all 3 properties;
- Speed + Security
- Speed + Decentralization
- Security + Decentralization
Some blockchains have claimed to have solved this problem but then it later becomes apparent that it falls into one of these categories. Two notable solutions to this problem are layer 2 blockchains and sharding.
Layer 2-based solutions are like blockchains that attach themselves to a main blockchain. It operates as a blockchain on its own but syncs its state to the main blockchain at regular time intervals or based on a specific protocol. A notable example of layer 2 is the lightning network which is built on top of bitcoin.
The sharding approach divides a blockchain into sub-blockchains each behaving as a full blockchain and syncing with other shards such that it appears to the user as one blockchain. Users are split into shards but they are not even aware. The near blockchain is a good example of a blockchain that is using sharding as part of its architecture.
Custody and Secret Recovery
In the blockchain, accounts can only be manipulated using private keys. Without their private key, a user cannot access their account or any funds in it. Since blockchain is decentralized, it also means that everyone is responsible for keeping safe their private key. Predictably, it is the case that most people are misplacing their private keys and hence losing access to their accounts and funds. Although there are good proposed solutions to this problem, it's still not solved yet since some of the solutions are either too complicated or impractical. It is necessary to point out that wallets that keep private keys on behalf of users are not considered a solution and such wallets are rather a virtual representation of blockchain wallets.
One interesting solution is social recovery where the private key is split and shared amongst individuals the user chooses. In the event the user loses access to their keys, all or some of the individuals can cryptographically combine their key shares to reproduce the actual key for the user. Note that none of the individuals can use their share to access the account since they only have part of the key.
Centralization
A blockchain becomes centralized if a few entities control most of the miners/validators in the network. With time this phenomenon is beginning to show up due to the high quitting rate of miners/validators and the ever higher barrier to becoming miner/validator. As time goes on it becomes difficult to run and maintain miner/validator software, it is difficult for new players to enter the game and some existing players are exiting because they’re not making enough profit. To become an Ethereum validator you need to have at least 32 Eth to get a small chance to create blocks. You should also be able to operate the mining software and or higher staff to ensure the continuous operation of the software. The stakes are even higher for Bitcoin with even higher expenses needed for the mining hardware and electricity power.
Inter Blockchain Communication (IBC)
This is a hot new problem in blockchain ecosystems where protocols are being drafted and implemented to provide a common interface with which all the blockchains can communicate with each other. An interesting solution in this space is Bridges and a blockchain that has also tried to address this problem is Cosmos. By using bridges and the Cosmos IBC, it can facilitate communication between different blockchains. The Cosmos blockchain is interesting and worth looking into.
Blockchains With Different Architectures
To conclude, almost all current blockchains are very similar in terms of architectural choices and algorithms. However, there are some blockchains with a notably different architecture from the others. I want to talk about three of those and point out where they are different.
- Stellar
- Cosmos
- Hedera
Stellar
Stellar is a blockchain specifically designed and built for money transfer. Where it differs the most in terms of its architecture is the consensus algorithm it uses. It uses SCP (Stellar Consensus Protocol) with the distinct property that a validator on the network must choose other validators in the network such that it will only trust information received from these validators. This is quite contrary to all the existing consensus protocols which eliminate trust. In addition, there are no incentives for being a validator on the Stellar network.
Cosmos
Cosmos blockchain is dubbed a blockchain of blockchains. It is a blockchain within which other blockchains can be created. This is not layer 2 but rather a fully extended blockchain within a blockchain. Each blockchain can have its protocols, and governance and even use a different consensus algorithm, the default is Tendermint.
Hedera
Technically this is not a blockchain since it doesn’t use the blockchain data structure. It uses another data structure called the hashgraph. However, it is also able to satisfy the technical requirements of blockchains and has its cryptocurrency.
The blockchain ecosystem is large and wide and interesting things are yet to happen. I created this post to give an overview of the software architecture of blockchains.
