a16z Crypto's latest research: What is the key to the large-scale application of DeFi?

Authors: PGarimidi, jneu_net, @MaxResnic

Compiled by: Jiahua, ChainCatcher

Blockchain can now genuinely claim that it has the capability to compete with existing financial infrastructure. Today's production systems can handle tens of thousands of transactions per second and are about to see an order of magnitude increase. However, beyond raw throughput, financial applications also require predictability. Whether it's a trade, a bid in an auction, or an options exercise, the normal operation of financial systems demands a definitive answer: when will this transaction be executed? If transactions face unpredictable delays (whether malicious or incidental), many applications will become unusable.

To make on-chain financial applications competitive, blockchains must provide short-term packaging guarantees, meaning that if a valid transaction is submitted to the network, it can ensure it is packaged as quickly as possible. For example, consider an on-chain order book. An efficient order book requires market makers to continuously provide liquidity by maintaining buy and sell orders for assets on the ledger.

The key challenge faced by market makers is to minimize the bid-ask spread while avoiding the risk of "adverse selection" due to quotes deviating from the market. To achieve this, market makers must constantly update their orders to reflect the state of the real world. For instance, if an announcement from the Federal Reserve causes asset prices to spike, market makers need to react immediately by updating their orders to the new prices. In this scenario, if the transaction to update the market maker's order does not land immediately, they will incur losses as arbitrageurs execute their orders at outdated prices. Market makers would then need to set wider spreads to reduce their risk exposure in such events, which in turn diminishes the competitiveness of on-chain trading venues.

Predictable transaction packaging provides market makers with a strong guarantee, enabling them to respond quickly to off-chain events and maintain the efficiency of on-chain markets.

What We Have vs. What We Need

Currently, existing blockchains only provide robust final packaging guarantees, typically effective within a few seconds. While these guarantees are sufficient for applications like payments, they are too weak to support large categories of financial applications where market participants need to react to information in real-time.

Taking the aforementioned order book as an example: for market makers, if arbitrageurs' trades can land in earlier blocks, then guaranteeing they are packaged "within the next few seconds" is meaningless. Without strong packaging guarantees, market makers are forced to widen their spreads and offer worse prices to users to cope with increased adverse selection risks. This, in turn, significantly reduces the attractiveness of on-chain trading compared to other venues that provide stronger guarantees.

To truly realize the vision of blockchain as modern infrastructure for capital markets, builders need to address these issues so that high-value applications like order books can thrive.

What Makes Achieving Predictability Difficult?

Strengthening the packaging guarantees of existing blockchains to support these use cases is a challenge. Some current protocols may rely on a node that can decide on transaction packaging at any given time (the "leader"). While this simplifies the engineering challenge of building high-performance blockchains, it also introduces a potential economic bottleneck where these leaders can extract value. Typically, during the window when a node is selected as a leader, they have complete power over which transactions to package into blocks. For a blockchain handling any scale of financial activity, the leader is in a privileged position. If this single leader decides not to package a transaction, the only remedy is to wait for the next leader willing to package that transaction.

In permissionless networks, leaders have the incentive to extract value, commonly referred to as MEV. MEV extends far beyond merely sandwiching AMM trades. Even if a leader can only delay transaction packaging by a few milliseconds, it can yield them significant profits and reduce the efficiency of the underlying applications. An order book that prioritizes processing only certain traders' transactions puts everyone else at a disadvantage. In the worst-case scenario, leaders may become so hostile that traders completely leave the platform.

Suppose there is an interest rate hike, and the ETH price immediately drops by 5%. Every market maker on the order book rushes to cancel their orders and place new ones at the new price. Meanwhile, every arbitrageur submits orders to sell ETH at outdated order prices. If this order book operates on a protocol with a single leader, that leader holds immense power. The leader can simply choose to censor all market makers' cancellation actions, allowing arbitrageurs to reap substantial profits. Alternatively, the leader might not directly censor cancellation actions but instead delay them until after the arbitrageurs' trades land. The leader could even insert their own arbitrage trades to fully exploit the price discrepancies.

Two Fundamental Demands

In the face of these advantages, active participation from market makers becomes economically unviable; they risk being taken advantage of whenever prices fluctuate. This issue boils down to the leader having too much privilege in two key aspects: 1) the leader can censor any other person's transactions, and 2) the leader can see others' transactions and submit their own transactions in response. Any one of these issues could prove disastrous.

An Example

We can pinpoint the problem precisely through the following example. Consider an auction with two bidders, Alice and Bob, where Bob is also the leader of the block in which the auction occurs. (The setup with only two bidders is for illustrative purposes; the same reasoning applies regardless of the number of bidders.)

The auction accepts bids during the time required to generate the block, assuming from time t=0 to t=1. Alice submits a bid bA at time tA, and Bob submits a bid bB at time tB > tA. Since Bob is the leader of this block, he can always ensure he acts last. Alice and Bob also have a continuously updated source of asset price truth they can read (e.g., the mid-price from a centralized exchange). At time t, let’s assume this price is pt. We assume that at any given time t, the market's expectation of the asset price at the end of the auction (t=1) is always equal to the current real-time price pt. The auction rules are simple: the bidder with the highest bid between Alice and Bob wins the auction and pays their bid amount.

The Demand for Anti-Censorship

Now let’s consider what happens when Bob can leverage his advantage as the leader of this auction. If Bob can censor Alice's bid, it’s clear the auction will collapse. With no other bidders, Bob only needs to bid an arbitrarily small amount to guarantee winning the auction. This results in the auction settling with essentially zero revenue.

The Hidden Demand

A more complex situation arises if Bob cannot directly censor Alice's bid but can still see Alice's bid before placing his own. In this case, Bob has a simple strategy. When he bids, he simply checks whether ptB > bA holds. If it does, then Bob's bid is just slightly higher than bA; if not, then Bob does not bid at all.

By executing this strategy, Bob puts Alice at a disadvantage of adverse selection. The only way Alice can win is if the price update causes her bid to ultimately exceed the expected value of the asset. Whenever Alice wins the auction, she anticipates losing money, making it better not to participate at all. With all competitors gone, Bob can again simply bid an arbitrarily small amount and win, while the auction effectively generates zero revenue.

The key point here is that the duration of this auction does not matter. As long as Bob can censor Alice's bid or see Alice's bid before placing his own, this auction is doomed to fail.

The same principle applies to any high-frequency trading asset environment, whether spot trading, perpetual contracts, or derivatives exchanges: if there exists a leader with the power that Bob has in this example, that leader could lead to a complete market collapse. To make on-chain products serving these use cases viable, they must never grant leaders such powers.

How Do These Issues Arise in Practice Today?

The above story paints a bleak picture for on-chain transactions on any permissionless single-leader protocol. However, many decentralized exchanges (DEXs) on single-leader protocols continue to maintain healthy trading volumes; why is that? In practice, a combination of two forces has offset the aforementioned issues:

• Leaders do not fully exploit their economic power because they themselves are often heavily invested in the success of the underlying blockchain;
• Applications have built workarounds to avoid being so vulnerable in the face of these issues.

While these two factors have kept decentralized finance (DeFi) operational thus far, they are insufficient for on-chain markets to truly compete with off-chain markets in the long run.

To qualify as a leader on a blockchain with substantial economic activity requires significant staking. Thus, either the leader must have a large amount of their own stake or possess enough reputation for other token holders to delegate their staking to them. In either case, large node operators are typically known entities whose reputations are at risk. Not only is it reputation, but this staking also means these operators have a financial incentive to ensure their blockchain runs well. Because of this, we have largely not seen leaders fully exploit their market power as described above—but that does not mean these issues do not exist.

First, relying on the goodwill of node operators through social pressure and appealing to their long-term incentives is not a solid foundation for the financial future. As the scale of on-chain financial activity increases, the potential profits for leaders also correspondingly rise. The more this potential grows, the harder it becomes socially to make leaders act against their immediate direct interests.

Second, the extent to which leaders can exploit their market power is a spectrum ranging from benign to leading to complete market collapse. Node operators can unilaterally push to utilize their power for higher profits. When some operators challenge the accepted baseline, others will quickly follow suit. The actions of a single node may seem trivial, but when everyone changes, the impact is evident.

Perhaps the best example of this phenomenon is Timing Games: leaders attempt to announce block production as late as possible while the protocol is still valid to earn higher rewards. When leaders become too aggressive, this can lead to longer block times and block skipping. While the profitability of these strategies is well-known, leaders choose not to engage in these games primarily to act as good stewards of the blockchain. However, this is a fragile social balance. Once a single node operator begins to play these strategies to earn higher rewards without any consequences, other operators will quickly join in.

Timing Games are just one example of how leaders can increase profits without fully exploiting their market power. Leaders can also take many other actions that increase their rewards at the expense of applications. Isolated, these actions may be feasible for applications, but ultimately the balance will tip to a point where the costs of being on-chain outweigh the benefits.

Another factor keeping DeFi operational is that applications shift important logic off-chain, only posting results on-chain. For example, any protocol that requires rapid auctions executes this off-chain. These applications typically run their required mechanisms on a set of permissioned nodes to avoid encountering malicious leaders. For instance, UniswapX runs its Dutch auctions off the Ethereum mainnet to complete trades, similarly, CowSwap runs its batch auctions off-chain.

While this works for applications, it places the value proposition of the underlying layer and on-chain builds in a precarious position. In a world where application execution logic is off-chain, the underlying layer becomes purely a settlement layer. One of the strongest selling points of DeFi is composability. In a world where all execution occurs off-chain, these applications essentially live in isolation. Relying on off-chain execution also adds new assumptions to the trust model of these applications. The operation of applications no longer solely depends on the activity of the underlying chain; this off-chain infrastructure must also function properly.

How to Achieve Predictability

To address these issues, we need protocols that satisfy two properties: consistent transaction packaging and ordering rules, as well as transaction privacy before confirmation (for strict definitions and extended discussions of these properties, see this paper).

Basic Demand 1: Anti-Censorship

We summarize the first property as short-term anti-censorship. If any transaction that reaches an honest node is guaranteed to be included in the next possible block, then the protocol is short-term anti-censorship:

Short-term Anti-Censorship: Any valid transaction that arrives on time at any honest node must be packaged into the next possible block.

More precisely, we assume the protocol runs on a fixed clock, with each block generated at a set time, for example, every 100 milliseconds. What we need to guarantee is that if a transaction arrives at an honest node at t=250ms, it will be included in the block generated at t=300ms. Adversaries should not have the discretion to selectively package certain transactions they hear while omitting others.

The spirit of this definition is that users and applications should have an extremely reliable way to land transactions at any point in time. There should not be a situation where a single node happens to drop packets (whether due to malice or simple operational failure) causing a transaction to fail to land. While this definition requires providing packaging guarantees for transactions arriving at any honest node, in practice, the overhead of achieving this may be too high. The important feature is that the protocol should be robust enough that the behavior of on-chain entry points is highly predictable and easy to reason about.

Permissionless single-leader protocols clearly do not satisfy this property because if at any point the single leader is a Byzantine node, there is no other way to land transactions. However, even a set of four nodes that can guarantee packaging transactions within each time period greatly improves the number of options available for users and applications to land transactions. Sacrificing a certain amount of performance for a protocol that can reliably allow applications to thrive is worthwhile. There is still more work to be done to find the right trade-off between robustness and performance, but the guarantees provided by existing protocols are insufficient.

Given that protocols can guarantee packaging, ordering is somewhat a natural consequence. Protocols can freely use any deterministic ordering rules they like to ensure consistent ordering. The simplest solution is to order by priority fees, or perhaps allow applications to flexibly order transactions interacting with their state. The best way to order transactions remains an active area of research, but in any case, ordering rules only make sense based on the landing of transactions that require ordering.

Basic Demand 2: Hiding

After short-term anti-censorship, the next most important property is that the protocol provides a form of privacy we call "hiding."

Hiding: Before the protocol finalizes the packaging of a transaction, no party other than the node receiving that transaction submission can obtain any information about that transaction.

Protocols with the "hiding" property may allow nodes to view all transactions submitted to them in plaintext but require the rest of the protocol to remain blind until consensus is reached and the order of transactions is determined in the final log. For example, the protocol may use time-lock encryption to hide the entire contents of a block until a certain deadline; or the protocol may use threshold encryption to immediately decrypt the block once a committee agrees it is irreversibly confirmed.

This means that nodes may abuse the information obtained from any transaction submitted to them, but the rest of the protocol only knows what they have reached consensus on afterward. By the time transaction information is disclosed to the rest of the network, the transaction has already been ordered and confirmed, so no other party can front-run it. To make this definition useful, it indeed means that multiple nodes can land transactions at any given time.

We abandon the use of a stronger concept where users only learn any information about their transactions before confirmation (e.g., in an encrypted memory pool) because the protocol needs to take some steps as a filter for garbage transactions. If the content of transactions is completely hidden from the entire network, then the network cannot filter out garbage transactions from meaningful ones. The only way to solve this problem is to leak some non-hidden metadata as part of the transaction, such as the address of the fee payer that will be charged regardless of whether the transaction is valid.

However, this metadata may leak enough information for adversaries to exploit. Therefore, we prefer a single node to have complete visibility of the transaction while other nodes in the network have no visibility of it. But this also means that for this property to be useful, users need to have at least one honest node as an on-chain entry point to land transactions at each time period.

A protocol that possesses both short-term anti-censorship and hiding provides an ideal foundation for building financial applications. Returning to our example of trying to run auctions on-chain, these two properties directly address the ways in which Bob could lead to market collapse. Bob can neither censor Alice's bid nor leverage Alice's bid to inform his own bid, which precisely resolves the issues in our previous example.

With short-term anti-censorship, anyone submitting a transaction (whether a trade or an auction bid) can be assured of immediate packaging. Market makers can change their orders; bidders can bid quickly; settlements can land efficiently. Users can be confident that any action they take will be executed immediately. This, in turn, will allow the next generation of low-latency real-world financial applications to be fully built on-chain.

To enable blockchains to truly compete with and even surpass existing financial infrastructure, we need to address far more than just throughput issues.