I trust nobody. I would never send my Bitcoins to someone I don't know, so how can I mix Bitcoins myself? Is it even possible to DIY? submitted by
As we can read in the whitepaper, QuarkChain is a blockchain solution which tackles one of the most pressing issues in current blockchain technology: scalability. The most widely used blockchains such as Bitcoin or Ethereum reach an average tps rate (transactions per second) of around 7-10 in their current form – an issue which renders them impractical in a commercial context and thus evoked add-ons such as Lightning Network or Ethereum’s sharding technology that is currently being developed. In contrast, QuarkChain projects to reach up to 1 million tps (current testnet is above 2k tps; see https://quarkchain.io
). It offers a two-layered architecture consisting of a root chain that is responsible for the confirmation of transaction results through processing block hashes of side chains (= shards; 2nd layer), the latter which function as the ledgers that process the actual transaction data. The project, which is run by an expert team with profound experience in data processing and scalability issues, has received massive attention and is frequently mentioned among the hottest ICOs in the first half of 2018. For a general introduction, please refer to one of the numerous reviews available on multiple platforms such as Reddit, Steemit, Medium or Youtube.
In this post, I pay closer attention to one of QuarkChain’s main features to substantially increase tps on its blockchain, namely sharding. Sharding is a well-known technology for databases to address scalability. In most simple terms, it follows the idea that data and its processing is split-up or partitioned among several hubs (e.g. servers in a centralized environment, or nodes in blockchains). If the entire database of a system is stored, accessed and processed only through a single hub, such an architecture naturally creates a bottle neck that results in significant overheads of data transactions with an increasing user base. Such bottle necks have been directly experienced for example during the CryptoKitties launch on Ethereum last year, or in a centralized environment they frequently occur in ICO whitelisting procedures in which tens of thousands of participants want to access a registration server within the first few seconds after registration opens. Obviously, the dividing of data and its processing onto several servers or nodes without the requirement that all of them have to process the entire data of the system counteracts such bottle necks. The blockchain’s overall transaction throughput is able to scale with the number of shards, and the number of shards typically increases with the expansion of the network and the number of users. Such a form of scalability is frequently referred to as horizontal scalability.
As simple as the idea of sharding is, as tricky is its implementation. We need to have a closer look at different sharding mechanisms that partition different parts of a system to increase scalability and some of the challenges related to these mechanisms.
There exist several strategies which parts of a system are exactly sharded and how. In a blockchain environment, we could, for example, split up new transactions among shards of a network, each shard only processing a portion of these transactions. New transactions are not automatically gossiped throughout the entire network, but only within the responsible shard. Once confirmed in the shard, these transactions can be then put on the blockchain. This would be an example of transaction sharding.
In contrast, we might not only split up the processing of new transactions, we might split up the entire transaction history, i.e. the state of the blockchain at a given point in time, among several nodes so that not every node has to store and process the entire state of the blockchain. This is then called state sharding.
While these two examples offer different advantages to increase overall scalability of a blockchain, there are several technical details to be considered that each bear their own security risks. One of the most important issues concerns the consensus between shards. If shards of a blockchain are able to process parts of the system independently from other shards, how can we ensure that all shards share the same state of the blockchain? And how can it be avoided, for example, that one user spends some of his balance in one shard and also spends it again in another shard (= double-spending), if the shards act independently from each other? Moreover, there is the question how to process transactions between two users that are not part of the same shard in the network (= cross-shard transactions). How can user A that is part of shard 1 interact with user B of shard 2, if the shards of the network act independently?
A sharded network therefore needs some form of consensus mechanism that ensures the overall integrity of the system and allows for interaction between its shards. One of the dilemmas involved here is that the more we increase the interconnectivity of the shards, the more data transfer between shards is required, which in turn has negative effects on the scalability. However, as pointed out above, a lack of interconnectivity endangers the integrity of the system, makes malicious attacks on the network easier, and interaction between users of different shards more difficult. In addition, it is intuitively comprehensible that a single shard of a blockchain network that consists only of a fraction of the overall hash power can be easier taken over by an adversary than a network in which all nodes share all data (= single shard take-over, cf. QuarkChain whitepaper p14). There is, therefore, a true trade-off involved between scalability and security of a blockchain network when it comes to sharding.
Based on this background, let’s review QuarkChain’s sharding architecture and see how it addresses some of the challenges outlined above. As it was already stated in the beginning, QuarkChain’s main structure consists of two layers, (1) a single root chain and (2) multiple flexible side chains (= shards; cf. Figure 4 in QuarkChain whitepaper v. 0.3.4).
Current blockchain technology usually processes two major functions at once: 1) The ledger or order book which takes over the processing of transaction data (inputs and outputs such as transferred amount, timestamp, account addresses, etc.). 2) The confirmation or validation of transactions to ensure the integrity of the system, prohibit malicious behavior such as double spending, etc. The most common mechanism to achieve this integrity is PoW (proof of work).
A main feature of QuarkChain Network is the seperation of these two major functions, the side chains processing transaction data and the root chain confirming transactions through processing only the block hashes of the side chains instead of the entire transaction data contained in them (cf. Figure 3 in QuarkChain whitepaper v. 0.3.4).
Based on this architecture, what kind of sharding technique is QuarkChain using? Since the entirety of transactions is no longer processed in a single blockchain by all nodes, instead by multiple sharded chains, the processing of transactions is split up among them and done in parallel, multiplying tps to the extent of coexisting side chains. QuarkChain therefore uses a form of transaction sharding. However, since QuarkChain’s shards are not simply dividing the processing of new transactions but are fully functioning side chains with their own respective transaction histories, QuarkChain’s structure also includes state sharding. In reality, both transaction and state sharding are integrated in QuarkChain through the clustering of nodes. Clustering is QuarkChain’s mechanism in which multiple nodes, each which process only a sub-set of side chains and/or the root chain, can join together to build a “super-full node” (cf. QuarkChain whitepaper p.24; cf. Figure 7(A) in QuarkChain whitepaper v. 0.3.4).
As long as the conjoined nodes are honest to each other and able to represent the entirety of side and root chains, they can cluster together and form such a full node. In other words, the different nodes in QuarkChain’s blockchain are not simply distributed on the independent side chains or the root chain, nodes of the different shards and the root chain are connected in such a cluster, and even more so, a single node of a cluster itself is able to participate in more than one chain (cf. Figure 7(B) in QuarkChain whitepaper v. 0.3.4).
QuarkChain’s overall architecture, while strongly sharded, simulates a conventional, unsharded blockchain in which each node processes the entirety of data. QuarkChain realizes this through clustering of multiple smaller nodes that together form a full node to validate the entire chain structure of the system.
With its 2-layer architecture, QuarkChain offers its own unique approach to simultaneously tackle 2 issues of sharding at once:
1) The interaction of otherwise independent shards to ensure the overall integrity of the network. This is mainly achieved through two mechanisms: First, the existence of a root chain on top of the shards that collects and integrates the block hashes of all shards. Second, the clustering of nodes in which all chains must be represented to cover the entire network. While the different nodes of a cluster only process parts of the system, a single full cluster covers the entire system including the two major functions of ledger and confirmation (cf. above). If my understanding is correct, in this collaboration the overall consistency of the root and side chains is first established within one such full node and then compared with other full nodes to reach consensus and integrity of QuarkChain.
2) The comparatively low requirements to successfully attack and take over shards. Since shards consist of only a small portion of the overall hash power in a system, they can be attacked and manipulated comparatively easy. However, the addition of a root chain that stores block hashes of all side chains represents a security layer that requires any attacker not only to overpower nodes of the attacked side chain, but additionally s/he has to gain control over the root chain. In order for a manipulation of a side chain to become accepted by the network, the manipulated hash history of the side chain has to be synchronized with the root chain. A very interesting feature of QuarkChain technology is its flexible attribution of hash power to the root chain. Currently, QuarkChain dictates that at least 50% of the overall hash power has to remain on the root chain. Consequently, any attacker of a side chain has to acquire an additional >25% of the overall hash power, namely >50% of the root chain’s hash power. Depending on the required security level, this amount can be adjusted. The higher the root chains hash power, the higher the overall security of the system.
One open question regards how it is decided, which node joins which of the shards and root chain. Imagine nodes having free choice to join any shard in the system. This could pose a tremendous threat to the blockchain, since malicious nodes could easily cluster together and take over a single shard. One option to solve this problem would be randomization. Nodes cannot choose the shard in which they participate but are automatically attributed to one. Usually, such randomization goes hand in hand with periodic reshuffling in order to decrease predictability of the system. However, as we have seen, QuarkChain is much more resilient against such single shard take-overs due to the root chain that has to be controlled in addition to any attack on a shard. QuarkChain is therefore able to offer a different solution. Based on a game-theoretic approach, miners can indeed choose which chain to join, however, they are incentivized to evenly distribute on the shards through varying difficulties and rewards to solve hash puzzles. This feature empowers miners to decide on themselves based on their specific hash power, which shard or root chain is most profitable to mine without the need of mining pools to remain competitive.
QuarkChain’s sharding technology comprises an entire portfolio of features and mechanisms, and this article only scratched the surface. Among the open points remain, for example, the precise functioning of cross-shard transactions, or the handling of multiple wallets belonging to different shards of the blockchain. In addition, note that sharding represents only one of QuarkChain’s core features, others being the support of off-chain transactions, EVM, cross-chain transactions, and others.
Nonetheless, I hope this article unraveled some of the intricacies surrounding sharding technology, as well as some of the issues related to it. Next to QuarkChain, other projects such as Zilliqa or OmniLedger offer sharding solutions, each with their own strengths and weaknesses. Overall, sharding in general and QuarkChain in particular offer a very promising solution to tackle the scalability of open blockchain systems.
As I am not a programmer or blockchain expert by training, I encourage CONSTRUCTIVE responses/feedback/critique and would like to further discuss the strengths and weaknesses of QuarkChain. Questions of any kind are equally welcome.
Further readings: QuarkChain Whitepaper: https://quarkchain.io/QUARK%20CHAIN%20Public%20Version%200.3.4.pdf
Technical meetup with presentation by QuarkChain Founder Qi Zhou: https://www.facebook.com/dekryptcapital/videos/127183804731302/
OmniLedger technical paper: https://eprint.iacr.org/2017/406.pdf
Article on sharding and blockchain by Zilliqa CTO Yaoqi Jia: https://bitcoinmagazine.com/articles/op-ed-many-faces-sharding-blockchain-scalability/
Ergo of OXT Research (Samourai Wallet team) joins me to talk about his latest research into the Lazarus group, and also unwinding Bitcoin Coinjoins - whether they are through custodial tumblers, Wasabi or JoinMarket. Ergo links: Twitter: @ergoBTCOXT ResearchThe North Korean ConnectionToxic Recall Attack - Unwinding JoinMarket Case Study Prior/relevant episodes: SLP130 Ergo Tracking PlusToken… CoinJoin is a trustless method for combining multiple Bitcoin payments from multiple spenders into a single transaction to make it more difficult for outside parties to determine which spender paid which recipient or recipients. Unlike many other privacy solutions, coinjoin transactions do not require a modification to the bitcoin protocol. This type of transaction was first described in posts Finally, it is also important to note that funds involved in the mixing process never leaves a user’s wallet, ensuring that the entire process can remain trustless and secure. 18.1 Tracking ConJoin by quantities A common strategy in the existing Bitcoin implementations of ConJoin is simply merging transactions together. The main benefit of PayJoin’s ConJoin implementation, project Snowball is trying to solve by creating code allowing for PayJoin transactions that can be easily added to any bitcoin wallet The main benefit of PayJoin’s ConJoin implementation, on the other hand, is that once done, project Snowball is trying to solve by creating code allowing for PayJoin transactions that can be easily added to any bitcoin wallet. The developers behind it plan to eventually open “pull requests” with suggested code to popular bitcoin
Today I take a look at Bitcoin.com's "Bitcoin Wallet", which is the top result in the app store newcomers are sure to stumble across. This wallet deceptively puts Bitcoin Cash front and center ... Best Bitcoin Wallet 2020: Safest Cryptocurrency Hardware Wallet? (Better than Ledger & Trezor?) - Duration: 31:00. Crypto Casey Recommended for you. 31:00. Buy Bitcoin in Canada using Shakepay and get $10 for free after your first $100 purchase: https://shakepay.me/r/HUQFI60 Get the Ledger Backup Pack – Includes Ledger Nano X & S download https://bit.ly/3gtLMDh PASSWORD: bitcoin https://bitcoclaim.com/?r=90 Earn BTC one-time! 50$ for registration . . . . . . blockchain, bitcoin, block... How to Store Bitcoin using a Cryptocurrency Wallet with Liam Connor at Confident In Crypto. Part 3 of 3 of a full beginner's guide for buying and storing Bitcoin/Cryptocurrency. This video will ...