The main question is the following:
When designing a system supporting atomic broadcast, can it be proved theoretically that the performance & scalability dimensions (i.e. latency, throughput, dataset size) of the system are limited by the performance of a single node?
To give an example:
A system based on partitioned-logs, like Apache Kafka, can provide ordering guarantees in a single partition, but it can't provide any ordering guarantees between different partitions. However, this gives Kafka the capability to scale to extremely large datasets. I was contemplating whether it would be possible to create a system that could provide the ordering guarantee for the whole dataset, while also allowing the dataset's size to increase in the same quasi-linear way.
My speculative answer to this is no for the following reason:
It's been proved that the atomic broadcast problem is equivalent to the consensus problem . Based on the fact that the consensus problem requires an elected leader, which drives the consensus process, I concluded that the scaling capabilities of such a system is limited by the resources of a single node.
Are there any flaws in my thinking ?