LOG REPLICATION IN RAFT VS KAFKA
DOI:
https://doi.org/10.24193/subbi.2020.2.05Keywords:
comparison, Raft, Kafka, consensus algorithm, replicated log.Abstract
The implementation of a fault-tolerant system requires some type of consensus algorithm for correct operation. From Paxos to Viewstamped Replication and Raft multiple algorithms have been developed to handle this problem. This paper presents and compares the Raft algorithm and Apache Kafka, a distributed messaging system which, although at a higher level, implements many concepts present in Raft (strong leadership, append-only log, log compaction, etc.).
This shows that mechanisms conceived to handle one class of problems (consensus algorithms) are very useful to handle a larger category in the context of distributed systems.
References
Apache Kafka. http://kafka.apache.org
C. McCabe, Kafka improvements proposals” (having status accepted 2020, Retrieved from https://cwiki.apache.org/confluence/display/KAFKA/KIP-500
D. Ongaro, J. Ousterhout (2013), In Search of an Understandable Consensus Algorithm, USENIX ATC 14, Proceedings of the 2014 USENIX Annual Technical Conference, June 2014 Pages 305-320, Retrieved from https://web.stanford.edu/ouster/cgibin/papers/raft-atc14
J. Kreps, Benchmarking Apache Kafka: 2 Million Writes Per Second (On Three Cheap Machines), Retrieved from https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-three-cheap-machines, Retrieved
J. Ousterhout, Lecture for the Raft User Study, March 2013, Retrieved from https://raft.github.io/slides/raftuserstudy2013.pdf
L. Lamport, The Part-Time Parliament, ACM Transactions on Computer Systems, 16 (2): 133-169. DOI:10.1145/279227.279229.
T. D Chandra, R. Griesemer, J. Redstone, Paxos made live: an engineering perspective. In Proc. PODC’07, ACM Symposium on Principles of Distributed Computing (2007), ACM, pp. 398-407.
W. Guozhang, J. Koshy, S. Subramanian, K. Paramasivam, M. Zadeh, N. Narkhede, J. Rao, J. Kreps, J. Stei, Building a Replicated Logging System with Apache Kafka, Proceedings of the VLDB Endowment, Retrieved from http://www.vldb.org/pvldb/vol8/p1654-wang.pdf, DOI:10.14778/2824032.2824063
W. Han Z. Shang, K. Wolter, Performance Prediction for the Apache Kafka Messaging System, 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City, DOI: 10.1109/HPCC/SmartCity/DSS.2019.00036
B. Reed, F Junqueira, A simple totally ordered broadcast protocol, Proceedings of the 2nd Workshop on Large-Scale Distributed Systems and Middleware, September 2008, Article No. 2, Pages 1-6, doi:10.1145/1529974.1529978,
A. Medeiros, ZooKeeper’s atomic broadcast protocol: Theory and practice, Helsinki University of Technology, 2012, Retrieved from https://www.semanticscholar.org/paper/ZooKeeper-
F. P. Junqueira, B. C. Reed and M. Serafini, Zab: High-performance broadcast for primary-backup systems, 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks (DSN), Hong Kong, 2011, pp. 245-256, DOI: 10.1109/DSN.2011.5958223
P. Hunt, M. Konar, F. Junqueira, B. Reed, ZooKeeper: wait-free coordination for internet-scale systems, USENIXATC’10: Proceedings of the 2010 USENIX conference on USENIX annual technical conference, June 2010, Pages 11
H. Yusuf1, I Surjandari. (2020). Comparison of Performance Between Kafka and Raft as Ordering Service Nodes Implementation in Hyperledger Fabric. International Journal of Advanced Science and Technology, 29(7s), 3549-3554. Retrieved from http://sersc.org/journals/index.php/IJAST/article/view/17652
H. Yusuf, I Surjandari, Comparison of Performance Between Kafka and Raft as Ordering Service Nodes Implementation in Hyperledger Fabric, International Journal of Advanced Science and Technology Vol. 29, No. 7s, (2020), pp. 3549-3554, ISSN: 2005-4238 IJAST
C. Wang, X. Chu, Performance Characterization and Bottleneck Analysis of Hyperledger Fabric, arXiv:2008.05946v1 [cs.DC]
C McCabe, Kafka improvements proposals (having status accepted), 2020, https://cwiki.apache.org/confluence/display/KAFKA/KIP500
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2020 Studia Universitatis Babeș-Bolyai Informatica
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
