Performance Benchmarking for NoSQL Database Management Systems

Camelia-Florina ANDOR

doi:10.24193/subbi.2021.1.02

Authors

Camelia-Florina ANDOR Department of Computer Science, Faculty of Mathematics and Computer Science, Babes-Bolyai University, Cluj-Napoca, Romania. camelia.andor@ubbcluj.ro

DOI:

https://doi.org/10.24193/subbi.2021.1.02

Keywords:

NoSQL database, performance benchmarking, MongoDB, Cassandra.

Abstract

NoSQL database management systems are very diverse and are known to evolve very fast. With so many NoSQL database options available nowadays, it is getting harder to make the right choice for certain use cases. Also, even for a given NoSQL database management system, performance may vary significantly between versions. Database performance benchmarking shows the actual performance for different scenarios on different hardware configurations in a straightforward and precise manner. This paper presents a NoSQL database performance study in which two of the most popular NoSQL database management systems (MongoDB and Apache Cassandra) are compared, and the analyzed metric is throughput. Results show that Apache Cassandra outperformes MongoDB in an update heavy scenario only when the number of operations is high. Also, for a read intensive scenario, Apache Cassandra outperformes MongoDB only when both number of operations and degree of parallelism are high.

Received by the editors: 18 April 2021.

2010 Mathematics Subject Classification. 68P15, 68P99. 1998 CR Categories and Descriptors. H.2.1 [Database Management]: Logical design - Data models; H.2.4 [Database Management]: Systems - Distributed databases, Parallel databases

References

C.-F. Andor and B. Pˆarv. NoSQL Database Performance Benchmarking - A Case Study. Studia Informatica, LXIII(1):80–93, 2018.

Apache Cassandra. http://cassandra.apache.org/. Accessed: 2021-02-14.

Performance Analysis: Benchmarking a NoSQL Database on Bare-Metal and Virtualized Public Cloud - Aerospike NoSQL Database on Internap Bare Metal, Amazon EC2 and Rackspace Cloud. http://pages.aerospike.com/rs/229-XUE-318/images/Internap_CloudSpectatorAerospike.pdf. Accessed: 2021-03-24.

B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. Benchmarking Cloud Serving Systems with YCSB. Proceedings of the 1st ACM symposium on Cloud computing, pages 143–154, 2010.

Fixstars. GridDB and Cassandra Performance and Scalability. A YCSB Performance Comparison on Microsoft Azure. Technical report, Fixstars Solutions, 2016.

A. Gandini, M. Gribaudo, W. J. Knottenbelt, R. Osman, and P. Piazzolla. Performance Evaluation of NoSQL Databases. EPEW 2014: Computer Performance Engineering, Lecture Notes in Computer Science, 8721:16–29, 2014.

J. Klein, I. Gorton, N. Ernst, P. Donohoe, K. Pham, and C. Matser. Performance Evaluation of NoSQL Databases: A Case Study. Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, pages 5–10, 2015.

MongoDB. https://www.mongodb.com/. Accessed: 2021-02-14.

NoSQLBench. https://github.com/nosqlbench/nosqlbench. Accessed: 2021-03-24.

Stress Test for Couchbase Client and Cluster. https://docs.couchbase.com/sdk-api/couchbase-c-client/md_doc_cbc-pillowfight.html. Accessed: 2021-03-21.

The cassandra-stress tool. https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/tools/toolsCStress.html. Accessed: 2021-03-21.

YCSB MongoDB Performance Testing. https://www.mongodb.com/blog/post/performance-testing-mongodb-30-part-1-throughput-improvements-measured-ycsb. Accessed: 2021-03-24.