Maciej Besta, Robert Gerstenberger, Marc Fischer, Michał Podstawski, Nils Blach, Berke Egeli, Georgy Mitenkov, Wojciech Chlapek, Marek Michalewicz, Hubert Niewiadomski, Jürgen Müller, Torsten Hoefler:
The Graph Database Interface: Scaling Online Transactional and Analytical Graph Workloads to Hundreds of Thousands of Cores
(In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'23), presented in Denver, CO, USA, Association for Computing Machinery, ISBN: 979-8-400701-09-2, Nov. 2023) Best Paper Finalist
Graph databases (GDBs) are crucial in academic and industry applications. The key challenges in developing GDBs are achieving high performance, scalability, programmability, and portability. To tackle these challenges, we harness established practices from the HPC landscape to build a system that outperforms all past GDBs presented in the literature by orders of magnitude, for both OLTP and OLAP workloads. For this, we first identify and crystallize performance-critical building blocks in the GDB design, and abstract them into a portable and programmable API specification, called the Graph Database Interface (GDI), inspired by the best practices of MPI. We then use GDI to design a GDB for distributed-memory RDMA architectures. Our implementation harnesses one-sided RDMA communication and collective operations, and it offers architecture-independent theoretical performance guarantees. The resulting design achieves extreme scales of more than a hundred thousand cores. Our work will facilitate the development of next-generation extreme-scale graph databases.
@inproceedings{besta2023gdi, author={Maciej Besta and Robert Gerstenberger and Marc Fischer and Michał Podstawski and Nils Blach and Berke Egeli and Georgy Mitenkov and Wojciech Chlapek and Marek Michalewicz and Hubert Niewiadomski and Jürgen Müller and Torsten Hoefler}, title={{The Graph Database Interface: Scaling Online Transactional and Analytical Graph Workloads to Hundreds of Thousands of Cores}}, year={2023}, month={Nov.}, booktitle={Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC'23)}, location={Denver, CO, USA}, publisher={Association for Computing Machinery}, isbn={979-8-400701-09-2}, source={http://www.unixer.de/~htor/publications/}, }
