Design and Evaluation of Parallel Hashing over Large-scale Data

Long ChengLong Cheng, Spyros KotoulasSpyros Kotoulas, Tomas E. WardTomas E. Ward, Georgios TheodoropoulosGeorgios Theodoropoulos

Long Cheng, Spyros Kotoulas, Tomas E. Ward, Georgios Theodoropoulos
Design and Evaluation of Parallel Hashing over Large-scale Data
Proc. 21st IEEE International Conference on High Performance Computing (HiPC'14), 1-10, December 2014. IEEE

Details
Bibtex

KurzfassungAbstract
High-performance analytical data processing systems often run on servers with large amounts of memory. A common data structure used in such environment is the hash tables. This paper focuses on investigating efficient parallel hash algorithms for processing large-scale data. Currently, hash tables on distributed architectures are accessed one key at a time by local or remote threads while shared-memory approaches focus on accessing a single table with multiple threads. A relatively straightforward “bulk-operation” approach seems to have been neglected by researchers. In this work, using such a method, we propose a high-level parallel hashing framework, Structured Parallel Hashing, targeting efficiently processing massive data on distributed memory.

We present a theoretical analysis of the proposed method and describe the design of our hashing implementations. The evaluation reveals a very interesting result - the proposed straightforward method can vastly outperform distributed hashing methods and can even offer performance comparable with approaches based on shared memory supercomputers which use specialized hardware predicates. Moreover, we characterize the performance of our hash implementations through extensive experiments, thereby allowing system developers to make a more informed choice for their high-performance applications.
Weitere Informationen unter:Further Information: Link
Forschungsgruppe:Research Group: Wissensbasierte SystemeKnowledge-Based Systems

@inproceedings{CKWT2014,
  author    = {Long Cheng and Spyros Kotoulas and Tomas E. Ward and Georgios
               Theodoropoulos},
  title     = {Design and Evaluation of Parallel Hashing over Large-scale Data},
  booktitle = {Proc. 21st {IEEE} International Conference on High Performance
               Computing (HiPC'14)},
  publisher = {IEEE},
  year      = {2014},
  month     = {December},
  pages     = {1-10},
  doi       = {10.1109/HiPC.2014.7116909}
}