Robust and Skew-resistant Parallel Joins in Shared-nothing Systems

Long ChengLong Cheng, Spyros KotoulasSpyros Kotoulas, Tomas E. WardTomas E. Ward, Georgios TheodoropoulosGeorgios Theodoropoulos

Download

Long Cheng, Spyros Kotoulas, Tomas E. Ward, Georgios Theodoropoulos
Robust and Skew-resistant Parallel Joins in Shared-nothing Systems
Proc. 23rd ACM International Conference on Information and Knowledge Management (CIKM'14), October 2014. ACM

Details
Bibtex

KurzfassungAbstract
The performance of joins in parallel database management systems is critical for data intensive operations such as querying. Since data skew is common in many applications, poorly engineered join operations result in load imbalance and performance bottlenecks. State-of-the-art methods designed to handle this problem o�er signi�ficant improvements over naive implementations. However, performance could be further improved by removing the dependency on global skew knowledge and broadcasting. In this paper, we propose PRPQ (partial redistribution & partial query), an e�fficient and robust join algorithm for processing large-scale joins over distributed systems. We present the detailed implementation and a quantitative evaluation of our method. The experimental results demonstrate that the proposed PRPQ algorithm is indeed robust and scalable under a wide range of skew conditions. Speci�cally, compared to the state-of-art PRPD method, we achieve 16% - 167% performance improvement and 24% - 54% less network communication under diff�erent join workloads.
Weitere Informationen unter:Further Information: Link
Projekt:Project: DIAMOND
Forschungsgruppe:Research Group: Knowledge SystemsKnowledge-Based Systems

@inproceedings{CKWT2014,
  author    = {Long Cheng and Spyros Kotoulas and Tomas E. Ward and Georgios
               Theodoropoulos},
  title     = {Robust and Skew-resistant Parallel Joins in Shared-nothing
               Systems},
  booktitle = {Proc. 23rd {ACM} International Conference on Information and
               Knowledge Management (CIKM'14)},
  publisher = {ACM},
  year      = {2014},
  month     = {October}
}