Inproceedings4048: Unterschied zwischen den Versionen
Aus International Center for Computational Logic
Long Cheng (Diskussion | Beiträge) (Die Seite wurde neu angelegt: „{{Publikation Erster Autor |ErsterAutorNachname=Cheng |ErsterAutorVorname=Long }} {{Publikation Author |Rank=2 |Author=Spyros Kotoulas }} {{Publikation Author…“) |
Long Cheng (Diskussion | Beiträge) Keine Bearbeitungszusammenfassung |
||
| Zeile 17: | Zeile 17: | ||
{{Inproceedings | {{Inproceedings | ||
|Referiert=1 | |Referiert=1 | ||
|Title= | |Title=QbDJ: A Novel Framework for Handling Skew in Parallel Join Processing on Distributed Memory | ||
|To appear=0 | |To appear=0 | ||
|Year= | |Year=2013 | ||
|Month= | |Month=November | ||
|Booktitle=Proc. | |Booktitle=Proc. 15th IEEE International Conference on High Performance Computing and Communications (HPCC'13) | ||
|Pages= | |Pages=1519-1527 | ||
|Publisher=IEEE | |Publisher=IEEE | ||
}} | }} | ||
{{Publikation Details | {{Publikation Details | ||
|Abstract=The performance of parallel distributed data management systems becomes increasingly important with the rise of Big Data. Parallel joins have been widely studied both in the parallel processing and the database communities. Nevertheless, most of the algorithms so far developed do not consider the data skew, which naturally exists in various applications. State of the art methods designed to handle this problem are based on extensions to either of the two prevalent conventional approaches to parallel joins - the hash-based and duplication-based frameworks. In this paper, we introduce a novel parallel join framework, query-based distributed join (QbDJ), for handling data skew on distributed architectures. Further, we present an efficient implementation of the method based on the asynchronous partitioned global address space (APGAS) parallel programming model. We evaluate the performance of our approach on a cluster of 192 cores (16 nodes) and datasets of 1 billion tuples with different skews. The results show that the method is scalable, and also runs faster with less network communication compared to state-of-art PRPD approach in [1] under high data skew. | |||
|Abstract= | |Download=2013-A-Novel.pdf | ||
|Link=http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6832096 | |||
|Download= | |DOI Name=10.1109/HPCC.and.EUC.2013.214 | ||
|Link=http:// | |||
|DOI Name=10.1109/ | |||
|Projekt=DIAMOND | |Projekt=DIAMOND | ||
|Forschungsgruppe=Information Systems | |Forschungsgruppe=Information Systems | ||
}} | }} | ||
Version vom 21. Oktober 2014, 23:35 Uhr
QbDJ: A Novel Framework for Handling Skew in Parallel Join Processing on Distributed Memory
Long ChengLong Cheng, Spyros KotoulasSpyros Kotoulas, Tomas E. WardTomas E. Ward, Georgios TheodoropoulosGeorgios Theodoropoulos
Long Cheng, Spyros Kotoulas, Tomas E. Ward, Georgios Theodoropoulos
QbDJ: A Novel Framework for Handling Skew in Parallel Join Processing on Distributed Memory
Proc. 15th IEEE International Conference on High Performance Computing and Communications (HPCC'13), 1519-1527, November 2013. IEEE
QbDJ: A Novel Framework for Handling Skew in Parallel Join Processing on Distributed Memory
Proc. 15th IEEE International Conference on High Performance Computing and Communications (HPCC'13), 1519-1527, November 2013. IEEE
- KurzfassungAbstract
The performance of parallel distributed data management systems becomes increasingly important with the rise of Big Data. Parallel joins have been widely studied both in the parallel processing and the database communities. Nevertheless, most of the algorithms so far developed do not consider the data skew, which naturally exists in various applications. State of the art methods designed to handle this problem are based on extensions to either of the two prevalent conventional approaches to parallel joins - the hash-based and duplication-based frameworks. In this paper, we introduce a novel parallel join framework, query-based distributed join (QbDJ), for handling data skew on distributed architectures. Further, we present an efficient implementation of the method based on the asynchronous partitioned global address space (APGAS) parallel programming model. We evaluate the performance of our approach on a cluster of 192 cores (16 nodes) and datasets of 1 billion tuples with different skews. The results show that the method is scalable, and also runs faster with less network communication compared to state-of-art PRPD approach in [1] under high data skew. - Weitere Informationen unter:Further Information: Link
- Projekt:Project: DIAMOND
- Forschungsgruppe:Research Group: Information Systems„Information Systems“ befindet sich nicht in der Liste (Computational Logic, Automatentheorie, Wissensverarbeitung, Knowledge-Based Systems, Knowledge Systems, Wissensbasierte Systeme, Logische Programmierung und Argumentation, Algebra und Diskrete Strukturen, Knowledge-aware Artificial Intelligence, Algebraische und logische Grundlagen der Informatik) zulässiger Werte für das Attribut „Forschungsgruppe“.Knowledge-Based Systems
@inproceedings{CKWT2013,
author = {Long Cheng and Spyros Kotoulas and Tomas E. Ward and Georgios
Theodoropoulos},
title = {QbDJ: A Novel Framework for Handling Skew in Parallel Join
Processing on Distributed Memory},
booktitle = {Proc. 15th {IEEE} International Conference on High Performance
Computing and Communications (HPCC'13)},
publisher = {IEEE},
year = {2013},
month = {November},
pages = {1519-1527},
doi = {10.1109/HPCC.and.EUC.2013.214}
}
