Linked data partitioning for RDF processing on Apache Spark

Amir Hossein Atashkar; Nasser Ghadiri; Mehdi Joodaki

doi:10.1109/ICWR.2017.7959308

Linked data partitioning for RDF processing on Apache Spark

Amir Hossein Atashkar, Nasser Ghadiri, Mehdi Joodaki

Western Sydney University

Research output: Chapter in Book / Conference Paper › Conference Paper › peer-review

5 Citations (Scopus)

Abstract

![CDATA[RDF models are widely used in the web of data due to their flexibility and similarity to graph patterns. Because of the growing use of RDFs, their volumes and contents are increasing. Therefore, processing of such massive amount of data on a single machine is not efficient enough, because of the response time and limited hardware resources. A common approach to overcome this limitation is cluster processing and huge datasets could benefit distributed cluster processing on Apache Hadoop. Because of using too much of hard disks, the processing time is usually inadequate. In this paper, we propose a partitiong approach based on Apache Spark for rapid processing of RDF data models. A key feature of Apache Spark is using main memory instead of hard disk, so the speed of data processing in our method is improved. We have evaluated the proposed method by runing SQL queris on RDF data which partitioned on the cluster and demonstrates improved performance.]]

Original language	English
Title of host publication	Proceedings of the 3rd International Conference on Web Research (ICWR), Tehran, Iran, 19-20 April, 2017
Publisher	IEEE
Pages	73-77
Number of pages	5
ISBN (Print)	9781538604205
DOIs	https://doi.org/10.1109/ICWR.2017.7959308
Publication status	Published - 2017
Event	International Conference on Web Research - Duration: 19 Apr 2017 → …

Conference

Conference	International Conference on Web Research
Period	19/04/17 → …

Access to Document

10.1109/ICWR.2017.7959308

Cite this

@inproceedings{2a64c66937394e8ba8d2403ecfcced90,

title = "Linked data partitioning for RDF processing on Apache Spark",

abstract = "![CDATA[RDF models are widely used in the web of data due to their flexibility and similarity to graph patterns. Because of the growing use of RDFs, their volumes and contents are increasing. Therefore, processing of such massive amount of data on a single machine is not efficient enough, because of the response time and limited hardware resources. A common approach to overcome this limitation is cluster processing and huge datasets could benefit distributed cluster processing on Apache Hadoop. Because of using too much of hard disks, the processing time is usually inadequate. In this paper, we propose a partitiong approach based on Apache Spark for rapid processing of RDF data models. A key feature of Apache Spark is using main memory instead of hard disk, so the speed of data processing in our method is improved. We have evaluated the proposed method by runing SQL queris on RDF data which partitioned on the cluster and demonstrates improved performance.]]",

author = "Atashkar, {Amir Hossein} and Nasser Ghadiri and Mehdi Joodaki",

year = "2017",

doi = "10.1109/ICWR.2017.7959308",

language = "English",

isbn = "9781538604205",

pages = "73--77",

booktitle = "Proceedings of the 3rd International Conference on Web Research (ICWR), Tehran, Iran, 19-20 April, 2017",

publisher = "IEEE",

note = "International Conference on Web Research ; Conference date: 19-04-2017",

}

TY - GEN

T1 - Linked data partitioning for RDF processing on Apache Spark

AU - Atashkar, Amir Hossein

AU - Ghadiri, Nasser

AU - Joodaki, Mehdi

PY - 2017

Y1 - 2017

N2 - ![CDATA[RDF models are widely used in the web of data due to their flexibility and similarity to graph patterns. Because of the growing use of RDFs, their volumes and contents are increasing. Therefore, processing of such massive amount of data on a single machine is not efficient enough, because of the response time and limited hardware resources. A common approach to overcome this limitation is cluster processing and huge datasets could benefit distributed cluster processing on Apache Hadoop. Because of using too much of hard disks, the processing time is usually inadequate. In this paper, we propose a partitiong approach based on Apache Spark for rapid processing of RDF data models. A key feature of Apache Spark is using main memory instead of hard disk, so the speed of data processing in our method is improved. We have evaluated the proposed method by runing SQL queris on RDF data which partitioned on the cluster and demonstrates improved performance.]]

AB - ![CDATA[RDF models are widely used in the web of data due to their flexibility and similarity to graph patterns. Because of the growing use of RDFs, their volumes and contents are increasing. Therefore, processing of such massive amount of data on a single machine is not efficient enough, because of the response time and limited hardware resources. A common approach to overcome this limitation is cluster processing and huge datasets could benefit distributed cluster processing on Apache Hadoop. Because of using too much of hard disks, the processing time is usually inadequate. In this paper, we propose a partitiong approach based on Apache Spark for rapid processing of RDF data models. A key feature of Apache Spark is using main memory instead of hard disk, so the speed of data processing in our method is improved. We have evaluated the proposed method by runing SQL queris on RDF data which partitioned on the cluster and demonstrates improved performance.]]

UR - https://hdl.handle.net/1959.7/uws:68938

U2 - 10.1109/ICWR.2017.7959308

DO - 10.1109/ICWR.2017.7959308

M3 - Conference Paper

SN - 9781538604205

SP - 73

EP - 77

BT - Proceedings of the 3rd International Conference on Web Research (ICWR), Tehran, Iran, 19-20 April, 2017

PB - IEEE

T2 - International Conference on Web Research

Y2 - 19 April 2017

ER -

Linked data partitioning for RDF processing on Apache Spark

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this