SLA-aware and deadline constrained profit optimization for cloud resource management in big data analytics-as-a-service platforms

Discovering optimal data analytics solutions to extract value from data for better and faster decision making is essential for many application domains, especially in the big data era. Big data analytics typically requires a tremendous amount of computational resources to process large data volumes that can be very expensive and time consuming. Our research focuses on providing optimization solutions for Analytics-as-a-Service (AaaS) platforms that automatically and elastically provision cloud resources to execute queries guaranteeing Service Level Agreements (SLAs) across a range of Quality of Service (QoS) requirements. We propose admission control and resource scheduling algorithms for AaaS platforms to maximize profits while providing time-minimized query execution plans to meet user demands and expectations. To enable timely responses as required for many domains, the algorithms utilize data splitting-based query admission and resource scheduling offering parallel processing on the split datasets. Extensive experiments are conducted to evaluate the algorithm performance compared to state-of-the-art optimization algorithms. Experimental results show that our algorithms perform significantly better from a range of perspectives, including increasing query admission rates and creating higher profits, whilst supporting efficient resource configurations that are able to support big data processing demands under tight deadlines.