Hybrid workflow provisioning and scheduling on edge cloud computing using a gradient descent search approach

The dramatic growth of the Internet of Things (IoT) technology in many application domains, ranging from intelligent video surveillance, smart retail to the Internet-of-Vehicles brings new computation challenges for rationalized utilization of computing resources. IoT application execution refers to hybrid processing model of stream and batch to achieve data analytics objectives. Hybrid workflow execution combines the challenges of latency-sensitive and resource-intensive processing. To resolve these challenges, we proposed a two stages hybrid workflow scheduling framework on edge cloud computing. In the first stage, we proposed a resource estimation algorithm based on a linear optimization approach, the gradient descent search (GDS) and in the second stage, we adopted a cluster-based provisioning and scheduling technique on heterogeneous edge cloud resources. This work provides a multi-objective optimization model for execution time and monetary cost under constraints of deadline and throughput. Results demonstrated the framework performance in controlling the execution of hybrid workflows by an efficient tuning for stream processing parameters, such as arrival rate and processing throughput. Under working constraints, the proposed scheduler provides significant improvement for large hybrid workflows in terms of execution time and monetary cost with an average of 8% and 35%, respectively.