Stochastic recursive algorithms for networked systems with delay and random switching : multiscale formulations and asymptotic properties

Motivated by consensus control of networked systems with communication latency and randomly switching topologies, this paper studies stochastic approximation (SA) algorithms for systems with time delays and randomly switching dynamics. To accommodate realistic time delay systems, our formulation of the discrete-time systems does not impose bounds on delays when the sampling intervals become small. The switching dynamics are modeled as a finite-state Markov chain. The transition probability matrix of the Markov chain contains a small parameter, termed the transition frequency parameter, while the SA algorithm defines its updating speed by another small parameter called the adaptation stepsize. The interplay of the two parameters introduces multiscale system dynamics, whose interaction with time delay further complicates system analysis. Using weak convergence methods, convergence of the algorithm is obtained. The limit behavior of the scaled estimation errors is also analyzed. It is shown that, depending on relative scales between the transition frequency and adaptation stepsize, the SA algorithms demonstrate fundamentally different asymptotic behaviors. Such behaviors are rigorously characterized and illustrated by simulation.