Distributed asynchronous deterministic and stochastic gradient optimization algorithms
John N. TsitsiklisDepartment of Electrical Engineering, Laboratory for Electromagnetic Research, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USADimitri P. BertsekasDepartment of Electrical Engineering, Laboratory for Electromagnetic Research, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USAMichael AthansDepartment of Electrical Engineering, Laboratory for Electromagnetic Research, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, USA
1986en
ABI
Abstract
We present a model for asynchronous distributed computation and then proceed to analyze the convergence of natural asynchronous distributed versions of a large class of deterministic and stochastic gradient-like algorithms. We show that such algorithms retain the desirable convergence properties of their centralized counterparts, provided that the time between consecutive interprocessor communications and the communication delays are not too large.
Identifiers
Citations and references
Cited by 90 references