Optimization for Data-Driven Learning and Control

Volume 108, Issue 11 | November 2020

Guest Editors:

Usman A. Khan, Waheed U. Bajwa, Angelia Nedić, Michael G. Rabbat, and Ali H. Sayed

Special Issue Papers

Scanning the Issue

By U. A. Khan, W. U. Bajwa, A. Nedi ́c, M. G. Rabbat, and A. H. Sayed

A General Framework for Decentralized Optimization With First-Order Methods

By R. Xin, S. Pu, A. Nedi ́c, and U. A. Khan

This article presents a general framework for distributed first-order methods, for minimizing a finite sum of functions, that is applicable to both undirected and directed graphs. Such problems have found a significant interest in control, signal processing, and estimation, and more recently in large-scale data science and machine learning problems.

Decentralized Zeroth-Order Constrained Stochastic Optimization Algorithms: Frank–Wolfe and Variants With Applications to Black-Box Adversarial Attacks

By A. K. Sahu and S. Kar

This article presents an overview of the recent work in the area of distributed zeroth-order optimization, focusing on constrained optimization settings and algorithms built around the Frank–Wolfe framework.

Stochastic Quasi-Newton Methods

By A. Mokhtari and A. Ribeiro

This article discusses recent developments to accelerate convergence of stochastic optimization through the exploitation of second-order information and shows applications in the context of predicting the click-through rate of an advertisement displayed in response to a specific search engine query.

Primal–Dual Methods for Large-Scale and Distributed Convex Optimization and Data Analytics

By D. Jakoveti ́c, D. Bajovi ́c, J. Xavier, and J. M. F. Moura

This article focuses on the augmented Lagrangian method (ALM), where a constrained optimization problem is solved with a series of unconstrained subproblems, with respect to the original (primal) variable, while the constraints are controlled via dual variables.

Distributed Optimization, Averaging via ADMM, and Network Topology

By G. França and J. Bento

This article reviews recent research quantifying the influence of the network topology on the convergence behavior of distributed methods and further explores the connections between the alternating direction method of multipliers (ADMM) and lifted Markov.

Distributed Optimization for Robot Networks: From Real-Time Convex Optimization to Game-Theoretic Self-Organization

By H. Jaleel and J. S. Shamma

This article presents a collection of state-of-the-art results for distributed optimization problems arising in the context of robot networks, with a focus on two special classes of problems, namely, real-time path planning for multirobot systems and self-organization in multirobot systems using game-theoretic approaches.

Variance-Reduced Methods for Machine Learning

By R. M. Gower, M. Schmidt, F. Bach, and P. Richtárik

This article discusses stochastic variance-reduced optimization methods for problems where multiple passes through batch training data sets are allowed.

Scaling-Up Distributed Processing of Data Streams for Machine Learning

By M. Nokleby, H. Raja, and W. U. Bajwa

This article reviews recently developed methods that focus on distributed training of large-scale machine learning models from streaming data in the compute-limited and bandwidth-limited regimes, with an emphasis on convergence analysis that explicitly accounts for the mismatch between computation, communication, and streaming rates, and that provides sufficient conditions for order-optimum convergence.

Advances in Asynchronous Parallel and Distributed Optimization

By M. Assran, A. Aytekin, H. R. Feyzmahdavian, M. Johansson, and M. G. Rabbat

This article focuses on asynchronous parallel and distributed methods for large-scale optimization problems in machine learning, where the processors may maintain an inconsistent view of the optimization variables.

Time-Varying Convex Optimization: Time-Structured Algorithms and Applications

By A. Simonetto, E. Dall’Anese, S. Paternain, G. Leus, and G. B. Giannakis

This article reviews a broad class of algorithms for time- varying optimization with an emphasis on both algorithmic development and performance analysis.

Graph Learning Under Partial Observability

By V. Matta, A. Santos, and A. H. Sayed

This article examines the network tomography problem and considers the question: How much information can one glean about the underlying graph topology by observing the behavior of certain distributed optimization methods over the graph nodes?

Accelerated First-Order Optimization Algorithms for Machine Learning

By H. Li, C. Fang, and Z. Lin

This article provides a comprehensive survey of accelerated first-order methods with a particular focus on stochastic algorithms and further introduces some recent developments on accelerated methods for nonconvex optimization problems.