Advances in Machine Learning and Deep Neural Networks

Volume 109, Issue 5 | May 2021

Guest Editors:

Rama Chellappa, Sergios Theodoridis, and Andre van Schaik

Special Issue Papers

Scanning the Issue

By R. Chellappa, S. Theodoridis, and A. van Schaik

Toward Causal Representation Learning

By B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio

This article reviews fundamental concepts of causal inference and relates them to crucial open problems of machine learning, including transfer learning and generalization, thereby assaying how causality can contribute to modern machine learning research.

Optimism in the Face of Adversity: Understanding and Improving Deep Learning Through Adversarial Robustness

By G. Ortiz-Jiménez, A. Modas, S.-M. Moosavi-Dezfooli, and P. Frossard

This article presents an in-depth review of the field of adversarial robustness in deep learning and provides a self-contained introduction to its main notions.

Graph Neural Networks: Architectures, Stability, and Transferability

By L. Ruiz, F. Gama, and A. Ribeiro

This article deals with graph neural networks (GNNs) that operate on data supported on graphs.

Mathematical Models of Overparameterized Neural Networks

By C. Fang, H. Dong, and T. Zhang

The focus of this article is on theoretical developments concerning the analysis of overparameterized neural networks.

Mad Max: Affine Spline Insights Into Deep Learning

By R. Balestriero and R. G. Baraniuk

In this article, the bridge between deep networks (DNs) and approximation theory via spline functions and operators is rigorously established.

Tropical Geometry and Machine Learning

By P. Maragos, V. Charisopoulos, and E. Theodosis

This article deals with tropical geometry that has recently emerged as a tool in the analysis and extension of several classes of problems in both classical machine learning and deep learning.

A Unifying Review of Deep and Shallow Anomaly Detection

By L. Ruff, J. R. Kauffmann, R. A. Vandermeulen, G. Montavon, W. Samek, M. Kloft, T. G. Dietterich, and K.-R. Müller

This article deals with application of deep learning techniques to anomaly detection. Furthermore, connections between classic “shallow” and novel deep approaches are established, and it is shown how this relation might cross-fertilize or extend both directions.

Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

By J. Park, S. Samarakoon, A. Elgabli, J. Kim, M. Bennis, S.-L. Kim, and M. Debbah

The goal of this article is to provide a holistic overview of relevant communication and machine learning (ML) principles, and thereby present communication-efficient and distributed ML frameworks with selected use cases.

A Review of Deep Learning in Medical Imaging: Imaging Traits, Technology Trends, Case Studies With Progress Highlights, and Future Promises

By S. K. Zhou, H. Greenspan, C. Davatzikos, J. S. Duncan, B. van Ginneken, A. Madabhushi, J. L. Prince, D. Rueckert, and R. M. Summers

In this article, the authors highlight both clinical needs and technical challenges in medical imaging and describe how emerging trends in deep learning are addressing these issues.

Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications

By M.-Y. Liu, X. Huang, J. Yu, T.-C. Wang, and A. Mallya

This article provides an overview of generative adversarial networks (GANs) with a special focus on algorithms and applications for visual synthesis.

Tensor Methods in Computer Vision and Deep Learning

By Y. Panagakis, J. Kossaifi, G. G. Chrysos, J. Oldfield, M. A. Nicolaou, A. Anandkumar, and S. Zafeiriou

This article provides an overview of tensors and tensor methods in the context of representation learning and deep learning, with a particular focus on visual data analysis and computer vision applications.

Computational Media Intelligence: Human-Centered Machine Analysis of Media

By K. Somandepalli, T. Guha, V. R. Martinez, N. Kumar, H. Adam, and S. Narayanan

The topic treated in this article is the application of deep learning algorithms, combined with audio-visual signal processing, to analyze entertainment media such as film/TV.

Advancing Neuromorphic Computing With Loihi: A Survey of Results and Outlook

By M. Davies, A. Wild, G. Orchard, Y. Sandamirskaya, G. A. Fonseca Guerra, P. Joshi, P. Plank, and S. R. Risbud

This article provides a survey of results obtained to date with Intel’s Loihi across the major algorithmic domains under study, including deep-learning approaches as well as novel approaches that aim to harness the key features of spike-based neuromorphic hardware more directly.

Brain-Inspired Learning on Neuromorphic Substrates

By F. Zenke and E. O. Neftci

This article provides a mathematical framework for the design of practical online learning algorithms for neuromorphic substrates.