Self-supervised methods in representation learning with residual networks (ResNets) have made strong progress in recent years, but still trail state-of-the-art supervised learning performance on ImageNet classification benchmarks. …

Fine-tuning pretrained large-scale deep neural networks (NN) for downstream tasks has become the status quo in the deep learning community. …

This year marks the 10th anniversary of the epoch-making paperImageNet Classification with Deep Convolutional Neural Networks and the associated model’s convincing victory in the ImageNet Large Scale Visual Recognition Challenge. …

Running embedded machine learning (ML) systems on edge devices has become increasingly attractive in recent years, as migrating such systems from the cloud can improve privacy, latency, security and accessibility. …

Most AI researchers agree the trusty keyboard will not be the preferred human-machine interface of the future. While smart device-based voice interfaces are now increasingly popular, direct, brain-computer interfaces (BCIs) have tremendous potential in terms of speed and efficiency, with possible applications in neuroscience research, medical devices, and even videogames…

The practice of fine-tuning pretrained large neural language models (LMs) for specific downstream tasks has enabled countless performance breakthroughs in natural language processing (NLP) in recent years. This paradigm has also inspired machine learning researchers to explore such models’ ability to generalize by avoiding the memorization of their training data.

…

The emergence in recent years of powerful vision-language pretraining models has significantly boosted performance on a range of image-to-text generation tasks. The development of large-scale pretraining models for text-to-image synthesis however remains relatively underexplored in the machine learning research community.

In the new paper ERNIE-ViLG: Unified Generative Pre-training for Bidirectional…

Studies have shown that humans naturally learn languages at an early age largely through linguistic interactions with caregivers such as parents and teachers. …