Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
Data can be represented as a vector in a continuous space, but often it is concentrated in a narrow area. Can it be improved?
Mike Erlihson
Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration
Augmenting images training data to add variability doesn't always end well since the augmentation changes the semantic content of the image. Here's how to solve it.
Perceiver: General Perception with Iterative Attention
To overcome Transformers' squared complexity (w.r.t input length), the Perceiver article here offers a novel method to learn the QKV matrices. Check it out!
Discriminator Rejection Sampling
A summary of a method to improve GAN generated images by exploiting accumulated "information" from the Discriminator during the GAN training process.
Sharpness-Aware Minimization for Efficiently Improving Generalization
This very relevant article suggests an unusual method to improve neural network regularization abilities. I believe that this method has a great potential to enter the standard neural network training toolkit. I was also impressed by the detailed comparison to other methods.