Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features.
Also cited · not yet reviewed (2)
- Instance discrimination2018 · cited 20×, 2 in Method“To improve the inferior efficiency, Wu et al. cvpr18nce propose to set up a memory bank to store the instance features 𝐟if_{i} calculated in the previous step.”From this paper · §Proposed Method
- Knowledge Distillation2015 · cited 1×, 1 in Method“where τ\tau is the temperature parameter controlling the concentration level of the sample distribution arxiv15temp. 𝐯iT𝐟jv_{i}^{T}{f_{j}} measures the cosine similarity between the feature 𝐟jf_{j} and the ii-th memo…”From this paper · §Proposed Method
Led to
- MoCo2019 · cited 6×, 5 in Method“As the focus of this paper is not on designing a new pretext task, we use a simple one mainly following the instance discrimination task in Wu2018a, to which some recent works Ye2019; Bachman2019 are related.”From MoCo · §Method
Abstract
This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space. Motivated by the positive concentrated and negative separated properties observed from category-wise supervised learning, we propose to utilize the instance-wise supervision to approximate these properties, which aims at learning data augmentation invariant and instance spread-out features. To achieve this goal, we propose a novel instance based softmax embedding method, which directly optimizes the `real' instance features on top of the softmax function. It achieves significantly faster learning speed and higher accuracy than all existing methods. The proposed method performs well for both seen and unseen testing categories with cosine similarity. It also achieves competitive performance even without pre-trained network over samples from fine-grained categories.