Zhuo's Blog

Tutorial: https://www.youtube.com/watch?v=eYhvJR4zFUM

Segment Anything

Training language models to follow instructions with human feedback

BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision

Something like

• CLIPasso
  The goal is that given an image, generate a stroke image with several simple curves to represent this image.
  Based on some initial points, curves can be drawn to generate a simple stroke image. The function from points to the stroke image is differentiable. Then to make sure the stroke image represent the given image, two kinds of losses are designed. One loss is based on CLIP, that the CLIP image embedding from the two images should be similar. The other is geometric loss, that the features in the shallow layers of a VGG/ResNet of these two images should be similar.

1. Language-driven Semantic Segmentation

Overview

Let’s quickly go through the new models related to network compression published at CVPR 2019, ICLR 2019 and ICML 2019. Some works needs to be read and understood more carefully.

Overview

In this post, we are going to look at the following neural network models: MobileNet v1^[1] & v2^[2], SqueezeNet^[3], ShuffleNet v1^[4] & v2^[5], NasNet^[6]. We consider the following questions: