Skip to main content

DeepFashion2: A Versatile Benchmark for Fashion Image Understanding


-By Yuying Ge1, Ruimao Zhang
, Lingyun Wu2, Xiaogang Wang
, Xiaoou Tang1, and Ping Luo
The Chinese University of Hong Kong
2SenseTime Research


Even as fashion image analysis gets more traction from today’s image recognition researchers, understanding fashion images remains challenging for real-world applications due to large deformations, occlusions, and discrepancies in clothing across domains and between consumer and commercial images.
DeepFashion is a large-scale clothes database introduced last year by a research team from the Chinese University of Hong Kong (CUHK). The dataset contains over 800k diverse fashion images, each labeled with 50 categories, 1,000 descriptive attributes, bounding boxes and clothing landmarks.
DeepFashion was a solid foundation, but it left a number of areas for improvement. It was limited to a single clothing-item per image, sparse landmarks (4~8 only), and had no per-pixel masks. CUHK researchers recently teamed up with Chinese AI giant SenseTime to develop a greatly improved iteration in DeepFashion2, a large-scale benchmark with comprehensive tasks and annotations of fashion image understanding.
DeepFashion2 contains 491K images of 13 popular clothing categories. A full spectrum of tasks are defined, including clothes detection and recognition, landmark and pose estimation, segmentation, as well as verification and retrieval. All these tasks are supported by rich annotations.
The dataset also includes a total of 801K images of pieces of clothing. Each item is labeled with scale, occlusion, zooming, viewpoint, bounding box, dense landmarks, and per-pixel mask. These items can be categorized as 43.8k clothing identities, where a clothing identity represents a class of apparel with nearly identical cuts, patterns, and designs. Images of the same clothing identities are taken from buyers and sellers, where an item from the buyer and an item from the seller forms a pair.
Researchers say the work makes three main contributions:
  1. Compared with other clothes datasets, DeepFashion2 annotations are at least 3.5× those of DeepFashion, 6.7× of ModaNet, and 8× of FashionAI.
  2. A full spectrum of tasks is carefully defined on the proposed dataset.
  3. Researchers extensively evaluated Mask R-CNN with DeepFashion2. A novel Match R-CNN is also proposed to aggregate all the learned features from clothes categories, poses, and masks to solve clothing image retrieval in an end-to-end manner.
333The research team believes the rich data and labels of DeepFashion2 will accelerate the development of future algorithms to understand fashion images. The paper DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images is on arXiv

Comments

Popular posts from this blog

DDSP: DIFFERENTIABLE DIGITAL SIGNAL PROCESSING

 -By Google Research {jesseengel,hanoih,gcj,adarob}@google.com Paper Link Code Demo Abstract Most generative models of audio directly generate samples in one of two domains: time or frequency. While sufficient to express any signal, these representations are inefficient, as they do not utilize existing knowledge of how sound is generated and perceived. A third approach (vocoders/synthesizers) successfully incorporates strong domain knowledge of signal processing and perception, but has been less actively researched due to limited expressivity and difficulty integrating with modern auto-differentiation-based machine learning methods. In this paper, we introduce the Differentiable Digital Signal Processing (DDSP) library, which enables direct integration of classic signal processing elements with deep learning methods. Focusing on audio synthesis, we achieve high-fidelity generation without the need for large autoregressive models or adversarial losses, demonstrating that DDSP enable...

Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network

By - Siyang Lu, Xiang Wei, Yandong Li, Liqiang Wang Department of Computer Science, University of Central Florida, Orlando, FL, USA School of Software Engineering, Beijing Jiaotong University, China Paper Link Previous week we did a scan of the survey of Anomaly detection   , that covers breath of the topic (Types of Anomalies, Types of Models and their applications). In this summary paper, we look in to the core Heavy lifting part of Detecting Anomaly in Big Data System Logs Using Convolutional Neural Network. Abstract Nowadays, big data systems are being widely adopted by many domains for offering effective data solutions, such as manufacturing, healthcare, education, and media. Big data systems produce tons of unstructured logs that contain buried valuable information. However, it is a daunting task to manually unearth the information and detect system anomalies. A few automatic methods have been developed, where the cutting-edge machine lear...

Capsule Neural Networks

-By Sara Sabour, Nicholas Frosst,  Geoffrey E. Hinton  Google Brain Toronto Paper Link Abstract   A capsule is a group of neurons whose activity vector represents the instantiation parameters of a specific type of entity such as an object or an object part. We use the length of the activity vector to represent the probability that the entity exists and its orientation to represent the instantiation parameters. Active capsules at one level make predictions, via transformation matrices, for the instantiation parameters of higher-level capsules. When multiple predictions agree, a higher level capsule becomes active. We show that a discrimininatively trained, multi-layer capsule system achieves state-of-the-art performance on MNIST and is considerably better than a convolutional net at recognizing highly overlapping digits. To achieve these results we use an iterative routing-by-agreement mechanism: A lower-level capsule prefers to send its output to higher...