Open World Entity Segmentation
We introduce a new image segmentation task, called Entity Segmentation (ES), which aims to segment all visual entities (objects and …
DocEdit: Language-guided Document Editing
Professional document editing tools require a certain level of expertise to perform complex edit operations. To make editing tools …
User-Entity Differential Privacy in Learning Natural Language Models
In this paper, we introduce a novel concept of user-entity differential privacy (UeDP) to provide formal privacy protection …
LayerDoc: Layer-wise Extraction of Spatial Hierarchical Structure in Visually-Rich Documents
Digital documents often contain images and scanned text. Parsing such visually-rich documents is a core task for automating document …
MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding
Document images are a ubiquitous source of data where the text is organized in a complex hierarchical structure ranging from fine …
Delving into OOD Detection with Vision-Language Representations
Recognizing out-of-distribution (OOD) samples is critical for machine learning systems deployed in the open world. The vast majority of …
DocLayoutTTS: Dataset and Baselines for Layout-informed Document-level Neural Speech Synthesis
We propose a new task of synthesizing speech directly from semi-structured documents where the extracted text tokens from OCR systems …
DocTime: A Document-level Temporal Dependency Graph Parser
We introduce DocTime - a novel temporal dependency graph (TDG) parser that takes as input a text document and produces a temporal …
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns
This work presents one of the first comprehensive studies on different sparse attention patterns in Transformer models. We first …
FedKC: Federated Knowledge Composition for Multilingual Natural Language Understanding
Multilingual natural language understanding, which aims to comprehend multilingual documents, is an important task. Existing efforts …
Unsupervised Cross-lingual Image Captioning
Most recent image captioning works are conducted in English as the majority of image-caption datasets are in English. However, there …
Interactive Image Generation with Natural-Language Feedback
Using natural-language feedback to guide image generation and manipulation can greatly lower the required efforts and skills. This …
Unified Pretraining Framework for Document Understanding
Document intelligence automates the extraction of information from documents and supports many business applications. Recent …
Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU models
Recent studies indicate that NLU models are prone to rely on shortcut features for prediction. As a result, these models could …
SelfDoc: Self-Supervised Document Representation Learning
We propose SelfDoc, a task-agnostic pre-training framework for document image analysis. Because documents are multimodal displays and …
Multi-Scale Aligned Distillation for Low-Resolution Detection
In instance-level detection tasks (e.g., object detection), reducing input resolution is an easy option to improve runtime efficiency. …
Exploiting Semantic Embedding and Visual Feature for Facial Action Unit Detection
Recent study on detecting facial action units (AU) has utilized auxiliary information (i.e., facial landmarks, relationship among AUs …
Self-Supervised Relationship Probing
Structured representations of images according to visual relationships are beneficial for many vision and vision-language applications. …
Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction
The explosion of video data on the internet requires effective and efficient technology to generate captions automatically for people …
Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioning
The prevalent approach to the image captioning is an encoder-decoder framework, where the combination of convolutional neural networks …
Resilient Load Restoration in Microgrids Considering Mobile Energy Storage Fleets: A Deep Reinforcement Learning Approach
Mobile energy storage systems (MESSs) provide mobility and flexibility to enhance distribution system resilience. The paper proposes a …
Bridging images and natural language with deep learning
We, as humans, can easily use our vision and language capabilities to accomplish a wide variety of tasks that combine the image and the …
Watch It Twice: Video Captioning with a Refocused Video Encoder
With the rapid growth of video data and the increasing demands of various applications such as intelligent video search and assistance …
Unpaired Image Captioning via Scene Graph Alignments
Most of the existing deep learning based image captioning methods are fully-supervised models, which require large-scale paired …
Scene Graph Generation with External Knowledge and Image Reconstruction
Scene graph generation has received growing attention with advancement image understanding tasks such as object detection, attributes …
Unpaired Image Captioning by Language Pivoting
Image captioning is a multimodal task involving computer vision and natural language processing, where the goal is to learn a mapping …
Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models
Textual-visual cross-modal retrieval has been a hot research topic in both computer vision and natural language processing communities. …
Stack-Captioning: Coarse-to-Fine Learning for Image Captioning
The existing image captioning approaches typically train a one-stage sentence decoder, which is difficult to generate rich fine-grained …
An Empirical Study of Language CNN for Image Captioning
Language Models based on recurrent neural networks have dominated recent image caption generation tasks. In this paper, we introduce a …
Recent Advances in Convolutional Neural Networks
In this paper, we provide a broad survey of the recent advances in convolutional neural networks. We detailize the improvements of CNN …
