Publications | Jiuxiang Gu

Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspective (arXiv 2024)

Jiuxiang Gu, Yingyu Liang, Zhenmei Shi, Zhao Song, Yufa Zhou

Toward Infinite-Long Prefix in Transformer (arXiv 2024)

Jiuxiang Gu, Yingyu Liang, Zhenmei Shi, Zhao Song, Chiwun Yang

Toffee: Efficient Million-Scale Dataset Construction for Subject-Driven Text-to-Image Generation (arXiv 2024)

Yufan Zhou, Ruiyi Zhang, Kaizhi Zheng, Nanxuan Zhao, Jiuxiang Gu, Zichao Wang, Xin Eric Wang, Tong Sun

Tensor attention training: Provably efficient learning of higher-order transformers (arXiv 2024)

Jiuxiang Gu, Yingyu Liang, Zhenmei Shi, Zhao Song, Yufa Zhou

TRINS: Towards Multimodal Language Models that Can Read (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024)

Ruiyi Zhang, Yanzhe Zhang, Jian Chen, Yufan Zhou, Jiuxiang Gu, Changyou Chen, Tong Sun

Self-Cleaning: Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances (Findings of the Association for Computational Linguistics (NAACL) 2024)

Zhendong Chu, Ruiyi Zhang, Tong Yu, Rajiv Jain, Vlad Morariu, Jiuxiang Gu, Ani Nenkova

Selective reflection-tuning: Student-selected data recycling for llm instruction-tuning (Findings of the Association for Computational Linguistics (ACL) 2024)

Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Jiuxiang Gu, Tianyi Zhou

SOHES: Self-supervised Open-world Hierarchical Entity Segmentation (The Twelfth International Conference on Learning Representations 2024)

Shengcao Cao, Jiuxiang Gu, Jason Kuen, Hao Tan, Ruiyi Zhang, Handong Zhao, Ani Nenkova, Liang-Yan Gui, Tong Sun, Yu-Xiong Wang

MMR: Evaluating Reading Ability of Large Multimodal Models (arXiv 2024)

Jian Chen, Ruiyi Zhang, Yufan Zhou, Ryan Rossi, Jiuxiang Gu, Changyou Chen

Lrm: Large reconstruction model for single image to 3d (The Twelfth International Conference on Learning Representations (ICLR) 2024)

Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, Hao Tan

LLaVA-Read: Enhancing Reading Ability of Multimodal Language Models (arXiv 2024)

Ruiyi Zhang, Yufan Zhou, Jian Chen, Jiuxiang Gu, Changyou Chen, Tong Sun

ImageFolder: Autoregressive Image Generation with Folded Tokens (arXiv 2024)

Xiang Li, Hao Chen, Kai Qiu, Jason Kuen, Jiuxiang Gu, Bhiksha Raj, Zhe Lin

Fourier circuits in neural networks: Unlocking the potential of large language models in mathematical reasoning and modular arithmetic (arXiv 2024)

Jiuxiang Gu, Chenyang Li, Yingyu Liang, Zhenmei Shi, Zhao Song, Tianyi Zhou

Fast John Ellipsoid Computation with Differential Privacy Optimization (arXiv 2024)

Jiuxiang Gu, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song, Junwei Yu

Exploring the frontiers of softmax: Provable optimization, applications in diffusion model, and beyond (arXiv 2024)

Jiuxiang Gu, Chenyang Li, Yingyu Liang, Zhenmei Shi, Zhao Song

DocSynthv2: A Practical Autoregressive Modeling for Document Generation (arXiv 2024)

Sanket Biswas, Rajiv Jain, Vlad I Morariu, Jiuxiang Gu, Puneet Mathur, Curtis Wigington, Tong Sun, Josep Lladós

DocScript: Document-level Script Event Prediction (Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING) 2024)

Puneet Mathur, Vlad I Morariu, Aparna Garimella, Franck Dernoncourt, Jiuxiang Gu, Ramit Sawhney, Preslav Nakov, Dinesh Manocha, Rajiv Jain

Customization assistant for text-to-image generation (Proceedings of the IEEE onference on Computer Vision and Pattern Recognition 2024)

Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Tong Sun

Conv-basis: A new paradigm for efficient attention inference and gradient computation in transformers (arXiv 2024)

Jiuxiang Gu, Yingyu Liang, Heshan Liu, Zhenmei Shi, Zhao Song, Junze Yin

Commit: Coordinated instruction tuning for multimodal large language models (arXiv 2024)

Junda Wu, Xintong Li, Tong Yu, Yu Wang, Xiang Chen, Jiuxiang Gu, Lina Yao, Jingbo Shang, Julian McAuley

Category-Aware Active Domain Adaptation (Forty-first International Conference on Machine Learning 2024)

Wenxiao Xiao, Jiuxiang Gu, Hongfu Liu

ARTIST: Improving the Generation of Text-rich Images by Disentanglement (arXiv 2024)

Jianyi Zhang, Yufan Zhou, Jiuxiang Gu, Curtis Wigington, Tong Yu, Yiran Chen, Tong Sun, Ruiyi Zhang

ADOPD: A Large-Scale Document Page Decomposition Dataset (The Twelfth International Conference on Learning Representations 2024)

Jiuxiang Gu, Xiangxi Shi, Jason Kuen, Lu Qi, Ruiyi Zhang, Anqi Liu, Ani Nenkova, Tong Sun

A Multi-LLM Debiasing Framework (arXiv 2024)

Deonna M Owens, Ryan A Rossi, Sungchul Kim, Tong Yu, Franck Dernoncourt, Xiang Chen, Ruiyi Zhang, Jiuxiang Gu, Hanieh Deilamsalehy, Nedim Lipka

Reflection-tuning: Data recycling improves llm instruction-tuning (Workshop on Instruction Tuning and Instruction Following at NeurIPS 2023)

Ming Li, Lichang Chen, Jiuhai Chen, Shwai He, Heng Huang, Jiuxiang Gu, Tianyi Zhou

Llavar: Enhanced visual instruction tuning for text-rich image understanding (arXiv 2023)

Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, Tong Sun

LayerDoc: layer-wise extraction of spatial hierarchical structure in visually-rich documents (Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV) 2023)

Puneet Mathur, Rajiv Jain, Ashutosh Mehra, Jiuxiang Gu, Franck Dernoncourt, Quan Tran, Verena Kaynig-Fittkau, Ani Nenkova, Dinesh Manocha, Vlad I Morariu, others

DocEdit: language-guided document editing (Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) 2023)

Puneet Mathur, Rajiv Jain, Jiuxiang Gu, Franck Dernoncourt, Dinesh Manocha, Vlad I Morariu

Aims: All-inclusive multi-level segmentation for anything (Advances in Neural Information Processing Systems (NeurIPS) 2023)

Lu Qi, Jason Kuen, Weidong Guo, Jiuxiang Gu, Zhe Lin, Bo Du, Yu Xu, Ming-Hsuan Yang

A Critical Analysis of Document Out-of-Distribution Detection (Findings of the Association for Computational Linguistics (EMNLP) 2023)

Jiuxiang Gu, Yifei Ming, Yi Zhou, Jason Kuen, Vlad Morariu, Handong Zhao, Ruiyi Zhang, Nikolaos Barmpalios, Anqi Liu, Yixuan Li, others

User-Entity Differential Privacy in Learning Natural Language Models (IEEE International Conference on Big Data (Big Data) 2022)

Phung Lai, NhatHai Phan, Tong Sun, Rajiv Jain, Franck Dernoncourt, Jiuxiang Gu, Nikolaos Barmpalios

UNISON: Unpaired cross-lingual image captioning (Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) 2022)

Jiahui Gao, Yi Zhou, LH Philip, Shafiq Joty, Jiuxiang Gu

Towards language-free training for text-to-image generation (Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2022)

Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, Tong Sun

Tigan: Text-based interactive image generation and manipulation (Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) 2022)

Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Chris Tensmeyer, Tong Yu, Changyou Chen, Jinhui Xu, Tong Sun

Open-vocabulary instance segmentation via robust cross-modal pseudo-labeling (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022)

Dat Huynh, Jason Kuen, Zhe Lin, Jiuxiang Gu, Ehsan Elhamifar

Open world entity segmentation (IEEE Transactions on Pattern Analysis and Machine Intelligence 2022)

Lu Qi, Jason Kuen, Yi Wang, Jiuxiang Gu, Hengshuang Zhao, Philip Torr, Zhe Lin, Jiaya Jia

Meta spatio-temporal debiasing for video scene graph generation (European Conference on Computer Vision (ECCV 2022)

Li Xu, Haoxuan Qu, Jason Kuen, Jiuxiang Gu, Jun Liu

MGDoc: Pre-training with multi-granular hierarchy for document image understanding (Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP) 2022)

Zilong Wang, Jiuxiang Gu, Chris Tensmeyer, Nikolaos Barmpalios, Ani Nenkova, Tong Sun, Jingbo Shang, Vlad I Morariu

Learning the Visualness of Text Using Large Vision-Language Models (Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP) 2022)

Gaurav Verma, Ryan A Rossi, Christopher Tensmeyer, Jiuxiang Gu, Ani Nenkova

Learning adaptive axis attentions in fine-tuning: Beyond fixed sparse attention patterns (Findings of the Association for Computational Linguistics (ACL) 2022)

Zihan Wang, Jiuxiang Gu, Jason Kuen, Handong Zhao, Vlad Morariu, Ruiyi Zhang, Ani Nenkova, Tong Sun, Jingbo Shang

Improving the reliability for confidence estimation (European Conference on Computer Vision (ECCV) 2022)

Haoxuan Qu, Yanchao Li, Lin Geng Foo, Jason Kuen, Jiuxiang Gu, Jun Liu

FedKC: Federated knowledge composition for multilingual natural language understanding (Proceedings of the ACM Web Conference (ACM Web)) 2022)

Haoyu Wang, Handong Zhao, Yaqing Wang, Tong Yu, Jiuxiang Gu, Jing Gao

Ei-clip: Entity-aware interventional contrastive learning for e-commerce cross-modal retrieval (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022)

Haoyu Ma, Handong Zhao, Zhe Lin, Ajinkya Kale, Zhangyang Wang, Tong Yu, Jiuxiang Gu, Sunav Choudhary, Xiaohui Xie

Doctime: A document-level temporal dependency graph parser (Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics (NNACL) 2022)

Puneet Mathur, Vlad Morariu, Verena Kaynig-Fittkau, Jiuxiang Gu, Franck Dernoncourt, Quan Hung Tran, Ani Nenkova, Dinesh Manocha, Rajiv Jain

DocLayoutTTS: Dataset and Baselines for Layout-informed Document-level Neural Speech Synthesis. (INTERSPEECH 2022)

Puneet Mathur, Franck Dernoncourt, Quan Hung Tran, Jiuxiang Gu, Ani Nenkova, Vlad I Morariu, Rajiv Jain, Dinesh Manocha

Delving into out-of-distribution detection with vision-language representations (Advances in neural information processing systems (NeurIPS) 2022)

Yifei Ming, Ziyang Cai, Jiuxiang Gu, Yiyou Sun, Wei Li, Yixuan Li

Ca-ssl: Class-agnostic semi-supervised learning for detection and segmentation (European Conference on Computer Vision (ECCV 2022)

Lu Qi, Jason Kuen, Zhe Lin, Jiuxiang Gu, Fengyun Rao, Dian Li, Weidong Guo, Zhen Wen, Ming-Hsuan Yang, Jiaya Jia

Bit-aware randomized response for local differential privacy in federated learning ( 2022)

Phung Lai, Hai Phan, Li Xiong, Khang Tran, My Thai, Tong Sun, Franck Dernoncourt, Jiuxiang Gu, Nikolaos Barmpalios, Rajiv Jain

Unidoc: Unified pretraining framework for document understanding (Advances in Neural Information Processing Systems (NeurIPS) 2021)

Jiuxiang Gu, Jason Kuen, Vlad I Morariu, Handong Zhao, Rajiv Jain, Nikolaos Barmpalios, Ani Nenkova, Tong Sun

Towards interpreting and mitigating shortcut learning behavior of NLU models (Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NNACL) 2021)

Mengnan Du, Varun Manjunatha, Rajiv Jain, Ruchi Deshpande, Franck Dernoncourt, Jiuxiang Gu, Tong Sun, Xia Hu

Selfdoc: Self-supervised document representation learning (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021)

Peizhao Li, Jiuxiang Gu, Jason Kuen, Vlad I Morariu, Handong Zhao, Rajiv Jain, Varun Manjunatha, Hongfu Liu

Multi-scale aligned distillation for low-resolution detection (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2021)

Lu Qi, Jason Kuen, Jiuxiang Gu, Zhe Lin, Yi Wang, Yukang Chen, Yanwei Li, Jiaya Jia

Exploiting semantic embedding and visual feature for facial action unit detection (Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR 2021)

Huiyuan Yang, Lijun Yin, Yi Zhou, Jiuxiang Gu

Self-supervised relationship probing (Advances in Neural Information Processing Systems (NeurIPS) 2020)

Jiuxiang Gu, Jason Kuen, Shafiq Joty, Jianfei Cai, Vlad Morariu, Handong Zhao, Tong Sun

Resilient load restoration in microgrids considering mobile energy storage fleets: A deep reinforcement learning approach (2020 IEEE Power & Energy Society General Meeting (PESGM) 2020)

Shuhan Yao, Jiuxiang Gu, Huajun Zhang, Peng Wang, Xiaochuan Liu, Tianyang Zhao

Finding it at another side: A viewpoint-adapted matching encoder for change captioning (Proceedings of the European Conference on Computer Vision (ECCV) 2020)

Xiangxi Shi, Xu Yang, Jiuxiang Gu, Shafiq Joty, Jianfei Cai

Watch It Twice: Video Captioning with a Refocused Video Encoder (Proceedings of the ACM International Conference on Multimedia (MM) 2019)

Xiangxi Shi, Jianfei Cai, Shafiq Joty, Jiuxiang Gu

Unpaired Image Captioning via Scene Graph Alignments (Proceedings of the IEEE International Conference on Computer Vision (ICCV) 2019)

Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Handong Zhao, Xu Yang, Gang Wang

Scene graph generation with external knowledge and image reconstruction (Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) 2019)

Jiuxiang Gu, Handong Zhao, Zhe Lin, Sheng Li, Jianfei Cai, Mingyang Ling

Video Captioning with Boundary-aware Hierarchical Language Decoding and Joint Video Prediction (Neurocomputing 2018)

Xiangxi Shi, Jianfei Cai, Jiuxiang Gu, Shafiq Joty

Unpaired image captioning by language pivoting (Proceedings of the European Conference on Computer Vision (ECCV) 2018)

Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Gang Wang

Stack-Captioning: Coarse-to-Fine Learning for Image Captioning (Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI). 2018)

Jiuxiang Gu, Jianfei Cai, Gang Wang, Tsuhan Chen

Recent advances in convolutional neural networks (Pattern Recognition 2018)

Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, Gang Wang, Jianfei Cai, others

NTU ROSE Lab at TRECVID 2018: Ad-hoc Video Search and Video to Text. (TRECVID 2018)

Muhammet Bastan, Xiangxi Shi, Jiuxiang Gu, Zhao Heng, Chen Zhuo, Dennis Sng, Alex C Kot

Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models (Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018)

Jiuxiang Gu, Jianfei Cai, Shafiq Joty, Li Niu, Gang Wang

An empirical study of language cnn for image captioning (Proceedings of the IEEE International Conference on Computer Vision (ICCV) 2017)

Jiuxiang Gu, Gang Wang, Jianfei Cai, Tsuhan Chen