Avatar

Jaemin Cho

UNC NLP Group

I’m a PhD student at MURGe-Lab, UNC NLP group, advised by Prof. Mohit Bansal. Prior to UNC, I did machine learning research at AI2, Naver, and SNU.

I’m interested in building reliable machine learning systems that understand various types of real-world data, such as language, vision, music and dance.

News

Interests

  • Machine Learning
  • Natural Language Processing
  • Multimodal Learning
  • Generative Model

Education

  • M.S. in Computer Science, 2022

    University of North Carolina at Chapel Hill

  • B.S. in Industrial Engineering, 2018

    Seoul National University

Publications

TVLT: Textless Vision-Language Transformer

Vision-and-Language modeling without text, by using a transformer which takes only raw visual and audio inputs - NeurIPS 2022
Preprint Code

LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning

LST brings Memory efficiency into Parameter-efficient transfer learning - NeurIPS 2022
Preprint Code

Fine-grained Image Captioning with CLIP Reward

CLIP as reward function for fine-grained image captioning - Findings of NAACL 2022
Preprint Code

VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks

Adapter-based Parameter-Efficient Training for V&L tasks - CVPR 2022
Preprint Code

DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generative Transformers

Probing the Reasoning Skills and Social Biases of Text-to-Image Models
Preprint Code

MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding

A question answering benchmark on real-world news articles for multi-media and multi-hop reasoning - AAAI 2022
Preprint

VidLanKD: Improving Language Understanding via Video-Distilled Knowledge Transfer

Video-based grounding can improve diverse NLU tasks - NeurIPS 2021
Preprint Code

Unifying Vision-and-Language Tasks via Text Generation

Tackle different V&L tasks via text generation with a single unified architecture - ICML 2021
Preprint Code

X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers

Generate image from text by predicting masked patches with multi-modal transformers - EMNLP 2020
Preprint Code Project

Mixture Content Selection for Diverse Sequence Generation

Separate Diversification from Generation to improve both diversity and accuracy in sequence generation - EMNLP 2019
Preprint Code

A Hierarchical Latent Structure for Variational Conversation Modeling

Propose a hierarchical VAE model and utterance drop regularization to mitigate posterior collapse problem - NAACL 2018 (Oral)
Preprint Code Video

Experience

 
 
 
 
 

Research Intern

Microsoft Research

May 2022 – Aug 2021 Redmond, WA (Remote)
Working on

  • Vision and Language
 
 
 
 
 

Research Intern

Adobe Research

May 2021 – Aug 2021 San Jose, CA (Remote)

Publications

Worked on

  • Vision and Language

Advisors

 
 
 
 
 

PhD Student

UNC NLP Group

Aug 2020 – Present Chapel Hill, NC

Publications

Working on

  • Vision and Language

Advisor

 
 
 
 
 

Predoctoral Young Investigator

Allen Institute for AI

Sep 2019 – Jul 2020 Seattle, WA

Publications

Worked on

  • Vision and Language

Advisor

 
 
 
 
 

Visiting Scholar

SNU Music & Audio Research Group

Jun 2019 – Jul 2019 Seoul, Korea

Worked on

  • Human Pose Estimation
  • Music / Dance Understanding

Advisor

 
 
 
 
 

AI Residency

NAVER Clova

Jul 2018 – Mar 2019 Seongnam, Korea

Publications

Worked on

  • Generative Model
  • Question Answering / Generation
  • Summarization

Advisor

 
 
 
 
 

Research Internship

SNU Vision & Learning Lab

Mar 2017 – May 2018 Seoul, Korea

Publications

Worked on

  • Conversation Modeling
  • Image Captioning
  • Generative Model
  • Uncertainty Estimation

Advisor

 
 
 
 
 

Research Internship

HKUST

Jan 2017 – Feb 2017 Hong Kong

Worked on

  • AST-based Python-Pseudocode Translation

Advisor

 
 
 
 
 

NLP Engineer

DataNada

Aug 2016 – Dec 2016 Seoul, Korea

Developed

  • ADA, Chatbot for 8Percent, the first Chatbot in the FinTech industry in Korea
  • Korean Dependency Parser

Advisor

 
 
 
 
 

Research Internship

Polytechnique Montréal

Jan 2015 – May 2015 Montreal, Canada

Worked on

  • Mixed Integer Programming
  • Patient Classification
  • Nursing Assignment

Advisor