Avatar

Jaemin Cho

UNC NLP Group

I’m a PhD student at MURGe-Lab, UNC NLP group, advised by Prof. Mohit Bansal. Prior to UNC, I did machine learning research at AI2, Naver, and SNU.

I’m interested in building reliable machine learning systems that understand various types of real-world data, such as language, vision, music and dance.

News

Interests

  • Machine Learning
  • Natural Language Processing
  • Multimodal Learning
  • Generative Model

Education

  • M.S. in Computer Science, 2022

    University of North Carolina at Chapel Hill

  • B.S. in Industrial Engineering, 2018

    Seoul National University

Publications

Visual Programming for Text-to-Image Generation and Evaluation

Interpretable/explainable visual programming frameworks for T2I generation (VPGen) and evaluation (VPEval)

Self-Chained Image-Language Model for Video Localization and Question Answering

To handle video QA, we self-chain BLIP-2 for 2-stage inference (localize+QA) & refining localization via QA feedback

Hierarchical Video-Moment Retrieval and Step-Captioning

HiREST is a holistic, hierarchical benchmark of multimodal retrieval and step-by-step summarization for a video corpus - CVPR 2023

TVLT: Textless Vision-Language Transformer

Vision-and-Language modeling without text, by using a transformer which takes only raw visual and audio inputs - NeurIPS 2022 (Oral)

Unifying Vision-and-Language Tasks via Text Generation

Tackle different V&L tasks via text generation with a single unified architecture - ICML 2021

Mixture Content Selection for Diverse Sequence Generation

Separate Diversification from Generation to improve both diversity and accuracy in sequence generation - EMNLP 2019

A Hierarchical Latent Structure for Variational Conversation Modeling

Propose a hierarchical VAE model and utterance drop regularization to mitigate posterior collapse problem - NAACL 2018 (Oral)

Experience

 
 
 
 
 

Student Researcher

Google Research

May 2023 – Aug 2023 Austin, TX
Working on

  • Vision and Language
 
 
 
 
 

Research Intern

Microsoft Research

May 2022 – Aug 2022 Redmond, WA (Remote)

Publications

Worked on

  • Vision and Language

Advisors

 
 
 
 
 

Research Intern

Adobe Research

May 2021 – Aug 2021 San Jose, CA (Remote)

Publications

Worked on

  • Vision and Language

Advisors

 
 
 
 
 

PhD Student

UNC NLP Group

Aug 2020 – Present Chapel Hill, NC

Publications

Working on

  • Vision and Language

Advisor

 
 
 
 
 

Predoctoral Young Investigator

Allen Institute for AI

Sep 2019 – Jul 2020 Seattle, WA

Publications

Worked on

  • Vision and Language

Advisor

 
 
 
 
 

Visiting Scholar

SNU Music & Audio Research Group

Jun 2019 – Jul 2019 Seoul, Korea

Worked on

  • Human Pose Estimation
  • Music / Dance Understanding

Advisor

 
 
 
 
 

AI Residency

NAVER Clova

Jul 2018 – Mar 2019 Seongnam, Korea

Publications

Worked on

  • Generative Model
  • Question Answering / Generation
  • Summarization

Advisor

 
 
 
 
 

Research Internship

SNU Vision & Learning Lab

Mar 2017 – May 2018 Seoul, Korea

Publications

Worked on

  • Conversation Modeling
  • Image Captioning
  • Generative Model
  • Uncertainty Estimation

Advisor

 
 
 
 
 

Research Internship

HKUST

Jan 2017 – Feb 2017 Hong Kong

Worked on

  • AST-based Python-Pseudocode Translation

Advisor

 
 
 
 
 

NLP Engineer

DataNada

Aug 2016 – Dec 2016 Seoul, Korea

Developed

  • ADA, Chatbot for 8Percent, the first Chatbot in the FinTech industry in Korea
  • Korean Dependency Parser

Advisor

 
 
 
 
 

Research Internship

Polytechnique Montréal

Jan 2015 – May 2015 Montreal, Canada

Worked on

  • Mixed Integer Programming
  • Patient Classification
  • Nursing Assignment

Advisor