){kind=avatar}
Jaemin Cho
PhD student @ UNC Chapel Hill
I am a PhD student at UNC Chapel Hill (MURGe-Lab, UNC NLP group), advised by Prof. Mohit Bansal. I am supported by Bloomberg Data Science Ph.D. Fellowship.
While at UNC, I have spent my summer time at Bloomberg (in 2024), Google Research (in 2023), Microsoft Research (in 2022) and Adobe Research (in 2021). Prior to UNC, I did machine learning research at AI2, Naver, and SNU.
I am interested in building reliable machine learning systems that understand various types of real-world data. This includes but not limited to:
- Single model that handles various data and tasks (e.g., VL-T5 / X-LXMERT / TVLT).
- Reducing the cost for model training (e.g., Ctrl-Adapter / SELMA / VL-Adapter / LST).
- Evaluation metrics and benchmarks (e.g., DALL-Eval / VPEval / DSG / LayoutBench).
- Leveraging reasoning capabilities / world knowledge of foundation models to new domains (e.g., LM for image generation / LM for video generation / LM for RL environment generation / CLIP as a reward function).
News
Education
Ph.D. in Computer Science, 2025 (expected)
University of North Carolina at Chapel Hill
M.S. in Computer Science, 2022
University of North Carolina at Chapel Hill
B.S. in Industrial Engineering, 2018
Seoul National University
Selected Publications
Interpretable/explainable visual programming frameworks for T2I generation (VPGen) and evaluation (VPEval) - NeurIPS 2023
All Publications
Quickly search over publications.
Also see my Google Scholar for a more complete list.
Using LLM (GPT-4) to generate a ‘diagram plan’ for fine-grained layouts (object/text labels/arrows, etc.) and render in either raster images (via diffusion) and vector graphics (via PowerPoint / Inkscape or any tools). - COLM 2024
EnvGen is a novel framework that uses LLMs to adaptively create training environments to help smaller embodied RL agents learn useful skills that they are weak at. - COLM 2024
Using LLM (GPT-4) to generate a ‘video plan’ for consistent multi-scene video generation - COLM 2024
a New Diagnostic Benchmark (LayoutBench) and a new Baseline model (IterInpaint) for Layout-Guided Image Generation - CVPR Workshop 2024 (Oral)
Combining the best of both of specialist (e.g., SimpleClick) and generalist (e.g., SAM) designs to achieve all of low latency, high quality, and diverse prompts - CVPR 2024
CRG is a training-free method that guides VLMs to help understand the visual prompts, by contrasting the outputs with & without visual prompts. - ECCV 2024
Interpretable/explainable visual programming frameworks for T2I generation (VPGen) and evaluation (VPEval) - NeurIPS 2023
To handle video QA, we self-chain BLIP-2 for 2-stage inference (localize+QA) & refining localization via QA feedback - NeurIPS 2023
HiREST is a holistic, hierarchical benchmark of multimodal retrieval and step-by-step summarization for a video corpus - CVPR 2023
Vision-and-Language modeling without text, by using a transformer which takes only raw visual and audio inputs - NeurIPS 2022 (Oral)
A question answering benchmark on real-world news articles for multi-media and multi-hop reasoning - AAAI 2022
Text-to-Image Generation via predicting vector-quantized image patches with multimodal LMs - EMNLP 2020
