Danny To Eun Kim
김도은
toeunkim{at}cmu{dot}edu

I'm a first year PhD student at CMU Language Technologies Institute (LTI), advised by Prof. Fernando Diaz. My research interests span the fields of Natural Language Processing (NLP) and Information Retrieval (IR), with recent emphasis on retrieval-enhanced machine learning (REML) and known-item retrieval. I like to draw inspiration from a broader context, including areas like psychology, cognition, and neuroscience.

Prior to CMU, while I was in the UK, I was a graduate researcher at Web Intelligence Group, University College London (UCL), where I received M.Eng. in Computer Science in 2022. I was working with Prof. Emine Yilmaz and Prof. Aldo Lipani on Conversational AI and User Simulation. Also, I was a member of both Knowledge Graphs and NLP interest group at The Alan Turing Institute.

During the summer of 2022, I interned at Raft as a Machine Learning Engineer, automating the paperwork in freight forwarding industry by OCR and NLP.
From May 2021, I joined team Condita as a main developer to compete in the first Amazon Alexa Prize TaskBot Challenge, where I spent significant amount of time, constructing a real-time, multi-modal, knowledge-intensive, and interactive conversational assistant; our team has made to the quarterfinals.
During my undergraduate degree, I worked with Prof. Marianna Obrist at UCL (Human-Computer) Interaction Centre, finding ways to cluster text-based stories by authors' smell experience.

I take pride in being one of the early members of the UCL Artifical Intelligence Society, and during my three years of active involvement, I had the privilege of founding the first Machine Learning tutorial series, which has become an annual tradition.

Check out my blog page or pictures for more!

CV  /  Google Scholar  /  Blog  /  Twitter  /  GitHub

Follow @TEKnologyy

Current Research:

• Retrieval-Enhanced Machine Learning (REML) or RAG
• Tip of the tonuge (known-item) retrieval

Theses

• Master's: Multi-Task Neural User Simulator for Task Oriented Dialogue System
• Bachelor's: Exploring the Potential of Automating the Process of Clustering Smell Stories

Conversational User Simulator that 1) generates user-side utterance, 2) predicts user's next action and 3) satisfaction level by multi-task learning. SOTA in Satisfaction and Action prediction in USS dataset

We present COoking-aNd-DIy-TAsk-based (Condita) task-oriented dialogue system, for the 2021 Alexa Prize TaskBot Challenge. Condita provides an engaging multi-modal agent that assists users in cooking and home improvement tasks, creating a memorable and enjoyable experience to users. We discuss Condita's state machine like architecture and analyze the various conversational strategies implemented that allowed us to achieve excellent performance throughout the competition.

Spin-off research from the Alexa Prize TaskBot Challenge. Assisting users to cook is one of these tasks that are expected to be solved by intelligent assistants, where ingredients and its corresponding attributes, such as name, unit, and quantity, should be provided to users precisely and promptly. To provide an engaged and successful conversational service to users for cooking tasks, we propose a new ingredient parsing model.

There is still a lack of literature and its reviews on structured data modelling via diffusion models, compared to other data modalities such as visual and textual data. To address this gap, we present a comprehensive review of recently proposed diffusion models in the field of structured data. First, this survey provides a concise overview of the score-based diffusion model theory, subsequently proceeding to the technical descriptions of the majority of pioneering works that used structured data in both data-driven general tasks and domain-specific applications. Thereafter, we analyse and discuss the limitations and challenges shown in existing works and suggest potential research directions.

In the NeurIPS 2022 IGLU Challenge NLP Task, we address two key research questions: 1) when should the agent ask for clarification, and 2) what clarification questions should it ask. In this report, we briefly introduce our methods for the classification and ranking task. For the classification task, our model achieves an F1 score of 0.757, which placed the 3rd on the leaderboard. For the ranking task, our model achieves about 0.38 for Mean Reciprocal Rank by extending the traditional ranking model. Lastly, we discuss various neural approaches for the ranking task and future direction.

• Can Dialogue Systems Have Theory-of-Mind Ability?
  Presented at the NLP Interest Group
  at The Alan Turing Institute.
  24th November, 2022
  Recording / Slides


• Personal Knowledge Graphs: Towards Better Conversational Assistants
  Presented at the 5th Meet-up of Knowledge Graphs Interest Group
  at The Alan Turing Institute.
  27th October, 2022
  Recording / Slides

• Intensive Python Programming Course (for MSc students' dissertation)

• CEGE0096: Geospatial Programming (Fall 2022)
  • Coursework marking automation [interesting solution in my blog post]
• CEGE0004: Machine Learning for Data Science (Spring 2023)
• COMP0071 Software Engineering (Spring 2023)
• COMP0189: Applied Artificial Intelligence (Spring 2023)
• Leading practical classes [GitHub Repo]

• Helping 1^st year CS students with programming (C, Java, Python, Haskell)

• Season 20/21 : [GitHub] [Lecture Recordings]
• Season 21/22 : [GitHub] [Pictures in blog]

You can know me better from my CV

Jump to the top of this page.
Website source code is adapted from here.