Speaker: Jiepu Jiang, Ph.D. candidate in Computer Science, University of Massachusetts Amherst

When: 11:00AM ~ 11:50AM, March 5, 2018

Where: CSB130

Abstract:Search engines often require large-scale human annotation to improve and evaluate their services. In this talk, we introduce research towards reducing such human labor requirements, especially methods without using private user data. We first present a method called similarity-based distant supervision, which automatically generates text retrieval training labels in a new environment using only open data sources such as Wikipedia and existing test collections. The method can also be refined substantially with the help of only a few human judgments, which by themselves are insufficient to train search engines directly. Second, we also introduce methods for evaluating user experience with a new search engine service --- intelligent voice assistant --- without asking users' explicit feedback. The techniques predict user experience based on behavioral data and user modeling. The methods can diagnose the goodness of a running system instantly without interrupting users and shows excellent generalizability to new, unseen intelligent assistant tasks comparing to offline evaluation methods such as counting precision and recall.

Bio:Jiepu Jiang is a Ph.D. candidate in Computer Science at the University of Massachusetts Amherst. His research aims to help people better access and use information, including both designing effective systems to support information access and studying human factors during this process. He has been regularly publishing and serving in top information retrieval and data mining conferences. He received the best student paper award from CHIIR 2017 for his work on understanding dynamics of search result judgments in information retrieval. He also received a Ph.D. in Library & Information Science from the University of Pittsburgh.