Robin Jia (USC)- Auditing, Understanding, and Evaluating Large Language Models

Abstract: The widespread adoption of large language models places a responsibility on the AI research community to rigorously study and understand them. In this talk, I will describe my group’s research on analyzing language models’ training data, internal mechanisms, and downstream behavior. First, I will discuss two complementary approaches to auditing the usage of copyrighted data for language model training. Next, I will describe my group’s work on understanding how language models work internally, focusing on a case study of how analyzing model subcomponents can improve in-context learning. Finally, I will highlight our collaborative efforts to characterize LLMs’ strengths and weaknesses in medical and robotics applications.