Chi Han (UIUC)- Internal Workings of Foundation Models: Diagnosing and Adapting Internal Representations

Abstract: While foundation models (FMs) continue to revolutionize natural language processing and AI applications, tracing, locating, and precisely addressing their limitations remains a major challenge. The development of future language models would greatly benefit from a structural understanding of FMs. This presentation brings together several recent papers that systematically explain the internal representations of FMs from both theoretical and empirical perspectives. These works offer preliminary characterizations of the roles and adaptation of internal components: (1) how positional representations hold clues for resolving the context length limitations of FMs, (2) how word representations can be used as steers for generation control, and (3) how cross-modal representations can be best aligned for scientific discovery. Together, they provide insights into addressing inherent limitations of FMs in a principled and efficient way, and point to a promising future of developing a modular “anatomy” for foundation models.