My research focuses on knowledge and dynamics of language models - spanning their fine-tuning efficiency, training dynamics, and mergeability. I will begin by introducing Fast Forward, a simple optimization strategy that significantly accelerates large segments of low-rank training to save computational costs. Next, I will explore the training dynamics of large language models, revealing the fundamental disconnect between how internal parametric knowledge and external knowledge are acquired and forgotten over trainin.. We will then transition to the biological domain to examine the training dynamics of protein language models. Finally, I will conclude by exploring the underlying causes of model mergeability, demonstrating how a base model's prior knowledge influences the success of merging multiple fine-tuned models into a single multitask model. Together, these works provide deeper insights into the mechanics of model knowledge and learning and offer practical methods for more efficient and robust AI development.