Fine‑Tuning
LLMsCornell Course

In this course, you will discover how to adapt and refine large language models (LLMs) for tasks beyond their default capabilities by creating curated training sets, tweaking model parameters, and exploring cutting-edge approaches such as preference learning and low-rank adaptation (LoRA). You'll start by fine-tuning a base model using the Hugging Face API and analyzing common optimization strategies, including learning rate selection and gradient-based methods like AdaGrad and ADAM.

As you progress, you will evaluate your models with metrics that highlight accuracy, precision, and recall, then you'll extend your techniques to include pairwise preference optimization, which lets you incorporate direct user feedback into model improvements. Along the way, you'll see how instruction-tuned chatbots are built, practice customizing LLM outputs for specific tasks, and examine how to set up robust evaluation loops to measure success.

By the end of this course, you'll have a clear blueprint for building and honing specialized models that can handle diverse real-world applications.

You are required to have completed the following courses or have equivalent experience before taking this course:

• LLM Tools, Platforms, and Prompts
• Language Models and Next-Word Prediction