Episode Notes:
Aurimas Griciūnas is the Chief Product Officer of Neptune.AI, a startup building experiment tracking tools for foundation model training.
*Sections*
Overview of LLM Ops and Experiment Tracking - 00:01:42
Challenges of Scaling LLMs: From ML to LLM Ops - 00:02:14
Scale and Complexity of LLM Clusters and Training - 00:06:58
Frontier Models and Training Cycles - 00:09:25
Enterprise Lessons from LLM Ops - 00:12:10
Branching Experiments and Fine-Tuning Models - 00:13:50
The Importance of Fine-Tuning in Enterprises - 00:17:30
Visualization Challenges at Scale - 00:18:23
Tracking and Fault Tolerance in LLM Ops - 00:22:59
Evaluation Problems in GenAI and LLM Apps - 00:27:00
Experimentation in Agentic AI Systems - 00:29:08
Three Key Enterprise LLM Ops Considerations - 00:33:15
The Future of Scaling LLM Ops - 00:34:39
Closing Discussion: Full Fine-Tuning and Ambitious Enterprise Models - 00:40:29
28 сен 2024
