Excellent. There are a lot of teachers out here that would like you to make dummy proof instructions on building the following: 1) A pipeline for converting pdfs of reference material into USE or some other encoding paragraph by paragraph 2) a pipeline for converting pdfs of our existing tests and quizzes into fine tuning data for the academic language models I am asking you to help us build 3) a voice or text to paragraph-in-reference-material language model And if you a feeling really super: a model that generates common place journals like Erasmus Darwin's Great Book, i.e. quotation commentary quotation commentary ad infinitum
there should be more information on how to run distributed training on pytorch using vertex ai pipelines, there are just a bunch of workarounds and not well organised notebooks