Thank you for this tutorial.

Bro, your statement from 05:22 is completely wrong and misguiding. LoRA is used for finetuning LLM models, when full-finetuning is not possible. It does so by freezing all model weights, and incorporating and training low-rank matrices(A*B) in Attention modules. LoRA speeds up training and reduces memory requirements but does not provide a speedup during inference. If LLM model is too large to be handled by LoRA due to GPU memory limitations, Quantized LoRA is used to finetune the model. Overall, QLoRA is a more advanced solution when LoRA alone cannot handle large models for finetuning.

is there is a multimodal llm can fine-tuning for sentiment analysis from text, image, video and audio ?

The quantisation of changing number format applies only to the result of activation function or also to the individual weights ? Where we apply this quantisation in the NN

Cool video....Some test would be appreciated. Also, maybe you can include the qwen vision models, especially.

you can fine-tuning this multimodal llm for sentiment analysis

Thanks! this is an awesome explanation

where to draw these kinda charts ?. could you tell me. itll be helpful

Dude change the battery in your smoke detector

thank you. I have a question ! do we use dspy for development or production ?

nice i made this before ! A model which picks the correct model ! BUT : then i decided tat a 1b agent can be the router model ! Then i decided that models as TOOLS ! so once you create an anthrpic as a tool , it will select the anthropic insteads ! i think its all about understanding the power of Tools andeven graphs and nodes : If we create some graphs then their start point are the tool ! SO: the docstring methodolgy is the best version of the tool calling method ! , perhaps with a react type framwork ( epecally when using tools ) by creating details docstring and example in the docstring , ach tool added will be woven into the prompt ! so the aim is to create model ( or tune one ) to use the react framwork as well as selecting tools ! -- I think that higging face agents is the methodology which is correct because we ca host models on hugging face .. and hit those spaces ! ... Spaces as TOOLS !.. SO again we see tools takinng a front role as the main prompt is to select the correct tool for the intent: also train for slot filling and intent detection (hf dataset ) .... the routing method was very good learning execises ! ... but it also needs the pydantic to send back the coreect route to select , when it could be done via a tool which is already preprogrammed iin to the library ( stoping reason )...

Thank you for that video. I've been studying this model and helped me a lot.

How did you push your own custom dataset on huggingface?

i always got this message ImportError: cannot import name 'load_flow_from_json' from 'langflow' (unknown location) already clone from github using windows

fp16 also has a bigger mantissa than bfloat which benefits normalized or bounded activation functions (e.g. sigmoid)

Well the graphs at 2:18 are incorrect, sigmoid and tanh have different ranges, so the output gate should have range - 1 to 1 (tanh)

I don’t understand why you say that LoRA is fast for inference… in any case you need to forward through the full rank pretrained weights + low-rank finetuned weights.

Thank you, that's a beautiful explanation! One thing I struggle understanding, is the term quantization blocks in 4:30 - why we need several of them. In my understanding from the video, we ponder about using 3 blocks of 16 bits to describe a number. Which is 48 bits and is more expensive than 32-bit float. But couldn't we just use 16*3 = 48 bits per number instead? Using 48 bits (without splitting it) would give us a very high precision within [0,1] range, due to powers-of-two I did ask GPT, and it responded that there exists a 'Scale Factor' and a 'Zero-Point', which are constants that shift and stretch the distribution in 6:02 Although I do understand these might be those quantization constants, - I am not entirely sure what the 64 blocks described in the video are 6:52 Is this because of the Rank of Matrix-Decompositions is 1, with 64 entries in both vectors?

Awesome explanation! ❤

🎉❤

Hi, when i am having 100k pdf documents and i store all the embedding into vector store without following any chunking. Now if i want to retrieve using prompt how can we augment relevant information on such an huge un-chunked vector? Please suggest what is the best way to handle this problem? Please help some references as well along with your inputs

I wish I was good at math to understand this stuff.

Great video! How good is it compared to I-JEPA?

very well explained.

Just start watching but... signed bye is -128..127 not 127..127. google 2's complement to see why.

Hi, what is reason you first convert it to FP16.gguf not directly to 8 bit.

I'm getting error RuntimeError: No GPU found. A GPU is needed for quantization.

I have a question. after embedding we still have the same number of features x1->x4, and say that they have dimension is 1x10, means 10 features each. W* is 4x4 right! My question is X is 4x10 and X^T is 10x4. How we do dot product W . X^T as the dimension is (4x4) . (10x4)! or I am missing something?

Thank you. Can you share the notebook please?

The training of artificial neural networks is computationally expensive for several key reasons: Large number of parameters: Deep neural networks often have millions or even billions of parameters (weights and biases) that need to be optimized during training. Updating such a large number of parameters requires significant computational resources. Iterative process: Training typically involves many iterations over the entire dataset (epochs) to gradually adjust the parameters. This iterative nature makes it time-consuming, especially for large datasets. Backpropagation: The backpropagation algorithm, used to calculate gradients and update weights, requires forward and backward passes through the network for each training example. This process is computationally intensive, especially for deep networks. Matrix operations: Neural network computations involve many matrix multiplications and other mathematical operations, which are computationally expensive, especially as network size increases. Large datasets: Training on large datasets, which is often necessary for good performance, requires processing massive amounts of data repeatedly. Hyperparameter tuning: Finding optimal hyperparameters (e.g., learning rate, network architecture) often involves training multiple models with different configurations, multiplying the computational cost. Complex architectures: Advanced architectures like convolutional neural networks or recurrent neural networks involve specialized operations that add to the computational complexity. Gradient descent optimization: The optimization process itself, typically using variants of gradient descent, requires many small updates to the parameters, each requiring computation of gradients. These factors combined make the training of artificial neural networks a computationally intensive task, often requiring specialized hardware like GPUs to accelerate the process.

Hyperparameter tuning: Finding optimal hyperparameters (e.g., learning rate, network architecture) often involves training multiple models with different configurations, multiplying the computational cost. Complex architectures: Advanced architectures like convolutional neural networks or recurrent neural networks involve specialized operations that add to the computational complexity.

What makes the training of artificial neural networks computationally expensive The training of artificial neural networks is computationally expensive for several key reasons: 1. Large number of parameters: Deep neural networks often have millions or even billions of parameters (weights and biases) that need to be optimized during training. Updating such a large number of parameters requires significant computational resources. 2. Iterative process: Training typically involves many iterations over the entire dataset (epochs) to gradually adjust the parameters. This iterative nature makes it time-consuming, especially for large datasets. 3. Backpropagation: The backpropagation algorithm, used to calculate gradients and update weights, requires forward and backward passes through the network for each training example. This process is computationally intensive, especially for deep networks. 4. Matrix operations: Neural network computations involve many matrix multiplications and other mathematical operations, which are computationally expensive, especially as network size increases. 5. Large datasets: Training on large datasets, which is often necessary for good performance, requires processing massive amounts of data repeatedly. 6. Hyperparameter tuning: Finding optimal hyperparameters (e.g., learning rate, network architecture) often involves training multiple models with different configurations, multiplying the computational cost. 7. Complex architectures: Advanced architectures like convolutional neural networks or recurrent neural networks involve specialized operations that add to the computational complexity. 8. Gradient descent optimization: The optimization process itself, typically using variants of gradient descent, requires many small updates to the parameters, each requiring computation of gradients. These factors combined make the training of artificial neural networks a computationally intensive task, often requiring specialized hardware like GPUs to accelerate the process. Citations: [1] Training Neural Networks | Machine Learning - Google for Developers developers.google.com/machine-learning/crash-course/training-neural-networks/video-lecture [2] Various Optimization Algorithms For Training Neural Network towardsdatascience.com/optimizers-for-training-neural-network-59450d71caf6?gi=ea8e0c3dd721 [3] 5 Algorithms to Train a Neural Network - DataScienceCentral.com www.datasciencecentral.com/5-algorithms-to-train-a-neural-network/ [4] The differences between Artificial and Biological Neural Networks towardsdatascience.com/the-differences-between-artificial-and-biological-neural-networks-a8b46db828b7 [5] [PDF] Introduction to Neural Network Algorithm einsteinmed.edu/uploadedFiles/labs/Yaohao-Wu/Lecture%209.pdf

Thank you sire

Very detailed and informative vedio！

Is there any type of mathematical proof or at least reasoning that the weight of pretrained Neuron Network Weights are normal distributed? This in essense is the foundational data point they are using. And yes, very well done - I just found you via good old Google Search looking for QLoRA. Thanks for investing your time to bring these concepts closer to the community and people.

Great job mate, keep going. Could you please update on current research for proof-of-concept CLIP/text based prompting for SAM? Note one suggestion kindly: Don't make the text animation bounce. It's very distracting when trying to read the text. Maybe you can try other kind of animation or just keep it simple. Even other kinds of objects in flowchart bounce when they appear. Please avoid this bounce animation.

Super in depth and specific, thank you!!!

Nice video. When can I find the model downloaded in colab? Can you help me?

But won't it create the problem of overfitting

this is better explained than what the inventor of Lora itself explained in his video.

😊😊😊😊

😊😊😊😊

😊😊😊😊

😊😊😊😊😊

very informative videos. I couldnt find video 4 and 5. pls share the link with me.

這支影片普遍地解釋了 DSPy 的重要事項。先玩過了之後，有一點經驗但是充滿疑問之時，就可以來看看這支影片，解答諸多疑點。

audio and video out of sync

Amazing video! I'm wondering that for the model architecture images you provided, are the spatial and temporal layers only talking about the architecture of the U-net block? Would the rest of the Video LDM model be the same?

Thank you so much for providing this video!!!!!