Image Classification Using Vision Transformer | ViTs 

Can u send exact link where I could download dataset?

I will try my best

Glad you enjoyed it!

Glad you think so!

Same for me

I will try after finishing my pipelined work.

Sure

@@CodeWithAarohi mam i downloaded the going_modular but still geeting the going_modular error. can you please guide us how to use this going_modular properly after downoading

I will try

Thank you so much, we are waiting your brilliant video@@CodeWithAarohi

This can happen for various reasons, and here are some steps you can take to diagnose and potentially fix the issue: Smaller batch sizes can sometimes lead to numerical instability. Try increasing the batch size to see if it has an impact on the problem. Implement gradient clipping to limit the magnitude of gradients during training. This can prevent exploding gradients, which can lead to "NaN" values in the loss. The learning rate used in your optimization algorithm might be too high, causing the model's weights to diverge during training. Try reducing the learning rate and experiment with different values to find the appropriate one for your model. Regularization techniques like L1 or L2 regularization can help stabilize training. Consider adding regularization to your model to prevent overfitting.

yes

@CodeWithAarohi how can we use the going_modular in google colab i tried but i don t know how

@tajikhaoula8068 copy going_modular folder in your google drive and then import it

​@@CodeWithAarohi hello maam it didn't worked for me maybe im missing some steps could you please make a video on how to import it in Jupiter or google colab.

aarohisingla1987@gmail.com

Please check the repo, this folder is already there.

@@CodeWithAarohi Thank you!

@@MonishaRaja hi can you please tell me how did you run it in colab

I will try!

I never did that but will surely try.

Welcome 😊

Post full error message.

universe.roboflow.com/enrico-garaiman/flowers-y6mda/dataset/7

To reduce the output sequence from the transformer encoder to a single token representation by selecting the first token and passing it through a classifier.

@@CodeWithAarohi Can we not combine all the tokens together into one with cat + lin or sum? Intuitively, they all contain contextual information, so would that be a bad idea?

universe.roboflow.com/enrico-garaiman/flowers-y6mda

aarohisingla1987@gmail.com

You have to copy this going_modular folder in your current working directory. This folder is available here: github.com/AarohiSingla/Image-Classification-Using-Vision-transformer

Get it from roboflow universe

Sure, Thanks!

@@CodeWithAarohi I have Q, I use colab for this code, every codes runs well but i cannot import going_modular. how can i deal with this?

@@잇준-v7m same issue are you able to solve?

Have you tried working with data with more than 3 channels?

did it works? i want to do my fyp on vit thats why

Copy the going_modular folder and helper.py file from this link and paste it in the directory where your jupyter notebook is: github.com/AarohiSingla/Image-Classification-Using-Vision-transformer

You can download the going_modular folder from github.com/AarohiSingla/Image-Classification-Using-Vision-transformer

​@@CodeWithAarohi you make a video of how to install the going modular im fresher to it.

Sure!

universe.roboflow.com/enrico-garaiman/flowers-y6mda/dataset/7

@@CodeWithAarohi How can I get the two folders for daises and dandelions from this link?

Going_modular is a folder present in my repo. You need to download it and put it in your current working directory.

​@@CodeWithAarohi very nice video but you did not explain what "going_modular.going_modular import engine" it is and where you got it from ??

Noted

Do you have code to calculate precision, recall, F1 score in vision transformer. Please reply

Sure

Yes, Do you have code to calculate precision, recall, F1 score?

Because there is an extra dimension that gets added as a class.

github.com/AarohiSingla/Image-Classification-Using-Vision-transformer

@@CodeWithAarohi Thanks!

@@CodeWithAarohi Hey idk why, but it still says this : Invalid Notebook missing attachment: image.png

have u try re run the script from the beginning ?

Glad it was helpful!

You can download that from github.com/AarohiSingla/Image-Classification-Using-Vision-transformer

@@CodeWithAarohi Hello mam, first of all great video and amazing explanation of ViT. going_modular package is not compatible with my python version. I tried all other option to install it from git, using pip install but still problem persist. Plz help... i am beginner in ViT rest of the code works perfect.

I am running code in Jupyter Notebook with Python 3.12.2

You just need to install torch, torchaudio, torchvision and torchinfo. THis is the version I am using: pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url download.pytorch.org/whl/cu116 pip install torchinfo

Yes, you can work with rectangular images in Vision Transformers (ViTs), but you're correct that padding may not be the best solution, especially if it introduces a lot of empty space. You can resize your rectangular images to a square shape before inputting them into the ViT. Or you can crop your rectangular images to a square shape, preserving the most important parts of the image.

You have to download the going_modular folder from my github repo and paste it in your working directory. github.com/AarohiSingla/Image-Classification-Using-Vision-transformer

custom_image_path = "test_img.jpg" # Predict on custom image pred_and_plot_image(model=vit, image_path=custom_image_path, class_names=class_names)

@@CodeWithAarohi creating a database before right?

Ok next time

I have used jpg format.

These are 2 different architectures. Vision Transformers are more advanced and powerful but require more computational resources and are more complex to implement and fine-tune. YOLOv8 is promoted for its speed, resource efficiency, ease of use, and strong community support, making it ideal for real-time object detection and deployment on edge devices.

github.com/AarohiSingla/Image-Classification-Using-Vision-transformer

I will try!

Glad it was helpful!

Thank you for your appreciation. CNNs (Convolutional Neural Networks) operate on local features hierarchically, extracting patterns through convolutional layers, while ViTs (Vision Transformers) process global image structure using self-attention mechanisms, treating image patches as tokens similar to text processing in transformers.

@@CodeWithAarohi thanks for your kind reply. Love from Lahore Pakistan

going_modular is a folder. You need to put it in your current working directory and please check the path of it.

Glad it helped you!

There isn't a strict minimum requirement for running Vision Transformers. But just to give you an idea- Use a CUDA-enabled GPU (e.g., NVIDIA GeForce GTX/RTX), at least 16GB of RAM (32GB recommended for larger models)

Glad you enjoyed it!

You're most welcome

The original ViT paper used a fixed-size patch (e.g., 16x16 pixels), but you can experiment with different patch sizes based on your dataset and task. Larger patches may capture more global features but require more memory. 2- The number of Transformer blocks in your model. Deeper models may capture more complex features but also require more computational resources. 3- The dimensionality of the hidden representations in the Transformer. Larger hidden sizes may capture more information but also increase computational cost. 4- The number of parallel attention mechanisms in the Transformer block. Increasing the number of heads can help capture different aspects of relationships in the data. YOu can make changes in learning rate, drop out, weight decay, batch size, Optimizer also.

This is just for demo purpose. You can create your model from scratch or you can can use the pretrained models and fine tune it with your custom dataset.

I will try.

You're welcome!

going_modular is a fodler in my github repo. Place this folder in your google drive and then run your colab

ok ma'am let me try.. thank you@@CodeWithAarohi

@@CodeWithAarohi mam still its not working. can you please help?

Never tried it. but, I think you can use.

@@CodeWithAarohi can I conntact you I need your help?

@@CodeWithAarohi did the images should have special dimanation?

You can get it from here: universe.roboflow.com/search?q=flower%20classification

Glad my video helped you!

Will try.

public.roboflow.com/classification/flowers_classification/3

ViT (Vision Transformer) models are primarily designed to work with square images but ViT for non-square images is possible, but it requires some modifications to the architecture and preprocessing steps. Regarding using pretrained ViT models for specific tasks, it can be a good starting point in many cases, especially if you have a limited amount of task-specific data.

You can download it from roboflow100

Will try

You are welcome

I will try to cover the topic.

I will try after finishing my pipelined work.

Yes , You can use ViT for Object detection

ViT is only for image classification, if you want to use vit architecture in image captioning, you need quite different model form. find google scholar and find the modified model for image captioning