📚 Paper Demystification: Ever stumbled upon an intriguing AI research paper and wondered what it all means? Join us as we dissect and explain the most groundbreaking papers, highlighting their real-world implications and practical significance.
🧠 Join Our AI Community: It's not just a channel - it's a community of AI enthusiasts and data science aficionados. Engage with us, ask questions, and connect with fellow learners who share your passion for AI and its endless possibilities.
🚀 Subscribe for AI Adventures: Ready to embark on an AI adventure? Hit the subscribe button and stay tuned for exciting explorations into the future of technology and intelligence.
So, if you're ready to embark on a journey of knowledge, innovation, and skill-building in the realm of AI, don't forget to hit that subscribe button and ring the notification bell. Let's explore the future, one algorithm at a time! 🚀🧠
Hi, thanks for the video. When I run main, I get a "RuntimeError: grad can be implicity created only for scalar outputs". Can anyone help me out how to solve this?
can you please help me with finetuning paddle ocr ? I watched all 4 videos of yours on that topic but I am getting many errors at the time of training, please help me
hello, this is very good video,now i have a question, Why use PaddleOCR to extract information from images when you can directly read and extract information from images using LayoutLMv3?
I'm getting this error when I run main.py : PreTokenizedEncodeInput must be Union[PreTokenizedInputSequence, Tuple[PreTokenizedInputSequence, PreTokenizedInputSequence]]
What version of transformers do you use? because I'm getting this error when I run main.py : ImportError: cannot import name 'PreTokenizedEncodeInput' from 'transformers' (C:\Users\khaou\AppData\Local\Programs\Python\Python312\Lib\site-packages\transformers\__init__.py)
After running the main.py file i am getting the below error, how can I resolve this?? ValueError: Expected input batch_size (1536) to match target batch_size (1024).
I'm getting error in Inference file in: predictions = op.argmax(-1).squeeze().toList() the errror is: AttributeError: 'tuple' object has no attribute 'argmax'. Please help. ASAP.
Hi sir i cloned the GitHub repo that you have provided and created virtual environment after running the command pip install paddleclas I am getting the error as below, I am trying to resolve it from past 2 days not solved can you please help error: command '/usr/bin/swig' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for faiss-cpu Running setup.py clean for faiss-cpu Failed to build faiss-cpu ERROR: Could not build wheels for faiss-cpu, which is required to install pyproject.toml-based projects
Hi Mani, I did launch the webserver with auth and I can access the images, uploaded the json, but in Label Studio, if I swap the ocr field to 'img' from 'string' it won't show the image, (brokenData)? Any idea?
Quick heads up, on the github repo, you have this image url creation defined twice: def create_image_url(filename): """ Label Studio requires image URLs, so this defines the mapping from filesystem to URLs if you use ./serve_local_files.sh <my-images-dir>, the image URLs are localhost:8081/filename.png Otherwise you can build links like /data/upload/filename.png to refer to the files """ return f'localhost:8080/{filename}' created twice
I'm trying this to execute this on colab..however getting following error while executing Main.py code block ......RuntimeError: grad can be implicitly created only for scalar outputs. how we can create this entire script for google colab
You can't put the whole main in a cell, because there is no main the scripts need to work into a succession, you have to break it down into functions and isolate input and output of files in the temporary folder or in you google drive that you need to mount and run everything sequentially. You are better off using his code, it needs a couple of tweaks but it's all together working. In the file that produces the label studio json for labelling there is twice a function for creating a url, url that you need to get right because it will be used by label studio to render your image.
Hi, can you help me Ididn't got single output for my dataset i have annotated and trained 100 images with 8 classes with 100 epochs but, I didn't got any result, can you modify that inference with pil image, because with matplotlib I didn't got any expected result view?? 😢
Thanks for tutorial: I am getting following error: Traceback (most recent call last): File "F:\PyCharmProjects\LayoutLMTrial\Inference.py", line 51, in <module> op = model(input_ids = inputs_ids.unsqueeze(0), File "F:\PyCharmProjects\LayoutLMTrial\venv\lib\site-packages\torch n\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "F:\PyCharmProjects\LayoutLMTrial\venv\lib\site-packages\torch n\modules\module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "F:\PyCharmProjects\LayoutLMTrial\trainer.py", line 33, in forward loss = loss_fn(output,lables) File "F:\PyCharmProjects\LayoutLMTrial\trainer.py", line 12, in loss_fn return nn.CrossEntropyLoss()(pred.view(-1,5),target.view(-1)) AttributeError: 'NoneType' object has no attribute 'view' can someone please help ?
Please check the json file you got from paddleocr output in that json file you need to do onehot encoding manually string to integer as explained in video. Let me know if it working or not. Thank for your support, Please subscribe the channel for more such videos.
@@AIOdysseyhub thanks for the response. i did exported to json-min format. changed the labels to integer manually. but still getting the error the only difference is that i have 5 classes and you demonstrated only 4 classes.
yeah, you can store the output in any of the structured data like dict using json library or pandas to store in Json or csv. Thank you for the support. Please subscribe to the channel for more such video.
Hi, We split into three sets (train, test, valid) if we want to test the model performance while training, by only splitting into two we are testing the model performance after training is done for that iteration. Thank you for the support. Please subscribe to the channel for more such videos.
Hi, Please check the code, In for loop you have not given the path of all images correct or looping related issue. Thank you for the support. Please subscribe the channel for more such video and support.
been heavily invested my time into OCR and ML these few days. been lucky also able to came across this , as I'm searching also for tools to label my financial document
Thank you very much for the support and sorry for the late reply. Please subscribe to the channel for more such videos. Please let me know what more related videos I can upload where you have difficulty..
Hi AI Odyssey. If i had 6 classes , what would be the appropriate changes to make in the inference file ?. Specifically in those lines of code : one_class = concat_torch[torch.where((concat_torch[:,4]==1) & (concat_torch[:,3]==0) & (concat_torch[:,2]==0))] two_class = concat_torch[torch.where((concat_torch[:,4]==2) & (concat_torch[:,3]==0) & (concat_torch[:,2]==0))] three_class = concat_torch[torch.where((concat_torch[:,4]==3) & (concat_torch[:,3]==0) & (concat_torch[:,2]==0))] four_class = concat_torch[torch.where((concat_torch[:,4]==4) & (concat_torch[:,3]==0) & (concat_torch[:,2]==0))] finl = torch.row_stack((one_class, two_class, three_class, four_class)) unique_ = torch.unique(finl, dim=0) plot_img(test_dict['img_path'], unique_[:, :4] ,unique_[:, 4].tolist(), unique_[:, 4].tolist(), width_scale, height_scale). Your response will be highly appreciated. Thanks in advance.
add this in respective line and test it four_class = concat_torch[torch.where((concat_torch[:,4]==4) & (concat_torch[:,3]==0) & (concat_torch[:,2]==0))] five_class = concat_torch[torch.where((concat_torch[:,4]==5) & (concat_torch[:,3]==0) & (concat_torch[:,2]==0))] Six_class = concat_torch[torch.where((concat_torch[:,4]==6) & (concat_torch[:,3]==0) & (concat_torch[:,2]==0))] finl = torch.row_stack((one_class, two_class, three_class, four_class, five_class, Six_class)) Hope this will help you, Let me know if you have any issues. Thank you, Please subscribe to the channel for more such videos.
I'm getting following error while training model "ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`labels` in this case) have excessive nesting (inputs type `list` where type `int` is expected)."
@@AIOdysseyhub Hi Mani i was also facing the same error "ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`labels` in this case) have excessive nesting (inputs type `list` where type `int` is expected)." I have also tried this solution but its not working for me . i already did the one hot encoding as mentioned in the video plz help with this.
Halo bro, can you help me? File "C:\Users\Admin\PycharmProjects\LayoutLMV3_Fine_Tuning-main\src\Inference.py", line 22, in <module> test_dict, width_scale, height_scale = dataSetFormat(image) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Admin\PycharmProjects\LayoutLMV3_Fine_Tuning-main\src\utils.py", line 71, in dataSetFormat test_dict['bboxes'].append(scale_bounding_box(process_bbox(item[0]), width, height)) ^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Admin\PycharmProjects\LayoutLMV3_Fine_Tuning-main\src\utils.py", line 58, in process_bbox return [box[0][0], box[1][1], box[2][0]-box[0][0], box[2][1]-box[1][1]] ~~~~~~^^^ TypeError: 'float' object is not subscriptable
the co_ord is not a list its float object, Please check the co_ord and print the co_ord before where it has used and track back the co_ord variable where it changing to float value. let me know if this help you or not
Hi bro, can you help me? If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`labels` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
Please track back to which line the error was raising, based on that line we can check where we are doing mistake. If it's not helping, please let me know. Thank you
I have resolved this issue, it is in the input json file, instead of the Key Label being of type str I have not changed it to type int. Thank you very much! Can I contact you via social media?@@AIOdysseyhub
@@AIOdysseyhub facing same issue. Please help Some weights of LayoutLMv3ForTokenClassification were not initialized from the model checkpoint at C:/Users/AshwariyaSah/ASH/LayoutLMV3_Fine_Tuning/inputs/layoutlmv3Microsoft and are newly initialized: ['classifier.bias', 'classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. Some weights of LayoutLMv3ForTokenClassification were not initialized from the model checkpoint at ../inputs/layoutlmv3Microsoft and are newly initialized: ['classifier.bias', 'classifier.weight'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference. 0%| | 0/4 [00:00<?, ?it/s]
@@AIOdysseyhub File "C:\Users\AshwariyaSah\ASH\LayoutLMV3_Fine_Tuning\src\main.py", line 35, in <module> train_loss = train_fn(dataload, model, optimizer) File "C:\Users\AshwariyaSah\ASH\LayoutLMV3_Fine_Tuning\src\engine.py", line 9, in train_fn for data in tqdm(data_loader, total=len(data_loader)): File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\tqdm\std.py", line 1182, in __iter__ for obj in iterable: File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\torch\utils\data\dataloader.py", line 630, in __next__ data = self._next_data() File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\torch\utils\data\dataloader.py", line 674, in _next_data data = self._dataset_fetcher.fetch(index) # may raise StopIteration File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\torch\utils\data\_utils\fetch.py", line 51, in <listcomp> data = [self.dataset[idx] for idx in possibly_batched_index] File "C:\Users\AshwariyaSah\ASH\LayoutLMV3_Fine_Tuning\src\loader.py", line 32, in __getitem__ encoding = self.processor( File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\transformers\models\layoutlmv3\processing_layoutlmv3.py", line 122, in __call__ encoded_inputs = self.tokenizer( File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\transformers\models\layoutlmv3\tokenization_layoutlmv3_fast.py", line 330, in __call__ return self.batch_encode_plus( File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\transformers\models\layoutlmv3\tokenization_layoutlmv3_fast.py", line 412, in batch_encode_plus return self._batch_encode_plus( File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\transformers\models\layoutlmv3\tokenization_layoutlmv3_fast.py", line 670, in _batch_encode_plus return BatchEncoding(sanitized_tokens, sanitized_encodings, tensor_type=return_tensors) File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\transformers\tokenization_utils_base.py", line 223, in __init__ self.convert_to_tensors(tensor_type=tensor_type, prepend_batch_axis=prepend_batch_axis) File "C:\Users\AshwariyaSah\.pyenv\pyenv-win\versions\3.10.0\lib\site-packages\transformers\tokenization_utils_base.py", line 764, in convert_to_tensors raise ValueError( ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (`labels` in this case) have excessive nesting (inputs type `list` where type `int` is expected).
Hi brother, I have exported the json file of label studio, now I want to use it for training on Paddle, I hope you can support me, thank you very much.
Yeah, you can but check if 5000 pdf are mandatory it will unnecessarily increase compute power, if redundance pdf are there like with same layout etc, reduce the size to train in less time.
hi bro ianm getting this repo erro:"huggingface_hub.utils._validators.HFValidationError: Repo id must use alphanumeric chars or '-', '_', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: '../input_files'."
Hi, Trace back to the code line on which line you are getting this error, I am also not sure where does this comes from so trace it and let me know. Thank you
hi, this is resolved ,i iried with same kind of images and the data set used for this same about 10 but still the model is not detecting need your help
brother ...i created virtual environment but still facing issue while installing paddleocr it is because or myMupdf library ...please address this if possible
Delete the current virtual env and create a new env and first install paddle libraries as mentioned in video then check if got installed or not properly then install mupdf libraries, I have installed multiple time, for me there was no issue it should be same for you as well, THanks for reaching out, Please subscribe to the channel and if your issue does not solve please let me know. Thank you 😊😊😊😊
