Тёмный
No video :(

robots.txt | What, Why, and How | रोबोट्स फाइल क्या है? 

Amit Tiwari
Подписаться 257 тыс.
Просмотров 55 тыс.
50% 1

robots.txt ya kai baar log galti se ise robot.txt bhi bol dete hai, ek tarah ki instruction file hai jo koi bhi search engine bot sabse pahle apki website par dhoodhta hai aur padhta hai.
Is file kya use hai, kaise ye apki website ko safe kar sakti hai, uske SEO me help kar sakti hai, ye jante hai aaj ke video me.
Useragent List www.robotstxt....
=================
Please share the video to help your channel.
Google My Business Explained In Hindi - • How To Use Google My B...
Local SEO In Hindi - • Local SEO Profession...
Google Ads/Adwords - • Negative Keywords Kya Hai
Google Search Console - • Google Search Console ...
Google Analytics - • Google Analytics In Hi...
Marketing & Advertising Of Restaurants - • How To Get More Custom...
========================
Contact Us
========================
Call/Whatsapp: +91 8282824232 [9 AM - 9 PM Indian Standard Time, Sunday Off]
Website: www.amittiwari.net
Facebook: teamamittiwari
Twitter: teamamittiwari
Instagram: teamamittiwari
#robots.txt #robots #robot.txt

Опубликовано:

 

5 сен 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 157   
@sumankhanal2325
@sumankhanal2325 4 года назад
After watching one video on your RU-vid channel after this I watch all video
@waqarkhalid7548
@waqarkhalid7548 3 года назад
acha sir ap sa pochna ha ka ya file upload kahn karni ha ? ya ya code kahn lekhna ha robots .txt ka
@thirdeyeanalysis650
@thirdeyeanalysis650 4 года назад
Amit Sir aap desh sewa kar rahe ho....Salut hai aapko....India ko world me no 1 banane me yogdan dene ke liye Dhanyawad....jai hind.
@pankaj.yadav1
@pankaj.yadav1 4 года назад
amit sir app ne bhut axe se btaya hai ...aur important jankari di hai ...thanku
@kanchandobhal
@kanchandobhal 3 года назад
All videos are informative and very useful.
@anakshidhar4925
@anakshidhar4925 2 года назад
Finally I got the correct concept for robot.txt file . Thank you so much Sir.
@ecommercepassionskills
@ecommercepassionskills 4 года назад
love from PAKISTAN. Your video forced me to subscribe to your channel. Very informative video. keep it up sir.
@usman9420
@usman9420 4 года назад
hii
@RnTechTips777
@RnTechTips777 2 года назад
nyc infor sir thanks for sharing
@technoshamp2917
@technoshamp2917 4 года назад
Awesome sir, #thanku aapne toh saare points he clear kar diye explaining like you are teaching students in a institute. Thank you sir please haame aaise he content waali video diya kaaro.
@shivkumarsingh1812
@shivkumarsingh1812 4 года назад
Bahut acha rha video jankaribhari
@adeeali
@adeeali Год назад
Thanks a lot you explained this robots.txt file in very details thanks once again
@GREATRAJASTHANTech
@GREATRAJASTHANTech 5 лет назад
Nice Video Brother ...
@AmitTiwari
@AmitTiwari 5 лет назад
Thanks bhai sa.
@GREATRAJASTHANTech
@GREATRAJASTHANTech 5 лет назад
@@AmitTiwari bhai sa bahut achhe video banate ho mai sare Video Dekhta hu .... Seriously I like Your Content And Quality
@AmitTiwari
@AmitTiwari 5 лет назад
I try to do my best.
@parmakharbmotivation5442
@parmakharbmotivation5442 7 месяцев назад
Nice sir ❤
@informativevideos5976
@informativevideos5976 2 года назад
Very helpful in detail...Best teacher
@NehaSingh-mt3bm
@NehaSingh-mt3bm 3 месяца назад
thanks for such video this was helpful want more content from u on digital marketing nd seo, i just hate wscubetech videos those are just spam
@user-eu1ny8un7z
@user-eu1ny8un7z 2 года назад
super
@manishjaisinghani4941
@manishjaisinghani4941 5 лет назад
Thankx... Learnt something new.
@OnlineDigitalZone
@OnlineDigitalZone 4 года назад
Very nice video
@sandeep_rathore21
@sandeep_rathore21 3 года назад
Amazing video with lot of information
@gadgetpursuit8688
@gadgetpursuit8688 2 года назад
awesome video explanation.
@viralshortquotes
@viralshortquotes 5 лет назад
Very nice sir aj to Cute lag rhy hian shirt b VIP ha..👌
@AmitTiwari
@AmitTiwari 5 лет назад
haha, thanks!
@viralshortquotes
@viralshortquotes 5 лет назад
@@AmitTiwari sir ap whatsapp no send me plz.
@prati6150
@prati6150 3 года назад
If this is practically then it will be more beneficial.
@nhcrypto
@nhcrypto 5 лет назад
Thanks.
@firugs
@firugs 5 лет назад
Another super helpful video. Thank you. 🙏
@musicallifestyle8199
@musicallifestyle8199 2 года назад
Very good tutorial
@swapneelkumarbankar6341
@swapneelkumarbankar6341 4 года назад
Thanks for explaining so nicely..
@vishwanath7567
@vishwanath7567 3 года назад
Amazing way of explanation 😊
@sudipkumarbose3806
@sudipkumarbose3806 5 лет назад
Very informative sir....plz make a detailed video on structured data....thanks in advance.
@AmitTiwari
@AmitTiwari 5 лет назад
There are couple of videos about schema data ru-vid.comsearch?query=Schema%20data and I am going to upload one more about adding schema data through google tag manager.
@dancpechanc
@dancpechanc 4 года назад
Greate Job.
@AbhishekPandeyg
@AbhishekPandeyg 4 года назад
Nice video.
@SthitaPrangyaMohantySEO
@SthitaPrangyaMohantySEO 2 года назад
blogger ke liye best robots rules mention kijiye plesae.
@rinkuchoudhary226
@rinkuchoudhary226 3 года назад
Very helpful
@dholaramdholaram330
@dholaramdholaram330 3 года назад
Thank you so much
@shoaibumar360
@shoaibumar360 3 года назад
really great video , subscribed ur channel
@AmitTiwari
@AmitTiwari 5 лет назад
robots.txt, what it is and how to use it.
@AmitTiwari
@AmitTiwari 4 года назад
We have talked about sitemap in this video too ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-CWuxWsNrlvM.html
@mantoshsharma5143
@mantoshsharma5143 4 года назад
robots.txt file ko kaisa deka aur kaha par hota ha?
@user-of3fi7zm8y
@user-of3fi7zm8y 4 года назад
very nice video!
@gypsygram3090
@gypsygram3090 4 года назад
Thank you sir
@mkshaikh5150
@mkshaikh5150 5 лет назад
nice sir, ek video short me "SEO" ke sabhi services par upload kare. (SEO Complete Campaign Kya Hota Hai?) jisse hume ye understand ho ki SEO actual kya hai jisse bahot users ko help milengi Thanks...
@souvikmanna7307
@souvikmanna7307 3 месяца назад
Sir category and tags ko kaise disallow kare ? Kya privacy-policy, contact us and about us - is type ke page ko disallow karna chahiye ? ❤
@simrangupta6235
@simrangupta6235 3 года назад
I liked your robots.txt :D #masoom
@inventibletech3768
@inventibletech3768 5 лет назад
Sir, Content syndication pe dedicated video banaeye.... Plez
@AmitTiwari
@AmitTiwari 5 лет назад
Jarur. Thanks for the suggestion.
@AKMMAITHILIBHOJPURI
@AKMMAITHILIBHOJPURI 3 года назад
prefix करनेके बाद। HTML tag कि कोड को के निचे सेट करने बाद भी verify failed बतारहा है। 3 wabsid मे सहीसे करके देख लीया नहीं हो रहा है। पिलिज भाई कोई अछि तरीका बताइए। बहोत परेसान हु ईसे। 0
@syednajmishah5928
@syednajmishah5928 4 года назад
Great (y)
@niralaaloof-laila3803
@niralaaloof-laila3803 3 года назад
Sir disallow ka mtlb hota hai k us particular robot ko us particular page pe crawl krne se mna krna
@AmitTiwari
@AmitTiwari 3 года назад
Yes, exactly.
@dailylyricshub9603
@dailylyricshub9603 3 года назад
Sir You deserve more subs 💕
@sukhvindersingh2249
@sukhvindersingh2249 3 года назад
My mind is blow
@amitshivapal959
@amitshivapal959 3 года назад
Sar mein digital marketing offline Sinha chahta hun aapke pass
@aditya02041991
@aditya02041991 4 года назад
Tag and category pages pe video banao.. Should we allow robots to index tag and category pages..??
@amanpreet_singh
@amanpreet_singh 5 лет назад
how to submit par bhi video banaiye
@AmitTiwari
@AmitTiwari 5 лет назад
You can simply upload it on the root folder of your website.
@pleasesmile8847
@pleasesmile8847 3 года назад
Sir kindly ye batade Jo 5xx errors hai jaise ke apne kaha keh host se baat kare lekin me blogger pe work karta hoo to kaise aur kis se contact karoo.
@RahulHChauhan
@RahulHChauhan 5 лет назад
Hi Amit, Can you please create video on crawl budget and use of JavaScript in SEO?
@AmitTiwari
@AmitTiwari 5 лет назад
Yes! I want to. I was waiting for someone to ask. Let's talk about some of these concepts pretty soon. Thank you.
@RahulHChauhan
@RahulHChauhan 5 лет назад
@@AmitTiwari Thank you
@SonuKumar-rj6nl
@SonuKumar-rj6nl 3 года назад
Dono concept alg hai kya?
@Factboy_360
@Factboy_360 3 года назад
Sir please reply.... Agar robot.txt file me.. Disallow : /search Ho to Iska matlab kya hota hai
@hemant3339
@hemant3339 4 года назад
Robots.txt file me bina puri jankari ke chedkhane karna mehaga pad sakta hai
@anaghavaidya7065
@anaghavaidya7065 3 года назад
Good afternoon sir, If i don't want to crawl a page for example contact us page. So I have to write like Disallow: /contact us Right?
@waqarkhalid7548
@waqarkhalid7548 3 года назад
acha sir ap sa pochna ha ka ya file upload kahn karni ha ? ya ya code kahn lekhna ha robots .txt ka
@mamtavkogz
@mamtavkogz 4 года назад
Sar Mera Google search console mein verify nahin Ho pa raha hai problem Aisa ho raha hai ki jab bhi main domain name dalti hun aur usko verify Karti Hun tu vah godaddy par jata hai but vahan par verify nahin ho raha hai please iska jarur reply kariye ga
@pavanpatil8047
@pavanpatil8047 4 года назад
ये फाइल कैसे बनाएं और कैसे अपने website मे add करे
@waqarkhalid7548
@waqarkhalid7548 3 года назад
acha sir ap sa pochna ha ka ya file upload kahn karni ha ? ya ya code kahn lekhna ha robots .txt ka
@rjshivam4732
@rjshivam4732 2 года назад
Sir robot TXT file server par b update kar di h aur wordpress par b lekin link se chek krta hu tho old update ko hi dikhata h
@jaisrivastava2200
@jaisrivastava2200 2 года назад
It mean robots.txt files mein hum crawler ko crawl krne se rok rahe hai.......... or robots tag mein crawl krne se nai rok rahe sirf no index krne ke liye instructions de rhe hai ?
@dharendrakumar4358
@dharendrakumar4358 4 года назад
Allow: /wp-admin/admin-ajax.php iska kya mtlab samjhu sir?
@AmitTiwari
@AmitTiwari 4 года назад
is instruction se pahle ek aur instruction hoga disallow: /wp-admin/. Is pahle instruction se aap search engine crawlers ko wp-admin folder ke andar aane se rokte hai aur fir dusre instruction allow: /wp-admin/admin-ajax.php se search engine crawlers ko sirf is ek file, admin-ajax.php, ko access karne ki permission dete hai.
@cruxweldtechshorts4405
@cruxweldtechshorts4405 5 лет назад
Hi Amit, may I know the software which you use to edit your videos?
@AmitTiwari
@AmitTiwari 5 лет назад
Premiere pro
@poojari18
@poojari18 2 года назад
6:46 - Its colon, not semi colon.
@SonuKumar-rj6nl
@SonuKumar-rj6nl 3 года назад
Sir robots.txt or google search console me xml create krna ek he hai????
@AmitTiwari
@AmitTiwari 3 года назад
.xml file sitemap hoti hai aur robots.txt instruction file hoti hai. Robots file website ka baap hota hai jiski baat sunni jaruri hai warna nateeja kharab hota hai. Sitemap website ki maa hai, jo pyaar se rasta batati hai aur kuch galat ho jaye to bhi kam chal jata hai.
@SonuKumar-rj6nl
@SonuKumar-rj6nl 3 года назад
@@AmitTiwari sir me rankmath me ise mange kr skta hu?
@gurusingh6996
@gurusingh6996 4 года назад
what is the difference between disallowing and no index. plz reply
@pranaliwankar1470
@pranaliwankar1470 Год назад
Then what is the use of allow in robots.txt
@mazedaarvideo
@mazedaarvideo 4 года назад
Meta robot, no index no follow k upar ek video plz
@AmitTiwari
@AmitTiwari 4 года назад
Noted. Thanks for the suggestion.
@mazedaarvideo
@mazedaarvideo 4 года назад
Welcome
@mkshop9534
@mkshop9534 4 года назад
#tag kaise lgta h RU-vid pe title ke upper
@shubhamupadhyay6185
@shubhamupadhyay6185 4 года назад
Hello Amit Tiwari, Can plz suggest what is the result if disallowed 3urls and also submit a sitemap in robots.txt still, crawler goes through disallowed URLs.....
@AmitTiwari
@AmitTiwari 4 года назад
If you disallow a URL in robots file then it won't be indexed but if any other page is linking to it then it might get indexed so if you really want any particular page to not get indexed, use no-index tag on the page(s) itself.
@shubhamupadhyay6185
@shubhamupadhyay6185 4 года назад
@@AmitTiwari implementing this conditions aren't we trying to confuse crawler First we say not to crawl Second we are submitting sitemap......!!!!
@androidmobilegames8081
@androidmobilegames8081 Год назад
*? Or *?* In dono k kya mutlab hota h in robots.txt file mei.
@neyondiscord6068
@neyondiscord6068 3 года назад
sir ye same thing wordpress pa use hota hai kya ?
@shivajitiwariofficial
@shivajitiwariofficial Год назад
bhai kha lagana hai aur kaise y kaun batayga
@rabbiyakhan1203
@rabbiyakhan1203 2 года назад
That means this file is JUST for the bots/crawlers and not for the normal visitors??
@nevergiveup6536
@nevergiveup6536 3 года назад
hi Only admin menu bar icons are visible. All menu names are disappeared on admin menu side bar on word press kindly help
@KHTAMSING
@KHTAMSING 4 года назад
Sirr Redirect error show ho rha hai mune serch consol main kaise fix kru ese
@learnwebcodelearnwebcode6123
@learnwebcodelearnwebcode6123 5 лет назад
Agr main asa likhu robots.txt file main User-Agent : * Allow : / To ya sub page ko index kare ga na ya right hain ya rong
@AmitTiwari
@AmitTiwari 5 лет назад
Yes, ye bhi sahi hai. Kyuki aap disallow ki jagah allow instruction ko use kar rahe hai aur " / " laga kar pure root folder ki permission de rahe hai.
@learnwebcodelearnwebcode6123
@learnwebcodelearnwebcode6123 5 лет назад
Thank you sir for your reply mare ko kuch din robots.txt ko la kar total e confused tha par ab pura clear hu.
@ZIGCOMPUTERS
@ZIGCOMPUTERS 4 года назад
Meri website ZIGCOMPUTERS.com ka robots.txt is tarah ka hai User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Is it Correct?
@SarkarBLR
@SarkarBLR 3 года назад
How to implement on practically
@vinay00128
@vinay00128 4 года назад
Hello sir whatever you said i undesrastand on issue i m facing . i have this url in warning @t​ i think it was not needed in my case to disable tags and categories now what to do please resolve it
@realtorfiles
@realtorfiles 2 года назад
Bro please teach me
@AkashBhatiJodhpur
@AkashBhatiJodhpur 5 лет назад
what happens if we not mention robots.txt in the website which have multiple urls
@AmitTiwari
@AmitTiwari 5 лет назад
Agar aap koi robots file nahi dete hai to searchbots apki puri website ko apni marji se crawl karega. Chhoti sites par crawl budget ka utna asar nahi hota hai so no problem there.
@AkashBhatiJodhpur
@AkashBhatiJodhpur 5 лет назад
@@AmitTiwari aesi web site jisme 1500-2000 url ho, jaisa ke meine tumse pucha tha ?
@AmitTiwari
@AmitTiwari 5 лет назад
Oh, aise site ke case me apko dekhna hoga ki kaun se pages important nahi hai aur unhe disallow karna hoga taki apka crawl budget important pages ke liye bacha rahe. Agar kinhi pages me duplicate content hai to unhe bhi disallow kar sakte hai.
@radioshofi8735
@radioshofi8735 5 лет назад
robot.txt blocked please help me bhai
@AmitTiwari
@AmitTiwari 5 лет назад
please provide more information or your website's link via FB or Twitter. contact details are in the description of the video.
@brightfun1
@brightfun1 4 года назад
Mujhe ek pdf aaya h robot.txt ka, plz mujhe aap ye btayein k q aaya or iske baad mai kya kru.. Reply sir plz
@panchayatpress9819
@panchayatpress9819 4 года назад
Robot.txt file ko WordPress me kase find kare
@truefacts4063
@truefacts4063 4 года назад
sir robot txt file ko on kiya aur kuch nai likha toh kya hoga ? aur custom robot header tags me bhi on kiya lekin usme b kuch select nai kiya like nodp no index sirf on karke rakh diya toh kya meri website google search me show hogi ?
@sherry13646
@sherry13646 4 года назад
if a open cart file than what will be the disallow code
@Trendzya
@Trendzya 5 лет назад
Ones again two important information Amit ji please make video how to stop spam email ( my website is e-commerce) Thanks in advance 🌹👍
@AmitTiwari
@AmitTiwari 5 лет назад
kya apki emails spam me ja rahi hai ya apko website ki wajah se kafi spam emails mil rahi hai?
@Trendzya
@Trendzya 5 лет назад
@@AmitTiwari mujhe Apne Support wale email par regularly kafi fake emails received ho rahe hey
@AmitTiwari
@AmitTiwari 5 лет назад
Aap contact form ke liye RECAPTCHA use kar sakte hai jisse bots form fill na kar paye aur contact information section ko normal html ki jagah par javascript se load kar sakte hai.
@Trendzya
@Trendzya 5 лет назад
@@AmitTiwari okay noted I am not technical person but main yah Kara Sakta hu Thank you so much Amit ji 🌹👍
@anuragbansal5410
@anuragbansal5410 2 года назад
No: 'noindex' detected in 'X-Robots-Tag' http header i am getting this issue please help
@nirmalkholiya6647
@nirmalkholiya6647 4 года назад
Hello Sir meri wp-includes file google bot ne crawl karke index kar li thi jiski vajah se search console me 5×× error show ho rha tha. Ab sir mene use robots.txt file me add kar diya tha lekin abhi bhi wo files access ho rahi hai. Kya karu sir?
@designdrona5708
@designdrona5708 4 года назад
Disallow and noindex wali bat samaj nhi aayi
@rafiuali2425
@rafiuali2425 7 месяцев назад
robot.txt file edit kase hoga
@OMGtv-OneMinuteGuideTV
@OMGtv-OneMinuteGuideTV 4 года назад
I see callfailcause.txt file in my mobile. This us getting created automatically. Can you please tell me what it is and how can I stop it?
@sams5110
@sams5110 3 года назад
Crawler: Unknown error What's happening Our crawler can't access this page because of the issues listed. Note, these are not policy violations. What you can do Make sure you familiarize yourself with the way that the AdSense crawler works. For instance, the AdSense crawler indexes by URL and it won't access pages or directories prohibited by a robots.txt file. It's also different from the Google crawler. *Please help yeh mere AdSense account ke har page per araa hai error* kayse solve Kare sir please help
@quickinteriorpvtltd3755
@quickinteriorpvtltd3755 5 лет назад
Nice video Amit Sir, but I've one query. How to use robot.txt in chat bot. Where crawler not collect our clients personal chatting information. Thanks,
@KishoreGupta89
@KishoreGupta89 5 лет назад
Google crawler not read our chat messages. We can check from Google cached pages text version. @Amit- Sir i am right?
@AmitTiwari
@AmitTiwari 5 лет назад
@@KishoreGupta89 yes, correct!
@ShayariKaKhajana
@ShayariKaKhajana 4 года назад
User-agent:* Allow:/ Ho to usaka matlab kya hoga ?
@AmitTiwari
@AmitTiwari 4 года назад
iska matlab hai sare search engines apki puri website ko crawl kar sakte hai.
@ShayariKaKhajana
@ShayariKaKhajana 4 года назад
@@AmitTiwari thex 1) Disallow: 2) Allow:/ Ye dono saman hai
@NarendraBorse
@NarendraBorse 4 года назад
How to block ID dynamic pages in robots.txt file, Plz help
@AmitTiwari
@AmitTiwari 4 года назад
Use screaming frog to find the parameters of your dynamic page and then use wildcard to block them.
@NarendraBorse
@NarendraBorse 4 года назад
Thank you
@yt_souvik
@yt_souvik 4 года назад
Crawl-delay not working
@Premanandji1-x7e
@Premanandji1-x7e 2 года назад
Google pe mera robots txt error 404 arha h
@amoldutal775
@amoldutal775 5 лет назад
Html is good ya php??
@AmitTiwari
@AmitTiwari 5 лет назад
Both, it depends how the good coder is. Untidy code will be bad whether it is html or PHP.
@nhcrypto
@nhcrypto 5 лет назад
Where is link for Search Engine Bots?
@AmitTiwari
@AmitTiwari 5 лет назад
Sorry, I missed. Here is the link and fixed the description too www.robotstxt.org/db.html
@jahidparves5850
@jahidparves5850 3 года назад
Indian government 😂😂😂😂. Chaal copy mar dete hain 😂😂😂
@KoyalPk-gp1jn
@KoyalPk-gp1jn Год назад
Disallow: /cgi-bin/ what is the purpose of code in this robots.txt file
Далее
What Is Robots.txt | Explained
9:29
Просмотров 22 тыс.
Custom Robots.txt for Fast Google Indexing in 2024
19:09