Тёмный

Stealing Storage from Telegram 

Dev Detour
Подписаться 23 тыс.
Просмотров 184 тыс.
50% 1

By popular demand.
Discord: / discord
Github for code: github.com/devdetour/Telegram...
Ko-Fi: ko-fi.com/devdetour
Chapters:
00:00 Last Time
00:40 Shameless plug
0:42 Why I Didn't Do That
1:58 What/Why Telegram
2:32 Not Banned Yet
2:43 Teldrive
3:20 FUSE programs
4:36 Implementing It
6:18 Fixing It
6:34 Writing is Hard
9:48 Reading is Also Hard
10:52 Performance
11:29 TY, CYA

Наука

Опубликовано:

 

19 май 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 322   
@shayhan6227
@shayhan6227 3 месяца назад
This guy is the reason free services eventually start to cost money.
@R_Samnan
@R_Samnan 20 дней назад
Yea specially if its out in the public then alot of people will do the same, just because something is free doesnt mean it has no expenses behind it
@devdetour
@devdetour 3 месяца назад
1 correction, when I mention at 10:37 that Telegram is doing some encryption stuff and maybe that's why it's slow, this is incorrect. Though Telegram DOES support encryption, this was not an encrypted chat, so that wouldn't make things slow. Another clarification, I knew what RAR was before my last video, just not that it could break files into chunks and password protect them. I guess I phrased this badly :)
@Yoshidzo
@Yoshidzo 26 дней назад
The thing you mentioned is actually about end-to-end encryption (E2EE). Every chat is encrypted by default. The difference is whenever this encryption is E2E (telegram secret chats) or client-server (default).
@DarkxPunk
@DarkxPunk 20 дней назад
And to add, the data is not only encrypted to the server, it’s is obfuscated by breaking it into pieces and storing it on multiple servers so if only one server is access the data is junk.
@rayanfernandes2631
@rayanfernandes2631 3 месяца назад
2 years ago I wanted build the exact same fuse mount thingy with telegram, but gave up since it was very very very much ambitious for my knowledge in systems file i/o operations. Really informative content, Keep em coming. Enjoyed every bit! ❤
@Yuki-HoYo
@Yuki-HoYo 3 месяца назад
I was kinda surprise when I found your channel. You’re underrated for the type of content you’re making
@Spiderfffun
@Spiderfffun 3 месяца назад
ty you reminded me to subscribe to this amazing creator
@suvantmsuvantheminecraftia7407
@suvantmsuvantheminecraftia7407 3 месяца назад
I JUST saw your part 1 video of stealing storage from discord and your part 2 is up by the time I finished the video(uploaded an hour ago)!
@prawnproductions234
@prawnproductions234 3 месяца назад
The algorithm has brought me back. Thank you.
@UtkarshKoppikar
@UtkarshKoppikar 3 месяца назад
Can you open source this? Would like to contribute. Or you have already and I haven't reached that point in the video
@devdetour
@devdetour 3 месяца назад
Yeah! GitHub repo link is in the description. Contributions very much welcome!
@sketchychillandchill
@sketchychillandchill 3 месяца назад
​@@devdetourand the discord fs?
@-john-9438
@-john-9438 3 месяца назад
Me not knowing how Github works (Im not a programmer)
@guinea_horn
@guinea_horn 3 месяца назад
​@@-john-9438then why comment
@xynocode
@xynocode 3 месяца назад
this project may help me archiving my interpolated video project when I don't have any available HDD storage. thanks
@jamlie977
@jamlie977 3 месяца назад
that's amazing lmao to be honest, this approach is much easier than the previous one. it took me around 8 seconds to think of a way to create this, but the only difference is that i would have to use something like a /sync to send the new pictures/videos
@ohmygosh6176
@ohmygosh6176 2 месяца назад
Anyday Google Drive, Dropbox, or other cloud storage can change their policy or go down. That is why it is always a good idea to have multiple backup plans.
@JamesSmith-ix5jd
@JamesSmith-ix5jd 3 месяца назад
Telegram is slow for non premium especially in the evening (in Europe), sometimes downloading a simple 20MB video takes forever.
@luizansounds
@luizansounds 9 дней назад
Honestly just for the unlimited file size, it's worth paying the premium just for the speed, specially if you compress everything in 2gb chunks for backup
@smith3463
@smith3463 3 месяца назад
You're amazing. I had always given up on my dream of becoming a programmer, but you sparked it once more.
@GreatTS
@GreatTS 2 месяца назад
Same here. First I watched your video related to Discord unlimited storage and then realised why didn't i used telegram and then i upgraded my code for telegram.
@TunisiaSparx
@TunisiaSparx 3 месяца назад
awsome i always wanted to do omething like that ! uploading photos to some place and showing them dynamically from my deployed web app
@phizicks
@phizicks 3 месяца назад
I would've written each chunk as their own file and have a metadata file (FAT) for each message id part to the file. if you overwrite, you replace the initial X block files and delete the rest if they are smaller, or create new block files if they are bigger.
@killervacuum
@killervacuum 3 месяца назад
i use telegram as a backup mainly for the photos from my phone but also other important files. the main drawback is that the download and upload speeds can be quite slow sometimes.
@CodeF53
@CodeF53 3 месяца назад
7z supports encrypting, splitting, and compressing files into chunks and has better compression ratios than rar. I would also use the brotli fork, as it has an even better compression algorithm.
@Kalphalus
@Kalphalus 3 месяца назад
On discord, why dont use another channel to save the name and the file ids? Or at least make a backup of the id list every once in a while on Google Drive, Dropbox, (Telegram, Discord,) etc.
@Bromon655
@Bromon655 3 месяца назад
People who experiment with these free storage tactics are a large part of the reason why tech companies have started putting up paywalls for their services. Server space doesn’t grow on trees.
@Kipruto254
@Kipruto254 3 месяца назад
This is why we can't have good things. Telegram will likely impose a restrictions after this.
@Spiderfffun
@Spiderfffun 3 месяца назад
@@Kipruto254 didn't this guy mention something like this existing already?
@kaidenrogers
@kaidenrogers 3 месяца назад
@@Kipruto254 Telegram just deletes old files when they start running out of space. Your uploaded files aren't guaranteed to live there forever. The faster people use it up like this, the faster they delete. So it's really no sweat off their back. Other than increased bandwidth usage, but we saw Telegram's mitigation for that, they throttle it, hoping you'll upgrade to premium.
@R_Samnan
@R_Samnan 20 дней назад
exactly dude, free storage doesnt mean it auto generates itself with no costs infinitely
@kaidenrogers
@kaidenrogers 20 дней назад
@@R_Samnan It's a loss leader. And it's not like it's guaranteed storage. If they need more space, they just delete your old files. The storage may not auto generate, but it does regenerate.
@JyothisJayanth
@JyothisJayanth 3 месяца назад
btw the teldrive owner got his own rclone fork that supports teldrive. I was using rclone mount and it was working perfectly fine :)
@excessiveanalysis2982
@excessiveanalysis2982 3 месяца назад
One thing I noticed in your design layout was the decision to encrypt each segment of the total file separately. While this does not affect functionality, it can impact the information security of the uploaded data. By encrypting each chunk separately, with what I assume is the same symmetric key, you run the risk of crypto-analysis on the file chunks revealing data on the key and or plaintext. The reason for this is due to the fact, because of the segmented nature of the encryption, you are effectively re-using the same key for multiple encryptions. While this is not a huge deal practically, it is still a possible attack vector. A simple solution would be to simply encrypt the entire file once, and then split the encrypted data afterwards. Other than that (unrealistic, you used CBC mode by the looks of it, so IV helps a lot) security vulnerability in a scheme that was made for fun, amazing video! I must say, I love over-the-top systems that solve problems in a way-more-than-necessary manner simply because it can 😂
@skygnd
@skygnd 3 месяца назад
I’m from the future, you’ll have at least 500k subs by the end of this year man
@oussama7132
@oussama7132 3 месяца назад
can you reduce the background music volume next time?
@devdetour
@devdetour 3 месяца назад
Thanks for the feedback, I'll keep this in mind. Still trying to figure out the right audio levels :)
@toggenburger
@toggenburger 3 месяца назад
@@devdetour please dont go way lower. often videos have comedically quiet background music. Guess its a matter of preference
@Girgoo
@Girgoo 3 месяца назад
I would love to see a fuse system that cache usb-drive metadata only. Meaning I dont have to connect the usb-drive to my computer to check what files I had on it last time it was connected.
@thewhitefalcon8539
@thewhitefalcon8539 3 месяца назад
I think Windows has that and calls it hierarchical storage management
@Slada1
@Slada1 3 месяца назад
Everything from voidtools can do that too
@supernenechi
@supernenechi 3 месяца назад
Yay, got into the video! Great video dude!
@kabul891234
@kabul891234 3 месяца назад
NIce, just installed teldrive, just make sure to use your own api id and hash
@KaptanUfuk
@KaptanUfuk 2 месяца назад
yes. i found this method about 2,5 years ago but i never automated it, i couldnt do it. i was waiting for someone to do this application. thanks
@Twisted_Code
@Twisted_Code 3 месяца назад
FUSE sounds like it might be relevant to a problem I've had on my to do list to solve. Is this the same term used by the Windows ecosystem for the concept?
@MightyElemental
@MightyElemental 3 месяца назад
sounds great for a backup system that has no guarantee the data will remain there 😅 Those file upload speeds don't seem right though... Would adding more upload/download workers improve the performance? The data may need to be split into chunks for each worker, but it may be faster overall?
@AtaGunZ
@AtaGunZ 3 месяца назад
I'm pretty sure it is rate limited per account or even per IP
@edicarlosmartins1562
@edicarlosmartins1562 3 месяца назад
Hey man! Could you mentioned in the description that people also can TELEGRAM-UPLOAD from Pypi?
@weak
@weak 3 месяца назад
i never thought of telegram for storage
@wrathofainz
@wrathofainz 3 месяца назад
There are no good fuse tutorials in yt search, you should add fuselib or something to the tags or title of the video. I intend to use this for a personal server of mine if i can figure out how to get it working on my linux machine. Theres already some fusefs stuff going on under the hood so it can't be too bad... right? Edit: i downloaded the example script amd your telegram fusefs script as examples so hopefully I can make sense of it with those.
@ErikUden
@ErikUden 3 месяца назад
Sure, these are “bad” download and upload speeds if compared to transferring files to a local drive, but what about enabling it to sync just like Nextcloud or OneDrive does? Wouldn't it be awesome to use this in order to have locally stored files backed up to the cloud. I could assume the Nextcloud client would be a huge help to implement something like this here! What do you think?
@sassas4557
@sassas4557 3 месяца назад
TIP: Try to send all the zipped files togheter and to download all of them togheter and then allow the user to modify, etc., so the flow should look like "read:download all (storing in temp)->check if they are all (by seeing witch are the missing ones checking their names from the DB file)->once they're all merge the file->show the file", " write: if the new file was used less then x time ago (ex. 2 hours ago), wait; else, move the file to temp->split in chunks->check and write to db->upload->check with db". Like 3 months ago, I tried to do the same thing by hand, separating the files with Winrar and uploading them slowly, and it worked, but if I have to upload a lot of things, it could get messy. PS: We also have to think about resuming upload after power-up and checking partially uploaded files during shoutdown, but it can be resolved by leaving the files in temp until they're fully uploaded with the checks mentioned before and other checks when booting and shutting down the PC (we write the queue of files to upload and download somewhere in the database to know what we were doing before the shoutdown). Nice video, keep on going man, I love these kind of things. Sorry for my bad english btw.
@realcheez795
@realcheez795 3 месяца назад
this is amazing
@ManIkWeet
@ManIkWeet 3 месяца назад
A lot of specialized file download software can download a file using multiple connections simultaneously, maybe you can use that to improve speeds more?
@MrImodre
@MrImodre 3 месяца назад
fsync/fflush: exists devdetour: reinvented cache goes brrrrr
@devdetour
@devdetour 3 месяца назад
I did actually look at pyfuse3's fsync(), but I don't think it behaves like a cache. As far as I can tell from the pyfuse3 docs (www.rath.org/pyfuse3-docs/operations.html#pyfuse3.Operations.fsync), it just passes the fh, not actually the data, so a separate cache is still needed - and I couldn't actually get pyfuse3 to call into fsync() in testing, so I kept it in release(). Maybe not optimal, but it works!
@kleins0
@kleins0 3 месяца назад
Very interesting video again!
@maciej.opalinski
@maciej.opalinski 3 месяца назад
The caching scheme you implemented does not let you read files if they don't fit in RAM. Fuse manages data in chunks to allow working on files larger than free memory.
@davia.sampaio8633
@davia.sampaio8633 3 месяца назад
This is actually good if you dont want certain files on four system. This way you can just drop them on a "cloud" that has no connection to you
@kairu_b
@kairu_b 3 месяца назад
Another great video!
@Girgoo
@Girgoo 3 месяца назад
Maybe it is better if you just sync the file in the background, possible make small chunks to have multiple upload run simultaneously? Now with 2 different providers(telegram + discord) you can actually have raid 1?
@DanDrakeV
@DanDrakeV 3 месяца назад
This is so good!
@Sircliffe
@Sircliffe 3 месяца назад
Store files on RU-vid with digital data modes like OFDM-3500.
@nameq
@nameq 3 месяца назад
On a TDLib level for telegram you can request parts of the file instead of the whole file from the storage
@nourmagdi6031
@nourmagdi6031 3 месяца назад
i loved the Rick and Morty reference
@DealazonDaily
@DealazonDaily 3 месяца назад
I have an idea can you create an GUI for telegram only for uploading the files like if the members is free the file limit will be 2GB I'd the premium users is 4GB limit so if this idea create an GUI project is bit more powerful to use ??❤
@nand3kudasai
@nand3kudasai 3 дня назад
KISS instead of fighting against the cache. Upload the chunks and not the whole files. That would improve random access times in case you want to read or start reading from a random part. Pretty much the same way FAT stores files in chunks. It might even help if you implement some sort of parallelization in the pipeline. Ive seen mny of these tools, i find them interesting, i remember there was one to store files on email accounts (back when goggle falsely advertised infinite storage). Good job on using py fuse. Like others i would have used 7z to split and encrypt.
@nand3kudasai
@nand3kudasai 3 дня назад
It will also improve your ram usage problem.theres a reason why libfuse uses samll chunks and youre basically going against that.
@ayte1549
@ayte1549 3 месяца назад
It'd be fun to have a whole OS stored in telegram, and have your bootloader fetch if form there every time you boot up the PC.
@luizansounds
@luizansounds 9 дней назад
Dude making a suggestion of how a data center can boot x3
@rovicrosales5547
@rovicrosales5547 3 месяца назад
I have a question?? Does the files last as long as the telegram exists?
@ErikUden
@ErikUden 3 месяца назад
Imagine you could sync with this like with Nextcloud ^ ^
@notchieuwu
@notchieuwu 3 месяца назад
i LOVE fuse stuff, i've tried to do a similar thing in windows with terabox because the client sucks, but it didn't work out
@D0Samp
@D0Samp 3 месяца назад
I see we have advanced pretty far from the time I used a free ad-supported web hoster with filesize limits and a PHP script to concatenate split files.
@mehregankbi
@mehregankbi 3 месяца назад
did you also purchase premium? it's cheap iirc and it says speed does increase for premium users.
@parkour_pain
@parkour_pain 3 дня назад
You mentioned having to do some unconventional things to get the library to work for dlscord, what might those things be? I’m working on a different project but I’m needing to call functions in my bot too and I’m struggling
@parkour_pain
@parkour_pain 3 дня назад
I’ll take any help I can get I would appreciate it so much I am very desperate
@lancemarchetti8673
@lancemarchetti8673 3 месяца назад
Great video!
@erikehlers7006
@erikehlers7006 3 месяца назад
I have noticed, that the download speed is kinda slow even when you download files normaly. So I think it actually has to do something with their servers
@MrPeepa
@MrPeepa 3 месяца назад
Why not use read write streams for uploading and downloading. And then in the pipeline add a counter for the amount of chunks that got written or read.
@juuujuuu1564
@juuujuuu1564 3 месяца назад
What about pings? You can give them data and they'll send it back. Just do that a lot of times a second sending data to different IPs and you're storing data.. over internet wires!
@Spiderfffun
@Spiderfffun 3 месяца назад
I swear I saw a video like that already.
@juuujuuu1564
@juuujuuu1564 3 месяца назад
@@Spiderfffun Please link it if you can find it! Would be interesting.
@Jsephos
@Jsephos 3 месяца назад
@@juuujuuu1564 ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-JcJSW7Rprio.html
@realminecraaftt
@realminecraaftt 3 месяца назад
@@juuujuuu1564 ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-JcJSW7Rprio.html
@devdetour
@devdetour 3 месяца назад
Yes, someone has done this! ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-JcJSW7Rprio.html. It's really really interesting!
@SMASHTheGamingCanuck
@SMASHTheGamingCanuck 3 месяца назад
Wonder if Guilded would be faster then discord as it offers 500mb file upload limit
@this-one
@this-one 3 месяца назад
No need for Samba on your Linux host, if you want to run it on windows there is WinFsp, which is basically FUSE for Windows.
@heated1333
@heated1333 3 месяца назад
How is this not a negative for telegram? I love telegram as a service, I don't think I'm a fan of potentially them bearing more load then they need to when the provide such an amazing service so cleanly. Is it reasonable to ask to rethink this? Maybe do it to someone like Apple?
@hoangduong5954
@hoangduong5954 2 месяца назад
What about creating a bunch of google/microsoft account and store files in there?
@_Karlsson
@_Karlsson 3 месяца назад
Telegram download is slower than upload on purpose to decrease people misusing the service as much. But it's still really useful in a pinch.
@abasgames
@abasgames 3 месяца назад
you can store your data on RU-vid by uploading videos and use them as filesystem
@LovelyNucleus
@LovelyNucleus 3 месяца назад
2:50 Literally me with all of my projects 😭
@myfather7893
@myfather7893 3 месяца назад
wasn't there Unlim Cloud app which used telegram for storgae. Or did it stopped working?
@ALoonwolf
@ALoonwolf 3 месяца назад
I didn't have enough storage space so I used RU-vid to store all my videos, including my own film which took months of tedious work to make. Then RU-vid deleted my channel, destroying everything. Don't trust online storage!
@talhahussains1b47
@talhahussains1b47 3 месяца назад
start teaching coding ngl your really fun to watch❤❤
@zine_eddinex24
@zine_eddinex24 3 месяца назад
Cool keep up 🆙 👍 😅
@tutacat
@tutacat 3 месяца назад
Actually, HTTP supports something called download range, which only downloads the requested range from a file.
@_varianta007
@_varianta007 2 месяца назад
I do the same thing for years with files, links, music etc
@algj
@algj 3 месяца назад
You used FUSE and still - no random reads/writes, missed a huge opportunity to allow streaming media with low latency, torrent seeding and many other things.
@undead2146
@undead2146 3 месяца назад
tbh when you live in a country where an SSD costs an entire months paycheck, a free but shitty cloud service doesn't seem that bad.
@despareint
@despareint 3 месяца назад
Till Telegram bans your account for whatever TOS violation, you may appeal but your files are gone Other issue is capped download speed
@reilshat
@reilshat 3 месяца назад
The download speed in telegram is limited just so that you don't use it as a file storage
@flamingkillermc2806
@flamingkillermc2806 3 месяца назад
Didn't know about winRAR and telegram 💀💀💀💀
@ohhim.
@ohhim. 3 месяца назад
Oh man do we need a windows version of this
@brennanlaurent4748
@brennanlaurent4748 3 месяца назад
What about steam could you store files on steam using there cloud save feature
@MccTube
@MccTube 3 месяца назад
I love it when you do this lol
@matheuswmac
@matheuswmac 3 месяца назад
bro, the worst day was when I lost my group with my files and there was no backup, (I was banned from my group somehow) so I don't trust Telegram much for backups
@kaidenrogers
@kaidenrogers 3 месяца назад
Yeah, Telegram doesn't guarantee they'll keep your messages. That's how they can offer storage for "unlimited" files. If they start running out of space, they start deleting old files.
@Chicken_Soy
@Chicken_Soy 3 месяца назад
thank you
@AnoshterHaar
@AnoshterHaar 2 месяца назад
Does this provide any form of redundancy? Cause why would you otherwise even bother the slightest... There are zip programs out there making raid like redundancy possible for zip files. Multipar for example.
@GabrielTobing
@GabrielTobing 3 месяца назад
Pirates: Rarrrrrrrrr XD
@hwtw
@hwtw 3 месяца назад
How about make telegram and discord on a raid like thing?
@simon_roy
@simon_roy 3 месяца назад
Can you get ban from telegram for doing this ?
@JellyRockFN
@JellyRockFN 3 месяца назад
Is this safe to store content in telegram?
@hanspeter9636
@hanspeter9636 3 месяца назад
Can it run DOOM?
@raceup
@raceup 3 месяца назад
I think the download speed on telegram can be improved by buying telegram premium..
@InfinitycgIN
@InfinitycgIN 3 месяца назад
yeah, tried it some time ago, teledrive has very bad speed in general, got myself a 4tb drive instead
@tomerbs440
@tomerbs440 3 месяца назад
any windows alternative to FUSE?
@Omaryllo
@Omaryllo 3 месяца назад
I'm surprised you didn't write any streaming logic. I suppose a 2GB max memory cache isn't the worst, but it could also perform way better
@JyothisJayanth
@JyothisJayanth 3 месяца назад
Yep, telegram supports that too and it works really well
@RajveerSingh-vf7pr
@RajveerSingh-vf7pr 3 месяца назад
Oh My god, I havn;t seen the video but i have a very advance app for this.. No matter the file size, and all parts are hashed in a db,
@bootmii98
@bootmii98 3 месяца назад
Stealing Storage from RU-vid next? Is there any stego in this video?
@frankruan6842
@frankruan6842 3 месяца назад
The reason why Telegram download is so slow is that Telegram limits regular users' download speed cuz they want to make use buy their premium subscription. That was intentional.
@turikhay
@turikhay 3 месяца назад
Telegram is pretty aggressive when it comes to bandwidth or API usage.
@kaidenrogers
@kaidenrogers 3 месяца назад
Bandwidth, sure. But API usage? I don't think they're any worse than most other platforms out there.
@esecallum
@esecallum 3 месяца назад
MEGA does 50 gb ?
@F0reverYoung
@F0reverYoung 3 месяца назад
it good for another backup
@Hoopaugi
@Hoopaugi 3 месяца назад
I completely forgot rar files were a thing. Don't really run into them outside of torrenting games.
@czypherth
@czypherth 3 месяца назад
what if you emulate how ssd works
@user-wz7sx2ii3w
@user-wz7sx2ii3w 3 месяца назад
Telegram really slow sometimes in uploading/downloading
Далее
Stealing Storage from Discord
7:39
Просмотров 1,6 млн
Why Are Open Source Alternatives So Bad?
13:06
Просмотров 312 тыс.
How A Steam Bug Deleted Someone’s Entire PC
11:49
Просмотров 842 тыс.
i cant stop thinking about this exploit
8:40
Просмотров 285 тыс.
The $0 Home Server
16:53
Просмотров 734 тыс.
how to be a Linux user
5:01
Просмотров 11 тыс.
4 Billion If Statements | Prime Reacts
9:47
Просмотров 464 тыс.
Why You Can't Visit 💀🎺.com
8:50
Просмотров 482 тыс.
STOP using Cloud Storage! Do this instead:
8:44
Просмотров 771 тыс.
What's Wrong With This Hello World?
8:58
Просмотров 40 тыс.
подключение в противофазу
0:56
Просмотров 262 тыс.