Тёмный

How To Stream Large Files Over TCP In Golang 

Anthony GG
Подписаться 61 тыс.
Просмотров 31 тыс.
50% 1

► Join my Discord community for FREE education 👉 / discord
► Become a Patreon for exclusive tutorials 👉 / anthonygg_
► Follow me on Twitter 👉 / anthdm
► Follow me on GitHub 👉 github.com/anthdm
In this Golang tutorial, I will teach you how you can stream large files over a TCP network in Golang.
#golang

Опубликовано:

 

30 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 56   
@anthonygg_
@anthonygg_ Год назад
► Join my Discord community for free education 👉 discord.com/invite/bDy8t4b3Rz ► Become a Patreon for exclusive tutorials👉 www.patreon.com/anthonygg_ ► Buy me a coffee 👉 donate.stripe.com/aEU2a6ayH2uCa3u4gg Thanks for watching
@joaomendoncayt
@joaomendoncayt 4 месяца назад
"just use little-endian and drink margaritas at the beach" "not too big because my pc is pepega" these videos are just perfect, perfect pace, perfect content, perfect tone, perfect accent
@user-fr6gw8xr4b
@user-fr6gw8xr4b 2 месяца назад
Great video. I saw a few problems with the final solution. In the sendFile you append the size of file to the conn. In readLoop you read that size in the loop which is problem. And even if the size is in out of the loop, this will try to copy the entire byte data from the conn into buf in one go.
@fredericcordier1519
@fredericcordier1519 Год назад
this video is really great and very educational with the different steps/tests. Thanks for your work, the days are too short to watch all your videos :)
@anthonygg_
@anthonygg_ Год назад
Thank you!
@ArtemTimofeev-iw5cl
@ArtemTimofeev-iw5cl Год назад
man... i cant express how much this will do for my cdn in golang... Dear Anthony, bless you😃
@anthonygg_
@anthonygg_ Год назад
Your welcome my man!
@bashscript2805
@bashscript2805 Год назад
скинь ссылочку на cdn
@otaxhu8021
@otaxhu8021 8 месяцев назад
Your videos are very helpful, no doubt this is the best golang content creator
@Alibek11221
@Alibek11221 9 месяцев назад
Like your work! Thank you! Learned a lot through your videos
@hebozhe
@hebozhe Месяц назад
Go beast! I needed this locally for a 20GB corpus, so thanks for the tips.
@bjugdbjk
@bjugdbjk Год назад
Awesome , Thnx for the amazing Go content.
@Nexjsdeveloper
@Nexjsdeveloper Год назад
i need to see ur video over and over! Perfectly fine.
@jatindersinghaujla
@jatindersinghaujla Год назад
Before watching this video I just listen what streaming is but today I have seen how actually it works. This streaming concept clear but buffer is and how stream it. Great we are expecting more exciting and interesting videos.
@user-hn9xf3oj9h
@user-hn9xf3oj9h Месяц назад
thank you for uploading great video!
@shoebilyas9664
@shoebilyas9664 Месяц назад
your channel is a rare gem
@anthonygg_
@anthonygg_ Месяц назад
Ty
@vaibhavmewada4508
@vaibhavmewada4508 2 месяца назад
nice video anthony
@lowe7372
@lowe7372 11 месяцев назад
You are the best!
@Andika-hk6rd
@Andika-hk6rd 5 месяцев назад
I'm new to programming and I think this is cool, gonna save the video, I think it's useful :)
@oscarllamas
@oscarllamas Год назад
Great as always
@9scott291
@9scott291 Год назад
Thank you for sharing
@MaybeAwab
@MaybeAwab Год назад
the best
@Chxrli3
@Chxrli3 20 дней назад
Thank you so much for this video :) its really helpful. Do you have any links or videos about parsing from the loop?
@aianshu
@aianshu 2 месяца назад
Awesome
@ruslangabitov5202
@ruslangabitov5202 5 месяцев назад
Thank you for your video. How will this transmission work if the sender don't know the file size? Is it possible just send the EOF in the end of transmission? Does conn send EOF when the connection closed?
@fullstack_journey
@fullstack_journey Год назад
Time to build my own OTT platform startup
@ollegio
@ollegio 2 месяца назад
But your first implementation is also a streaming one: you are not reading whole file into memory, you have a fixed buffer which is updated on each Read. I think io.Copy just does the same thing
@anthonygg_
@anthonygg_ 2 месяца назад
I think io copy is bit more effiecient under the hood.
@deNudge
@deNudge Год назад
Why are you reading the file size within the for loop?
@L0wPressure
@L0wPressure Год назад
Yes, it leads to neverending stream of random bytes. Get it above the for loop and it will do what's intended.
@simo_the_goat
@simo_the_goat 2 месяца назад
MR Anthony I really appreciate your content. Please I have a small question I want to know why I opened my activity monitor and my RAM is getting too large even though It's streaming
@oleks19999
@oleks19999 3 месяца назад
I just don't get one thing, what condition will stop the for loop ? Cause we want to stop it at some point right ?
@jondoe79
@jondoe79 Год назад
Anthony is a Go Master ultimatum
@anthonygg_
@anthonygg_ Год назад
The undisputed.
@wMwPlay
@wMwPlay Год назад
Can I download twitch stream and stream it to youtube upload API directly, using this method?
@anthonygg_
@anthonygg_ Год назад
That will involve some more work my man
@sovrinfo
@sovrinfo Год назад
Awesome video. Big thanks. I have a question, if I set size: 400000 program exit with code 1. Error message: recieves 400000 bytes over the network 2022/12/16 20:50:03 EOF Process finished with the exit code 1
@anthonygg_
@anthonygg_ Год назад
Could be a read or write timeout. You can configure that
@sovrinfo
@sovrinfo Год назад
@@anthonygg_ Thanks
@tonycheng1614
@tonycheng1614 Год назад
I am very new in go. Can you specify the advantages of streaming file data comparing non-streaming file data in transfer? Thank you sir.
@anthonygg_
@anthonygg_ Год назад
Non streaming is saving the complete file in memory. Hence 30 gb files will not work on a simple server
@MarinescuEdward
@MarinescuEdward 3 месяца назад
@@anthonygg_ even with the second appraoch if you send 30 gb file it will still allocate 30 gb of memory `binary.Read(con, binary.LittleEndian, &size)` since size is 30 gb
@ghostbuster5266
@ghostbuster5266 Год назад
You need read file size before loop 😅
@adamsribz
@adamsribz 11 месяцев назад
Would be amazing if you could share github snippets with your videos
@anthonygg_
@anthonygg_ 11 месяцев назад
Im getting paid way to less for that. This is free content made in my free time. Leeched by many. Returned by a few.
@adamsribz
@adamsribz 11 месяцев назад
@@anthonygg_ I feel you! Your hard work shows even in your free stuff.
@madhudson1
@madhudson1 3 месяца назад
the for loop on the read with the end solution is a bit redundant i think. Wouldn't this solution just endlessly (for huge files) read into memory anyway? the Buffer will just fill up continuously, instead of allowing you to process x number of bytes?
@anthonygg_
@anthonygg_ 3 месяца назад
I need to check this, you could be right
@headlikeahole6548
@headlikeahole6548 6 месяцев назад
How does the method "binary.Read" know where the size data is and where the file data is?
@twitchizle
@twitchizle 5 месяцев назад
Server sends, filesize. Binary reads filesize. Server sends filedata. Binary read file data.
@tanko.reactions176
@tanko.reactions176 11 месяцев назад
brother, this is the second time i see you sending stuff over the network without taking into account that the operating system can not always send N-bytes just becuase you wish to send N-bytes. it may send N/2.. or much less. if you wish to stream, you have to keep track of a pointer to how much has been send. if N/2 has been send, then you send from N/2 until N.. you can not declare the chunk to be sent, if only a piece of it may be sent. the issue does not occur locally, it occurs when you do this over the net!
@anthonygg_
@anthonygg_ 11 месяцев назад
What are you talking about my man. Read net.Conn.
@dazealex
@dazealex 7 месяцев назад
I have no idea what the dude is saying either. RTFM @@tanko.reactions17
@mjhsaja
@mjhsaja 5 месяцев назад
rand.Reader is missing
Далее
How To Refactor A Golang Project By Using Interfaces
18:38
Coding a Web Server in 25 Lines - Computerphile
17:49
Просмотров 330 тыс.
🛑 до конца!
00:12
Просмотров 26 тыс.
How To Build A Custom TCP Server In Golang!?
19:32
Просмотров 26 тыс.
How To Use The Context Package In Golang?
17:03
Просмотров 56 тыс.
Golang pointers explained, once and for all
13:49
How To Use Goroutines For Aggregating Data In Golang?!
17:15
Making Minimalist Chat Server in C on Linux
14:28
Просмотров 40 тыс.
Golang Channels Or Wait Groups? Let Me Explain.
18:32
A Practical Example How To Use Interfaces In Golang
14:42
This Is The BEST Way To Structure Your GO Projects
11:08
My Viewers DDoSed my Go App
2:36:31
Просмотров 53 тыс.
Why I Use Golang In 2024
9:21
Просмотров 276 тыс.