Тёмный
No video :(

justforfunc #37: sync.Pool from the pool 

justforfunc: Programming in Go
Подписаться 40 тыс.
Просмотров 18 тыс.
50% 1

It's summer and I love hanging out by the pool ...
so it's just logic that I would make an episode about sync.Pool!
In this episode we learn how to measure how many memory allocations a program is performing and how to reduce them thanks to sync.Pool, as well as how to identify which piece of your code is
allocating the memory thanks to pprof.
source code: github.com/cam...

Опубликовано:

 

25 авг 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 51   
@OrElimelech
@OrElimelech 6 лет назад
Watching the 3 of you just laughing and having fun was the best part of the episode :)
@MffnMn
@MffnMn 6 лет назад
Hey, love your content, but I'm not sure this video demonstrates how pools are useful, since the main performance gain comes from adopting the smaller struct for the payload. I ran the benchmark with a normal allocation to for data data := &pullRequest{} at the beginning of the function and this produced a 246 allocs/op, i.e the pool is only saving one allocation for us. Could you do a follow up comment or video on some cases where pools give a significant help?
@Phscyoman
@Phscyoman 6 лет назад
I was about to comment the same thing.
@JustForFunc
@JustForFunc 6 лет назад
Sure, I actually wanted to make sure people didn't get the idea that pools are magic and you should use them everywhere (they're not) I'll add the new episode somewhere
@AlexeyZhuchkov
@AlexeyZhuchkov 6 лет назад
I also think that in this example it's better to use var data pullRequest Since it could be allocated on stack and stack memory is much faster than heap and does not require GC at all. However, judging by allocations, even in such a simple example, the compiler decided to allocate this memory on the heap. It would be interesting to know why.
@joonasfi
@joonasfi 5 лет назад
Awesome seeing you goofing around with my favourite container hero, Jessfraz!
@marinprcela2778
@marinprcela2778 5 лет назад
Maybe it would be fair to do step one (without pool) but with smaller struct as in step 5. Your parents’ house looks lovely! ❤️
@jub0bs
@jub0bs 3 года назад
Exactly!
@neknarqo
@neknarqo 6 лет назад
Run the benchmarks from the episode with Go 1.11beta1 and got "64918 B/op 22 allocs/op". Twenty-TWO allocations!
@JustForFunc
@JustForFunc 6 лет назад
Waaaaaaat, that's amazing 😮
@user-gb9ik8qq9w
@user-gb9ik8qq9w 2 года назад
You are awesome! Thank you for all your videos! I wish to meet you someday) from Russia with love))
@wilkins7407
@wilkins7407 6 лет назад
"Okay, let's do it once more where you actually say it" 😂
@_thehunter_
@_thehunter_ 4 года назад
thank you so much, for enlightening about pool with example.. I wasted hrs to understand this
@chneau
@chneau 6 лет назад
lovely ending :)
@pjox
@pjox 6 лет назад
I am extremely disappointed, I saw the thumbnail and I thought you were going to do the whole video with a pair of cute huge glasses on, but it was just clickbait and you broke my heart :'( . Other than that, amazing video as always. Thanks!
@JustForFunc
@JustForFunc 6 лет назад
Now I need to buy humongous sunglasses
@pjox
@pjox 6 лет назад
I agree! It should still be summer when the next episode of JustForFunc comes out :D
@johnbalvin5401
@johnbalvin5401 6 лет назад
Isn't the bechmark compare wrong? At the end it takes less resourses because you are only decoding one field and not because Sync.Pool
@neelesh2023
@neelesh2023 3 года назад
awesome
@azzalos
@azzalos 6 лет назад
Would've been interesting to benchmark with RunParallel in this situation, since requests would also come in like that.
@rodrigolopes10
@rodrigolopes10 6 лет назад
Hey man, what is this "whoseport" command? I was trying to find but I didn't find anything. Thanks man.
@deeeeeds
@deeeeeds 6 лет назад
Add this to your ~/.bashrc file: alias whoseport="lsof -i -n -P | grep $1" The command will slightly differ if you're not on a Mac.
@jimcanoa
@jimcanoa 6 лет назад
A linux version fwiw whoseport() { PID=$(sudo ss -H -p -l -4 "sport = :$1"|sed "s/^.*pid=\(.*\),.*$/\1/") ps w $PID }
@Rodkranz
@Rodkranz 6 лет назад
thanks man =)
@JustForFunc
@JustForFunc 6 лет назад
➜ which whoseport whoseport () { lsof -i ":$1" | grep --color=auto --exclude-dir={.bzr,CVS,.git,.hg,.svn} LISTEN }
@rodrigolopes10
@rodrigolopes10 6 лет назад
Really thanks man, it will be useful a lot, I didn't know that.
@easternsunking255
@easternsunking255 6 лет назад
does golang has the method like c using "memset" to quickly reset the object to zero?
@zohebakhtar1172
@zohebakhtar1172 4 года назад
hi , i have GRPC client and i am storing client objects in a map and i am reusing the connection , is a good idea to create sync.pool for this case. how can i do that
@petersafari9882
@petersafari9882 6 лет назад
Would using jsoninter instead of encoding/json have any effect on the metrics?
@hualetwang3105
@hualetwang3105 6 лет назад
赞,准备让我同事用这种方式优化一下我们的一个go库 😊
@robertoferrer535
@robertoferrer535 6 лет назад
Buenisimo como siempre
@TheSTVentertainment
@TheSTVentertainment 5 лет назад
In the doc: "Any item stored in the Pool may be removed automatically at any time without notification. If the Pool holds the only reference when this happens, the item might be deallocated. A Pool is safe for use by multiple goroutines simultaneously." Can you explain this to me? Really appreciate it.
@easternsunking255
@easternsunking255 6 лет назад
the example to use sync.pool github.com/Netflix/rend/blob/master/handlers/memcached/batched/conn.go#L189
@hoaibaonguyen477
@hoaibaonguyen477 6 лет назад
Hi Francesc, thanks for the video. However I'm still not clear why in the last (5th) step, no reset is correct while in the 3rd and 4th step, you must reset the data in the pool. I can't see the difference between those steps. Could you clarify more? Thank you.
@JustForFunc
@JustForFunc 6 лет назад
The webhook always contains the field, so resetting it is unnecessary But it wouldn't hurt to set it to 0, or even keep the *int and set it to nil (one extra allocation)
@hoaibaonguyen477
@hoaibaonguyen477 6 лет назад
JustForFunc: Programming in Go ah I see. Thank you for a great video.
@Kinagi
@Kinagi 6 лет назад
Can't wait to start using sync pool in my work! Thank you!
@komuwairagu3942
@komuwairagu3942 6 лет назад
hi @Campoy, regarding the fact that you need to 'zero'(for lack of a better term) out the thing you get from a Pool before using it; I was looking at the documentation for sync.Pool and I don't see it mentioned. Is that a bug in the documentation, or did I misread it?
@Kirides
@Kirides 6 лет назад
Well, you take something, write into it, read a value and return it back to the pool. The data you wrote into it, is still there, so the next function that uses the pool might get a pre-filled object and if it does not validate it correctly you run into issues. Usually pools are used for things like buffers where you rent a []byte, write into it and use it like this written := w.Write(buf, []byte("Hello! .... x100")) // this could be a serializer that reuses the given array val := r.Read(buf, 0, written) return val
@komuwairagu3942
@komuwairagu3942 6 лет назад
thanks,
@musale2277
@musale2277 6 лет назад
It's really nice seeing a comment from a familiar name :D Hi Komu!
@mishasawangwan6652
@mishasawangwan6652 6 лет назад
interesting i wrote a webhook bot for fun awhile back .. i have a question though, in the payload you use pointer to string and pointer to int.. why are these pointers instead of value types? thanks for the video
@vdemario
@vdemario 6 лет назад
Usually that's for the json package to put nil in the field when it's not in the payload.
@13TheGUNNER
@13TheGUNNER 6 лет назад
So that you can differentiate between a integer coming as 0 vs the integer not coming at all in the payload
@JustForFunc
@JustForFunc 6 лет назад
That's exactly why
@mishasawangwan6652
@mishasawangwan6652 6 лет назад
Subhojit Dutta damn that’s crazy thanks for the info
@dimitris5032
@dimitris5032 6 лет назад
And I thought you would be doing this in a pool also wearing the sunglasses
Далее
Only I get to bully my sister 😤
00:27
Просмотров 17 млн
justforfunc #29: dependency injection in a code review
28:09
Never write another loop again (maybe)
10:48
Просмотров 250 тыс.
Go 1.20 Memory Arenas Are AMAZING | Prime Reacts
16:38
justforfunc #9: The Context Package
36:27
Просмотров 140 тыс.
Golang Error Handling Is Better Than You Think!
18:53
Всё про конкурентность в Go
23:36
justforfunc #27: two ways of merging N channels
16:15
Only I get to bully my sister 😤
00:27
Просмотров 17 млн