How we program multicores - Joe Armstrong

Подписаться 1,5 тыс.

Просмотров 70 тыс.

0% 0

When we write a program, we just want it to run faster when we run it on a multicore. If I have a 10 core computer I just want it to run 10 times faster, if I have a 100 core computer it should run 100 times faster. When we program in Erlang this is approximately true. Our goal is that applications run 0.75 x N times faster on an N-core computer. In this talk I'll explain how we do this.
Joe Armstrong is the inventor of the Erlang programming language. He has a PhD in computer science from the KTH Royal Institute of Technology in Stockholm. He has worked in industry, research and as an entrepreneur and is the author of several books on concurrent programming.

Наука

Опубликовано:

9 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 95

@RFC3514 4 года назад

<a href="#" class="seekto" data-time="1369">22:49</a> - This is the bit where he told us the real secret of parallel programming.

@birrextio6544 4 года назад

I think he said that google does it wrong :-)

@vladirimus2689 4 года назад

Any way to find out what he said during that time interval? Is there a transcript?

@vladirimus2689 4 года назад

No transcript found if you search on Google. I sense foul play.

@0x44Monad 3 года назад

Very odd...

@rickacton7540 8 месяцев назад

HAHAHA

@fosres 3 года назад

I am truly sorry this legend of a human being passed away. It would be awesome if he was around. A benevolent speaker. An amazing writer. And the inventor of one of the best programming languages to conquer the problems real-time systems face everyday.

@DuTuben32 2 года назад

He held a course in Erlang at my university. The man had a galaxy sized brain.

@fosres 2 года назад

@@DuTuben32 It must have been amazing having him as a professor. Thanks for sharing.

@llothar68 Год назад

Only 68 years old. I now start to worry about myself. We are all mortals even if youtube helps to extend our recognized life a few more years.

@nathanlewis42 5 лет назад

RIP Joe Armstrong

@amenostalgique 7 лет назад

Erlang is my new religion

@danielsmith5626 17 дней назад

<a href="#" class="seekto" data-time="1368">22:48</a> The one true computer scientist. He knew the way.

@MrDigitsu 2 года назад

A bow to the legend. These are the real heroes of the world.

@senthilramalingam9500 7 лет назад

Awesome. Joe, the man who nullifies my current thought process and makes me think in a different perspective and define a problem correctly before choosing a solution path.

@rgw5991 Месяц назад

and you are still a humunculous

@rgw5991 Месяц назад

how fat???

@ximalas 5 лет назад

Joe mentioning Alan Kay's thoughts about throwing everything away and beginning anew and Facebook's lack of reply on what to do with what we leave behind, this all reminds me of a Star Trek episode or two where parts of our history was simply lost. So maybe we should start over again before it's too late.

@laurelsporter4569 2 года назад

Star Trek? What about China and Russia, that actively destroyed history? What about the Dark Ages, where the Church's expansion, combined with difficult daily lives, made people not care of their old history, and often probably reuse items it was recorded with? What about the height of and fall of Egypt? Despite their attempts at extensive record-keeping, there are huge gaps. What of the Native American mound cultures? It's like one day they went from high stone age empires to scavenging tribal scavenging warlords, and none have any - public, at least - stories to tell about that collapse. It's abnormal for us to keep good records.

@JohnBat26 7 лет назад

Amazing presentation !

@MCLooyverse 3 года назад

The way I think about multi-threaded, parallel, distributed, or whatever-you-like programming (which I've done practically none of) is that you have some problem to solve, so you say to your buddy "Hey, get me the solution to A." then you get the answer to B, then when you're both done, you combine your answers into the solution to your original problem. Of course, some problems can't be "factored" like that (maybe you could call them "prime"?), which I'd imagine is where he's going with this talk (which I obviously haven't finished yet).

@mohamedfouad2304 4 года назад

its videos like this that makes me keep the youtube app

@MultiNnn999 7 лет назад

Awesome talk! More programmers need to know this information.

@vaishakm6 4 года назад

Hello Joe, Hello Mike, Hello Mike, Hello Robert

@nero1375 3 года назад

ohhh I see what you did here! ahahahaha

@AlinNemet 7 лет назад

great stuff

@user-og7ev8uo8b 8 дней назад

absolute legend

@BryonLape 3 года назад

Why is it that in so many of these tech conference videos the audio is horrible?

@venkatrajanala6841 5 лет назад

Rip Joe !

@morthim 4 года назад

audio cuts out

@garyda6605 2 года назад

Genius

@shamzfk 7 лет назад

Just as a side note, why Go is on a Shared Memory side of things? at <a href="#" class="seekto" data-time="300">5:00</a> It does allow it, thats true, but it also discourages it while providing goroutines and channels as the means to implement ideas of CSP which both Go and Erlang are based on.

@neuronist 5 лет назад

exactly, Go does allow you to pass messages and the scheduling is comparable to erlang, but its still very viable, easy and normal to use shared memory models in go, which would also - for me - put it generally more on the shared memory model places, even though you can opt for a different way. I am still not sure if that is a good thing necessarily and go should have some opinion on that part.

@aimp4598 3 года назад

Goroutines and channels do nothing to discourage shared memory. Rather, its lack of constraints facilitates the sharing of memory. To suggest it shouldn't fit on the shared memory side would be like suggesting that Go shouldn't be considered an imperative language because it has higher order functions. Yes, you could write in a purely functional style, but it's not a fundamental characteristic of the language.

@GertVeltink Месяц назад

@shamzfk I know I am late to the party, but I agree with you on the position of Go in the diagram. One of the Go mantras is: “Don't communicate by sharing memory, share memory by communicating.”. Yes memory is shared, but it is done in such a way, that if the message memory is with the receiver it is logically no longer with the sender. However, I have to put a side note on the CSP remark. CSP uses synchronous communication, the Actor model only supports asynchronous communication, which makes a large difference! The beauty of Go channels is that they support both synchronous (= unbuffered) and asynchronous (= buffered) communication. One final remark on the diagram. I’m amazed that Scala isn’t mentioned on the right side, because it clearly supports the actor model.

@chenzhixiang3116 Месяц назад

I would prefer a new approach: To program in a distributed promise style use language native async/await syntax, which then could be transformed(lower down) by a smart compiler into message passing style... MQ is still imperative, Declarative is better.

@bwfrieds 6 лет назад

The wisdom at <a href="#" class="seekto" data-time="1020">17:00</a> stuck me hard. Not sure why. Should have been obvious to me.

@dlwatib 5 лет назад

The problem is that computations are not grains of sand. They have data dependencies between them. They're more like spiders sitting in the middle of spider webs connected to other spider webs with other spiders sitting in the middle of their webs. And some spiders are bigger than others and have much longer lives, but most spiders have very short lives so it's hard to fit them neatly into barrels because they keep disappearing.

@zomgdude1 3 года назад

@@dlwatib This proves Joe's point exactly. Programs are only like spiders in spider webs if they are modeled like this, it's not intrinsic. This sort of model (which is the most common one as evidenced by the graph of message passing model vs shared memory model), ends up making parallelism even more difficult than it already is. Erlang is specifically designed so that computations are, in fact, like grains of sand as he described, purposefully avoiding spider webs.

@oberguga 4 года назад

Maybe a stupid question... If we declare equality between pure function and processes can it works? Just put output of function to atomic buffer and work with futures by default.

@dennydravis8758 2 года назад

That's an interesting thought... Hmmm You'd have a bit of an issue with tiny functions - I mean after all a lot of functions are recursive in one way or another

@oberguga 2 года назад

@@dennydravis8758 recursive function is kinda special case that detected by most compillers, so it can work in one process. Other tiny functions like add(a,b) also kinda special case and can be inlined, just because it's size or because of some tag...

@sanderd17 5 лет назад

Does JS fit into the "Shared memory" group of concurrency? If you consider traditional JS, it's not paralel, and programming style usually forces you to keep memory in a closure, so not shared. When you add workers, those communicate with messages instead of memory.

@maximmatyunin9934 9 месяцев назад

Well, JS (Node.JS) is (mostly) single threaded. And you can pass references to objects around and change anything anywhere. Yes, it has workers, but those are usually used for some kind of big work, you don't usually spin thousands of workers in JS, you don't have something like OTP to manage them

@AndersJackson 3 года назад

At <a href="#" class="seekto" data-time="2850">47:30</a> he was talking about the future. About Corona, apparently. At least as most people thing it is.

@vimalk78 7 лет назад

where is sound at <a href="#" class="seekto" data-time="1370">22:50</a> ?

@zacharylarson1245 7 лет назад

That's where Joe slips up and accidentally reveals to the world that we are being visited by beings from another (concurrent? parallel?) Earth, where humanity took a twist in the history of programming languages and has hence found a way to bring harmony to their universe. The Ministry of Information of the Global Illuminati had to censure this most guarded secret.

@DanGK 7 лет назад

"...beautiful diagram, the way we program that... *hand waving* ...destroy the JVM... *broad handwaving* ...death, a side-affect of mutable human state... *outward, inward handwaving* ...writing a small program..."

@weneedtogoback8553 5 лет назад

@@zacharylarson1245 O_o das woke

@vadityag 3 года назад

What does he mean by No FP hardware(more CPUs instead)?

@0x44Monad 3 года назад

no useless floating point units

@whossname4399 7 лет назад

How is the sending the program to the data done in practice?

@MatthewHolevinski 5 лет назад

(disclaimer: I'm an idiot.) But I think a somewhat similar real world example of that might be Joyent's Manta offering.? It appears to be somewhere within that realm (from a cloud based perspective).

@skepticmoderate5790 5 лет назад

Javascript programs are sent to the browser to be evaluated all the time. Also, there was a time when we used to download programs to solve problems.

@rodjacksonx 4 года назад

They were angry it only went 33x faster rather than 64x faster on 64 cores? Yeah, that sounds like typical, pointy-haired management.

@derekfrost8991 4 года назад

If your language does not have a strategy for parallelism, then it's days are numbered. Even prolog can do this, albeit with libraries.. :)

@nolan412 3 года назад

"C is like airline." Thanks RU-vid captions.

@nolan412 3 года назад

I will assume that was a couple minutes of swearing at programming an Earth to Moon and back system.

@nolan412 3 года назад

OMG. Prescient at the end.

@LordZordid 4 года назад

Why exactly does quantum mechanical entanglement not allow the transmission of information faster than the speed of light if you could manipulate the particles?

@victoreijkhout7115 4 года назад

I agree that shared memory should be avoided at all costs. But then he mentions briefly that Erlang has "xtables" because you don't want to reproduce large databases. Ok, but now you need locks &c again to prevent conflicting accesses again to these big tables. And so you have all the problems of shared memory. Not?

@lucf8476 3 года назад

No - ETS tables use memory of course, but you interact with them in one of the same ways you'd interact with external storage, ie, you send messages to a process, they're just fast (though not strictly as fast as just munging memory directly). One of the problems in using shared memory is, for instance, a shared references to data that's been changed 'out from under you' and unexpected/undefined behavior, and the various ways you need to complicate your code to detect or prevent that, and that's a problem that doesn't happen with data that's copied out of an ets table and returned to you in a message. It's pretty fast, basically like Redis, but like Redis (unless you've embedded it), there'll be copying overhead so it's not a magic bullet -- but also like Redis, it's in the right order of magnitude to speed up a whole lot of things.

@bocckoka 5 лет назад

I don't understand the question. - Jon Gjengset (see evmap crate). RIP Joe!

@neuronist 5 лет назад

It's funny how he says that the notion of things happening at the same time doesn't exist in Physics and has an amazing example...Yet...isn't it the same in your computer, the cloud, anywhere? The fallacy of thinking something could be simultaneous when in fact nothing is but is only the observable part and we come up with solutions to agree on simultaneity (is that a word?) like consensus protocols and name it blockchain, but are in fact just a way to agree on something that does inherently not exist in reality.

@GeorgeTsiros Год назад

that example was quite weak. Of course there exists the concept of simultaneity in physics. He literally _based_ the example *on the two stars exploding simultaneously* once again: the basis of the example is that the two stars indeed exploded *simultaneously*

@moshedimawalaadormeo 3 года назад

Sad that he passed away sometime ago

@NormanZhou 7 лет назад

Elixir all the way

@monetize_this8330 5 лет назад

What I don't get is, if BEAM OTP is a runtime system executing on an OS that is basically multi-threaded, how does it wrangle control away from the kernel scheduler? When my application gets scheduled by the OS, I have no idea how many cores, I might or might not, have given to me. Not constant either. I'd rather not even think about parallelism at all. It's a job for the language syntax and cpu/gpu features. Like matching parallel constructs to GPU features automagically.

@hidragerrum 7 лет назад

God, he even considering the death of his spouse. LOL

@MatthewHolevinski 5 лет назад

That's when you know you have a great speaker :)

@camilojazzfernandes 4 года назад

hahaha ... 3d args could seem silly in xd p ... hahaha ...

@goverdhank 7 лет назад

the talk (though good) was a glorified advertisement of Erlang - and, should have been titled on those lines !

@MatthewHolevinski 5 лет назад

I feel like he answered the title at 17:42, and as for an advertisement it to me sounded like he was bluntly honest about it not being the end all be all to all the worlds problems, but expressed it was a helpful tool.

@dlwatib 5 лет назад

All Joe's talks are ads for Erlang.

@john-danson3113 5 лет назад

Why are we still talking about cores when thread lock is the issue. Thread to Ram ratio is more important. That and the correct o/s and language / library and I do not mean Linux and c++. I am about to benchmark GO on Lambda versus C# on lambda and we'll see which wins. Yes this is nonsensical, no the metrics have it.

@tobiasfuchs7016 7 лет назад

I like and support Erlang in general, but this sounds ... off. The claim in the description is dubious. So, Amdahl's Law does not exist for Erlang? A general claim of linear speedup is implausible - no matter the programming model, just because of basic math. Or I'm mistaken and all algorithms can be formulated in Erlang without synchronization and sequential sections. <a href="#" class="seekto" data-time="2083">34:43</a> ... oh, I see, they refer to "massively parallel" (also known as "embarassingly parallel") problems and aim at 0.75 x N speedup. That's underwhelming. And to look down on shared memory parallelism is en vogue because oh my, the complexity, but presumably Erlang is subject to the same memory hierarchy as any other application, so how is data locality maintained or exploited in Erlang? If the answer is "we don't care" then all statements on robust performance are bogus. Or restricted to some problem domains, or anecdotic. "36 computers" is, to keep it in the message passing domain, a debugging setup for MPI applications at best.

@llothar68 7 лет назад

Amdahls Law is in practice the least of the problems. And often can be worked around when looking at the real world and rewriting the process instead of scaling a bad requirement. Like youtube view counters which do not need to update in sync (just sync some day) or all the NoSQL used now after they found out that a single SQL database is stupid for the real usecases. The very few places where Amdahls Law is relevant like adding number on your bank account or zipping a file for ONE user can be handled with fast hardware. And it's just one bankacount or one user.

@tobiasfuchs7016 7 лет назад

That's true, but not about my point, I was just using Amdahl as an example from the top of my head. Also, you sometimes just can't cheese it by relaxing the requirements, like in system programming. There is no "eventual consistency" in laser control loops.

@MatthewHolevinski 5 лет назад

agreed, dragon wrestling requires adequate amounts of wrestling.

@juansanchezgonzales3934 5 лет назад

@@llothar68 r

@skepticmoderate5790 5 лет назад

0.75 * N is underwhelming? What are you smoking?

@yorik2588 5 лет назад

Unfortunately, or fortunately, the problem that Erlang address is nothing which can't be handled by c++ and the like, provided you throw enough money and time at it. Facebook chat was written in Erlang but has now been rewritten in c++. Reason: engineers were not quite comfortable in Erlang and they had a lot of existing threading libraries in c++. So I think, what he says is very right, but big companies would not likely adopt Erlang. So as an engineer, is it worth learning Erlang ? Yes: If you are into programming for the love of it, and are not merely learning Erlang for career progression. But practically speaking, to go corporate, learning Erlang would not add a lot of value. Try convincing the business division that they should switch to Erlang.

@pietroviske 5 лет назад

Nowadays programmers are turning into Elixir, which is being highly successful by packaging all the power of Erlang with a better syntax, and the community is just awesome. A lot of new companies are adopting it and I believe we'll see much more in the future.

@RsZ789 4 года назад

@Sergio Díaz Nila Do you think Elixir can ever be used as a replacement for C++? The fact that it runs on BEAM means it will never have direct access to the low level of the system like C++ can.But, is that really necessary?

@newsgroupsusenet5486 3 года назад

@@RsZ789 You've never heard of Native Implemented Functions (NIFs) in Erlang. With NIFs you can call C/C++/Rust from Erlang/Elixir

@Luix 5 лет назад

golang maybe the solution...?

@nolan412 3 года назад

Looks like Erlang is still winning benchmarks.

@jessethompson2950 6 лет назад

"blah blah blah simultaneous, doesn't mean anything in Physics, we don't have the language for this (drones on for five minutes about exploding stars).." Look. Dude. Just say "Spacelike" and keep on walking. Okay? If they ask "What's spacelike mean?" Just say "Oh, that's just 'simultaneous'" for smart people, *and it's equally applicable to this topic*. ;P

@petertomov5728 4 года назад

No. This is a lecture. The explanation is structured, fairly self-contained precisely because of how he declares a term that he's going to use, explains it and then uses it. It's precise and correct, how an explanation for a large audience of people with mixed backgrounds should be. If your approach to programming is the same as your outlook on this lecture, you should listen carefully once more to this lecture and others, similar in nature. Simultaneous means "happening at the same time" Parallel means "happening at the same time as but not being affected nor affecting other instances" Concurrent means "happening at the same time, running separately from other instances while accessing one or more shared resources"

@pearz420 2 года назад

Yes, let's make something less clear and more nonsensical for the sake of *your* attention span.

@llothar68 5 лет назад

The problem of Erlang is that the language and the language implementation totally suck. They should move at least away from interpreter to full compiled language making it 10times faster in each thread.

@laurelsporter4569 2 года назад

HiPE been included with the standard runtime since 2001. It's just not a big concern to most users, as practical slowdowns usually occur elsewhere.

@MultipleObjectSelector Год назад

It's interpreted *byte code* with a very efficient decoder and a JIT. These days most ventures needn't be concerned with performance juicing: the law of diminishing returns applies here, and presently the merits of the current design (code server in particular comes to mind) broadly outnumber and outweigh the demerits.

@frankfahrenheit9537 4 года назад

<a href="#" class="seekto" data-time="2602">43:22</a> don't watch if you are interested in doing numerical stuff. It's called here "nitty gritty in-memory stuff", thanks for wasting my time. I think everybody with an IQ>120 can "invent" a system which is described here, able to do some clint-server stuff. Many cores, every core has local memory. Trivial.