How Memory Usage Slows Down Your Programs 

It’s amazing how fast you can make a program. I was writing a program that will seek out a certain line, but it was extremely slow. The solution was to put the entire file into an array, and the program went from a couple minutes to a single second.

Yes indeed. The more you understand how the underlying layers work (and don't work), the more often you can see opportunities to improve things.

I thinking about how to make DOP and OOP help each other. DOT is more about how to actually place data in memory, but it lacks readability in complex things. OOP on the other hand can be done in C with structs and pointers to real objects, but i curious how much it will cost if i will use OOP-ish structs in runtime.

i feel like OOP doesnt necessarily stop you from optimizing. you can use OOP in many ways, and C++'s creator talks a lot about performance and reliability for systems, so i doubt using OOP makes things worse (many of C++'s abstractions have no overhead)

@@badwolf8112 Using OOP doesn't stop you from optimizing, but it's just that things like for example virtual function calls can both bring the performance down and make it much harder to debug. Think of a line, where OOP is on the left and Data Oriented Design is on the right. The left is generalized to be very extensible, but can be slower and harder to debug. The right is more specialized to a specific task using knowledge of what you will *actually* use it for in practice, and can be faster and easier to debug. This principle of flexibility vs excellence also holds for a lot of other things, like AI: you either have an AI that is meh at a ton of different tasks, or have a state-of-the-art chess AI that excels at the one thing it was made for. If you try to design a real-life factory that is able to create *any* type of car, then your factory will be incredibly complicated and slow. The clue is that the more you decide to constrain what your program is capable of doing, the more performant and simpler you will be able to write it. OOP applies less constraints, which comes at several costs. This is an overgeneralization, because pretty much every programming technique has its place, but look up "object oriented programming vs data oriented design" for countless articles and videos if you want to know more.

OOP is for long term adaptability & encapsulation, when you're solving real problems correctness is key. Requirements change with time and you may need to add features and data types later, which were not even considered when the project began without updating 100's of files. A data oriented design works well for things like games, the data tables might even be extracted and processed from build edit tools that use objects to produce the level data that the runtime uses for high efficiency max fps with specificity not flexibility.

Good topic idea. Thanks! In C, arrays are laid out in a specific way. So, this example is going to see pretty consistent performance across nearly all machines, since the same elements will be close to each other each time. But, yeah, there definitely are situations where performance will be more machine specific. I'll see what I can do.

I once took some MATLAB code written by a client, made a series of tweaks to it, and showed it running a lot faster. I did all my testing on a Linux box. Unfortunately, the client needed to deploy it on a Windows installation. And on that machine, all the versions ran at pretty much the same speed. ∗Sigh∗ ...

Yes this professor is amazing

Thanks, Bill.

Yes it does. Your cache line is usually 64 Bytes long (could differ especially on embedded systems) meaning every time you access somewhere in memory it loads 64B into the cache. If your malloc'ed data is larger than you will have a delay each time you enter a new section that wasn't loaded into cache. Also note that whatever you load into cache will be removed as soon as new data is coming in but this behavior depends on your cache sizes and strategy.

Probably. I'll take a look.

Allocating the space would take extra time, but once you have the memory allocated, I don't know that it would be any slower. Maybe a little if having the accesses further apart affects cache use, but I would expect them to be nearly the same (except for the extra cost of the malloc/free calls). I guess I'll have to try it out.

Thank you, Dr. @@JacobSorber. In summary, execution would slow because of the allocation time for each cell but not because of memory access time, right?

It would be slower due to additional inderection. And allocator could give you any address so locality goes out of the window. That's a serious problem in Node based data structures like trees and such

Caching happens at a lower level-at the load/store level. It doesn't know anything about whether the variable is global, local, static...whatever. If your program accesses memory it be cached.

@@JacobSorber so when you store memory in heap using malloc the memory also gets cached? I quite don't understand when memory is stored in cache or not

@@leokiller123able It does when you use it.

@@JacobSorber okay so then if all memory gets cached when you use it, why do people say that accessing memory on heap is slower that on stack? If it goes in cache anyway shouldn't it be the same speed resuired to access it?

@@leokiller123able This comes down to two things. First, there's the cost of allocating and freeing memory. That adds overhead. Second, the heap can get a bit fragmented with bits of data scattered all over the place. The call stack is a simple linear data structure, and all of your local data for a function call will all be colocated in its stack frame. So, you often get better locality (hence better cache performance).

It would.

I haven't. I'll add it to the list and see what I can do.

You are welcome.