C# LINQ Performance Tips #1 - Let keyword & Custom Lookup 

Hey, thumbs up for all the effort. I would like to point out that LINQ is designed for readability, not performance, and it's excellent in situations where you need to apply complex transformations over not very big collections. Optimizing where optimizations are not need is, in the best case, harmless.

Honestly, the best LINQ performance tip is to not use LINQ. In many cases it's a compromise between readability vs performance, choosing between LINQ and a custom loop that does exactly what you need. There are, of course, cases where LINQ provides nice additions (e.g. null-checks) without doing anything too crazy on top of that (like .Any()). Just be mindful when using methods like GroupBy() which can be up to 9 times slower than a simple foreach.

Filter_ToLower_04 has another difference from the first three: It doesn't return the result. So it doesn't have to copy the reference back to/off the stack, which means less memory access, etc. I'm pretty sure that's why it's faster than version 03, and not because of the change in comparison order. Though you're still correct that the number of comparisons in the respective lists will have an impact on a more elaborate app.

At 11:50 where you talk about LINQ's Last() method, you say that LINQ doesn't check to see if the enumerable is actually a list. That's incorrect. LINQ does that actually: github.com/microsoft/referencesource/blob/master/System.Core/System/Linq/Enumerable.cs#L1091 And in the next line, you can see it actually calls the index: if (count > 0) return list[count - 1]; github.com/microsoft/referencesource/blob/master/System.Core/System/Linq/Enumerable.cs#L1093 More importantly, your code works faster also because you don't check like LINQ to see if the list has any elements in it before accessing the index. This isn't the main cause why your code is faster, but it's important to note. The performance penalties you're seeing are of different reasons, not because LINQ doesn't check if the instance is a list

Hello, I might be wrong but your example with GroupBy_O1 might be totally incorrect. By using your group by LINQ query you only building expression tree which is not materialised anywhere (you are not calling .ToList() after query or just .Count()) so your mesurement only shows how long it takes to build the LINQ expression for the runtime. I is never executed. I tried that example myself and with .ToList() call it takes almost 9 times more time than withou it. I think it might be a mistake on your side because you used .ToList() call in examples with filtering

When trying to do ECS in c# some time ago, I hit a wall when trying to provide a nice way of enumerating quickly throught various data buckets in an easy way. Mainly because C# can't inline lambdas. (Yet??). For instance: void IterateOnDataBuckets_Slow(Action iterator) { for (...) { // Imagine a more complex iterator. iterator(someData[i]); } } interface DataBucketIterator { void Evaluate(ref Data data); } void IterateOnDataBuckets_SillSlow(DataBucketIterator iterator) { for (...) { // Imagine a more complex iterator. iterator.Evaluate(ref someData[i]); // still quite slow. } } void IterateOnDataBuckets_FastAndInlineable(T iterator) where T : DataBucketIterator { for (...) { // Imagine a more complex iterator. iterator.Evaluate(someData[i]); // Since it's know what T is, it's able to inline this function. } } But of course, having to write a type just to enumearte the data, so I stopped the ECS experiement there. Wondering if you have any experience with that sort of problem?

I would emphasize the point that if you're examining millions of records, those 5ms in your test case will add up to a significant savings in time.

LINQ query syntax is very ugly to read, the method chain version is much more readable and modern in my opinion: transactions.Where(x => ...).Selec(x => x...);

Great tips, just be aware about coughing all the time, it is really annoying when someone uses headphones.

Great tips,,, thank you

Very good video.

For the ToLower bit: I am missing the option of first uniting the collections to be filtered by into a single one. That should also limit the number of ToLower calls to 1.

i there any wy to replacement of .contains. for large DB performance issue

Great video. I would only recommend to work on a proper naming convention, as you named all of the tested methods with quite generic (nothing saying) names.

in groupByLookup you dont deal with collisions,and when i see the imp of groupBy provided by Microsoft it look very similar to yours , so why the performance gap is hug ?!

What is diff between method 3 and 4.

With the let example I wonder what the performance would be like using LINQ method syntax i.e. transactions.Where(t => { var nameLowerCase = t.Name.ToLower(); if.... }).Select(...

I am 639th subscriber

Why dont you use BenchmarkDotnet to measure?

Anybody else cringe when he coughs? Hope OP is ok