SVD: Image Compression [Matlab] 

Thank you so much!!

Very very great lecture series. In my opinion, what humans/students need the most (water, air, food, and lecture series like this) should be available to all.

Thank you so much for the very kind words!!

Glad it was helpful!

Thanks!

He responded to another comment on this. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-H7qMMudo3e8.html&lc=UgybqQYO8S_PsQOrkvV4AaABAg.96mgj1_NYPW98T05U7uNgu

You're very welcome!

Thanks for the kind words and great question. We actually just finished up a 2nd edition, and realized that the discussion of cumulative energy needed to be cleared up. So your confusion is probably because it was a bit confusing... Technically, the cumulative energy would be computed by adding up the sum of the *squares* of the singular values, although most of the time we just add up the sum of singular values. Not a huge difference, but important to make units match up. And in that case, the spectrum does have a similar interpretation as *power* in the power spectral density with the Fourier transform. And yes indeed, this should all approach a normalized sum of 1 when we have all of the modes included. I will do my best to start posting errata soon to clarify some of these points. Usually would be posted on databookuw.com (and you can find the pdf at databookuw.com/databook.pdf ... not updated yet, but soon)

@@Eigensteve Wow, thank you for taking the time to answer my question! I must admit, I think there are a range of topics I need to delve further into to support my understanding of some of these concepts, but your response gives me a great point of departure for developing my intuition about this type of analysis. I guess I'm also asking too because, as I learn more, and watch more of these videos, I am trying to file away the sort of "magical" bits of knowledge as well as the sort of immediately "practical" bits (i.e "always graph the semilogy values of..")...not that the practical is any less magical...

You are very welcome

Interesting, I don't think I've ever seen the correct one then. Was this always the case? Anyway, in this case we know it's real numbers only, so still correct. In the words of the documentation: "When no complex elements are present, A' produces the same result as A.'."

@@_J_A_G_ yes. it was always the case and for real numbers it doesn't matter

@@liorcohen4212 Addendum for future readers: Apparently complex conjugate was anyway the right thing to do. So code was right also for complex numbers and lecture was simplifying. This came up in later lecture. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-46Hpy4FiGls.html

I'd like to add that if you attempt to do this on a random noise matrix, for example, you do get the correct metric when squaring S. Try this in Matlab: clear; clc; close all; X=2*randn(10000,100); %Generates random noise with variance = 4. Rectangular matrix assures we'll not mess up the row/columns. Variance 4 assures squaring changes the outcome w.r.t. not squaring [U,S,V]=svd(X,'econ'); %Regular SVD X_energy_indiv=diag(X*X'); %x1.*x1 is the energy of the first pixel, for example. For 10000x100, We get 10000 entries averaging 400 each. Dividing by the number of snapshots (400/100=4) we get the variance X_energy_total=sum(X_energy_indiv); %sum of the energies of each snapshot gives total energy. Should give about 10000*400=4e6 S_energy_nonsquared=sum(diag(S)); %should give about 2e4 S_energy_squared=sum(diag(S).^2); %should give exactly X_energy_total (4e6) (I think this is correct)

@@fzigunov Wondering the same, but the good news is, plotting on a log scale shouldn't really change the shape of the curves, just the units up to a constant factor at least I hope so! The cumulative plot might not have the right shape, and we depend on the percentage thresholds so I am a bit confused about that one

@@arminth4117 The problem is that it is very common for people to quote something like "modes 1 to M contain X% of the total energy" or "mode M contains X% of the energy in the flow". Therefore, the energy metric matters quite a bit. About the plot shape; I think it is quite a secondary feature when analyzing POD results. The whole point of POD is to provide a more understandable description of what a complex system is fundamentally doing, so the shape only matters to give you a sense that the higher order modes can indeed be discarded (or not).

He responded to another comment on this. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-H7qMMudo3e8.html&lc=UgybqQYO8S_PsQOrkvV4AaABAg.96mgj1_NYPW98T05U7uNgu

@@_J_A_G_ ty mate!

See discussion in other comment: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-QQ8vxj-9OfQ.html&lc=UgySxL8I3zJqiqPtUlt4AaABAg.9Owk8AGSt769ijFmwq9Vli

SVD Algoritham using Original Image Compression Make it Decompression Image Poor quality Disadvantage

Both original and reconstucted image are 2000x1500 pixels. This is (nx*ny) in both cases, so not where to look to save storage. The compression idea is to look at the right side of the equation and instead store the U,S,V matrices. The "recipient" would from those do the work to reconstruct Xapprox after loading the data. When discarding parts of those matrices (to keep only r columns of U and r rows of V) you get a lossy compression. The smaller r, the less data to store, but also a less accurate reconstruction. The key insight is that the vectors are already ordered by importance, so it's easy to include only as many as you need. The "title" calculation has r*(nx+ny) for the stored data size. I think (as he indicated verbally) it should be r*(nx+1+ny) to also include the S matrix (S is diagonal, so the r-by-r matrix is zeroes everywhere but the diagonal, so I agree that it's a very small correction). If the r=nx from the economy SVD transform, storing the U,S,V directly is no win, but as shown in the video r can be lowered quite a lot without visible degradation or even more if that is acceptable. PS. IMHO, this video is mainly part of the intro for SVD, explaining the concept of rank reduction. It's not a literal "how to compress images" tutorial.

Check out the links at databookuw.com

I'm pretty sure they just mirror the video in post-production

on twitter @eigensteve