This is exactly what I was looking for. Whenever I tried to research into the transpose I always got confused by all this talk of dual spaces and whatnot, this is the clearest explanation yet.
I remember asking my linear algebra teacher this exact thing and he just looked at me weird and said, "you just change rows and column". I stopped asking him stuff after that.
Such an amazing video. I'm shocked that you only have 924 subscribers. You explained this so well, and so elegantly, it really is truly amazing. Linear algebra is so beautiful. Thank you.
I realised at 11:00 that this falls out really nice in ga. Applying a linear transformation that is an orthogonal transformation is the same as applying a rotor. Applying a rotor looks like: a' = RaR~ if R is a unit versor, a is the original vector, a' is the transformed vector and ~ represents reversion operation If R isn't unit then a' = RaR^-1 as this expands to a' = (RaR~)/RR~ It seems like the transpose is analagous to reversion which is pretty cool.
It’s interesting there doesn’t seem to be a spot in the formula for sigma - I guess that makes sense because GA can’t really do axis-specific stretching with the basic operations?
My first linear algebra course was very abstract, and while I loved it for that, transpose was always so opaque. The = definition is so ungodly complicated and opaque, and while we could use it algebraically, we never understood what it meant. Having this video then would have saved me so much headache. At first I was a little confused why you were talking so much about preserving dot products, but when you introduced vbar->v and I saw where the forumula was headed it absolutely blew my mind. I also totally agree with your conventions - marking x and v as different like with that bar would have saved me so many headaches too. I also saw some places mention you needed *two* scalar products for the transpose which always confused me - but that notation makes it so clear why: one is a scalar product in the input space, and one’s in the output space. TL;DR awesome video and now I’m gonna rewatch in 3 times to make sure I didn’t miss anything 😊
Oh, and I love how clear this makes the spectral theorem too! When you mentioned the case where r1 = r2^-1, I literally thought to myself “oh, I bet that means the eigenvectors are nice, since it’s rotated axis aligned scaling” before even realizing that simplifying the formula would make it obvious that the matrix is symmetric. So cool!
This is an excellent video. My linear algebra studies took me through the "Linear Algebra Done Right" approach, and so we didn't use many matrices, which had its pros and cons. It is funny to me that this kind of study can often leave student without a solid grasp of what is really going on unless they put in the extra effort. For me, it wasn't until I did a course on Fourier Analysis (which was really just a functional analysis course in disguise imo) that I really had to understand a lot of this stuff algebraically. While the geometric understanding is helpful, it is funny how much it doesn't matter later on. Math is weird.
I guess it does matter. For me, a geometric intuition helps me understand what exactly is going on and eventually over time, the visual aspects become so engrained in memory and automated that working with more abstract algebraic terms is more convenient. The visual intuition becomes a subconscious way to process the information even though we use algebraic representations consciously.
This is such a great video, it really helped give me an intuition of what transposes and SVDs are. If only I had watched this video before I took my linear algebra final 2 days ago...
Thanks @arbodox . Your comment is valuable as someone who is fresh out of a Linear Algebra final exam, for the key areas of your focus in this video 👍.
Just had my exam about computer vision, which relies in many cases on SVD and all sorts of transformation matrices. This video brings much clarity, if only I saw this before my exam 😅
hi there, fellow math educator :) this is my first time watching your channel. great visuals and explanations! my only criticism is that the music was a bit loud and distracting. at the very least i'd say reduce the music volume (or do away with it), and if you keep music, i'd choose music with much lower tempo. the choice for this video felt a bit too 'fast' personally. but otherwise, great content! i'm looking forward to future videos. best of luck :)
Interestingly, this "sort of inverse" behavior (the fact that the transpose is an involution for finite dimensional vector spaces) but where "adding, subtracting or multiplying an element and its dual (=transpose) respectively gives a sort of symmetry, antisymmetry or symmetric square" finds other analogies throughout mathematics. By this I mean, the sum A + Aᵀ = 2 sym(A) is symmetric, the difference A - Aᵀ = 2 skew(A) is antisymmetric, and the product AAᵀ is symmetric, and in fact a "sort of squaring" of A, called the Gram matrix of A (and which is very important in the SVD). One simple analogical case is the complex conjugate (and you have a similar analogy with your four A, Aᵀ, A⁻¹ and (A⁻¹)ᵀ, with respectively z, z̄, z⁻¹ and z̄⁻¹ , neatly making an axis-aligned rectangle on a circle of radius |z|). The sum z + z̄ = 2 Re(z) is real, the difference z - z̄ = 2 Im(z) is imaginary, and the product zz̄ = |z|² is real, and in fact a "sort of squaring" of z, called the quadratic norm of z. Of course, this also applies to conjugate transposes (of which the above case is just a 1×1 example), etc. There's also things to say about any integral operator of the form = ∫ f(x) __ dx (which is pretty much the transpose of the function f seen as a vector, as it acts as a covector (linear form) over function spaces (which are just infinite dimensional vector spaces)), and you similarly have interesting duality properties of some linear operators, etc, but that's worth a video of its own, surely. Great video in any case, beautiful work. Thanks a lot !
I have thoroughly watched the video once time and I would rewatch again when I have free time ‘cause some of the concepts I don’t entirely understand 😊 Good video appreciate your work
I often find math videos hard to follow, but I really like this one! My question is why is preserving the dot product in orthogonal transformation special? Is there any more interesting properties we can get out of it?
Most matrices do not preserve the dot product, that’s what makes it special. Another special property that arises from preserving dot product: Orthogonal matrices are linear transformations that simply rotate/reflect a vector, and don’t change the length of it (orthogonal matrices are norm-preserving). As an example, suppose v = (1, 1) and we apply the orthogonal matrix A that rotates 45deg counterclockwise (what is this 2x2 orthogonal matrix A?). Then, Av = (-1,1) as expected. Notice how v and Av are the same norm. Now notice that the norm of a vector v is sqrt(v dot v). Thus the fact that orthogonal transformations preserve the dot product leads to our interesting property that orthogonal transformations are norm-preserving.
It is good that you talked about inverse transpose matrix and use SVD to show it. And I think there is a more visually intuitional way to show the geometric relationship between them. Say, if we have an full rank 3×3 matrix {a1 | a2 | a3}（a,b,c are vextors）and its inverse transpose matrix {a1' | a2' | a3'} ，we can found that : a1' is perpendicular to the plane of span {a2,a3} while the dot product of a1 and a1' is 1 a2' is perpendicular to the plane of span {a1,a3} while the dot product of a2 and a2' is 1 a3' is perpendicular to the plane of span {a2,a3} while the dot product of a3 and a3' is 1 In fact, in crystallography, if a1, a2, a3 are the basis of some crystal's lattice, then a1'， a2'， a3' happen to be the basis of its reciprocal lattice. However in crystallography textbooks, the relationship of inverse transpose hardly mentioned, rather, a1' , a2', a3' are defined as: a1' = (a2 × a3) / det {a1 | a2 | a3} a2' = (a1 × a3) / det {a1 | a2 | a3} a3' = (a1 × a2) / det {a1 | a2 | a3} See, in such definition we can easily get the perpendicular properties I have mentioned but hard to notice the inverse transpose nature of matrix {a1' | a2' | a3'} . Could you make an video to visualize the connection between reciprocal lattice, inverse transpose matrix and the intuition that inverse transpose matrix has vectors perpendicular just to vectors of the initial vectors? That really helps a lot.
If I studied more and think more like in this video instead of just memorizing the formula back to college, I would have became.... About the same me but slightly more intellectual superior.
Thank you for the video. Very clear explanation of transpose and SVD. I am motivated by applications… The SVD is so insanely powerful! Could you make a video that illustrates how the unitary rotation/scale/rotation of the SVD solve a problem? That would be so helpful! Thank you for sharing.
I don't really get the premise of the explanation. Wouldn't there be infinitely many new vectors v bar that satisfy x dot v = x bar dot v bar with x bar = A x and thus infinetly many matrices to get us there? What is so special about the transpose of the inverse of A then? It does of course satisfy the equation, but so could infinetly many other matrices, no?
For a given fixed x that is true, but if you want to use the same matrices for any arbitrary x and v, then you have to use the A and A-inverse-transpose.
Is the choice of x and v as names for the vectors instead of u and v a deliberate choise so that students don't mess up because u and v are very similarly written ? If yes, this is another proof of the care you put in the video who is very good
Having hard time with the background music. If you are not sure whether the music is loud or not, you should remove it. The content is nice though. Hope you do it better next time.
Okay, I really want to thank you for this content. It's incredible. I saw it in class and was a little lost as to the true meaning of these formulas. This video was perfect for that. Keep going 🫶🏻
