Channel update: Representation theory, AI, and more 

Every comment, every like, and every time you share one of our videos, is already a very real contribution to keeping the channel alive. Thank you very much for all those contributions. Oh, and it also feels nice to be called a gemstone from time to time 💎 😄

Glad you like our channel. Thanks!

That sounds like a really cool application in AI. When you say "valley" and "latent space", does it mean you are optimizing some cost function in a fitness landscape? I'd love to learn more.

@@AllAnglesMath To be clear, these are just concepts I'm playing around with as I try things. As for the jargon: yes, "latent space" is just the geometric representation of the co-domain matrix defining the fitness landscape. So in 2D say we have fitness as popularity of a cupcake against sugar content. There's a hump probably. With more ingredients, flour you have a 3d landscape. Add thousands of ingredients and you have a hyperdimensional landscape full of hills and valleys. With that in mind let's imagine an LLM considering the term "convention" which is a vector within a latent space. The meaning of that term (the direction of the vector) will shift if the term "traditional" appears or if the term "star trek" appears. The current solution for dealing with this is attention which is complex and expensive involving three additional matrices. Well, I was thinking of transforming the matrix specifically to construct an eigenvector in the landscape between the vector the vectors. That's the "valley" I was referring to. Like I say, just an idea I'm playing with.

@@davidmurphy563 It sounds really promising. I mean, if you could rotate your space to that eigenbasis, you might discover that each of the dimensions has a unique semantic meaning. The next question would then be: how many dimensions are enough to capture all the information? That would teach us something about the number of "big concepts" that are present in human language. Very interesting.

@@AllAnglesMath You know, I didn't think of rotation. There's the classic vector addition: "king - ""man" + "woman" = "queen" so you would presume rotation would be meaningful. Dimensionality is typically governed by tokenisation and there are loads of different strategies for that. The norm these days is 2048 bit. Plus, anything over 3 can't be pictured so I lump 4D and 555,555,555D into the same bucket. :))

We may make such a video some day, but for now the priority is on a lot more algebra.

It's about PCA (Principal Component Analysis). I don't know if that's related to eigenfaces.

Cool! We will need a new logo, with 2 eyes 👁👁