The Unreasonable Effectiveness of JPEG: A Signal Processing Approach 

Yes, this is a great tidbit of history that most people don't know! Fun fact you may already know: in video codecs such as H.264 and H.265, where compression ratios are really important in terms of saving bandwidth, most entropy based encoding is based on context-adaptive binary arithmetic coding (CABAC). The improvements of arithmetic coding vs Huffman coding were enough of an incentive for most developers of video codecs to implement this logic in both the encoding and decoding side.

@pyropulse Looks like one to me. A circular dependency: No one will use a feature that has no software support, and no-one will make software to support a feature that is never used. The problem cannot be solved because of a condition that can only be altered by solving the problem.

Sounds like someone working on serenity os has a chance to do things their way and support that format just because.

@@Bobbias is that the temple on thing

@@clown134 no, serenity is a different project to temple os. If temple os feels like an 80s os, serenity feels like a 90s os. It's basically a Unix style system with a windows 95 style desktop environment

Ha, this cracked me up! Great comment!

No NFT is save

@@Reducible qhy is only one coefficient in dct output if most of the cosine wave values were positive? Since the others were too low? But youd think some would still be positive and just lower positive value, no?

It should probably be phased out, really. JPEG's compression was cutting-edge when it came out, in 1992. There have many several attempt to replace it since then with more sophisticated compression that can achieve higher quality for the same size, but they've all failed because they can't compete with JPEG's universal support. The latest is WebP, which is making some progress because it has the giant of Google to promote it. JPEG2000 was a big flop. Though amusingly to me, every web browser today /does/ support it sort-of... not as a JPEG2000 file, but because it's one of the image compression methods supported within PDF files.

All/most of the image formats that followed are based on the basic idea of stacking waves, and even more so for the video formats. So in a way you are already paying homage to this genius design by watching this video! The JPEG people didn’t stop working on pictures after good ol’ JPEG either. Their latest JPEG XL comes with tricks to make it much more efficient in terms of beauty/fidelity-per-bit. It is able to go toe to toe with video-based image formats like HEIC, AV1F in terms of efficiency while staying easy on the CPU to encode and decode.

I agree. One thing I would have added though, is why we used sine instead of cosine. But perhaps if this is of interest to you, then you already know the answer why😉

If you've ever studied differential equations or Fourier series in general as a mathematics student, then the concepts also make a lot more sense intuitively than I expect they might as an EE student alone. 3Blue1Brown has some great videos on this.

totally agree. it's the visualization of behavior as parameters are varied that is so powerful. great job @Reducible!

That's great where did you do your degree I'm assuming you are indian

As a network engineer, thank you for not being a code monkey. Welcome to the Machine, priest. You're one of us!

Yup, there are quite a few on the list, but no promises on when I'll get to them :)

I didn't expect you to be here lol, might join one day making the same videos on this platform?

@@harryguanous7198 Welp, I am a cs major after all :p

@@vcubingx your name and profile picture are amazing

@@Reducible At 14:56 where isbthe corresponding negative DCt value?? Ther shoukd be one if there is a positive one..

XL sounds like the file sizes will be bigger

Opus is only used for voice streaming or voice compression, it's definitely not an mp3 killer

@@Dorumin It could be though. Opus performs really well at any bitrate, though it does excel especially at the low end. Outperforming MP3 isn't that impressive a performance though. There are lots of codecs that can make that claim. MP3 is just /old/.

@@Dorumin RU-vid also uses opus for many videos

I'm going to have to agree with the discussion above. Opus isn't a successor to MP3/AAC. AAC is reasonably considered a successor to MP3, but Opus is more of a peer to AAC.

I remember coding the DCT/IDCT functions in Borland Pascal two decades ago... it was a day of work - not full JPEG scheme, just playing with the coefficient (e.g. erasing them a seeing what it does) but quite some fun

I imagine it's not a good coding challenge, since there's not much elegancy you can bring here, often coding math-related stuff is tiresome and ugly, unless you use languages like Julia.

@@VivekYadav-ds8oz Well although this might actually be very true for someone, I personally feel kind of ok with coding applied math stuff. After all, coding a JPEG encoder/decoder is not only about coding a math part of it, it's also about engineering a software because that's what we, as programmers, do

@@kodirovsshik this. There's the beauty of engineering a complex function into code that is readable, easy to grasp, maintainable and performant.

@@kuma9239 true!!

So to summarize: you don't like JPEG the image format, but you do like JPEG the algorithm :p

@@susmitislam1910 yes haha

Theoretically, you have to give up some efficiency in order to compress any sort of information. The difficult part is about by keeping the data as original or comprehensible as possible, how much would you give up. But yeah, nowadays we have bigger and cheaper digital storage, so the problem of jpg is gradually getting noticed.

There are better alternatives to DCT available nowadays. For example, JPEG2000 uses wavelets: when you push the compression too far on these, instead of getting blocky like DCT, they become fuzzy, which is generally less objectionable.

There is nothing wrong With Lossy compression, i dont understand whats your problem, everything has Advantages and disadvantages

It's literally the most employee degree, currently.

Most of the things people interact with nowadays during their typical day is brought to you by electrical engineering and computer science. The fact that people take this for granted is a compliment to the fields, albeit can be frustrating at times. This also applies to many other fields of science, including chemistry, biology, math, physics, and all fields composed of those base sciences. Just think about the things you use, own, and touch, all created because of material science bred from those sciences.

It depends on what exactly you're majoring in. Computer science and software engineering are vast fields. I touched on those things because I liked doing usability stuff, but I could just as easily have avoided if I hadn't.

Yeah, very good point! Now that I think about it, you are right. I think I wanted some visual symmetry when I made it, but truth be told, it serves no purpose. Sometimes, when you are so deep into a project, you can forget how something so superficial can possibly lead to some confusion. Thanks for the feedback!

It's not only a discrete sequence (as the sequence of samples would be), but it's still discrete when you consider the extension to continuous signals. On the time domain you can interpolate using the cosines. Thus, on the left side I think the continuous line is helpful. In the frequency domain, you would still see discrete delta impulses, because the dct requires (assumes) your signal to be periodic. Other than that, great video! Thanks

@@Reducible I think the continuous line does serve some purpose. It makes it easier to understand why the discrete values are the way they are when you slowly shift the frequency between integers.

Though the interpolating values have no meaning, they serve as a reminder that cosines are in fact, continuous and not discrete. Also the signal transformation is better to visualize and understand with this in mind. This is ultimately what the step of quantization gets rid of, as it samples this continuous interval back to discrete space. In my mind it was definitely not in vain to have it included and visualized.

@@Reducible I think one problem is that it's a bit hard to spot the actual coefficient points because they're the same colour as the curve, which is why making the curve less bright is a valid solution.

Fun fact, I actually thought about explaining H.264 first, but then realized it would probably be a 2 hour video so I stuck with JPEG.

It's annoying that everything is forced through lossy photo compression, especially when pixel art is already so insanely compressible. A detailed 320x240 32-color piece can be 20KB, but it must be upscaled and converted to a fuzzy JPEG that's an order of magnitude larger than the original.

dirac impulse goes ↑

Lossy encoders were designed with mainstream usage in mind. You, as a professional with strict requirements and technical knowledge, are responsible to find another medium and suitable file format to carry your information. Although JPEG was somewhat forced onto Internet users as digital hardware and software grew in usability and prevalence, there was never a point in computer history when you couldn't use a lossless format or find another lossless workaround, mostly because raw solutions are FAR EASIER to implement and far more robust and cost effective, and rarely have anything to do with fashion or industry trends. I am a DTP professional and a graphic designer from the early 90's, I still remember IFF and PCX file formats on the Amiga. Don't mingle 'technology for the masses' with the 'technology as is'. Since the 2000's I remember people were struggling to find a good carrier for print-ready photography in certain workflows. TIFFs with ZIP compression were widely available and offered a superior lossless compression both in CMYK and RGB. We also had EPS DCS2 which would natively store grayscale color separations for high-resolution film development. Video and audio were something else due to monstrous demands on the memory for the time, but pixel art? Man. It all started from indexed palettes and simple pixel art. Why would it ever devolve into media intended for megapixels and high-freq noise? Vector graphics took more than 10 years to develop fully and it's still quite a niche technique if we look outside the DTP, but pixel art was there from the very beginning. Though, to be fair, I remember one historical gap. It was thanks to the holders of the LZW patent (used by GIF) on one side, and thanks to Apple pushing for high color palettes on the other, and so the browsers were caught between a rock and a hard place, but only browsers! It was some time before CompuServe finally got PNGs running throughout the ecosystem, in the late 90's. Though Microsoft always had the Bitmap format, the most native thing one can imagine, but it was completely discouraged on the Internet. In any case, since the 2000's, *having* to use JPEGs for anything it wasn't made for (high-res photos and common image interchange), was definitely not a thing if it ever was. Whoever had to mess around with upscaled JPEGs was someone who figured out stuff very wrongly.

That's because pixel art is pretty much the opposite of natural vision. It's also the reason why you can never use chroma subsampling on a PC monitor (it will screw up the GUI and text) and why desktop recordings without zoom look so horrible.

when saving JPEG, turn off the chroma subsampling (use 1:1) and your colour contrast will be retained - plus there is lossless JPEG scheme as well