Unbelievable 99% AI Model Compression Boosts Performance! [Paper Reading] 

Today we're reading the paper "The Truth is in There: Improving Reasoning in Language Modelswith Layer-Selective Rank Reduction" that compresses model size by 99% in select layers.
Join Agora, the open source AI research lab for daily paper readings at 10pm NYC Time:[ / discord ]
########### Notes ####################
Model compression: prune 90% of parameters for improved model generalization.
Selective pruning alone can improve model generalization without training.
Performance degradation can be minimized by pruning later layers instead of early layers.
Low approximation of weight matrices by reducing rank can offer accuracy benefits.
Spectral norm refers to the maximum singular value of a matrix.
Rank r approximation and SVD help find optimal solutions for matrix approximation.
Higher order components refer to singular vectors with larger singular values.
Rank reduction in MLP layers can be achieved by applying laser to specific components.
Greedy search can determine the amount of rank reduction across multiple layers.
Low rank approximation in specific layers of a transformer improves performance.
Matrices can be reduced to 99% of their original form for improved task performance.
Higher order components in weight matrices accumulate noisy answers during training.
Model architecture and structural choices affect the occurrence of this phenomenon.
Absolute weight pruning can effectively zero the bottom x% of matrix weights by their magnitude.