Thanks for the explanation. Apart from this, I read inside this paper about learned positional encoding also, where the author replaced the sinusoidal positional encoding with the learned positional encoding, however, the author got identical results. Can someone explain the working and about the learned positional encoding?
Here is a great debrief on that: (Folks that explain the positional embeddings much better than me). But let me know if you have a more specific question on that topic! datascience.stackexchange.com/questions/51065/what-is-the-positional-encoding-in-the-transformer-model
