It's actually quite common in voice acting. Because you're only recording your voice, you can basically do it anywhere and any time and just on your own. Also, Tom lives in New Jersey where the rest of the cast live in LA
Hinata Shoyou, the crystal gems (until Estelle moved back to London) would actually record a lot of episodes together. Zach and I think DeeDee have mentioned that recording for SU has been different then recording for other shows because they do work together