You have a lot of mechanical issues that make it harder for the controller to adjust the system stability, you can try a single flywheel first then you go for the second one, then you can apply sensor fusion technique later. try using better coupling system for motors and try motors with a well-lubricated gearbox.
It seems to me that the control loop of the top wheel is a bit lazy and sluggish. What others suggested, to add more inertia to the wheel could help bringing it back from a seriously tilted position, but keeping it in balance, it needs to finely and quickly react to miniscule angle deviations. I’d say, the dead band is too wide allowing too much tilt. Possibly the D term in the PID, but we can’t really see for sure as it either does not spin at all and falls then after a certain angle it tries with max. accel. and speed, but it’s too late then.
