The Wolly that fought against Ippo would lose to Alfredo, he lacks the necessary experience to fight at the world level. However, a more experienced version of Wolly would absolutely fucking utterly destroys Alfredo wirhout a shadow of a doubt. Completely different levels. Wolly is the most talented character of the series, what he did with Ippo after having only three fights (!!!) no one could do.
Wally got knocked tf out by Ricardo in the manga and he loses to Alfredo too. Peak vs Peak it's not even a close fight. Wally can have the edge in the starting rounds with his speed but Alfredo will either outbox with fast jabs and combinations or go in his deathgod mode and pummel Wally with body shots and take his head off with a left hook. Remember, Alf can take insane power so Wally can't earn his respect.