I had this book on my shelf for about five years before I finally finished it. I had started it a couple times, but put it down because the language was a bit taxing, even for a native english speaker. I agree with what you said about the book teaching you to read it as you go -- once I got into it, I was really into it and got used to the language quickly. Like you said it's a difficult read at first but very rewarding, and surprisingly funny at times.
Not yet, but I've started "dipping my toes" into it (picking it up in bookshops and reading a few passages at random) as I work my way toward actually reading it :)
Off topic: Season 2, Episode 4 of "The Big Door Prize" on Apple TV+ has a character that is 100% based on you. The guy looks exactly like you, and he's reading "Ulysses" for the seventh time. Sadly, the show isn't very well written, but you have now been immortalized on TV.