It's quite possible that the muddiness you're experiencing is a combination of scale length and pickup positioning + angle. My guitar is a 1987 Striker 100st with a vintage Super Distortion and a whole slew of other upgrades, and I keep it tuned to E flat, which is essentially equivalent to stepping to shortscale for any given note by moving it down one fret or ~1.5". The lower tuning or shorter scale length both move the natural attack harmonics present in the sound of the string along a smaller length, closer to the bridge, for a given note (such that a pickup in the same position picks up lower, 'darker' harmonics), as well as reducing the overall presence of those harmonics. I was very surprised at how dark the guitar sounded since the Super Distortion has a reputation of being quite bright, and even Strat-single-coil-like when wired in parallel. This didn't make sense to me until I realised just how far away it was from the bridge, being at a similar slant and very similar position to the humbucker in this Jag-Stang. Especially with the humbucker having an extra coil closer to the neck, it's halfway towards a middle pickup position, way further away than a Strat single coil, the humbucker on a typical HH Jaguar, or on a Les Paul, and combining this with the already darker lower tuning or shorter scale length creates a very dark sound. If you think about it, the 1st octave harmonic is 1/2 towards the bridge, the 2nd octave harmonic 3/4, the 3rd octave 7/8, the 4th octave 15/16, the 5th octave 31/32, and what harmonics your pickup is positioned under is going to determine what it picks up (this is also why a humbucker never quite sounds like a single coil, it is physically wider so it's positioned under more harmonic points). With bridge pickups, just moving them a couple millimeters can already make you pick up or drop a particular octave because of just how tightly those harmonics are spaced, so even having the pickup very slightly further away from the bridge can drastically darken the sound of an already dark shortscale guitar into near-unusability. Putting a neck humbucker there like you did is actually a pretty good solution, since those have the treble presence specifically to compensate for being further away from the bridge.
Lol, I was going to suggest you try a lower output humbucker, potentially with a ceramic magnet. I used to be a fan of high output buckers, but with modern amps, they have plenty of gain on tap, that they don't need the extra push of a super hot pickup. I find I get a much better result with a lower output pickup (14k max) and letting the amp doing the heavy lifting it was designed to do. If you still need more high end, try putting 1 meg pots in. The tone cap value doesn't have any effect until you start to roll back the tone knob. I like to use .015mfd caps for my tones. I never roll my tone more than halfway down, so the lower value cap gives me a more useable range of tones throughout the entire rotation of the pot. Good call on 11's for the short scale. I use beefy slinkys (11-52) on all of my 24" scale guitars. Perfect string tension when tuned to D# standard or drop C#, which is where I'm almost always tuned to. Cool you got it sorted out for the most part. I remember when the Jag-Stang came out. I was (still am) a big Nirvana fan, and I was pissed that they used Kurt's barely played and pretty much rejected custom guitar idea for a cash grab. I love Jags and Mustangs (I own three and two respectively), but still to this day haven't even tried a Jag-Stang, purely on principle, lol. I put a full size bucker in one and a hot rail in the other Mustang, so they already cover everything a Jag-Stang could do, and more, because I wired the bridge pickup switch to be humbucker series/off/humbucker parallel on one, then the other, the bridge pickup switch selects bridge/both/neck and the neck pickup switch as a dark switch (think jag rhythm circuit)/normal/out of phase when both pickups are on. Let me know if you ever want to mess around with the wiring, I could send you the diagrams I drew up for either configuration if you're interested.
I really think it might be due to the position of the pickup. I like to build and I’ve heard the difference that positioning can make. The further the bridge gets towards a more middle position, the more potential for weirdness imo. Tends to get ‘quacky’. I know that quack is not the sound you are getting…atleast not from what I can tell from the the video. Is the excessively muffled sound more apparent on the bass side? Anyway, it may be an idea to try 1000 meg pots?
