I made similar experience with a run of the mill Shure microphone.
I bought a cheap 18 buck microphone. https://www.thomann.de/intl/tbone_mb55.htm
That does not need as much gain.
Also I use OBS for streaming. And instead of running the mic through traktor I directly add it to OBS (But still through my s4 mk3) There I can add compressor and gain even more.
do say 'run of the mill' and Shure in the same sentence
It's just means is an ordinere microphone, use in many places. Does not mean it's bad. Like you said, it's more or less the industry standard.
It's also strange that a cheap option gives better results than a pro option
The cheap one probably has less impendence or something like that. I don't know much about microphones.
Do they use this community as a customer feedback and product development resource?
Yeah, naw .. yeah. Not really. Traktor team is very small and they don't even have time to look at longstanding bugs. Much less this.
Am I missing anything in my set up or doing anything wrong?
Sounds like you did everything right
Going direct through OBS is new to me and sounds promising.
That will only be useful if you stream / record a video.
https://support.native-instruments.com/hc/en-us/articles/360006980357-How-to-Stream-Your-TRAKTOR-Performance-to-YouTube-Live-or-Twitch
I had this problem of low gain in deck D Live Input with a Shure SM57. Thought maybe a battery-powered condenser would solve the problem, and it did. The microphone I am using now is a Rode NT3. Not an expensive mic. Takes a 9 volt battery. Sounds good. Gain on the meter is getting into the red so requires a small reduction in gain on the mixer.