Modelling Yamaha's classic PITCHFIX plugin from the year 2003.

StarRats

Does anyone have an idea how Pitchfix's formant correction module algorithm works? Is that even possible to model such thing in Reaktor?

colB

https://community.native-instruments.com/discussion/1998/modelling-yamahas-classic-pitchfix-plugin-from-the-year-2003

The engineers at Yamaha would have an idea, and would definitely be able to build it in Reaktor...

Trouble is that it's kinda difficult for anyone else to know for sure without reverse engineering the original plugin... which is probably against the licence agreement... etc. etc.

A wild guess would be some sort of FFT based thing that can separate, and individually manipulate the frequency envelope (for formants) and the frequency spectrum (for pitch)... That would mean some latency... so first thing would be to check and see what the latency figures are for the original plugin?

StarRats

There is a knob to set windowing. It seems that for maximum value formant tracking gets more precise and less 'worbling' which mean more latency, however i would never use it in realtime at all. Is it a guess that formants are correlated with envelopes? I would never guess that envelope has something to do with tone/timbre of voice. Sounds strange to me. Steinberg bought PitchFix back in 2009 and put it in Cubase as internal 'pitch correction' plugin but still max sampling rate is 192khz. Original Yamaha version is 96khz which is stunning for 2003 first release date. As for envelope - meldaproduction's mTransformer has envelope module which in some way distorts timbre but not in harmonics/overtones shifting way. Pitchfix does this shifting in a much different manner to my ears.

colB

Envelope in the frequency domain, not in the time domain.

An envelope in the frequency domain is acts as a filter. So e.g. can represent a humans vocal tract. If you just raise all frequencies equally, you change that and it souds weird.. the chipmunk effect.

But if you extract the frequency envelope, then raise the pitch of individual partials, the re-apply the frequency envelope, you can raise the pitch, but avoid the chipmunk effect. It also allows stuff like male to female, female to male etc.

Something like that

StarRats

Thx. So...how exactly envelope correlates to ezFFT phase and amplitude or vector and polar coords? Is there any other mean to dig into envelope in freq domain?...like partial framework or BFresiner ensemble from reaktor user library?

StarRats

10 years ago there was nowadays removed ensemble 'gTranscoder' . Maybe it has something to do with envelope because there is formant curve xy gui editor? Luckily i haven't erased it, version 1.1

colB

https://community.native-instruments.com/discussion/comment/11411#Comment_11411

Hmm, I'm not an expert in this area

first of all, you need to extract all the actual partials from the phase/amplitude information - this is not trivial, but there are various places on the net to find tutorials etc.

I suppose then you would use something like an envelope follower - but frequency domain - to plot the curve of the frequency response... just like with a time domain envelope follower, you would need the right attack/decay settings so localised changes are ignored and you get the overall shape...

I found a good explanation of the basics a while ago...

http://www.panix.com/~jens/pvoc-dolson.par

Towards the bottom there is a section on 'Pitch Transposition' with a diagram that shows the applications of the spectral envelope of the pre shifted sound onto the pitch shifted sound. I found this article very helpful - it's well written and easy to follow - a good tutorial, not someone writing a paper and trying to impress with lots of complicated maths ;)

Here's one that has more details on how to extract actual frequency partials from the FFT data

http://blogs.zynaptiq.com/bernsee/pitch-shifting-using-the-ft/

I haven't got round to implementing this yet, so can't vouch for the accuracy of the info in this blog!

Also check User Library for Pitchomat by Jan Brahler which uses fft phase vocoding IIRC

https://www.native-instruments.com/en/reaktor-community/reaktor-user-library/entry/show/6541/

His other uploads are also really good and worth studying

ANDREW221231

probably the easiest and possibly even most effective way to extract a spectral envelope is to just low pass filter the FFT

the you would want to linearize or whiten the frequency response of whatever you're applying it to

that's a bit trickier , high pass filtering would kinda get you there

I've used a linear predictive coding algorithm for this actually, directly on the FFT steam as well and it worked pretty well

then you would just convolve (multiply) the two together, with the spectral envelope shifted in time to shift the formants

you can find the linear predictive coding macro in my RIP the old forum thread

ANDREW221231

https://community.native-instruments.com/discussion/comment/11411#Comment_11411

it correlates to them over time in the case of ezFFT, the frequency domain is expressed in time like a repeating waveform. the envelope would be synced to the beginning and end of this waveform so it would draw a shape over the course of the frame, just the same as a normal envelope amplitude modulates a signal over time

if using polar coordinates you would just modulate them both the same

but, its better for music in every conceivable way to use amplitude and phase instead, for this you would just modulate the amplitude

amplitude controls frequency loudness over time, so you can effect the spectrum by drawing a shape over it time! phase is most likely left alone entirely for this type of thing, phase controls offset of the sine component in its cycle. say you had a 400hz component but every fft frame the phase is shifted back a little from where it was on the last frame, then you have effectively lowered the frequency, this is the main use of phase for musical fft but that's more advanced. you can have a world of fun messing with the volumes of frequency components and passing the phase straight to the inverse fft unchanged