How to consistently add/remove modulators to all groups?

Sometimes when I have "Edit all groups" enabled and add or remove a modulator, it doesn't seem to consistently apply that to all the groups.

For example, I have 45 groups and "Edit all groups" enabled. I've added an ADHSR Envelope Modulator targeting Volume.

If I delete the modulator then use find_mod and find_target functions in KSP to check for references to the mod and target, a few of the groups will still have the references to the mod (ENV_AHDSR) and target (ENV_AHDSR_VOLUME).

I do not see the "ghost" mod/target in the UI after deleting, unless I switch off "Edit all groups" and manually edit each group with the "ghost references" to remove them.

Are there scenarios where this is expected to a happen? Like maybe if I was accidentally just editing specific groups instead of all groups when I first added the mod/target, and that causes some kind of conflict in Kontakt when I try to delete them? What steps can I use to consistently add/remove mods/targets for all groups?