How to exclude custom claim provider from out-of-the-box SPFieldUser field

I created a custom claim provider, and I custom field type inheriting SPFieldUser.

The problem is, I don't want the claims from my custom claim provider to appear in SPFieldUser field, I want it only shown in my custom field that inheriting SPFieldUser. How?