The problem you face is that there's no API contract about this, so you can't exactly claim it isn't behaving as documented. Still, you could try a bug report, on the basis that what you're doing is based on a reasonable interpretation of the underlying mechanism.
Your description is a bit puzzling, though. If something is being cached, it can't be "an accessor implementation" for "the specified key", because it has to at least be class-specific. That suggests you should get your swizzled implementation if you replace the standard implementation before "setValue:forKey:" is first called for the class and key. While I can imagine general scenarios where you might swizzle accessors for classes you don't "own" after the original accessors have already been used, they're pretty scary. (For example, you have no control over whether the class uses accessors consistently, or uses a mixture of accessor methods and instance variables.)