From my experience this seems to be expected. The scheduler will only let your threads run on p-cores if the load is high enough. So as long as you don't have any audible drop-outs you should be fine.
KonradGlAudio
Users receive reputation points by contributing quality content to the forums.
Posts include post and replies published on the forums.
Post authors can mark replies as Accepted to indicate successful solutions.
Apple Developer Forums admins can mark replies as Apple Recommended to indicate an approved solution