Background:
Our CMIOExtensionDevice may change its supported formats in run time when video source changes. (i.e. set of resolutions change). A custom callback is triggered once this happens.
According to Apple doc CMIOExtensionStreamSource … “formats should not change during the life cycle of the associated stream” . So we can’t update formats of existing CMIOExtensionStreamSource once it has been created.
Initially, we thought stopping the stream -> removing the stream-> adding new stream -> starting the new stream would accomplish this.
CMIOExtensionStreamSource::stopStreamAndReturnError
CMIOExtensionDevice::removeStream
CMIOExtensionDevice::addStream
CMIOExtensionStreamSource::startStreamAndReturnError
but we ran into strange CMIO behaviors. In the os_log we could see error messages like
CMIOExtensionProvider.m:3085:-[CMIOExtensionProvider stopStreamForClientID:streamID:reply:]_block_invoke Invalid streamID
CMIOExtensionProviderContext.m:887:-[CMIOExtensionProviderContext stopStream:message:]_block_invoke stopStream failed Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list" UserInfo={NSLocalizedDescription=Invalid streamID}
CMIOExtensionSession.m:400:-[CMIOExtensionSessionStream stopStream:]_block_invoke stop stream index error Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list"
CMIO_DAL_CMIOExtension_Stream.mm:1981:Stop_block_invoke 37 unexpected Error Domain=NSOSStatusErrorDomain Code=-50 "paramErr: error in user parameter list"
CMIO_DAL_CMIOExtension_Device.mm:1565:StopStream 36 could not find stream with ID 37
It seemed like CMIO or AVFoundation was not made aware of such add/remove change.
On a different approach, it would work if in the CMIOExtensionProvider level we remove the old device and a create new device + stream with updated list of formats. However, such action will cause the client app to move to next available AVCaptureDevice (i.e FaceTime Camera) to preview, leading to a strange user experience.
DTS engineer recommendations:
I see, I agree that this would create a strange user experience, that is, reporting support for a format but not actually having hardware connected that supports that format.
Unfortunately there is no other option, you can provide the array of all possible formats, and then maintain a dynamic sub-set of these formats that you actually support at a given time. Then, if a format index is set (by whatever client app is using the device) that isn’t supported at a particular time, you could throw an error from setStreamProperties. Or you can tear everything down and create a new device + stream with the currently supported formats.
Beyond those options, I recommend that you file an enhancement request using Feedback Assistant, to explain to the engineering team your need for a way to update the formats array during the life cycle of the associated stream.
We filed an enhancement request a while ago in Feedback Assistant. Thought I might share this in the forum.
Post
Replies
Boosts
Views
Activity
Hello,
According to Create camera extensions with Core Media IO
from WWDC2002, one can use driverkit-extension (dext) to communicate with hardware. Does it mean there will be a .dext and a .systemextension in the SystemExtensions folder and they both need to be activated via user prompt separately?
We looked into Communicating Between a DriverKit Extension and a Client App
and USBApp as our starting point. With NewUserClient implemented, we could open our dext via IOServiceOpen in a user space program.
We have been unable to apply similar approach in a cameraextension code sample with the sandbox violation:
Violation: deny(1) iokit-open-user-client IOUserUserClient
In the plist we did add com.apple.security.temporary-exception.iokit-user-client-class entitlement array with IOUserUserClient like in our other test app.
Our goal is to let CMIOExtension exchange data / information with our usb device via function calls like IOConnectCallAsyncStructMethod upon successful opening our userClient
Has anyone run into similar problem or is this not the right way to do it?
Thanks for your time.