Solved. The problem was altool. I uploaded the exact same package via the Transporter app and it worked.
At some point in the last year altool changed. (I can only guess my issue had to do with the --upload-app option getting deprecated.) If you use the same command you used to (say, in a script, like you should), it will quietly corrupt your installer package and return success, instead of failing.
Impossible to debug because there is no warning or error, and you don't have access to both sides of the connection. There was no mention of this change to altool in the Xcode release notes, or at developer.apple.com/news...
Lesson is to just use the Transporter app instead of altool.
Post
Replies
Boosts
Views
Activity
You don't need to stop and start the stream at all. If the only thing that changed is the resolution, just go ahead and send the CMSampleBuffers with the new resolution to the CMIOExtensionStream. Client apps should handle this by reading resolution from the CMSampleBuffers they receive, not from the stream properties. Apple's apps (QuickTime, FaceTime) handle resolution changes just fine.
devicesWithMediaType is deprecated.
Use AVCaptureDeviceDiscoverySession:
let discoverySession = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInWideAngleCamera],
mediaType: .video,
position: .unspecified)
let devices = discoverySession.devices
and do it in a separate thread, it will take a second or two, not more.
Do you have the entitlement com.apple.security.device.microphone?
Did you check the value of granted?
If the user denies access once, they then need to go into Settings to grant access.
I learned a lot about CMBlockBuffer and CMSampleBuffer from this presentation: https://developer.apple.com/videos/play/wwdc2014/513/
See page 37 of the pdf.
CMSampleBuffer contains the CMBlockBuffer, plus metadata like timing info. CMBlockBuffer contains the actual data, which you can access with functions like CMBlockBufferGetDataPointer, CMBlockBufferGetDataLength, CMBlockBufferCopyDataBytes, etc.
Anyone here from Google: this happens when you try to manipulate rpaths on a bundle (MH_BUNDLE) rather than a shared library (MH_DYLIB).
Bundles sometimes have the same file extension as shared libraries (.dylib) but you can tell the difference with the file command:
% file x.dylib
x.dylib: Mach-O 64-bit dynamically linked shared library arm64
% file x.so
x.so: Mach-O 64-bit bundle arm64
The linker will add an rpath to a bundle no problem, but install_name_tool can't manipulate it.
You might be sending or consuming sample buffers repeatedly.
In your your device source (CMIOExtensionDeviceSource) implementation, you should have methods that start/stop the source stream and another pair that start/stop the sink stream.
@interface WhateverExtensionDeviceSource : NSObject<CMIOExtensionDeviceSource>
{
...
CMSampleBufferRef _sampleBuffer;
}
...
- (void)startStreaming;
- (void)stopStreaming;
- (void)startStreamingSink:(CMIOExtensionClient *)client;
- (void)stopStreamingSink:(CMIOExtensionClient *)client;
@end
Notice the reference to a sample buffer (_sampleBuffer).
The sink stream's "start" method should keep a reference to the client (the application feeding samples to the extension via the queue):
@implementation WhateverExtensionStreamSink
...
- (BOOL)authorizedToStartStreamForClient:(CMIOExtensionClient *)client
{
_client = client;
return YES;
}
- (BOOL)startStreamAndReturnError:(NSError * _Nullable *)outError
{
WhateverExtensionDeviceSource *deviceSource = (WhateverExtensionDeviceSource *)_device.source;
[deviceSource startStreamingSink:_client];
return YES;
}
In that method you should have a block running repeatedly on a timer. Here, you consume a sample buffer from the sink stream (consumeSampleBufferFromClient), keep a reference to it (_sampleBuffer = CFRetain(sample_buffer)), and notify the stream that you got the sample buffer (notifyScheduledOutputChanged):
- (void)startStreamingSink:(CMIOExtensionClient *)client
{
_streamingCounterSink++;
_timerSink = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, DISPATCH_TIMER_STRICT, _timerQueueSink);
dispatch_source_set_timer(_timerSink, DISPATCH_TIME_NOW, (uint64_t) (1e9 / (2 * kFrameRate)), 0);
dispatch_source_set_event_handler(_timerSink, ^{
[self->_streamSink.stream
consumeSampleBufferFromClient:client
completionHandler:^(
CMSampleBufferRef sample_buffer,
uint64_t sample_buffer_sequence_number,
CMIOExtensionStreamDiscontinuityFlags discontinuity,
BOOL has_more_sample_buffers,
NSError *error
) {
if (sample_buffer == nil)
return;
if (self->_sampleBuffer == nil) {
self->_sampleBuffer = CFRetain(sample_buffer);
}
CMIOExtensionScheduledOutput *output = [
[CMIOExtensionScheduledOutput alloc]
initWithSequenceNumber:sample_buffer_sequence_number
hostTimeInNanoseconds:...
];
[self->_streamSink.stream notifyScheduledOutputChanged:output];
}
];
});
dispatch_source_set_cancel_handler(_timerSink, ^{
});
dispatch_resume(_timerSink);
}
You have a similar block on a timer in the source stream's "start" method. There, you wait for the sample buffer reference to change from nil to non-nil, and pass it on to the source stream (sendSampleBuffer), and finally release _sampleBuffer and reset it to nil:
dispatch_source_set_event_handler(_timerSource, ^{
if (self->_sampleBuffer != nil) {
if (self->_streamingCounterSource > 0) {
CMTime time = CMClockGetTime(CMClockGetHostTimeClock());
Float64 ns = CMTimeGetSeconds(time) * 1e9;
CMSampleBufferRef sbuf = nil;
CMSampleTimingInfo timing_info = {
.presentationTimeStamp = time,
};
OSStatus err = CMSampleBufferCreateCopyWithNewTiming(
kCFAllocatorDefault,
self->_sampleBuffer,
1,
(const CMSampleTimingInfo []) {timing_info},
&sbuf
);
[self->_streamSource.stream
sendSampleBuffer:sbuf
discontinuity:CMIOExtensionStreamDiscontinuityFlagNone
hostTimeInNanoseconds:ns
];
CFRelease(sbuf);
}
CFRelease(self->_sampleBuffer);
self->_sampleBuffer = nil;
}
});
Notice:
_sampleBuffer is retained in the sink timer block and released in the source timer block;
the source and sink timers are set to twice the frame rate, so you don't miss a frame;
Funny... Photo Booth will accept either 420v or kCVPixelFormatType_32BGRA (maybe more?) in the stream source's formats property. You can even send it sample buffers with a format different from the one in .formats and it will work just fine.
But QuickTime only accepts kCVPixelFormatType_32BGRA in .formats. It too works even if you send it 420v sample buffers.
Lesson: declare kCVPixelFormatType_32BGRA in .formats, and send whatever you want. 🙂
Nevermind, the pixel format was not the issue. 420v works fine. Lots of reboots later I realized I wasn't setting the timing info right.
In case anyone else is doing a source-sink type of extension, make sure to update the timing info before sending the sample buffer to the source stream:
CMSampleBufferRef sbuf = NULL;
CMSampleTimingInfo timing_info = {
.presentationTimeStamp = CMClockGetTime(CMClockGetHostTimeClock()),
};
OSStatus err = CMSampleBufferCreateCopyWithNewTiming(
kCFAllocatorDefault,
self->_sampleBuffer,
1,
(const CMSampleTimingInfo []) {timing_info},
&sbuf
);
[self->_streamSource.stream sendSampleBuffer:sbuf ...];
CFRelease(sbuf);
To clarify, kCVPixelFormatType_32BGRA works fine but I'd rather stick to 420v because the conversion to 32BGRA doubles my app's CPU usage.
Get a list of cameras with AVCaptureDeviceDiscoverySession, find the one with the localizedName that you want, then get its ID using kCMIODevicePropertyDeviceUID.
Each stream should have its own streamID:
streamID:[[NSUUID alloc] init]
I just came across this same problem. The error was:
Error Domain=OSSystemExtensionErrorDomain Code=9 "(null)"
Very grateful for the solution @eskimo. Validation works now.
Just a note: don't forget your team id prefix (com.apple.developer.team-identifier)!
If your extension's entitlements file contains:
<key>com.apple.developer.team-identifier</key><string>1234567ABC</string>
<key>com.apple.security.application-groups</key>
<array>
<string>1234567ABC.group.com.example.myapp</string>
</array>
then your extension's Info.plist should contain:
<key>CMIOExtension</key>
<dict>
<key>CMIOExtensionMachServiceName</key>
<string>1234567ABC.group.com.example.myapp.service</string>
</dict>
Don't forget to unlock at the end:
CVPixelBufferUnlockBaseAddress(imageBuffer, 0);
The sampleSizeArray argument of CMSampleBufferCreate() is an array of size_t elements, but you are passing a pointer to an array.
Change &sampleSizeArray to simply sampleSizeArray in CMSampleBufferCreate().