CoreMediaIO Camera Extension: custom properties?

ObjC is what I haz. I believe you must first expose the key via the availableProperties method on your CMIOExtensionStreamSource.

From the header:

typedef NSString *CMIOExtensionProperty NS_TYPED_ENUM API_AVAILABLE(macos(12.3));

From the sample/template code:

- (NSSet<CMIOExtensionProperty> *)availableProperties {
//    return [NSSet setWithObjects:, CMIOExtensionPropertyStreamFrameDuration, nil];
CMIOExtensionProperty myCustomPropertyKey = @"4cc_cust_glob_0000";
    return [NSSet setWithObjects:
				CMIOExtensionPropertyStreamActiveFormatIndex,
				myCustomPropertyKey,
				nil];
}

In the client application you use the CoreMedia C API to set/get the property value:

CMIOObjectPropertyAddress myCustomPropertyAddress = {
	.Selector = FOUR_CHAR_CODE('cust')
	.Scope = kCMI0ObjectPropertyScopeGlobal,
	.mElement = kCMI0ObjectPropertyElementMain };

// CMIOObjectHasProperty(object,&myCustomPropertyAddress);
// CMIOObjectGetPropertyData(object, &myCustomPropertyAddress, 0, NULL, *propertyDataSize, propertyDataSize, propertyValue));

To see how to query the device list, open streams and retrieve property values, this sample may be useful:

https://github.com/bangnoise/cmiotest

When I attempt to access my custom property (on a stream), I get this error:

CMIO_DAL_CMIOExtension_Stream.mm:1165:GetPropertyData 50 wrong 4cc format for key 4cc_cust_glob_0000
CMIO_DAL_CMIOExtension_Stream.mm:1171:GetPropertyData unknown property error cust
CMIOHardware.cpp:328:CMIOObjectGetPropertyData Error: 2003332927, failed

This message is only triggered if I attempt to access the property from the client side. However, the os_log output shows the correct value for the property.

Error code 2003332927 is FOUR_CHAR_CODE('who?') which maps to kCMIOHardwareUnknownPropertyError

I have determined experimentally that under Objective-C (at least), the format of the key for a custom property is as follows:

CMIOExtensionProperty myCustomPropertyKey = @"cust_glob_0000";

The fcc_ prefix is either Swift-specific or superfluous. I'm on 12.3.

Still no data returned, the value I get is always 0.

I must recant my last comment. I was successful in retrieving custom properties on both Device and Stream using the following code:

const CMIOExtensionProperty CMIOExtensionPropertyCustomPropertyData_just = @"4cc_just_glob_0000";
const CMIOExtensionProperty CMIOExtensionPropertyCustomPropertyData_dust = @"4cc_dust_glob_0000";
const CMIOExtensionProperty CMIOExtensionPropertyCustomPropertyData_rust = @"4cc_rust_glob_0000";

+ (NSMutableDictionary*)sampleCustomPropertiesWithPropertySet:(NSSet<CMIOExtensionProperty>*)properties {
NSMutableDictionary* dictionary = [NSMutableDictionary dictionary];
	if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_just]) {
	CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue:@"Property value for 'just'"];
		[dictionary setValue:propertyState forKey:CMIOExtensionPropertyCustomPropertyData_just];
		}
	if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_dust]) {
	const size_t sData_length = 12;
	static const unsigned char sData[sData_length] = { 0xFE,0xED,0xFE,0xED, 0xFE,0xED,0xFE,0xED, 0xFE,0xED,0xFE,0xED };
	CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue:[NSData dataWithBytes:sData length:sData_length]];
		[dictionary setValue:propertyState forKey:CMIOExtensionPropertyCustomPropertyData_dust];
		}
	if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_rust]) {
	NSString* propertyValue = [NSString stringWithFormat:@"Custom property value for property '%@'.", CMIOExtensionPropertyCustomPropertyData_rust ];
	CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue:propertyValue];
		[dictionary setValue:propertyState forKey:CMIOExtensionPropertyCustomPropertyData_rust];
		}
	return dictionary;
}

Then for CMIOExtensionDeviceSource, I passed the custom property values as a dictionary to the initializer:

- (NSSet<CMIOExtensionProperty> *)availableProperties {
    return [NSSet setWithObjects:
			CMIOExtensionPropertyDeviceTransportType,
			CMIOExtensionPropertyDeviceModel,
			CMIOExtensionPropertyCustomPropertyData_just,
			CMIOExtensionPropertyCustomPropertyData_dust,
			CMIOExtensionPropertyCustomPropertyData_rust,
			nil];
}

- (nullable CMIOExtensionDeviceProperties *)devicePropertiesForProperties:(NSSet<CMIOExtensionProperty> *)properties
																	error:(NSError * _Nullable *)outError {
NSMutableDictionary* dictionary = [self sampleCustomPropertiesWithPropertySet:properties];
CMIOExtensionDeviceProperties *deviceProperties = [CMIOExtensionDeviceProperties devicePropertiesWithDictionary:dictionary];

And for CMIOExtensionStreamSource:

- (NSSet<CMIOExtensionProperty> *)availableProperties {
    return [NSSet setWithObjects:
						CMIOExtensionPropertyStreamActiveFormatIndex,
						CMIOExtensionPropertyStreamFrameDuration,
						CMIOExtensionPropertyCustomPropertyData_just,
						CMIOExtensionPropertyCustomPropertyData_dust,
						CMIOExtensionPropertyCustomPropertyData_rust,
						nil];
}

- (nullable CMIOExtensionStreamProperties *)streamPropertiesForProperties:(NSSet<CMIOExtensionProperty> *)properties
	error:(NSError * _Nullable *)outError {
NSMutableDictionary* dictionary = [self sampleCustomPropertiesWithPropertySet:properties];
CMIOExtensionStreamProperties* streamProperties = [CMIOExtensionStreamProperties streamPropertiesWithDictionary:dictionary];

This error message below is not related to the actual format of the key, the error is regarding the format of the propertyState (ie. the data associated with the key).

CMIO_DAL_CMIOExtension_Stream.mm:1165:GetPropertyData 50 wrong 4cc format for key 4cc_cust_glob_0000

I haven't tried on the Ventura beta but on 12.4 I am in the practice of rebooting after every update to my extension. If you're having problems it might be due to the fact that an old instance of your extension is being used by the system.

Replying to my own question with code to declare custom string and data properties on the camera extension, and getting/setting them on the app side in Swift. https://gist.github.com/ldenoue/84210280853f0490c79473b6edd25e9d

My question is about accessing the custom properties of provider (CMIOExtensionProviderSource).

I have successfully transfer some string data from host app to camera extension using custom properties of the device (CMIOExtensionDeviceSource) but I cannot find the way to access the custom property of Extension Provider (CMIOExtensionProviderSource).

I use this code to set the custom property of the device:

        let selector = FourCharCode("keep")
        var address = CMIOObjectPropertyAddress(selector, .global, .main)
        //deviceId: CMIOObjectID
        let exists = CMIOObjectHasProperty(deviceId, &address)
        if exists {
            var settable: DarwinBoolean = false
            CMIOObjectIsPropertySettable(deviceId,&address,&settable)
            if settable == false {
                return
            }
            var dataSize: UInt32 = 0
            CMIOObjectGetPropertyDataSize(deviceId, &address, 0, nil, &dataSize)
            var newCommand: CFString = "command string here" as NSString
            CMIOObjectSetPropertyData(deviceId, &address, 0, nil, dataSize, &newCommand)
        }

As I understand to set the property of provider I have to pass CMIOObjectId of Provider. I tried to use kCMIOObjectSystemObject, but unsuccessfully.

        //To use Extension Provider instead of Control Camera, Provider CMIOObjectId should be passed to CMIOObjectSetPropertyData instead of deviceId
        let exists = CMIOObjectHasProperty("ProviderId", &address)

        //this doesn't work, exists == false
        let exists = CMIOObjectHasProperty(kCMIOObjectSystemObject, &address)

Has someone used CMIOExtensionProviderSource custom properties and can help with that?

@DmitrijB Use this code to get the plugin ID to access custom provider properties.

CFStringRef plugInBundleID = CFStringCreateWithCString(NULL, "com.example.apple-CustomCamera.CameraExtension", kCFStringEncodingUTF8);
CMIOObjectID plugInID = kCMIOObjectUnknown;

AudioValueTranslation value;
value.mInputData = &plugInBundleID;
value.mInputDataSize = sizeof(CFStringRef);
value.mOutputData = &plugInID;
value.mOutputDataSize = sizeof(CMIOObjectID);

UInt32 used;
CMIOObjectPropertyAddress address;
address.mSelector = kCMIOHardwarePropertyPlugInForBundleID;
address.mScope = kCMIOObjectPropertyScopeGlobal;
address.mElement = kCMIOObjectPropertyElementMain;

CMIOObjectGetPropertyData(kCMIOObjectSystemObject, &address, 0, NULL, sizeof(AudioValueTranslation), &used, &value);

I am on macOS 13.2 with Xcode 14.2.

I was unable to get the custom property CMIOExtensionPropertyState of type NSDictionary and NSArray. (I was able to get values and sizes for type NSNumber, NSString, NSData).

the code failed on trying to determining the PropertyDataSize

CMIO_DAL_CMIOExtension_Device.mm:901:GetPropertyDataSize 46 wrong 4cc format for key 4cc_just_glob_0000 value {object1 = someValue;}
CMIO_DAL_CMIOExtension_Device.mm:906:GetPropertyDataSize unknown property error just
CMIOHardware.cpp:260:CMIOObjectGetPropertyDataSize Error: 2003332927, failed //('who?') kCMIOHardwareUnknownPropertyError

In CMIOExtensionDeviceSource...

const CMIOExtensionProperty CMIOExtensionPropertyCustomPropertyData_just = @"4cc_just_glob_0000";

- (NSSet<CMIOExtensionProperty> *)availableProperties
{
    return [NSSet setWithObjects:

            CMIOExtensionPropertyDeviceTransportType,
            CMIOExtensionPropertyDeviceModel,
            CMIOExtensionPropertyCustomPropertyData_just,
            nil];
}

- (nullable CMIOExtensionDeviceProperties *)devicePropertiesForProperties:(NSSet<CMIOExtensionProperty> *)properties error:(NSError * _Nullable *)outError
{
    CMIOExtensionDeviceProperties *deviceProperties = [CMIOExtensionDeviceProperties devicePropertiesWithDictionary:@{}];
    if ([properties containsObject:CMIOExtensionPropertyDeviceTransportType]) {
        deviceProperties.transportType = [NSNumber numberWithInt:kIOAudioDeviceTransportTypeVirtual];
    }

    if ([properties containsObject:CMIOExtensionPropertyDeviceModel]) {
        deviceProperties.model = @"SampleCapture Model";
    }

    if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_just]) {
        NSDictionary *numberDictionary = [[NSDictionary alloc] initWithObjectsAndKeys: @"someValue", @"object1", nil];
        CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue: numberDictionary];
      //NSArray *numberArray = [[NSArray alloc] initWithObjects:@"a", @"b", @"c", @"d", nil];
      //CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue: numberArray];

        [deviceProperties setPropertyState:propertyState forProperty:CMIOExtensionPropertyCustomPropertyData_just];
    }
    return deviceProperties;
}

In CMIO APIs

            CMIOObjectPropertyAddress propertyAddress = {
                (FOUR_CHAR_CODE( 'just' )),
                (kCMIOObjectPropertyScopeGlobal),
                (kCMIOObjectPropertyElementMain)
            };

            UInt32 dataSize = 0 ;
            UInt32 dataUsed = 0 ;
            OSStatus status = 0;

            status = CMIOObjectGetPropertyDataSize(deviceID, &propertyAddress, 0 , nil , &dataSize);
            ...
            //CFNumberRef value = NULL;
            //CFStringRef value = NULL ;
            //UInt8 value[16] = {0};
            status = CMIOObjectGetPropertyData(deviceID, &propertyAddress, 0 , nil , dataSize, &dataUsed, &value);

Has anyone encountered similar situation?

nullmc - yes, I've encountered this situation. Only numbers, data and strings are supported. Arrays and dictionaries are not.

I haven't specifically tried it but you should be able to serialize any NSCoding object into your NSData and then de-serialize it in the client extension.

@ssmith_c @DrXibber I appreciate the replies. I ran into another situation while experimenting custom props.

Is there ways to obtain a property's min and max from CMIOExtensionPropertyAttributes if a property is declared as

- (nullable CMIOExtensionDeviceProperties *)devicePropertiesForProperties:(NSSet<CMIOExtensionProperty> *)properties error:(NSError * _Nullable *)outError
{
    CMIOExtensionDeviceProperties *deviceProperties = [CMIOExtensionDeviceProperties devicePropertiesWithDictionary:@{}];

    if ([properties containsObject:CMIOExtensionPropertyCustomPropertyData_just]) {

        NSNumber *min = [[NSNumber alloc] initWithInt:0];
        NSNumber *max = [[NSNumber alloc] initWithInt:5];
        NSNumber *val = [[NSNumber alloc] initWithInt:3];
        NSArray *validRange = @[@0,@1,@2,@3,@4,@5];
        CMIOExtensionPropertyAttributes *propertyAttirbutes = 
        [CMIOExtensionPropertyAttributes propertyAttributesWithMinValue: min maxValue: max validValues: validRange readOnly:NO];

        CMIOExtensionPropertyState* propertyState = [CMIOExtensionPropertyState propertyStateWithValue:val  attributes:propertyAttirbutes];
        [deviceProperties setPropertyState:propertyState forProperty:CMIOExtensionPropertyCustomPropertyData_just];
    }
    return deviceProperties;
}

In the CMIO API I was able to get the read only boolean value by calling

CMIOObjectPropertyAddress propertyAddress = {
                (FOUR_CHAR_CODE( 'just' )),
                (kCMIOObjectPropertyScopeGlobal),
                (kCMIOObjectPropertyElementMain)};

Boolean settable;
status = CMIOObjectIsPropertySettable(deviceID, &propertyAddress, &settable);

However, calling things like (how we obtained property range for legacy CoreMediaIO plugin)

AudioValueRange myRange;
CMIOObjectPropertyAddress opa = {kCMIOFeatureControlPropertyAbsoluteValue, kCMIOObjectPropertyScopeGlobal, kCMIOObjectPropertyElementMain};
CMIOObjectPropertyAddress opa = {kCMIOFeatureControlPropertyNativeRange, kCMIOObjectPropertyScopeGlobal, kCMIOObjectPropertyElementMain};
result = CMIOObjectGetPropertyDataSize(deivceID, &opa, 0, NULL, &dataSize);
result = CMIOObjectGetPropertyData(deivceID, &opa, 0, NULL, dataSize, &dataSize, &myRange);

did not work and returned

Error: 2003332927, failed //('who?') kCMIOHardwareUnknownPropertyError

Does anyone know the how to obtain values in CMIOExtensionPropertyAttributes?

I am quite familiar with the older DAL architecture, having built a large number of camera plug-ins dating back to 2011. And before that also VDIG components which were the predecessor to DAL. I am not aware of any API in the C-language CoreMediaIO framework that restricts scalar property data to a specific value range. It's possible that this is part of some kind of validation layer new to CMIO Extensions aimed at preventing invalid values from being sent to your custom properties.

The available property set/get methods (such as you're using) are declared in the CoreMediaIO.framework CMIOHardwareObject.h I believe these are the only functions for accessing properties from the client side. Try sending an out-of-range value to your property declared with max and min and see if something pops up in the os_log

If you have a more complex property data type (such as your ranged values), my suggestion is to serialize it into a NSDictionary and then serialize the NSDictionary into an NSData object. Pass the NSData over the custom property connection. You could also use NSKeyedArchiver to flatten (into NSData) an arbitrary NSObject-derived class conforming to NSCoding. Or JSON data serialized in an NSString.

In my testing, older DAL plugins are already completely deprecated on MacOS Ventura (the WWDC presentation by Brad Ford says Ventura will be the last OS to support DAL). But it can be helpful to have a plugin built with the legacy DAL system so you can see exactly how the property data is communicated in the old system before trying to migrate to the new extension system. Monterey appears to be the last version of MacOS to support DAL CFPlugIn components and as such, probably the preferred OS for developing CMIO solutions.

I would recommend against using this sample code (below) for production because there are some serious race conditions in this code (ie. when multiple AVCaptureSession instances access the 'object property store' without any locks or queues. But for getting the gist of property data flow, this will give you both sides of the equation within a single process that Xcode can easily debug: https://github.com/johnboiles/coremediaio-dal-minimal-example

Once you have it completely grokked, then migrate to the new CMIO Extension property system.

There's a Swift adaptation of johnboiles sample code that's even more heinous because it pulls a Swift runtime into the host process - and that's an exciting party if the host was built with a different version of Swift. But if you're just using it for scaffolding, it may serve your needs.

@DrXibber I tried sending out of range value to the property. The min/max boundary didn't seem to have effect.

Calling

status = CMIOObjectSetPropertyData(devices[i], &propertyAddress, 0, nil, dataSize, &value)

with an out of range value will still trigger

- (BOOL)setDeviceProperties:(CMIOExtensionDeviceProperties *)deviceProperties error:(NSError * _Nullable *)outError 

in CMIOExtension. you can then manipulate the data with own internal logics.

However, the value in readOnly attribute in CMIOExtensionPropertyAttributes will be correctly reflected on corresponding CMIO APIs

...
Boolean settable;
status = CMIOObjectIsPropertySettable(deviceID, &propertyAddress, &settable);
status = CMIOObjectSetPropertyData(devices[i], &propertyAddress, 0, nil, dataSize, &value);

For anyone looking here who might benefit, it seems that you can use NSNumbers for custom CMIO properties under macOS 14.x, but that doing so will fail under macOS 12.x I detailed the issue in this post.

-- Mr. Fantastic