How to correctly create IOUserBlockStorageDevice through iouserclient?

I'm currently use IOUserBlockStorageDevice to simulate a ram disk. To simulate a pluggable ramdisk that can commute, I would like to use a hierarcy architecture.

However, something strange happened while I'm using hierarcy architecture.

The hierarcy architecture might shows as following in ioreg

...
+-o IOUserResources  <class IOUserResources, id 0x10000011b, registered, matched, active, busy 0 (571 ms), retain 9>
    | +-o IOUserDockChannelSerial  <class IOUserService, id 0x10000040c, registered, matched, active, busy 0 (0 ms), retain 8>
    | +-o MyService  <class IOUserService, id 0x10000040d, registered, matched, active, busy 0 (2 ms), retain 9>
    |   +-o MyUserClient  <class IOUserUserClient, id 0x100000760, registered, matched, active, busy 0 (2 ms), retain 9>
    |     +-o MyDevice  <class IOUserBlockStorageDevice, id 0x100000762, registered, matched, active, busy 0 (2 ms), retain 8>
    |       +-o IOBlockStorageDriver  <class IOBlockStorageDriver, id 0x100000763, registered, matched, active, busy 0 (1 ms), retain 8>
    |         +-o Testing Media  <class IOMedia, id 0x100000764, registered, matched, active, busy 0 (1 ms), retain 9>
    |           +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x100000765, registered, matched, active, busy 0 (0 ms), retain 6>
    +-o IOUserServer(com.apple.IOUserDockChannelSerial-0x10000040c)  <class IOUserServer, id 0x100000472, registered, matched, active, busy 0 (0 ms), retain 11>
    +-o IOUserServer(com.example.apple-samplecode.dext-to-user-client.driver-0x10000040d)  <class IOUserServer, id 0x100000474, registered, matched, active, busy 0 (0 ms), retain 13>

MyService is the loaded dext, which create an UserClient while a client connect.

MyUserClient has two interface: adddevice / remove which will add or remove a MyDevice instance.

MyDevice is the IOUserBlockStorageDevice create by MyuserClient.

I'm using the following plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>CFBundleDevelopmentRegion</key>
	<string>$(DEVELOPMENT_LANGUAGE)</string>
	<key>CFBundleExecutable</key>
	<string>$(EXECUTABLE_NAME)</string>
	<key>CFBundleIdentifier</key>
	<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
	<key>CFBundleInfoDictionaryVersion</key>
	<string>6.0</string>
	<key>CFBundleName</key>
	<string>$(PRODUCT_NAME)</string>
	<key>CFBundlePackageType</key>
	<string>$(PRODUCT_BUNDLE_PACKAGE_TYPE)</string>
	<key>CFBundleShortVersionString</key>
	<string>1.0</string>
	<key>CFBundleVersion</key>
	<string>1</string>
	<key>IOKitPersonalities</key>
	<dict>
		<key>testDevice</key>
		<dict>
			<key>CFBundleIdentifier</key>
			<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
			<key>CFBundleIdentifierKernel</key>
			<string>com.apple.kpi.iokit</string>
			<key>IOClass</key>
			<string>IOUserService</string>
			<key>IOMatchCategory</key>
			<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
			<key>IOProviderClass</key>
			<string>IOUserResources</string>
			<key>IOResourceMatch</key>
			<string>IOKit</string>
			<key>IOUserClass</key>
			<string>MyService</string>
			<key>IOUserServerName</key>
			<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
			<key>UserClientProperties</key>
			<dict>
				<key>MyDeviceProperties</key>
				<dict>					
					<key>CFBundleIdentifierKernel</key>
					<string>com.apple.iokit.IOStorageFamily</string>
					<key>IOClass</key>
					<string>IOUserBlockStorageDevice</string>
					<key>IOUserClass</key>
					<string>MyDevice</string>
				</dict>
				<key>IOClass</key>
				<string>IOUserUserClient</string>
				<key>IOUserClass</key>
				<string>MyUserClient</string>
			</dict>
		</dict>
	</dict>
	<key>OSBundleUsageDescription</key>
	<string></string>
</dict>
</plist>

and MyDevice is create by "auto kr = Create(this, "MyDeviceProperties", &client);" in MyUserClient.

To make sure the flow of MyDevice, I add an simple log (inside *** function) while entering each function implement.

Following is the system log after adddevice:

=== ExternalMethod AddDevice ===
2022-10-21 12:08:20.424783+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyUserClient ccc Inside ExternalMethod!!!!!
2022-10-21 12:08:20.424803+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyUserClient ExternalMethodType_AddDevice
2022-10-21 12:08:20.424812+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyUserClient  NewDevice
2022-10-21 12:08:20.424847+0800 0xcd3cd    Default     0x0                  0      0    kernel: (IOStorageFamily) INFO virtual bool IOUserBlockStorageDevice::init(OSDictionary *): Allocate resources
2022-10-21 12:08:20.425059+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice init.....
2022-10-21 12:08:20.425145+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice init done
2022-10-21 12:08:20.425152+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice Start
2022-10-21 12:08:20.425158+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) IOUserBlockStorageDevice::Start()
2022-10-21 12:08:20.425201+0800 0xcd3cd    Default     0x0                  0      0    kernel: (IOStorageFamily) INFO kern_return_t IOUserBlockStorageDevice::RegisterDext_Impl(): Registering dext
2022-10-21 12:08:20.425212+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice Reg service
2022-10-21 12:08:20.425270+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice Start() - Finished.
2022-10-21 12:08:20.425281+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyUserClient  NewDevice done
2022-10-21 12:08:20.425598+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside ReportRemovability
2022-10-21 12:08:20.425852+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside ReportWriteProtection
2022-10-21 12:08:20.425916+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside GetVendorString
2022-10-21 12:08:20.426022+0800 0xcd3cd    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside GetProductString
<log end, device stuch here>

The IOUserBlockStorageDevice stuck after calling GetProductString, not called GetDeviceParam, and the device size remains be zero in this case. I'm not sure why MyDevice stuck.

Could anyone help for this case? I'll post some detailed test in reply.

To clearify the problem, I make another test on IOUserBlockStorageDevice.

Using "SAME" MyDevice code, while I change my archiecture to single level(directly create MyDevice while dext load)

Following is the ioreg stack, which looks familiar in hierarcy architecture.

...
+-o IOUserResources  <class IOUserResources, id 0x10000011b, registered, matched, active, busy 0 (548 ms), retain 9>
    | +-o IOUserDockChannelSerial  <class IOUserService, id 0x10000040d, registered, matched, active, busy 0 (0 ms), retain 8>
    | +-o MyDevice  <class IOUserBlockStorageDevice, id 0x10000040e, registered, matched, active, busy 0 (2 ms), retain 9>
    |   +-o IOBlockStorageDriver  <class IOBlockStorageDriver, id 0x100000479, registered, matched, active, busy 0 (2 ms), retain 8>
    |     +-o Testing Media  <class IOMedia, id 0x10000047a, registered, matched, active, busy 0 (2 ms), retain 9>
    |       +-o IOMediaBSDClient  <class IOMediaBSDClient, id 0x10000047b, registered, matched, active, busy 0 (0 ms), retain 6>
    +-o IOUserServer(com.apple.IOUserDockChannelSerial-0x10000040d)  <class IOUserServer, id 0x100000474, registered, matched, active, busy 0 (0 ms), retain 11>

And the plist, just directly attach IOUserBlockStorageDevice

... (same as previous plist)
	<key>IOKitPersonalities</key>
	<dict>
		<key>MyService</key>
		<dict>
			<key>CFBundleIdentifier</key>
			<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
			<key>CFBundleIdentifierKernel</key>
			<string>com.apple.iokit.IOStorageFamily</string>
			<key>IOClass</key>
			<string>IOUserBlockStorageDevice</string>
			<key>IOMatchCategory</key>
			<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
			<key>IOProviderClass</key>
			<string>IOUserResources</string>
			<key>IOResourceMatch</key>
			<string>IOKit</string>
			<key>IOUserClass</key>
			<string>MyDevice</string>
			<key>IOUserServerName</key>
			<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>			
		</dict>
	</dict>
	<key>OSBundleUsageDescription</key>
	<string></string>
</dict>
</plist>

And the system log while dext install

2022-10-21 15:08:17.636661+0800 0xdd748    Default     0x0                  0      0    kernel: (IOStorageFamily) INFO virtual bool IOUserBlockStorageDevice::init(OSDictionary *): Allocate resources
2022-10-21 15:08:17.649681+0800 0xdd74a    Default     0x0                  0      0    kernel: Found class: IOUserBlockStorageDevice
2022-10-21 15:08:17.650548+0800 0xdd748    Default     0x0                  0      0    kernel: (IOStorageFamily) INFO virtual bool IOUserBlockStorageDevice::start(IOService *): Started
2022-10-21 15:08:17.651010+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice init.....
2022-10-21 15:08:17.651120+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice init done
2022-10-21 15:08:17.651137+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice Start
2022-10-21 15:08:17.651142+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) IOUserBlockStorageDevice::Start()
2022-10-21 15:08:17.651183+0800 0xdd74e    Default     0x0                  0      0    kernel: (IOStorageFamily) INFO kern_return_t IOUserBlockStorageDevice::RegisterDext_Impl(): Registering dext
2022-10-21 15:08:17.651223+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice Reg service
2022-10-21 15:08:17.651257+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice Start() - Finished.
2022-10-21 15:08:17.651304+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside GetDeviceParams <-----------calls here!!!!!
2022-10-21 15:08:17.651463+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside ReportRemovability
2022-10-21 15:08:17.651577+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside ReportWriteProtection
2022-10-21 15:08:17.651621+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside GetVendorString
2022-10-21 15:08:17.651657+0800 0xdd74e    Default     0x0                  0      0    kernel: (dext) AAABBB MyDevice inside GetProductString
2022-10-21 15:08:17.652767+0800 0xdd751    Default     0x0                  0      0    kernel: (IOStorageFamily) ERR virtual IOReturn IOUserBlockStorageDevice::doAsyncReadWrite(IOMemoryDescriptor *, uint64_t, uint64_t, IOStorageAttributes *, IOStorageCompletion *):
… more doAsyncReadWrite log

Yes, MyDevice work fine and calls GetDeviceParam now.

However, it's not what I want cause I want to use hierarcy architecture to add / remove device and control device size through other binary.

Dig into the difference between 2 logs from 2 hierarchies, these logs affect the behavior:

MyService / MyUserClient / MyDevice hierarchy:
2022-10-21 12:08:20.424847+0800 0xcd3cd    Default     0x0                  0      0    kernel: (IOStorageFamily) INFO virtual bool IOUserBlockStorageDevice::init(OSDictionary *): Allocate resources

MyDevice hierarchy:
2022-10-21 15:08:17.636661+0800 0xdd748    Default     0x0                  0      0    kernel: (IOStorageFamily) INFO virtual bool IOUserBlockStorageDevice::init(OSDictionary *): Allocate resources
2022-10-21 15:08:17.649681+0800 0xdd74a    Default     0x0                  0      0    kernel: Found class: IOUserBlockStorageDevice
2022-10-21 15:08:17.650548+0800 0xdd748    Default     0x0                  0      0    kernel: (IOStorageFamily) 
INFO virtual bool IOUserBlockStorageDevice::start(IOService *): Started

There’s no “kernel: (IOStorageFamily) INFO virtual bool IOUserBlockStorageDevice::start(IOService *): Started“ log in MyService / MyUserClient / MyDevice hierarchy.

I have searched it in github and found the code: https://github.com/apple-open-source/macos/blob/8ecaeeea2e342177fc8d87238569f3634efb26ad/IOStorageFamily/IOUserBlockStorageDevice_kext.cpp

In line 157, GetDeviceParams(&fDeviceParams); called, 157 is in the function of bool IOUserBlockStorageDevice::start(IOService *provider).

Why IOUserBlockStorageDevice::start(IOService *): Started not called in MyService / MyUserClient / MyDevice hierarchy? Maybe I miss some steps while create MyDevice?

How to make my IOUserBlockStorageDevice work fine (calls GetDeviceParam) in hierarcy architecture?

Thanks for everyone's help :D !

How to correctly create IOUserBlockStorageDevice through iouserclient?
 
 
Q