Post not yet marked as solved
Created 801108294
Thanks a lot!
-Steven
Post not yet marked as solved
Thanks, hope you had a great long weekend, yes, entire Quartz Window Services APIs are failing, but Quartz Display APIs are working, my XPC is pretty simple:
int main(int argc, const char* argv[])
{
#pragma unused(argc, argv)
LOGEX("start Service");
xpc_main( new_connection_handler );
exit(EXIT_FAILURE);
}
...
void xpcEventHandler(const xpc_connection_t conn, const xpc_object_t event)
{
xpc_type_t type = xpc_get_type(event);
if (type == XPC_TYPE_DICTIONARY) {
std::async(std::launch::async, handeClientRequest, conn, event);
} else {
if (event == XPC_ERROR_CONNECTION_INVALID) {
LOGEX("xpcEventHandler peer has closed the connection!!");
} else {
LOGEX("xpcEventHandler service will be terminated soon!!");
}
}
}
...
void new_connection_handler(const xpc_connection_t conn)
{
xpc_connection_set_event_handler(conn, ^(xpc_object_t event) {
xpcEventHandler(conn, event);
});
xpc_connection_resume((xpc_connection_t)xpc_retain(conn));
}
...
If it is not by design, what you guess AppKit didn't instantiate the Window Server could the reason,
Is there any API for instantiate ?
BTW, if Quartz Windows API call failed, is there something like Windows GetLastError which provide
the failure cause?
Thanks
Steven
Post not yet marked as solved
I'm using the process id now, the XPC can enumerate the WindowID with the AXUI API.
In the XPC, before dealing with the window scraping, I did check screencapture and it returns positive.
auto capture_able = CGPreflightScreenCaptureAccess();
if ( !capture_able)
{
LOGEX("This process doesn't have screen capture granted!");
return;
}
//XPC side check it is possitive.
And this XPC doesn't have problem in Scraping display with cgdisplaystream, only has problem with Quartz Window Service APIs, none of them behavior as expected.
Thanks
Steven
Post not yet marked as solved
I used to pass the CGWindowID from host to the XPC service, and both CGWindowListCreateImage and CGWindowListCreateImageFromArray won't grab any image (null returned), later I pass the process pid from the host to the XPC, then use
auto findWindowIds = [](uint32_t pId) -> CFArrayRef
{
auto appRef = AXUIElementCreateApplication(pId);
CFMutableArrayRef idArray = CFArrayCreateMutable(0, 0, nullptr);
CFIndex count = 0;
CFArrayRef windowArray = NULL;
auto err = AXUIElementGetAttributeValueCount(appRef, CFSTR("AXWindows"), &count);
if (err == kAXErrorSuccess && count)
{
AXUIElementCopyAttributeValues(appRef, CFSTR("AXWindows"), 0, count, &windowArray);
for (int idx = 0; idx < count; idx ++)
{
AXUIElementRef element = (AXUIElementRef)
CFArrayGetValueAtIndex(windowArray, idx);
CGWindowID temp = 0;
_AXUIElementGetWindow(element, &temp);
LOGEX("windowId: %u", temp);
CFArrayAppendValue(idArray, reinterpret_cast<void*>(temp));
}
}
SAFE_CFRELEASE(appRef);
return idArray;
};
It does get the correct windowId, anyway, CGWindowListCreateImage and CGWindowListCreateImageFromArray still cannot grab any image.
Post not yet marked as solved
Thanks Quinn, I believe what you said is right, could be my env issue, I have an App, due to it is using the third-party lib which doesn't support M1, I turn one of its feature into XPC service for performance, I managed to get the app built on M1 on Monterey 12.4(It has to use Xcode 11.7 to build the App, I build the XPC service with Xcode 13.4 for universal binary, this XPC service trigged by the App when using, dismissed by the App when stop using), on M1, the unsigned debug version cannot pass the screen record permission to the XPC service(Intel Mac doesn't have this issue), This caused that I cannot debug the code under M1 Mac, but release, signed and notarized client and the XPC service are working.
BTW, I'm facing another XPC service issue, I'm trying to scrape Window in the XPC services, somehow I cannot get image with the CGWindowID passed from host app, then I try to enumerate the WindowID to find the one I want to scrape, anyway both CGWindowListCopyWindowInfo and CGWindowListCreate are always returning null (in Intel Mac)
//Find WindowID of the process.
auto findWindowId = [](uint32_t pid) -> CGWindowID
{
CFArrayRef dict = CGWindowListCopyWindowInfo(kCGWindowListOptionAll, kCGNullWindowID);
//will return 0x0
SAFE_CFRELEASE(dict);
dict = CGWindowListCreate(kCGWindowListOptionAll, kCGNullWindowID);
//will return 0x0, this is just for testing
SAFE_CFRELEASE(dict);
return kCGNullWindowID;
};
auto windowId = findWindowId(GetSourceID());
Can we call CGWindowListCreate and CGWindowListCopyWindowInfo in XPC service? The host app has the screen capture permission.
-Steven
Post not yet marked as solved
{Looks bad in comment, put it here}
Apple says network.framework is better than OpenSSL+Socket, I have problem in proving that, anyone has official way of receiving the data?
void handleIncomingVideo( void* caller, nw_connection_t connection = nullptr )
{
// init processing
...
//handle incoming data
nw_connection_receive(
connection,
1,
bufLen,
^( dispatch_data_t content,
nw_content_context_t context,
bool is_complete,
nw_error_t receive_error ) {
auto isBufferBeginWithStartCode = []( const char* frame ) -> bool {
return ( frame[0] == 0x00 && frame[1] == 0x00 && frame[2] == 0x00 && frame[3] == 0x01 );
};
if (is_complete && content == nil) {
loggerEx(LOG_INFO, "Receiver done with video data!");
return;
}
if ( content != nullptr )
{
// handle received data here
....
// 2. schedule next incoming video recv
nw_retain( connection );
handleIncomingVideo( caller, connection );
}
else
{
// Context is nullptr, schedule the next recv
if ( receive_error == nullptr )
{
nw_retain( connection );
handleIncomingVideo( caller, connection );
}
}
} );
The video quality is not as stable as my old openssl + socket client, the same video(H264 stream) server, mostly the video quality is as expected, but sometime the video is blocky(missing data, not network issue since never happen with openssl+socket client), and sometime there is no video data come in, while old openssl+socket doesn't have this issue.
Am I using it correct?
That's a better idea, BTW, one question might be simple to you. when I call xpc_release(connection) after xpc_connection_cancel, it will crash and system complains misuse like
libxpc.dylib 0x00007fffd9152504 _xpc_api_misuse + 75
1 libxpc.dylib 0x00007fffd9141b7f _xpc_connection_last_xref_cancel + 52
2 libxpc.dylib 0x00007fffd9141b22 -[OS_xpc_connection _xref_dispose] + 17
If I remove xpc_release, it seems to be ok, and the app didn't enable ARC, does that mean if we called xpc_connection_cancel, even in non-ARC,
we don't need call xpc_release? my env is Monterey 12.2
Thanks
Steven
Thanks a lot, looks like I need find other way for it.
-Steven
Tried many options, almost cost me two days, finally got one which is working:
auto config_tls = ^(nw_protocol_options_t tls_options) {
auto sec_options = nw_tls_copy_sec_protocol_options(tls_options);
sec_protocol_options_set_min_tls_protocol_version(sec_options, tls_protocol_version_TLSv12);
sec_protocol_options_set_max_tls_protocol_version(sec_options, tls_protocol_version_TLSv12);
sec_protocol_options_append_tls_ciphersuite_group(sec_options, tls_ciphersuite_group_default);
sec_protocol_options_set_peer_authentication_required(sec_options, false);
};
One extra question, same code of screen capture, xpc module of standalone video tool box H264 encoder (running in native M1 or Intel mode), the video quality of Intel is very good, but with M1, the video is way too blurry, anyone has idea of why this is happening?
Post not yet marked as solved
I have the same problem, even adjust bitrate it will not work, I set the min bitrate to 2Mbps, the actual is about 0.5Mbps, I'm on Monterey 12.0.1
Great, CFRetain fixed the occasional crash issue.
Thanks to Quinn :)
Thanks, I posted the screenshot since it contains the error information which might help understanding the problem.
BTW, that surfaceObj was from callback block of
CGDisplayStreamCreateWithDispatchQueue, and its value is not null but not sure if it is valid.
auto dspref = CGDisplayStreamCreateWithDispatchQueue(
displayId,
captureWidth,
captureHeight,
'420v',
opts,
dispatch_queue_create("my_display_data", DISPATCH_QUEUE_SERIAL),
^( CGDisplayStreamFrameStatus status, uint64_t time, IOSurfaceRef surfaceObj, CGDisplayStreamUpdateRef ref )
{
'
Do we have any API to check if it is valid to avoid crash?
Do we need release this surfaceObj after sent out the XPC message? I suppose system do the maintenance job for the callback block.
Again, ARC was disabled.
-Steven
How do I avoid IOSurfaceCreateXPCObject crash? right now it is my headache issue.
Never mind, I'm sending surface instead and it is working
