Created 801108294
Thanks a lot!
-Steven
Post
Replies
Boosts
Views
Activity
Thanks, hope you had a great long weekend, yes, entire Quartz Window Services APIs are failing, but Quartz Display APIs are working, my XPC is pretty simple:
int main(int argc, const char* argv[])
{
#pragma unused(argc, argv)
LOGEX("start Service");
xpc_main( new_connection_handler );
exit(EXIT_FAILURE);
}
...
void xpcEventHandler(const xpc_connection_t conn, const xpc_object_t event)
{
xpc_type_t type = xpc_get_type(event);
if (type == XPC_TYPE_DICTIONARY) {
std::async(std::launch::async, handeClientRequest, conn, event);
} else {
if (event == XPC_ERROR_CONNECTION_INVALID) {
LOGEX("xpcEventHandler peer has closed the connection!!");
} else {
LOGEX("xpcEventHandler service will be terminated soon!!");
}
}
}
...
void new_connection_handler(const xpc_connection_t conn)
{
xpc_connection_set_event_handler(conn, ^(xpc_object_t event) {
xpcEventHandler(conn, event);
});
xpc_connection_resume((xpc_connection_t)xpc_retain(conn));
}
...
If it is not by design, what you guess AppKit didn't instantiate the Window Server could the reason,
Is there any API for instantiate ?
BTW, if Quartz Windows API call failed, is there something like Windows GetLastError which provide
the failure cause?
Thanks
Steven
I'm using the process id now, the XPC can enumerate the WindowID with the AXUI API.
In the XPC, before dealing with the window scraping, I did check screencapture and it returns positive.
auto capture_able = CGPreflightScreenCaptureAccess();
if ( !capture_able)
{
LOGEX("This process doesn't have screen capture granted!");
return;
}
//XPC side check it is possitive.
And this XPC doesn't have problem in Scraping display with cgdisplaystream, only has problem with Quartz Window Service APIs, none of them behavior as expected.
Thanks
Steven
I used to pass the CGWindowID from host to the XPC service, and both CGWindowListCreateImage and CGWindowListCreateImageFromArray won't grab any image (null returned), later I pass the process pid from the host to the XPC, then use
auto findWindowIds = [](uint32_t pId) -> CFArrayRef
{
auto appRef = AXUIElementCreateApplication(pId);
CFMutableArrayRef idArray = CFArrayCreateMutable(0, 0, nullptr);
CFIndex count = 0;
CFArrayRef windowArray = NULL;
auto err = AXUIElementGetAttributeValueCount(appRef, CFSTR("AXWindows"), &count);
if (err == kAXErrorSuccess && count)
{
AXUIElementCopyAttributeValues(appRef, CFSTR("AXWindows"), 0, count, &windowArray);
for (int idx = 0; idx < count; idx ++)
{
AXUIElementRef element = (AXUIElementRef)
CFArrayGetValueAtIndex(windowArray, idx);
CGWindowID temp = 0;
_AXUIElementGetWindow(element, &temp);
LOGEX("windowId: %u", temp);
CFArrayAppendValue(idArray, reinterpret_cast<void*>(temp));
}
}
SAFE_CFRELEASE(appRef);
return idArray;
};
It does get the correct windowId, anyway, CGWindowListCreateImage and CGWindowListCreateImageFromArray still cannot grab any image.
Thanks Quinn, I believe what you said is right, could be my env issue, I have an App, due to it is using the third-party lib which doesn't support M1, I turn one of its feature into XPC service for performance, I managed to get the app built on M1 on Monterey 12.4(It has to use Xcode 11.7 to build the App, I build the XPC service with Xcode 13.4 for universal binary, this XPC service trigged by the App when using, dismissed by the App when stop using), on M1, the unsigned debug version cannot pass the screen record permission to the XPC service(Intel Mac doesn't have this issue), This caused that I cannot debug the code under M1 Mac, but release, signed and notarized client and the XPC service are working.
BTW, I'm facing another XPC service issue, I'm trying to scrape Window in the XPC services, somehow I cannot get image with the CGWindowID passed from host app, then I try to enumerate the WindowID to find the one I want to scrape, anyway both CGWindowListCopyWindowInfo and CGWindowListCreate are always returning null (in Intel Mac)
//Find WindowID of the process.
auto findWindowId = [](uint32_t pid) -> CGWindowID
{
CFArrayRef dict = CGWindowListCopyWindowInfo(kCGWindowListOptionAll, kCGNullWindowID);
//will return 0x0
SAFE_CFRELEASE(dict);
dict = CGWindowListCreate(kCGWindowListOptionAll, kCGNullWindowID);
//will return 0x0, this is just for testing
SAFE_CFRELEASE(dict);
return kCGNullWindowID;
};
auto windowId = findWindowId(GetSourceID());
Can we call CGWindowListCreate and CGWindowListCopyWindowInfo in XPC service? The host app has the screen capture permission.
-Steven
{Looks bad in comment, put it here}
Apple says network.framework is better than OpenSSL+Socket, I have problem in proving that, anyone has official way of receiving the data?
void handleIncomingVideo( void* caller, nw_connection_t connection = nullptr )
{
// init processing
...
//handle incoming data
nw_connection_receive(
connection,
1,
bufLen,
^( dispatch_data_t content,
nw_content_context_t context,
bool is_complete,
nw_error_t receive_error ) {
auto isBufferBeginWithStartCode = []( const char* frame ) -> bool {
return ( frame[0] == 0x00 && frame[1] == 0x00 && frame[2] == 0x00 && frame[3] == 0x01 );
};
if (is_complete && content == nil) {
loggerEx(LOG_INFO, "Receiver done with video data!");
return;
}
if ( content != nullptr )
{
// handle received data here
....
// 2. schedule next incoming video recv
nw_retain( connection );
handleIncomingVideo( caller, connection );
}
else
{
// Context is nullptr, schedule the next recv
if ( receive_error == nullptr )
{
nw_retain( connection );
handleIncomingVideo( caller, connection );
}
}
} );
The video quality is not as stable as my old openssl + socket client, the same video(H264 stream) server, mostly the video quality is as expected, but sometime the video is blocky(missing data, not network issue since never happen with openssl+socket client), and sometime there is no video data come in, while old openssl+socket doesn't have this issue.
Am I using it correct?
Apple says network.framework is better than OpenSSL+Socket, I have problem in proving that, anyone has official way of receiving the data?
void handleIncomingVideo( void* caller, nw_connection_t connection = nullptr )
{
// init processing
...
//handle incoming data
nw_connection_receive(
connection,
1,
bufLen,
^( dispatch_data_t content,
nw_content_context_t context,
bool is_complete,
nw_error_t receive_error ) {
auto isBufferBeginWithStartCode = []( const char* frame ) -> bool {
return ( frame[0] == 0x00 && frame[1] == 0x00 && frame[2] == 0x00 && frame[3] == 0x01 );
};
if ( content != nullptr )
{
// handle received data here
....
// 2. schedule next incoming video recv
nw_retain( connection );
handleIncomingVideo( caller, connection );
}
else
{
// Context is nullptr, schedule the next recv
if ( receive_error == nullptr )
{
nw_retain( connection );
handleIncomingVideo( caller, connection );
}
}
} );
The video (H264 stream) quality is not as good as my old openssl + socket, and sometime cannot no video data come in, while old openssl+socket doesn't have this issue.
Am I using it correct?
Also, this problem
nw_protocol_boringssl_signal_connected(724) [C7:1][0x7f793592b300] TLS connected [version(0x0303) ciphersuite(TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) group(0x0017) signature_alg(0x0401) alpn(nil) resumed(0) offered_ticket(0) false_started(0) ocsp_received(0) sct_received(0) connect_time(96ms) flight_time(88ms) rtt(77ms) write_stalls(0) read_stalls(4)]
BTW, if we call
auto sec_options = nw_tls_copy_sec_protocol_options(tls_options);
Do we need nw_release(sec_options) after using sec_options?
That's a better idea, BTW, one question might be simple to you. when I call xpc_release(connection) after xpc_connection_cancel, it will crash and system complains misuse like
libxpc.dylib 0x00007fffd9152504 _xpc_api_misuse + 75
1 libxpc.dylib 0x00007fffd9141b7f _xpc_connection_last_xref_cancel + 52
2 libxpc.dylib 0x00007fffd9141b22 -[OS_xpc_connection _xref_dispose] + 17
If I remove xpc_release, it seems to be ok, and the app didn't enable ARC, does that mean if we called xpc_connection_cancel, even in non-ARC,
we don't need call xpc_release? my env is Monterey 12.2
Thanks
Steven
Thanks a lot, looks like I need find other way for it.
-Steven
Tried many options, almost cost me two days, finally got one which is working:
auto config_tls = ^(nw_protocol_options_t tls_options) {
auto sec_options = nw_tls_copy_sec_protocol_options(tls_options);
sec_protocol_options_set_min_tls_protocol_version(sec_options, tls_protocol_version_TLSv12);
sec_protocol_options_set_max_tls_protocol_version(sec_options, tls_protocol_version_TLSv12);
sec_protocol_options_append_tls_ciphersuite_group(sec_options, tls_ciphersuite_group_default);
sec_protocol_options_set_peer_authentication_required(sec_options, false);
};
BTW, I tried openssl s_client to connect the server, it works ok w/o handshaking complain.
openssl s_client -connect 192.168.68.107:7554
but nw_parameters_create_secure_tcp won't work
One extra question, same code of screen capture, xpc module of standalone video tool box H264 encoder (running in native M1 or Intel mode), the video quality of Intel is very good, but with M1, the video is way too blurry, anyone has idea of why this is happening?
I have the same problem, even adjust bitrate it will not work, I set the min bitrate to 2Mbps, the actual is about 0.5Mbps, I'm on Monterey 12.0.1