ScreenCaptureKit

RSS for tag

ScreenCaptureKit brings high-performance screen capture, including audio and video, to macOS.

Posts under ScreenCaptureKit tag

40 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

How to silence weekly/monthly screen capture access notification?
I'm running a launch agent in a CI node. The agent is responsible for launching CI build/test jobs. The agent, being the responsible process, has been granted kTCCServiceScreenCapture permission. With this in place I can run /usr/sbin/screencapture during CI test jobs, archiving the visual state of the CI machine if a test fails, which makes it easier to diagnose why the test failed. However with macOS 15 I get weekly/monthly notifications about the agent being able to record the screen. The general advice for this is that apps should migrate to ScreenCaptureKit, but I'm using a built in tool in macOS, /usr/sbin/screencapture, so how am I supposed to deal with that?
1
0
236
Aug ’24
updateConfiguration no work sometimes
I'm using screenCaptureKit for winodow capture. I build a filter like follow code, (I'm not usng independent window filter, because sometime I need capture multi windows in the same time) filter = [[SCContentFilter alloc] initWithDisplay:displayID includingWindows:includingWindows]; At begining, the capture works OK. When the target window's size or position changed, my code monitored this change and called updateConfiguration like below , I got completionHandler without error. [streamConfiguration setSourceRect:newRect]; [streamConfiguration setWidth:newWidth]; [streamConfiguration setHeight:newHeight]; [scStream updateConfiguration:streamConfiguration completionHandler:^(NSError *_Nullable error) { if (error) { // some error log } else { // update done } }]; But sometimes, it still output frame with old size, and the rect is still the old. And int some other cases, it works fine..... Is there any special work before call updateConfiguration to make it work ?
0
0
221
Aug ’24
Sending '$0' risks causing data races
I had no luck to compile a sample code provided by apple with Xcode 16.0 beta 5. ScreenCaptureKit demo (https://developer.apple.com/documentation/screencapturekit/capturing_screen_content_in_macos) The part it is failling is, streamOutput.capturedFrameHandler = { continuation.yield($0) } And the error message is Sending '$0' risks causing data races Task-isolated '$0' is passed as a 'sending' parameter; Uses in callee may race with later task-isolated uses Please enlighten me why this is an issue and how to avoid? Thanks in advance!
2
0
565
Aug ’24
How do we detect SCContentSharingPicker is cancelled?
Hello, I am trying to make use of SCContentSharingPicker for my app and I wonder how I can detect a close event of SCContentSharingPicker. I could open the picker screen with following simple code: SCContentSharingPicker.shared.isActive = true SCContentSharingPicker.shared.add(self) SCContentSharingPicker.shared.present() And I closed it with "Cancel" button located at the top right corner. Initially I was expecting to get a event through an observer like below but realised that it's called when a stream is canceled. extension ContentPickerButton: SCContentSharingPickerObserver { func contentSharingPicker(_ picker: SCContentSharingPicker, didCancelFor stream: SCStream?) { logger.info("Picker canceled for stream \(stream)") } I would like to get a picker close event so that I can deactivate the picker. (Otherwise, camera icon will stay alive at the tray.) How do we get a close event?
1
0
295
Aug ’24
ScreenCaptureKit com.apple.developer.persistent-content-capture entitlement?
One of my apps, Default Folder X, is an unconventional user of screen recording (and now ScreenCaptureKit). Part of its functionality is to add navigation controls to the Open and Save dialogs of other applications. It does this via the Accessibility API, and because of the limitations of that API, it sometimes has to actually pop up a menu in the target app's file dialog. To hide this from the user, it takes a screenshot of the Open or Save dialog and displays it in front of the dialog as a façade while it does its menu manipulation. Here's an example without the use of a captured image: https://www.stclairsoft.com/blog/wp-content/uploads/2024/08/EmptyFolderBehindTheCurtain.mov And an example with the façade: https://www.stclairsoft.com/blog/wp-content/uploads/2024/08/EmptyFolderWithScreenshot.mov This use case prevents me from using SCContentSharingPicker, as it's not a user-driven screen capture. Moreover, Sequoia b5's weekly screen recording reminders are popping up while the user is interacting with an Open or Save dialog, severely impacting his / her workflow. It appears that the Persistent Content Capture entitlement may prevent Sequoia from putting up the weekly warnings, though there's no documentation of the entitlement other than it being listed here: https://developer.apple.com/documentation/bundleresources/entitlements/com_apple_developer_persistent-content-capture. So my questions: Is that what the Persistent Content Capture entitlement does? Where is the form to request this entitlement? Without the entitlement, I can't see continuing the use of screen captures. And eliminating that will compromise the UI in my app in the way I've described above. It will also make Default Folder X unable to tailor its UI to match the Open and Save dialogs of the app it's enhancing (there's no API for it to use to get the light / dark mode of the window of another app, so it currently captures an image of the target file dialog to determine its UI mode). Thanks - Jon
2
4
575
1w
WindowServer daemon crashes
The scenario is quite simple run an application which uses [SCShareableContent getShareableContentExcludingDesktopWindows] and invoke captureImageWithFilter in completionHandler. delay invoking captureImageWithFilter for several seconds and switch user session before call it. The WindowServer crashes if app runs in inactive session. How to manage this issue correctly? Are there any way to avoid this crash?
1
0
293
Aug ’24
Apps using ScreenCaptureKit to capture screen don't capture some UI elements
Hello, I have a few apps that I use for screen recording/streaming like OBS as well as capturing screens to project into a VR headset (Immersed), and they use ScreenCaptureKit to record the full displays/all content. But when capturing that display, some application windows or UI elements, like in Microsoft Teams when you begin a screen sharing session and you get a control-bar overlay to manage sharing options (stop, start, etc), that particular element does not get captured by the recording app's capture (though other MS Teams windows and all the other applications on screen do). Another app that has this problem is the CleanShot X screen capturing app, where it's overlay UI elements don't get captured, but are still on the physical screen. This of course when using Mac displays in VR causes in issue where you can't see these particular CleanShot controls but they are there and intercepting mouse clicks/input. What would be causing only certain elements to not get captured when the recording app is telling ScreenCaptureKit to not exclude anything, and is there a property on these UI elements that the developer can "opt in" to get SCK to pick them up? I am trying to figure out what feedback to give to developers of programs that have this issue/if it's possible for them to modify their apps to change this behavior? Thanks!
0
0
303
Aug ’24
ScreenCaptureKit with dual monitors problem
I have a mac os app that uses screen capture logic. It was originally coded using the Quartz CG api: if let cgimage = CGDisplayCreateImage(CGMainDisplayID(), rect: cgRect) {...} and this worked as expected even when capturing a screen rect that began on secondary monitor and ended on primary monitor (or was entirely contained on secondary monitor). However now that API is deprecated and you're supposed to use ScreenCaptureKit instead. So I have attempted to convert the code. The trial code is: let scConfig = SCStreamConfiguration() scConfig.sourceRect = drect scConfig.width = Int(drect.width) scConfig.height = Int(drect.height) SCScreenshotManager.captureImage(contentFilter: sFilter, configuration: scConfig) {any,error in if let cgim = any { print("image dims \(cgim.width), \(cgim.height), requested: \(drect)") self.writeToFile2(cgim) } else { print("SCREEN CAP failed") } } ... where sFilter was previously set based on main screen display (with no exclusions). This code also "works" as long as the capture rect is entirely on primary monitor. But it fails if the rect spans both monitors or is fully contained on secondary monitor. (By fails I mean it produces empty image) So my question is: How to use ScreenCaptureKit to obtain screen shot of rectangle that spans dual monitors?
1
1
349
1d
Screen recording TCC permission overcome?
Hi, I am developing a background screen recording app and macOS Sequoia is constantly blocking the app after restart with a nag screen "Confirm to continue the screen recording". That's looking insane! I know that the built-in Grab Screenshot app is not requesting any permission and that is an anti-competitive beaviour! I have found inside the compiled code the following plist XML code that disables the software firewall TCC, implemented by Apple. It's undocumented for no reason. Anyone knows how to repeat this unblocking tweak?
3
0
396
Jul ’24
[macOS Sequoia b1-3] Screen recording permission for XPC service repeatedly requested
Hi, my app ScreenFloat can capture screenshots and record the screen (along with system- and microphone audio). It does this in an XPC service. On macOS Sequoia b1-3, recording does not work anymore (although permissions are granted to the app in System Preferences > Privacy & Security). Instead, I keep getting an error that my XPC service can access this computer's screen and audio. (of course, that's the point!) First of all, the screen is locked when the warning appears, clicks anywhere on the screen are not recognized. I have to hit Escape (or wait about a minute, at which point it resolves itself), to be able to click anywhere. Clicking on Continue To Allow doesn't do anything, either. The warning just re-appears every time. Do I need to add a new entitlement to my main app or the XPC service, or any new NSUsage strings to the InfoPlist.strings? How can I resolve this? Thank you, Matthias
16
1
1.8k
Aug ’24
What is the programmatic way to capture screen from a connected iOS device to the Mac?
What framework to use to capture screen of a device connected to the Mac in the way OBS or QuickTime Player does when an iOS device is connected to Mac via USB. I tried to list devices with AVFoundation and ScreenCaptureKit but only Mac camera, mic and displays are listed. When you select New Movie Recording in the QuickTime Player you can choose an Connected iPad or iPhone to record it's screan. Same with OBS. What is the way to do it in my own MacOS app?
2
0
539
Jul ’24
CGDisplayStream API hangs on MacOS15 for same name process
Hi, recently some users of my app upgrade MacOS to 15, and encounter hang when trying to get screen capture permission. After looking into this issue, I find out a repoducible way. If two processes of a same program both call CGDisplayStreamCreate/CGDisplayImageCreate API, the later one will hang(No need to be at almost same time). Here are my demo codes. #import <AVFoundation/AVFoundation.h> #import <CoreGraphics/CoreGraphics.h> #import <Foundation/Foundation.h> int main(int argc, const char *argv[]) { @autoreleasepool { CGDirectDisplayID displayID = CGMainDisplayID(); CGDisplayStreamRef stream = CGDisplayStreamCreate( displayID, 1, 1, kCVPixelFormatType_32BGRA, nil, ^(CGDisplayStreamFrameStatus status, uint64_t displayTime, IOSurfaceRef frameSurface, CGDisplayStreamUpdateRef updateRef){ }); if (!stream) { NSLog(@"Failed to create display stream!"); return 1; } else { NSLog(@"Create display stream success!"); CFRelease(stream); } } sleep(10); return 0; } The call of sleep is just to ensure I can run a second process of this program before the first one ends. It turns out that the first process checks the permission correctly and sleeps successfully. However the second process won't be able to print anything because it hangs when calling CGDisplayStreamCreate. Here is the output of sample: Call graph: 2546 Thread_1315735 DispatchQueue_1: com.apple.main-thread (serial) + 2546 start (in dyld) + 2840 [0x196f8f274] + 2546 main (in testapp) + 76 [0x100ef7e38] + 2546 SLDisplayStreamCreate (in SkyLight) + 316 [0x19d38163c] + 2546 SLSDisplayStreamCreateProxying (in ScreenCaptureKit) + 816 [0x231e2d614] + 2546 _dispatch_semaphore_wait_slow (in libdispatch.dylib) + 76 [0x19715e278] + 2546 _dispatch_sema4_timedwait (in libdispatch.dylib) + 64 [0x19715dc78] + 2546 semaphore_timedwait_trap (in libsystem_kernel.dylib) + 8 [0x1972ceda8] 2546 Thread_1315737 2546 start_wqthread (in libsystem_pthread.dylib) + 8 [0x19730b0f0] 2546 _pthread_wqthread (in libsystem_pthread.dylib) + 364 [0x19730c424] 2546 __workq_kernreturn (in libsystem_kernel.dylib) + 8 [0x1972d0a64] If I copy the program and run two different programs, the second program won't hang. This issue doesn't happen on MacOS early version. Can anybody help?
0
0
352
Jul ’24
updateContentFilter no work when swich capture window
Start carpture window A, it works fine. After some time, I want to switch to capture window B without stop the scstream. My code is like this : content_filter = [[SCContentFilter alloc] initWithDesktopIndependentWindow:winID_B]; [scStream_ updateContentFilter:content_filter completionHandler:^(NSError *_Nullable error) { if (error) { "some error log" } else { "update OK" } }]; I cant get the "update OK" log, but sometimes it still output frame of the old window A. Is there any special operation need do before call updateContentFilter ? Or, is my usage is not correct ?
0
0
290
Jun ’24
Filtering background process "windows" during capture
I am trying to take a screenshot of each running window on the screen independently (even if not in the foreground). I am using SCScreenshotManager's captureImageWithFilter and SCContentFilter's initWithDesktopIndependentWindow. When I do this for every SCWindow in SCShareableContent's windows, I do get all the actual windows, but also get many mostly blank windows or windows that no ordinary user would consider to be an actual window. These include dozens of windows with no title, some with "Focus Proxy" title, some for menu bar icons, the wallpaper, the desktop icons, etc. I've implemented a naive solution that filters all windows that have no title, owningApplications with no title or bundle ID in a hardcoded blocklist (e.g. "com.apple.controlcenter", "com.apple.wallpaper.agent") and that helps, but is far from robust and is naturally fragile. Is there a recommended way to distinguish actual application windows or an overall better approach here? Thanks!
1
0
484
May ’24
Screenshot with ScreenCaptureKit much larger than with Command-Shift-3
I am capturing a screenshot with SCScreenshotManager's captureImageWithFilter. The resulting PNG has the same resolution as the PNG taken from Command-Shift-3 (4112x2658) but is 10x larger (14.4MB vs 1.35MB). My SCStreamConfiguration uses the SCDisplay's width and height and sets the color space to kCGColorSpaceSRGB. I currently save to file by initializing a NSBitmapImageRep using initWithCGImage, then representing as PNG with representationUsingType NSBitmapImageFileTypePNG, then writeToFile:atomically. Is there some configuration or compression I can use to bring down the PNG size to be more closely in-line with a Command-Shift-3 screenshot. Thanks!
1
0
583
May ’24
IOSurface objects aren't released in ScreenCaptureKit
I use ScreenCaptureKit, CoreVideo, CoreImage, CoreMedia frameworks to capture screenshots on macOS 14.0 and higher. Example of creating CGImageRef: CVImageBufferRef cvImageBufferRef = ..; CIImage* temporaryImage = [CIImage imageWithCVPixelBuffer:cvImageBufferRef]; CIContext* temporaryContext = [CIContext context]; CGImageRef imageRef = [temporaryContext createCGImage:temporaryImage fromRect:CGRectMake(0, 0, CVPixelBufferGetWidth(cvImageBufferRef), CVPixelBufferGetHeight(cvImageBufferRef))]; I have the next results of profiling with XCode Instruments Memory Leaks & Allocations: there is constantly increasing memory usage, but no memory leaks are detected, and there are many calls to create IOSurface objects, that have been never released. The most part of memory - All Anonymous VM - VM: IOSurface. The heaviest stack trace: [RPIOSurfaceObject initWithCoder:] [IOSurface initWithMachPort:] IOSurfaceClientLookupFromMachPort I don't have any of IOSurface objects created by myself. There are low-level calls to it. In Allocation List I can see many allocations of IOSurface objects, but there are no info about releasing it. Due to this info, how can I release them to avoid permanent increasing memory consumption?
2
0
727
May ’24
Empty space with Screen Capture kit full screen window recording
I am using Screen Capture Kit to capture the windows and record it. But from macOS Sanoma onwards I see a wired behaviour when I try to capture the window which is in Full screen mode. The CMSampleBuffer returned by Screen capture kit has empty space at the top of the full screen window content. The ContentRect attachment in CMSampleBuffer includes this empty space. So there is no way to know what is the actual window content in the CMSampleBuffer. In the CaptureCample sample code provided by Apple it does not enumerate the Full screen windows. I made a change in that to enumerate full screen windows. The issue is reproduced in that also. Attaching the Image of showing the empty space. Has anybody encountered this issue?
4
0
584
Apr ’24