Empty space with Screen Capture kit full screen window recording

Question

tejusadiga2004 OP

Created Apr ’24

Replies 4

Boosts 0

Participants 2

I am using Screen Capture Kit to capture the windows and record it. But from macOS Sanoma onwards I see a wired behaviour when I try to capture the window which is in Full screen mode. The CMSampleBuffer returned by Screen capture kit has empty space at the top of the full screen window content. The ContentRect attachment in CMSampleBuffer includes this empty space. So there is no way to know what is the actual window content in the CMSampleBuffer.

In the CaptureCample sample code provided by Apple it does not enumerate the Full screen windows. I made a change in that to enumerate full screen windows. The issue is reproduced in that also.

Attaching the Image of showing the empty space. Has anybody encountered this issue? Screenshot 2024-04-24 at 6.01.40 PM.png

Boost

Answer 1

Pawel-Domas-dd OP

Apr ’24

We're seeing a very similar problem when capturing Chrome's full screen windows, it's reproducible usually when "Always Show Toolbar in Full Screen" is enabled:

Screenshot 2024-04-30 at 11.00.48 AM.png

There's transparent space in the top/right part of the window with the "sharing" icon in the top left. The main window's content is shifted to the bottom left.

0

Answer 2

tejusadiga2004 OP

Apr ’24

We are seeing this issue with all applications. Issue is that the content rect given by Screen Capture Kit in CMSampleBufer includes the empty space at the top. This makes it very difficult to select only the area where window content is present.

0

Answer 3

Pawel-Domas-dd OP

May ’24

Hey,

We've found a potential workaround which seems to fix the fullscreen Chrome case for us.

Try using setSourceRect on SCStreamConfiguration by passing the CGRect of the window you're trying to share. We obtain these from the SCWindow / frame property taken from the results of SCShareableContent getShareableContentWithCompletionHandler.

Even though the docs of setSourceRect say:

The system doesn’t reference this value when you capture a single window because it captures the full bounds of the window.

It seems like this makes the underlying logic reconsider something and excludes the padding area.

0

Answer 4

tejusadiga2004 OP

May ’24

It seems when window is in full screen the CMSampleBuffer returned has different video dimensions in CMVideoFormatDescription and the Content Rect.

CMSampleBuffer formatDescription = <CMVideoFormatDescription 0x6000027066d0 [0x7ff8558ff750]> { mediaType:'vide' mediaSubType:'420v' mediaSpecific: { codecType: '420v' dimensions: 3584 x 2240 } } sampleAttachmentsArray[0] = { sample 0: SCStreamUpdateFrameDisplayResolution = 2 SCStreamUpdateFrameContentRect = { Height = "1120.000020503998"; Width = "1724.261199951172"; X = 0; Y = 0; } }

It seems video dimension gives the correct rect which excludes the empty space. So having ContentRectHeight * DisplayRes - (ContentRectHeight * DisplayRes - VideoDimensionheight) from content rect will give the actual content rect.

0