AVDevice is ignoring 60fps

Question

maggus83 OP

Created Oct ’24

Replies 1

Boosts 1

Participants 1

Hello,

I try to get the Video from an HDMI USB capture card and show it in a PreviewLayer with 60fps. The device I am using (ShadowCast 2) is supporting 1080p with 60fps in "yuvs" and "420v".

This is my code with stripped away uninteresting stuff and removed error handling to build the previewLayer.

I am using the AVFrameRateRange because the capture device is not directly supporting 60.00 but <AVFrameRateRange: 0x600000875680 60.00 - 60.00 (1000000 / 60000240 - 1000000 / 60000240)> fps.

@Observable
final class AVFoundationService: AVService {
    // Live View
    private let session: AVCaptureSession = .init()
    
    var previewLayer: AVCaptureVideoPreviewLayer {
        let layer = AVCaptureVideoPreviewLayer(session: session)
        layer.videoGravity = .resizeAspect
        return layer
    }

    var activeVideoDevice: AVCaptureDevice? {
        // TODO: implement correct logic
        if let device = videoDevices.first(where: { $0.localizedName.contains("Shadow") }) {
            return device
        }
        
        return AVCaptureDevice.default(for: .video)
    }

    func setupStreamDemo(completion: @escaping (Error?) -> Void) {
        session.beginConfiguration()
               
        if let device = activeVideoDevice {
            do {
                let input = try AVCaptureDeviceInput(device: device)
                if session.canAddInput(input) {
                    session.addInput(input)
                } else {
                    print("explode")
                }

                for format in device.formats {
                    let dimensions = CMVideoFormatDescriptionGetDimensions(format.formatDescription)
                                        
                    if dimensions.width == 1920 && dimensions.height == 1080 && format.formatDescription.mediaSubType.description == "'yuvs'" {

                        let foundFPS = format.videoSupportedFrameRateRanges.first {
                            Int($0.minFrameRate) == 60 && Int($0.minFrameRate) == 60
                        }
                        
                        try device.lockForConfiguration()
                        
                        device.activeFormat = format
                        device.activeVideoMinFrameDuration = foundFPS!.minFrameDuration
                        device.activeVideoMaxFrameDuration = foundFPS!.minFrameDuration
                        
                        device.unlockForConfiguration()
                    }
                }
            } catch {
                return completion(error)
            }
        }
        
        session.commitConfiguration()
        session.startRunning()
        completion(nil)
    }
}

I am using the following code in SwiftUI to show the AVCaptureVideoPreviewLayer.

struct VideoPreviewView: NSViewRepresentable {
    private let previewLayer: AVCaptureVideoPreviewLayer

    func makeNSView(context: Context) -> NSView {
        let view = NSView()
        view.layer = self.previewLayer
        view.layer?.frame = view.bounds
        return view
    }

    func updateNSView(_ nsView: NSView, context: Context) {
        if let layer = nsView.layer as? AVCaptureVideoPreviewLayer {
            layer.session = self.previewLayer.session
        }
    }
}

When I now run my app, it will ignore whatever I set on device.activeVideoMinFrameDuration and/or device.activeVideoMaxFrameDuration. If I set it to 10 fps - it's running with 30, if I set 60 it is running with 30.

If I start in parallel to my app QuickTime and start a "Recording" from my USB Capture Card, it will switch to 60fps mode.

I am on Mac Sequoia 15.0 with Xcode 16.0.

What I am doing wrong?

Answered by maggus83 in 814371022

I found the reason/solution for this behavior.

You need to add an AVCaptureVideoDataOutput to your session.
On Mac you need to set the FPS on the AVCaptureVideoDataOutput too

#if os(OSX)
if let connection = videoDataOutput.connection(with: .video) {
    connection.videoMinFrameDuration = CMTimeMake(value: 1, timescale: Int32(fps))
    connection.videoMaxFrameDuration = CMTimeMake(value: 1, timescale: Int32(fps))
}
#endif

You need to change the AVCaptureSession.Preset. It defaults to high which I guess limits in an undocumented way the fps to 30. If you change it to hd4K3840x2160, hd1920x1080, ... it will run with the desired FPS.

I am really upset with the documentation from Apple. As a beginner it is really, really hard to debug problems like this. There is nowhere a hint, that the default profile high is limiting the "output" to 30 fps.

Boost

Answer 1

maggus83 OP

5d

Accepted Answer

I found the reason/solution for this behavior.

You need to add an AVCaptureVideoDataOutput to your session.
On Mac you need to set the FPS on the AVCaptureVideoDataOutput too

#if os(OSX)
if let connection = videoDataOutput.connection(with: .video) {
    connection.videoMinFrameDuration = CMTimeMake(value: 1, timescale: Int32(fps))
    connection.videoMaxFrameDuration = CMTimeMake(value: 1, timescale: Int32(fps))
}
#endif

You need to change the AVCaptureSession.Preset. It defaults to high which I guess limits in an undocumented way the fps to 30. If you change it to hd4K3840x2160, hd1920x1080, ... it will run with the desired FPS.

I am really upset with the documentation from Apple. As a beginner it is really, really hard to debug problems like this. There is nowhere a hint, that the default profile high is limiting the "output" to 30 fps.

0