I want to apply a SCNTechnique pipeline to the camera feed. To achieve this, I want to bring the camera input into the SceneKit world. The perfects API seems to be: let captureDevice = … scnScene.background.contents = captureDevice This is demonstrated in "SceneKit: What's New" (WWDC17) (at 44m19s) and is mentioned in the documentation of SCNMaterialProperty's contents. Instead of showing camera feed, it crashes with these messages: *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[AVCaptureVideoDataOutput setVideoSettings:] Unsupported pixel format type - use -availableVideoCVPixelFormatTypes' *** First throw call stack: (0x18993c7cc <REDACTED> 0x211e18488) libc++abi: terminating due to uncaught exception of type NSException Please advise. STEPS TO REPRODUCE Create a new Xcode project, starting from the SceneKit game template. Add Info.plist entry for NSCameraUsageDescription. Add a capture device property to GameViewController: class GameViewController: UIViewController { let captureDevice = AVCaptureDevice.default(for: .video) Set the background contents: scene.background.contents = captureDevice Run the app on device. PLATFORM AND VERSION iOS Development environment: Xcode 16.1, macOS 15.0.1. Run-time configuration: iOS 18.1

I'm trying to apply a Core Image filter to an UIImage. For that I want to get the CIImage format of the UIImage. I'm trying to obtain the CIImage of the UIImage as shown below. if let inputImage = self.orginalImageView.image{ if let ciImage = CIImage(image: inputImage){ print(ciImage) print(self.orginalImageView.image?.ciImage) } } } This method works. But one thing I noticed is that there is already a ciImage property and it inside UIImage and it is always nil. According to documentation ciImage The underlying Core Image data. var ciImage: CIImage? { get } Discussion If the UIImage object was initialized using a CGImage, the value of the property is nil. Does accessing image property of UIImage comes from CGImage so that the ciImage porperty is nil?

I am experiencing a bug when using a AVCapturePhotoBracketSettings object to capture a bracketed photo sequence on iPhone 16 Pro. Specifically, when I pass in an array of exposure values: [-x, 0, +x], where x >= 3. Specifically, the high exposure photo capture returns a black image. STEPS TO REPRODUCE Run the sample app I have provided on an iPhone 16 Pro Notice that bracketed images captured where the eV is set to [-3,0,+3], [-4,0,+4], or [-5,0,+5] return a black image for the high exposure photo. Notice that on other iOS devices (like iPhone 13 Pro), the high exposure photo is returned as high brightness as expected. I have also added two folders in the sample project that show screenshots of the bug: iPhone13Pro & iPhone16Pro Sample Project: https://www.icloud.com/iclouddrive/090O_68Z0Nh2UOxmPRwu56Tmw#Focused16ProBracketedCaptureBug

While customizing ImagePicker and using it, we find out that the metadata is not reflected normally and report it. The situation is as follows. The time or time zone of an image is changed in the Photos app. Changing the time zone of an image with an actual capture date of 2024:11:08 08:27:44 → 2024:11:07 17:27:44 Image data is extracted from a PHAsset using PHImageManager. The metadata is obtained from this image data. The time zone information exposed in the Exif tag information does not reflect the time or time zone changed in the Photos app. let asset: PHAsset = ... .... let options = PHImageRequestOptions() options.isSynchronous = true options.version = .current options.deliveryMode = .highQualityFormat options.resizeMode = .none options.normalizedCropRect = .zero options.isNetworkAccessAllowed = true options.progressHandler = { progress, error, _, _ in } PHImageManager.default().requestImageDataAndOrientation(for: asset, options: options) { imageData, uti, orientation, info in let cgImageSource = CGImageSourceCreateWithData(imageData! as CFData, nil) let properties = CGImageSourceCopyPropertiesAtIndex(cgImageSource!, 0, nil) as? Dictionary&lt;String, Any&gt; let exif = properties!["{Exif}"] let dictionary = exif as? Dictionary&lt;String, Any&gt; } Metadata Check In this case, it is reflected in the creationDate of PHAsset, so it can be somewhat compensated by forcibly replacing the metadata. However, because PHAsset does not include time zone information, when changing the time zone as well, it's impossible to calculate the correct time according to the time zone. PHPicker This issue is resolved when using the PHPickerResult provided by PHPicker. extension PhotosPickerViewController: PHPickerViewControllerDelegate { public func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) { ..... for result in results { let identifier = UTType.image.identifier if result.itemProvider.hasItemConformingToTypeIdentifier(identifier) { result.itemProvider.loadDataRepresentation(forTypeIdentifier: identifier) { data, error in guard let data = data, let cgImageSource = CGImageSourceCreateWithData(data as CFData, nil), let properties = CGImageSourceCopyPropertiesAtIndex(cgImageSource, 0, nil) as? Dictionary&lt;String, Any&gt;, let exif = properties["{Exif}"], let dictionary = exif as? Dictionary&lt;String, Any&gt; else { return } } } } } } Metadata Check Question I wonder why this happens, and if this is normal behavior. Instead of the System Picker that Apple provides as a base, I wonder if there is any way I can supplement it in that situation if I use a customizer.

iOS (Official) Photos app can display some EXIF-related metadata (e.g. camera and lens info, ISO, shutter speed, F-number) even when photos are offloaded to iCloud and the device is not connected to internet (e.g. airplane mode). However, with the Photos.framework, we need to download photos to retrive those metadata (which means it will not work with airplane mode). I tried the following methods, but none of those worked when photos were offloaded to iCloud and the device was in airplane mode: Requesting data with PHImageManager.default().requestImageDataAndOrientation Result: It does not return Data if the photo is not stored locally on the device, even with options.deliveryMode = .fastFormat Converting PHAsset#localIdentifier to an AssetsLibrary.framework URL (assets-library://asset/...) (I am aware that AssetsLibrary.framework is deprecated, but this was just a test.) Result: If PHImageManager does not returns Data, ALAsset#defaultRepresentation().metadata() returns an empty NSDictionary

To get calibration data during video recording, I use AVCaptureDepthDataOutput together with AVCaptureVideoDataOutput, synchronized via AVCaptureDataOutputSynchronizer (using the dataOutputSynchronizer method of CameraController in the example). The issue is that AVCaptureDepthDataOutput can only be used with .builtInLiDARDepthCamera, meaning it isn’t available for devices without LiDAR. Is it possible to obtain calibration data during video recording on devices without LiDAR, such as with .builtInWideAngleCamera? Can lensDistortionLookupTable and lensDistortionCenter be used to undistort the sampleBuffer I receive from AVCaptureVideoDataOutput?

My Camera app is repeatedly opening even though I am not taking any action to open it when I use iPhone. Today during a FaceTime call, the Camera app opened while the phone was unlocked without me touching anything. It didn’t end the FaceTime call, but just put the video on pause for the person I was speaking with. I force closed the Camera app, then it happened again a few minutes later. This has happened while using Google Maps and other apps as well, while the phone is unlocked. This is also happening while the phone is locked, just sitting on a table. All the sudden I look over and the screen is active showing the camera view. Today this has happened at least 20 times. I need to know how to stop it. I am on iOS 18.1 and enrolled in iOS 18 Public Beta. There are no pending software updates.

Hello, I have a problem reading a 2D data matrix type code with a camera. In the application, I use AVFoundation to operate the camera and work with 2D codes, and in the vast majority there is no problem with loading. Nothing special. I originally thought it might be a problem in my code, but I got the same result when I tried with the Camera app integrated in IOS. It can be seen that only the LiveText API for text recognition worked. But I am attaching the code with which the camera has a problem, even though the code looks perfectly fine at first glance. A classic handheld 2D code reader will read the code just fine. Can someone please explain to me why the camera, which normally reads these codes at the speed of light, sometimes has a problem with the codes? Thank you [Personal Information Edited by Moderator]

I created a locked camera capture extension as explained in Apple's documentation. I'm trying to explore the possibilities of using a bluetooth peripheral from that extension - anybody knows if this is possible? The CBCentralManagerDelegate reports .unsupported in func centralManagerDidUpdateState, even if I have provided all the permissions in Info.plist.

Hello, Faced with a really perplexing issue. Primary problem is that sometimes I get depth and video data as expected, but at other times I don't. And sometimes I'll get both data outputs for a 4-5 frames and then it'll just stop. The source code I implemented is a modified version of the sample code provided by Apple, and interestingly enough I can't re-create this issue with the Apple sample app. So wondering what I could be doing wrong? Here's the code for setting up the capture input. preferredDepthResolution is 1280 in my case. I'm running this on an iPad Pro (6th gen). iOS version 17.0.3 (21A360). Encounter this issue on iPhone 13 Pro as well. iOS version is 17.0 (21A329) private func setupLiDARCaptureInput() throws { // Look up the LiDAR camera. guard let device = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .video, position: .back) else { throw ConfigurationError.lidarDeviceUnavailable } guard let format = (device.formats.last { format in format.formatDescription.dimensions.width == preferredWidthResolution && format.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange && format.videoSupportedFrameRateRanges.first(where: {$0.maxFrameRate >= 60}) != nil && !format.isVideoBinned && !format.supportedDepthDataFormats.isEmpty }) else { throw ConfigurationError.requiredFormatUnavailable } guard let depthFormat = (format.supportedDepthDataFormats.last { depthFormat in depthFormat.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_DepthFloat16 }) else { throw ConfigurationError.requiredFormatUnavailable } // Begin the device configuration. try device.lockForConfiguration() // Configure the device and depth formats. device.activeFormat = format device.activeDepthDataFormat = depthFormat let desc = format.formatDescription dimensions = CMVideoFormatDescriptionGetDimensions(desc) let duration = CMTime(value:1, timescale:CMTimeScale(60)) device.activeVideoMinFrameDuration = duration device.activeVideoMaxFrameDuration = duration // Finish the device configuration. device.unlockForConfiguration() self.device = device print("Selected video format: \(device.activeFormat)") print("Selected depth format: \(String(describing: device.activeDepthDataFormat))") // Add a device input to the capture session. let deviceInput = try AVCaptureDeviceInput(device: device) captureSession.addInput(deviceInput) guard let audioDevice = AVCaptureDevice.default(for: .audio) else { return } // Configure audio input - always configure audio even if isAudioEnabled is false audioDeviceInput = try! AVCaptureDeviceInput(device: audioDevice) captureSession.addInput(audioDeviceInput) deviceSystemPressureStateObservation = device.observe( \.systemPressureState, options: .new ) { _, change in guard let systemPressureState = change.newValue else { return } print("system pressure \(systemPressureState.levelAsString()) due to \(systemPressureState.factors)") } } Here's how I'm setting up the output: private func setupLiDARCaptureOutputs() { // Create an object to output video sample buffers. videoDataOutput = AVCaptureVideoDataOutput() captureSession.addOutput(videoDataOutput) // Create an object to output depth data. depthDataOutput = AVCaptureDepthDataOutput() depthDataOutput.isFilteringEnabled = false captureSession.addOutput(depthDataOutput) audioDeviceOutput = AVCaptureAudioDataOutput() audioDeviceOutput.setSampleBufferDelegate(self, queue: videoQueue) captureSession.addOutput(audioDeviceOutput) // Create an object to synchronize the delivery of depth and video data. outputVideoSync = AVCaptureDataOutputSynchronizer(dataOutputs: [depthDataOutput, videoDataOutput]) outputVideoSync.setDelegate(self, queue: videoQueue) // Enable camera intrinsics matrix delivery. guard let outputConnection = videoDataOutput.connection(with: .video) else { return } if outputConnection.isCameraIntrinsicMatrixDeliverySupported { outputConnection.isCameraIntrinsicMatrixDeliveryEnabled = true } } The top part of my delegate implementation is as follows: func dataOutputSynchronizer( _ synchronizer: AVCaptureDataOutputSynchronizer, didOutput synchronizedDataCollection: AVCaptureSynchronizedDataCollection ) { // Retrieve the synchronized depth and sample buffer container objects. guard let syncedDepthData = synchronizedDataCollection.synchronizedData(for: depthDataOutput) as? AVCaptureSynchronizedDepthData, let syncedVideoData = synchronizedDataCollection.synchronizedData(for: videoDataOutput) as? AVCaptureSynchronizedSampleBufferData else { if synchronizedDataCollection.synchronizedData(for: depthDataOutput) == nil { print("no depth data at time \(mach_absolute_time())") } if synchronizedDataCollection.synchronizedData(for: videoDataOutput) == nil { print("no video data at time \(mach_absolute_time())") } return } print("received depth data \(mach_absolute_time())") } As you can see, I'm console logging whenever depth data is not received. Note because I'm driving the video frames at 60 fps, its expected that I'll only receive depth data for every alternate video frame. Console output is posted as a follow up comment (because of the character limit). I edited some lines out for brevity. You'll see it started streaming correctly but after a while it stopped received both video and depth outputs (in some other runs, it works perfectly and in some other runs I receive no depth data whatsoever). One thing to note, I sometimes run quicktime mirroring to see the device screen to see what the app is displaying (so not sure if that's causing any interference - that said I don't see any system pressure changes either). Any help is most appreciated! Thanks.

I extracted the gain map info from an image using let url = Bundle.main.url(forResource: "IMG_1181", withExtension: "HEIC") let source = CGImageSourceCreateWithURL(url! as CFURL, nil) let portraitData = CGImageSourceCopyAuxiliaryDataInfoAtIndex(source!, 0, kCGImageAuxiliaryDataTypeHDRGainMap) as! [AnyHashable : Any] let metaData = portraitData[kCGImageAuxiliaryDataInfoMetadata] as! CGImageMetadata Then I printed all the metadata tags func printMetadataProperties(from metadata: CGImageMetadata) { guard let tags = CGImageMetadataCopyTags(metadata) as? [CGImageMetadataTag] else { return } for tag in tags { if let prefix = CGImageMetadataTagCopyPrefix(tag) as String?, let namespace = CGImageMetadataTagCopyNamespace(tag) as String?, let key = CGImageMetadataTagCopyName(tag) as String?, let value = CGImageMetadataTagCopyValue(tag){ print("Namespace: \(namespace), Key: \(key), Prefix: \(prefix), value: \(value)") } else { } } } //Namespace: http://ns.apple.com/ImageIO/1.0/, Key: hasXMP, Prefix: iio, value: True //Namespace: http://ns.apple.com/HDRGainMap/1.0/, Key: HDRGainMapVersion, Prefix: HDRGainMap, value: 131072 //Namespace: http://ns.apple.com/HDRGainMap/1.0/, Key: HDRGainMapHeadroom, Prefix: HDRGainMap, value: 3.586325 I want to create a new CGImageMetadata and tags. But when it comes to the HDR tags. It always fails to add to metadata. let tag = CGImageMetadataTagCreate( "http://ns.apple.com/HDRGainMap/1.0/" as CFString, "HDRGainMap" as CFString, "HDRGainMapHeadroom" as CFString, .default, 3.56 as CFNumber ) let path = "\(HDRGainMap):\(HDRGainMapHeadroom)" as CFString let success = CGImageMetadataSetTagWithPath(metadata, nil, path, tag)// always false The hasXMP works fine. Is HDR a private dict for Apple?

I've requested the authentication in my main app. PHPhotoLibrary.requestAuthorization(for: .readWrite) { status in } Add the privacy description in both the main app and the extension. But No matter the device is locked or unlocked. When I call let fetchResult = PHAsset.fetchAssets(with: .image, options: nil) let count = fetchResult.count the count is always zero, even after a new photo is saved to the album in the same session.

Hello, I’ve encountered an issue where a green line appears only in the thumbnail view of an image in the iPhone photo gallery. The green line is not present in the actual image itself. Here are some additional details: • The issue occurs only when saving the image as a JPEG. When saved as PNG, the green line does not appear. • The green line also shows up when using the PHAsset method requestImage(for:targetSize:). Depending on the targetSize, the resulting image may contain the green line. • Interestingly, this issue does not appear on iPhone Xs Max running iOS 15.2.1. However, the green line does appear on iPhone 15 Pro Max running iOS 18.0.1 when viewing the same image. I have attached the problematic image for your reference. Following images are the screen captures that shows the issue occurring on my iPhone. iPhone 15 pro max (iOS 18.0.1) iPhone Xs Max (iOS 15.2.1 ) Could this be related to a display or gallery app issue on iOS? Any advice or solutions would be greatly appreciated. Thank you!

We are currently in the process of migrating our application from using ALAssetsLibrary to PHPhotoLibrary to ensure compatibility with the latest versions of iOS. However, we have noticed a discrepancy in the file sizes of images obtained using PHPhotoLibrary compared to those obtained using ALAssetsLibrary. Specifically, we would like to understand the following points: 1.Reason for File Size Differences: What are the reasons for the difference in file sizes between images obtained using ALAssetsLibrary and those obtained using PHPhotoLibrary? Could you provide detailed information on the settings and options in PHPhotoLibrary that affect the size and quality of the images? 2.Optimal Settings: What are the optimal settings in PHPhotoLibrary to obtain images with the same quality and file size as those obtained using ALAssetsLibrary? If possible, could you provide code examples or recommended option settings?

See Configuration Details at the end of this message. Despite numerous attempts, I have been unable to determine the correct syntax to fetch photo albums from my iPad Pro 13.0 using Xcode and Swift. All the photo album were synced to the iPad Pro 13-inch using the latest versions of Apple iTunes for Windows from an external Western Digital G-Drive hard drive (No iCloud). All synced albums appear under "From My Mac" on the iPad. I only want to access each album's photo and video count. See sample code snippet below. I have tried multiple subtype options and album types without success. Zero albums are always returned despite having around 3900 albums in the iPad Pro 13.0 photo library. Authorization to the photo library does not appear to be the problem. PHPhotoLibrary.requestAuthorization { status in if status == .authorized { let result = PHAssetCollection.fetchAssetCollections(with: .album, subtype: .any, options: nil) if result.count == 0 { print("No albums found.") return } } } Any help or suggestions would greatly be appreciated. ApplePhoto Configuration Details iPad Pro 13-inch (M4) (iPad16,6) iPadOS = 17.7 iCloud = Turned Off iPad Pro Photo Library Albums = 3900 iPad Pro Photo Library Photos = 118000 iPad Pro Photo Library Videos = 4800 MacOS = Sonoma 14.6.1 XCode Version = 16.0 Swift Version = 5.0 (Xcode Default) Microsoft Windows 10 Pro Version = 22H2 Apple iTunes for Windows = 12.13.4.4

In the WWDC 24 session "Use HDR for dynamic image experiences in your app" it's noted this is how you save edits for Adaptive HDR: SDR + HDR: writeHEIFRepresentation(of: sdrImage, to: url, colorSpace: p3Space, options: [.hdrImage: hdrImage]) SDR + Gain: writeHEIFRepresentation(of: sdrImage, to: url, colorSpace: p3Space, options: [.hdrGainMapImage: gainImage]) This won't compile because the format argument is missing. What format should be used? In the WWDC 23 session "Support HDR images in your app" RGBAf, RGBAh, and RGBA16, and RGB10 were mentioned but I'm not sure which one to use. If relevant, I'm editing photos from the user's photo library, so the image was probably taken on iPhone but perhaps not. Thanks!

Set 3 controls to the AVCaptureSession and remove them all. The number of controls in the session is indeed 0, but the camera controls button still shows the previous 3 controls. If it is only 3->2 or 3->1, it can be modified normally, 3->0 is not OK, 0->3 is OK. f (self.captureControl.zoom) { if (self.zoomScaleControl) { self.zoomScaleControl.enabled = false; [_session removeControl:self.zoomScaleControl]; } AVCaptureSlider *zoomSlider = [self.captureControl.zoom fetchCaptureSlider]; [zoomSlider setActionQueue:dispatch_get_main_queue() action:^(float zoomFactor) { @strongify(self); if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:tryChangeZoomScale:)]) { [self.dataOutputDelegate videoCaptureSession:self tryChangeZoomScale:zoomFactor]; } }]; self.zoomScaleControl = zoomSlider; } else { if (self.zoomScaleControl) { self.zoomScaleControl.enabled = false; [_session removeControl:self.zoomScaleControl]; } self.zoomScaleControl = nil; } if (self.captureControl.exposure) { if (self.exposureBiasControl) { self.exposureBiasControl.enabled = false; [_session removeControl:self.exposureBiasControl]; } AVCaptureSlider *exposureSlider = [self.captureControl.exposure fetchCaptureSlider]; [exposureSlider setActionQueue:dispatch_get_main_queue() action:^(float bias) { @strongify(self); if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:tryChangeExposureBias:)]) { [self.dataOutputDelegate videoCaptureSession:self tryChangeExposureBias:bias]; } }]; self.exposureBiasControl = exposureSlider; } else { if (self.exposureBiasControl) { self.exposureBiasControl.enabled = false; [_session removeControl:self.exposureBiasControl]; } self.exposureBiasControl = nil; } if (self.captureControl.len) { if (self.lenControl) { self.lenControl.enabled = false; [_session removeControl:self.lenControl]; } ORLenCaptureControlCustomModel *len = self.captureControl.len; AVCaptureIndexPicker *picker = [len fetchCaptureSlider]; [picker setActionQueue:dispatch_get_main_queue() action:^(NSInteger selectedIndex) { @strongify(self); if ([self.dataOutputDelegate respondsToSelector:@selector(videoCaptureSession:didChangeLenIndex:datas:)]) { [self.dataOutputDelegate videoCaptureSession:self didChangeLenIndex:selectedIndex datas:self.captureControl.len.indexDatas]; } }]; self.lenControl = picker; } else { if (self.lenControl) { self.lenControl.enabled = false; [_session removeControl:self.lenControl]; } self.lenControl = nil; } if ([_session canAddControl:self.zoomScaleControl]) { [_session addControl:self.zoomScaleControl]; } else { self.zoomScaleControl = nil; } if ([_session canAddControl:self.lenControl]) { [_session addControl:self.lenControl]; } else { self.lenControl = nil; } if ([_session canAddControl:self.exposureBiasControl]) { [_session addControl:self.exposureBiasControl]; } else { self.exposureBiasControl = nil; } if (_session.controlsDelegate == nil) { [_session setControlsDelegate:self queue:GetCaptureControlQueue()]; }

I am using PHImageRequestOptions and PHImageManager to load images to my app. I use version.original and resizeMode.none, version.original and resizeMode.extract. Both used to work well but since iOS18 version.original and resizeMode.extract doesn't work anymore. The images are loaded but the they are not shown. (Only the frames?) Anyone knows why? Thank you for reading.

Hi, After installing iPadOS18 Beta3 on my iPad 7th gen, the default camera app no ​​longer detects QR codes. I tried updating to Beta7, but the issue remained. Also, third-party apps that use AVCaptureMetadataOutput in AVFoundation Framework to detect QR codes also no longer work. You can reproduce the issue by running default camera app or the AVFoundation sample code from the Apple developer site on iPad 7th gen (iPadOS18Beta installed). https://developer.apple.com/documentation/avfoundation/capture_setup/avcambarcode_detecting_barcodes_and_faces Has anyone else experienced this issue? I would like to know if this issue occurs on other iPad models as well. This is similar to the following issue that previously occurred with iPadOS 17.4. https://support.apple.com/en-lamr/118614 https://developer.apple.com/forums/thread/748092

I'm using PhotoKit in macOS to add photos to the user's library. Experimenting with Shared Photo Library, it seems that these new photos always end up in the Personal Library, not the Shared Library. I'd like to get them into the Shared Photo Library somehow. Is this possible? Things I've considered: A variation/option for PHAssetChangeRequest.creationRequestForAsset: doesn't seem to exist A property of PHAsset: can't find anything A special PHAssetCollection that I could add to: again, doesn't seem to exist