I am looking to learn more about the new Capture Button controls for iPhone 16, and am working to adapt the AVCam Sample Code to support the Capture Button. While I believe I've followed the guidance in the Enhancing your app experience with the Camera Control documentation, I'm finding that while my AVCaptureControl items seem to be added to the capture session, the Capture Button does not ever do anything, nor are any of the delegate methods called. After I configure my capture session per the setupSession() method, I'm calling a method I added, func configureCameraControls(device:AVCaptureDevice): func configureCameraControls(device: AVCaptureDevice) { guard captureSession.supportsControls else { assertionFailure("App does not support camera control.") return } // Set the controls delegate captureSession.setControlsDelegate(controlsDelegate, queue: sessionQueue) // Begin configuring the capture session. captureSession.beginConfiguration() // Remove previously configured controls, if any. for control in captureSession.controls { captureSession.removeControl(control) } // Add a zoom control let systemZoomSlider = AVCaptureSystemZoomSlider(device: device) { zoomFactor in // TODO } // Create a control to adjust the device's exposure bias. let systemBiasSlider = AVCaptureSystemExposureBiasSlider(device: device) // Add a custom slider let focusSlider = AVCaptureSlider("Focus", symbolName: "scope", in: 0...1) focusSlider.setActionQueue(sessionQueue) { focusValue in // TODO } // Iterate over the passed in controls. for control in [systemZoomSlider, systemBiasSlider, focusSlider] { // Add the control to the capture session if possible. if captureSession.canAddControl(control) { captureSession.addControl(control) } else { print("Unable to add control \(control).") } } // Commit the capture session configuration. captureSession.commitConfiguration() } I define the controls delegate like so: final class CaptureControlsDelegate: NSObject, AVCaptureSessionControlsDelegate { func sessionControlsDidBecomeActive(_ session: AVCaptureSession) { } func sessionControlsWillEnterFullscreenAppearance(_ session: AVCaptureSession) { } func sessionControlsWillExitFullscreenAppearance(_ session: AVCaptureSession) { } func sessionControlsDidBecomeInactive(_ session: AVCaptureSession) { } } Which I instantiate earlier on in my app's lifecycle and make available to the CaptureService actor. I'm not sure if this snippet can provide enough detail to gather some help, but I can't quite fathom why the camera/capture pipeline works, but I'm not getting any functionality from the Capture Button nor is the AVCaptureSessionControlsDelegate ever having its methods called.

I was wondering of anyone had guidance on how to “livestream“ MV-HEVC content. More specifically, I have a left and right eye view for stereoscopic content (perhaps, for example, the views were taken from a stereoscopic video being passed through an AVPlayer). I know, based on sample code, that I can convert the stereoscopic video into a MV-HEVC file using AVAssetWriter. However, how would I take the stereoscopic video and encode it, in realtime, to a stream that could then leverage HLS Tools to deliver to clients? Is AVFoundation capable of this directly? Or is there an API within VideoToolbox that can help with this?

For those with existing projects that rely on a SceneDelegate and are primarily UIKit-based, I am curious how one might launch a window with a style of volumetric or an ImmersiveSpace. Is this possible without having a SwiftUI-based @Main entry point?

After taking a look at the Deliver Video Content for Spatial Experiences session, alongside the Destination Video sample code, I'm a bit unclear on how one might go about creating stereoscopic content that can be bundled up as a MV-HEVC file and played on Vision Pro. I see the ISO Base Media File Format and Apple HEVC Stereo Video format specifications, alongside new mvhevc1440x1440 output presets in AVFoundation, but I'm unclear what sort of camera equipment could be used to create stereoscopic content and how one might be able to create a MV-HEVC using a command-line tool that leverages AVFoundation/VideoToolbox, or something like Final Cut Pro. Is such guidance available on how to film and create this type of file? Thanks!

Summary: I am using the Vision framework, in conjunction with AVFoundation, to detect facial landmarks of each face in the camera feed (by way of the VNDetectFaceLandmarksRequest). From here, I am taking the found observations and unprojecting each point to a SceneKit View (SCNView), then using those points as the vertices to draw a custom geometry that is textured with a material over each found face. Effectively, I am working to recreate how an ARFaceTrackingConfiguration functions. In general, this task is functioning as expected, but only when my device is using the front camera in landscape right orientation. When I rotate my device, or switch to the rear camera, the unprojected points do not properly align with the found face as they do in landscape right/front camera. Problem: When testing this code, the mesh appears properly (that is, appears affixed to a user's face), but again, only when using the front camera in landscape right. While the code runs as expected (that is, generating the face mesh for each found face) in all orientations, the mesh is wildly misaligned in all other cases. My belief is this issue either stems from my converting the face's bounding box (using VNImageRectForNormalizedRect, which I am calculating using the width/height of my SCNView, not my pixel buffer, which is typically much larger), though all modifications I have tried result in the same issue. Outside of that, I also believe this could be an issue with my SCNCamera, as I am a bit unsure how the transform/projection matrix works and whether that would be needed here. Sample of Vision Request Setup: // Setup Vision request options var requestHandlerOptions: [VNImageOption: AnyObject] = [:] // Setup Camera Intrinsics let cameraIntrinsicData = CMGetAttachment(sampleBuffer, key: kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, attachmentModeOut: nil) if cameraIntrinsicData != nil { requestHandlerOptions[VNImageOption.cameraIntrinsics] = cameraIntrinsicData } // Set EXIF orientation let exifOrientation = self.exifOrientationForCurrentDeviceOrientation() // Setup vision request handler let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer, orientation: exifOrientation, options: requestHandlerOptions) // Setup the completion handler let completion: VNRequestCompletionHandler = {request, error in let observations = request.results as! [VNFaceObservation] // Draw faces DispatchQueue.main.async { drawFaceGeometry(observations: observations) } } // Setup the image request let request = VNDetectFaceLandmarksRequest(completionHandler: completion) // Handle the request do { try handler.perform([request]) } catch { print(error) } Sample of SCNView Setup: // Setup SCNView let scnView = SCNView() scnView.translatesAutoresizingMaskIntoConstraints = false self.view.addSubview(scnView) scnView.showsStatistics = true NSLayoutConstraint.activate([ scnView.leadingAnchor.constraint(equalTo: self.view.leadingAnchor), scnView.topAnchor.constraint(equalTo: self.view.topAnchor), scnView.bottomAnchor.constraint(equalTo: self.view.bottomAnchor), scnView.trailingAnchor.constraint(equalTo: self.view.trailingAnchor) ]) // Setup scene let scene = SCNScene() scnView.scene = scene // Setup camera let cameraNode = SCNNode() let camera = SCNCamera() cameraNode.camera = camera scnView.scene?.rootNode.addChildNode(cameraNode) cameraNode.position = SCNVector3(x: 0, y: 0, z: 16) // Setup light let ambientLightNode = SCNNode() ambientLightNode.light = SCNLight() ambientLightNode.light?.type = SCNLight.LightType.ambient ambientLightNode.light?.color = UIColor.darkGray scnView.scene?.rootNode.addChildNode(ambientLightNode) Sample of "face processing" func drawFaceGeometry(observations: [VNFaceObservation]) { // An array of face nodes, one SCNNode for each detected face var faceNode = [SCNNode]() // The origin point let projectedOrigin = sceneView.projectPoint(SCNVector3Zero) // Iterate through each found face for observation in observations { // Setup a SCNNode for the face let face = SCNNode() // Setup the found bounds let faceBounds = VNImageRectForNormalizedRect(observation.boundingBox, Int(self.scnView.bounds.width), Int(self.scnView.bounds.height)) // Verify we have landmarks if let landmarks = observation.landmarks { // Landmarks are relative to and normalized within face bounds let affineTransform = CGAffineTransform(translationX: faceBounds.origin.x, y: faceBounds.origin.y) .scaledBy(x: faceBounds.size.width, y: faceBounds.size.height) // Add all points as vertices var vertices = [SCNVector3]() // Verify we have points if let allPoints = landmarks.allPoints { // Iterate through each point for (index, point) in allPoints.normalizedPoints.enumerated() { // Apply the transform to convert each point to the face's bounding box range _ = index let normalizedPoint = point.applying(affineTransform) let projected = SCNVector3(normalizedPoint.x, normalizedPoint.y, CGFloat(projectedOrigin.z)) let unprojected = sceneView.unprojectPoint(projected) vertices.append(unprojected) } } // Setup Indices var indices = [UInt16]() // Add indices // ... Removed for brevity ... // Setup texture coordinates var coordinates = [CGPoint]() // Add texture coordinates // ... Removed for brevity ... // Setup texture image let imageWidth = 2048.0 let normalizedCoordinates = coordinates.map { coord -> CGPoint in let x = coord.x / CGFloat(imageWidth) let y = coord.y / CGFloat(imageWidth) let textureCoord = CGPoint(x: x, y: y) return textureCoord } // Setup sources let sources = SCNGeometrySource(vertices: vertices) let textureCoordinates = SCNGeometrySource(textureCoordinates: normalizedCoordinates) // Setup elements let elements = SCNGeometryElement(indices: indices, primitiveType: .triangles) // Setup Geometry let geometry = SCNGeometry(sources: [sources, textureCoordinates], elements: [elements]) geometry.firstMaterial?.diffuse.contents = textureImage // Setup node let customFace = SCNNode(geometry: geometry) sceneView.scene?.rootNode.addChildNode(customFace) // Append the face to the face nodes array faceNode.append(face) } // Iterate the face nodes and append to the scene for node in faceNode { sceneView.scene?.rootNode.addChildNode(node) } }

I currently have an app published in the App Store with an associated App Clip (the App Clip is configured in App Store Connect with a default App Clip Experience and an Advanced App Clip Experience). All debugging (whether from App Store Connect, or from Settings -> Developer -> App Clips Testing -> Diagnostics), confirms everything is configured properly (and more-so, the entire App Clip invocation experience is working great from Safari, QR code, and App Clip Code). My issues is when sharing a link via Messages. For example, let's say I'm sharing a URL of: https://myapp.com/events?eventID=123&userID=456. If I open this URL in Safari (without Private Browsing enabled), I see the banner to launch my App Clip. If I opt to launch the App Clip from the banner, it launches with the expected invocation URL, as noted above. If I share this website via Messages to someone else on an iPhone or iPad (running iOS 14+), the expected App Clip preview appears, and the receiver has an action button to launch my App Clip. However, when they launch the App Clip, all of the path/query parameters are missing (that is, the invocation URL appears as https://myapp.com). I'm wondering if this is intended behavior, or if the path I'm following isn't supported. Ideally, I would like my users to be able to visit the full URL, with query parameters, and if they opt to share that URL via Messages, the App Clip launches with the query parameters still in place to the receiver of the message. Thanks!

I am attempting to set up a Text property that shows a "timer" based countdown. My code is like so; VStack { 	 Text(Date().addingTimeInterval(600), style: .relative) } When I preview this code in a traditional SwiftUI view, the code appears as expected; in the middle of the canvas (as there are no vertical or horizontal spacers). Conversely, when I attempt to use the same code within a Widget, I find that the text is pushed all the way to the left side of the canvas, with no particular reason. Due to this, I have no way of centering the text. My only success in centering the text has been to embed in a HStack with multiple spacers; HStack { 		Spacer() 		Spacer() 		Spacer() 		Spacer() 		Text(Date().addingTimeInterval(600), style: .relative) } Is there any particular reason this would be the case? I've not found any documentation indicating that the manner in which WidgetKit views render Text would be any different than traditional SwiftUI views?

I am curious if there is suggested guidance on how to create "mock" Live Activities/ActivityKit for the sake of developing Lock Screen/Dynamic Island views in SwiftUI and taking advantage of SwiftUI Previews. For example, in the Display live data with Live Activities documentation, in the Create Lock Screen view section, demonstrates encapsulating the LockScreenActivityView in its own SwiftUI view. However, a subview of an ActivityConfiguration vends a generic context of type ActivityViewContext<MyActivityAttributes>, which does not seem to be able to be initialized directly. This makes it difficult to use SwiftUI Previews for building the Live Activity views. If I try to add a SwiftUI Preview; struct MyLockScreenLiveActivityView_Previews: PreviewProvider {     static var previews: some View {         MyLockScreenLiveActivityView(context: ...)     } } I am unsure how I would define a context that I could pass into the preview, as trying to manually define something like let context = ActivityViewContext<MyActivityAttributes> does not yield any accessible initializers to construct a mock object that conforms to ActivityViewContext. I might be missing something super simple, but would love any guidance, otherwise, I'm unable to use SwiftUI Previews for building the view.

I am facing an issue when building a project that builds without issue in Xcode 13.4.1, but hits a build error in Xcode 14. Specifically, this project uses CocoaPods and references a specific pod, "AdobeMobileSDK". In a sample project where I can replicate this issue, my Podfile looks like; target 'AdobeTestProject' do   # Comment the next line if you don't want to use dynamic frameworks   use_frameworks!   # Pods for AdobeTestProject   pod "AdobeMobileSDK" end I receive a build error reporting; Undefined symbols for architecture arm64:   "_OBJC_CLASS_$_ADBMobile", referenced from:       objc-class-ref in ContentView.o ld: symbol(s) not found for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation) After digging around a bit, I'm noticing that in Xcode 13.4, during the linking phase, Xcode issues a command that includes -lAdobeMobile. However, in Xcode 14, this is not the case. If I manually add -lAdobeMobile to the Other Linker Flags in my project settings, I can resolve this error, but this workaround is not suitable for all use cases (in a more complex project, adding -lAdobeMobile does not rectify the issue). I recognize this is likely some influenced by the SDK itself, but is there any documented change for Xcode 14 that would cause this build error only in Xcode 14 and not Xcode 13.4.1? This issue occurs regardless of whether Xcode 14 is running in Rosetta or not.

I have seen this question come up a few times here on Apple Developer forums (recently noted here - https://developer.apple.com/forums/thread/655505), though I tend to find myself having a misunderstanding of what technology and steps are required to achieve a goal. In general, my colleague and I are try to use Apple's Visualizing a Point Cloud Using Scene Depth - https://developer.apple.com/documentation/arkit/visualizing_a_point_cloud_using_scene_depth sample project from WWDC 2020, and save the rendered point cloud as a 3D model. I've seen this achieved (there are quite a few samples of the final exports available on popular 3D modeling websites), but remain unsure how to do so. From what I can ascertain, using Model I/O seems like an ideal framework choice, by creating an empty MDLAsset and appending a MDLObject for each point to, finally, end up with a model ready for export. How would one go about converting each "point" to a MDLObject to append to the MDLAsset? Or am I going down the wrong path?

Within my app, I have an Image that I am modifying with several modifiers to create an ideal appearance (code sample below). When taking this approach, I am finding that anything that is "underneath" the Image becomes unusable. In my case, I have a VStack with a Button and the Image. When the Image modifier of clipped() is applied, the Button becomes unusable (presumably because the Image is technically covering the button, but anything outside of the Image's frame is invisible). Is there a means of allowing an object below a clipped Image to still be functional/receive touches? VStack { &#9; Button(action: { &#9;&#9;&#9;print("tapped!") &#9; }, label: { &#9;&#9;&#9;Text("Tap Here") &#9; }) &#9; Image(uiImage: myImage) &#9;&#9;&#9;.resizable() &#9;&#9;&#9;.aspectRatio(contentMode: .fill) &#9;&#9;&#9;.frame(height: 150.0) &#9;&#9;&#9;.clipped() } I can confirm that if I change the aspectRatio to .fit, the issue does not appear (but, of course, my Image does not appear as I'd like it to). Subsequently, if I remove the .clipped() modifier, the issue is resolved (but, again, the Image then does not appear as I'd like it to).

As noted on the comparison page for Apple Watch - Series 6 - https://www.apple.com/watch/compare/, the U1 chip (Ultra Wideband) is a feature of the Apple Watch - Series 6. The WWDC 2020 session, Meet Nearby Interaction - https://developer.apple.com/videos/play/wwdc2020/10668/, does imply that this functionality exists on devices with a U1 chip, though the NearbyInteraction framework appears unavailable in watchOS. Can anyone confirm whether NearbyInteraction is available for watchOS?

Many apps that I download from the App Store seem to be adding shortcuts to the Shortcuts app, without me ever setting up a voice command. I was under the impression that to add a shortcut to the Shortcuts app, a user would need to create a voice command, via INUIAddVoiceShortcutViewController, which would then add the shortcut to the Shortcuts app. This is how I am currently adding a shortcut in my app, but am wondering how I could go about offering shortcuts in the Shortcuts app without needing to call INUIAddVoiceShortcutViewController?  let activity = NSUserActivity(activityType: "com.example.shortcut")  activity.title = "Sample Shortcut"  activity.userInfo = ["speech" : "This is a sample."]  activity.isEligibleForSearch = true  activity.isEligibleForPrediction = true activity.persistentIdentifier = "com.example.shortcut.myshortcut"            self.view.userActivity = activity  activity.becomeCurrent()              let siriShortcut = INShortcut(userActivity: activity)              // Setup view controller  let viewController = INUIAddVoiceShortcutViewController(shortcut: siriShortcut)              // Setup modal style  viewController.modalPresentationStyle = .formSheet              // Setup delegate  viewController.delegate = self              // Show view controller  DispatchQueue.main.async {     self.present(viewController, animated: true, completion: nil)  }

I have been exploring the sample image capture app, as well as command-line tool, for object capture. I've not yet figured out the best practices for capturing the bottom of images. For example, I have been attempting to demonstrate this with a sneaker. Per the WWDC21-10076 session, I have been circling around my object, taking photos using the sample capture app, using the automatic capture mode. While this is creating a 3D model, during my capture, I also turn my sneaker over, and capture the bottom. However, when my 3D model is created via the command-line tool, the "bottom" of the sneaker is always missing. Is there a given configuration when creating the PhotoGrammetrySession.Configuration that would be ideal for also including photos of the bottom of objects? While I pause, rotate my object to show the bottom, and continue capturing, I find that the bottom of the object is nearly always missing, despite many image captures that do include the bottom.

I'm curious if anyone has discovered a way to determining if their Messages app is in landscape left or landscape right? I've seen this topic come up in other discussions, but have not seen a resolution. Since Messages Extensions do support use of the camera and AVFoundation, I've been unable to set my video orientation as I'd usually use UIDevice.current.orientation to determine the orientation. Messages Extensions consistently report an unknown orientation, rather than Face Up, Face Down, Portrait, Landscape Left, Landscape Right, etc.I've been able to use a helpful suggestion of someone here to determine portrait vs. landscape by checking the following in my viewDidLayoutSubviews();if UIScreen.main.bounds.size.width &lt; UIScreen.main.bounds.size.height { // Portrait } else { // Landscape }This, however, results in things working well in portrait, but can result in rotated or upside down images in landscape since I cannot set landscape left or landscape right (or portrait upside down on iPad, for that matter).Thanks!