I have a simple visionOS app that uses a RealityView to map floors and ceilings using PlaneDetectionProvider and PlaneAnchors.
I can look at a location on the floor or ceiling, tap, and place an object at that location (I am currently placing a small cube with X-Y-Z axes sticking out at the location).
The tap locations are consistently about 0.35m off along the horizontal plane (it is never off vertically) from where I was looking.
Has anyone else run into the issue of a spatial tap gesture resulting in a location offset from where they are looking?
And if I move to different locations, the offset is the same in real space, so the offset doesn't appear to be associated with the orientation of the Apple Vision Pro (e.g. it isn't off a little to the left of the headset of where I was looking).
Attached is an image showing this. I focused on the corner of the carpet (yellow circle), tapped my fingers to trigger a tap gesture in RealityView, extracted the location, and placed a purple cube at that location.
I stood in 4 different locations (where the orange squares are), looked at the corner of the rug (yellow circle) and tapped. All 4 purple cubes are place at about the same location ~0.35m away from the look location.
Here is how I captured the tap gesture and extracted the 3D location:
var myTapGesture: some Gesture {
SpatialTapGesture()
.targetedToAnyEntity()
.onEnded { event in
let location3D = event.convert(event.location3D, from: .global, to: .scene)
let entity = event.entity
model.handleTap(location: location3D, entity: entity)
}
}
Here is how I set the position of the purple cube:
func handleTap(location: SIMD3<Float>, entity: Entity) {
let positionEntity = Entity()
positionEntity.setPosition(location, relativeTo: nil)
...
}
I am working with MeshAnchors, and I am having troubles getting to the classification of the triangles/faces.
This post references the MeshAnchor.Geometry, and that struct does have a property named "classifications", but it is of type GeometrySource. I cannot find any classification information in GeometrySource. Am I missing something there?
I think I am looking for something of type MeshAnchor.MeshClassification, but I cannot find any structs with this as a property.
Post not yet marked as solved
I see example code converting the results of a SpatialTap to a SIMD3 location. For example, from WWDC session Meet ARKit for spatial computing:
let location3D = value.convert(value.location3D, from: .global, to: .scene)
What I really want is a simd_float4x4 that includes orientation of the surface that the tap gesture/cast collided with?
My goal is to place an object with its Y-axis along the normal of the surface that was tapped.
For example, in the referenced WWDC session, they create a CollisionComponent from the MeshAnchor data. If that mesh data is covering a curved couch cushion, I would like the normal from that curved cushion (i.e., the closest triangle approximating it).
Is this possible?
My planned fallback is to only use planes for collision surfaces for tap gestures, extract the tap gesture value's entity (which I am hoping is the plane), and grab its transform for the orientation information.
I am hoping Apple has a simple function call that is more general than my fallback approach.
Post not yet marked as solved
In a progressive ImmersiveSpace, I created an object (a cylinder) and applied an OcclusionMaterial to it. It does hide my virtual content behind it, but does not show the content of my room. The cylinder just appears black.
In progressive (or full?) ImmersiveSpace, is it possible to apply occlusion material (or something else), so I can see the room behind the virtual content?
Basically, I want to punch a hole through the virtual content and see the room behind it.
As a practical example, imagine being in a progressive ImmersiveSpace, but you have a plane with an occlusion mesh applied to it above your Apple Magic Keyboard so you can see your keyboard.
Is this possible?
Post not yet marked as solved
Does RealityKit support a clipping plane, where I can define a plane and have all content on one side of the plane not rendered?
Post not yet marked as solved
I've added a simple visionOS Portal to an app's initial WindowGroup (a window with an attached portal is all that is displayed), but I've had troubles adding a portal to an ImmersiveSpace.
For example, using the boilerplate code that Xcode creates for a mixed spatial experience, I'd like to turn on & off the ImmersiveSpace which has a portal in it.
So far, the portal isn't showing up.
Is it possible to add a portal to an ImmersiveSpace? Are there any restrictions on where portals can be added?
When testing In-App Purchases in Xcode with a .storekit file, I can delete past purchase transactions, so I can re-test the purchase experience.
I've switched to using a Sandbox tester and made purchases. However, I cannot find how to delete previous purchase transactions made in the sandbox so I can re-run the tests.
Is this possible?
Post not yet marked as solved
When defining a volumetric WindowGroup, I can set the defaultSize().
It is possible to set a different volume size when opening a window with openWindow()?
In my use case, I want to display potentially different models that are of different sizes inside the volumetric window, and I want to preserve each model's size. I would like to create a volumetric window that is optimally sized for each model.
Alternatively I could create a volumetric window that is large enough to fit the largest model, and then reposition smaller models inside the volume to be at the front & bottom of the volume, but I haven't figured out how to do that either (Post on that question)
Post not yet marked as solved
I am having troubles placing a model inside a volumetric window.
I have a model - just a simple cube created in Reality Composer Pro that is 0.2m on a side and centered at the origin - and I want to display it in a volumetric window that is 1.0m on a side while preserving the cube's origin 0.2m size.
The small cube seems to be flush against the back and top of the larger volumetric window.
Is it possible to initially position the model inside the volume?
For example, can the model be placed flush against the bottom and front of the volumetric window?
(note: the actual use case is wanting to place 3D terrain (which tends to be mostly flat like a pizza box) flush against the bottom of the volumetric window)
I just downloaded the latest Xcode beta, Version 15.0 (15A240d) and ran into some issues:
On start up, I was not given an option to download the Vision simulator.
I cannot create a project targeted at visionOS
I cannot build/run a hello world app for Vision.
In my previous Xcode-beta (Version 15.0 beta 8 (15A5229m)), there was an option to download the vision simulator, and I can create projects for the visionOS and run the code in the vision simulator.
The Xcode file downloaded was named "Xcode" instead of "Xcode-beta". I didn't want to get rid of the exiting Xcode, so I selected Keep Both. Now I have 3 Xcodes in the Applications folder
Xcode
Xcode copy
Xcode-beta
That is the only thing I see that might have been different about my install.
Hardware: Mac Studio 2022 with M1 Max
macOS Ventura 13.5.2
Any idea what I did wrong?
Post not yet marked as solved
I have a strange warning in Xcode associated with ARKit. It isn't a big issue because there is a work around, but I am curious why this is happening and how I can avoid it in the future.
I opened up an old AR project in Xcode, and the editor gave a strange error message on the "import ARKit" line saying
Cannot load underlying module for 'ARKit'
Despite the error message, the code continues to build and run.
I've quit & restarted Xcode, rebooted the Mac, and even deleted and and redownloaded Xcode, but the error/warning was still there.
Upon some additional testing, I discovered that I only get this message when targeting "iOS 16.2" but not when targeting 16.0, 16.1, 16.3, or 16.4. (I did not try pre-iOS 16)
Any idea why my Xcode no longer likes ARKit on iOS 16.2 on my Mac?
Development platform:
Xcode: Version 14.3.1 (14E300c)
macOS: 13.4 (22F66)
Mac: Mac Studio 2022
iOS on iPhone 14 Pro Max: 16.5 (20F66)
Screenshot:
I have an ARView in nonAR cameraMode and a PerspectiveCamera. When I rotate my iPhone from portrait to landscape mode, the size of the content shrinks.
For example, the attached image shows the same scenes with the phone in portrait and landscape modes. The blue cube is noticeable smaller in landscape. The size of the cube relative to the vertical space (i.e., the height of the view) in each situation is consistent.
Is there a way to keep the scene (e.g., the cube) the same size whether I am in portrait or landscape mode?
Post not yet marked as solved
I’m embarking on a new project that will involve animating 3D faces & mouths. I’m looking at using ARFaceAnchors and blendShapes to capture data that will be used to animate the models’ facial expressions.
I have a few basic questions:
(1) As far as I can tell, Apple has not supported exporting Memojis to rigged 3D models. Is this still the case?
(2) I did find one web site that said Apple’s AvatarKit is now public, but everywhere else I’ve checked, it is still a private framework (and Xcode complains). Is AvatarKit still private?
(3) It looks like all 52 blendShapes for an ARFaceAnchor are updated every frame, which updates 60 times a second This is 3120 data points per second. Are there any best practice guides to reduce the data? For example, “These 10 blendShapes capture the most important features for animating a face.”
(4) It appears that visionOS does not support ARFaceAnchor. If I want to present a remote user as a Memoji (or other rigged model) in a shared experience, is there any way to do that at the current time?
Post not yet marked as solved
This is probably a minor point because it wouldn't affect distributed binaries, but I thought I'd mention it in case the behavior is unexpected.
After watching WWDC 20202 Explore logging in Swift, I tried some simple examples in a Mac command-line app. I was surprised to see the strings were all printed just fine. There was no redaction. At least when running the program from Xcode.
(Even using the old os_log() approach showed the strings without needing to add %{public}@.)
However, if I run the program from a Terminal shell, the string arguments are properly redacted.
I actually like this behavior (showing more while running in Xcode), but I thought I'd just raise the issue. Sample code and screenshot from Console are shown below.
import Foundation
import os
let logger = Logger(subsystem: "com.example.logging_test", category: "hello")
let greeting = "Hello"
let personName = "World"
logger.log("\(greeting), \(personName)")
logger.log("\(greeting, privacy: .private), \(personName)")
logger.log("\(greeting, privacy: .public), \(personName)")
os_log("%@, %@", greeting, personName)
Post not yet marked as solved
When setting ARView's environment camera feed exposure to a negative value to make the camera feed dimmer, for example
arView.environment.background = .cameraFeed(exposureCompensation: -3)
can this negatively affect ARKit's ability to track the device's localization and mapping capability?
That is, is the device's use of the camera for SLAM purposes independent of the exposureCompensation value?
Todd2.