For apps built specifically for the new Lidar sensor that have little to no use on devices without it, is there an appropriate Required device capabilities string?
Post
Replies
Boosts
Views
Activity
The new object capture API is really quite remarkable, and I can already think of several possible use cases and pipelines for it.
It is great that even though it can work with just 2D images the API can use metadata like a depth maps, lens data, gravity direction, and GPS location to create a better understanding of the scene and reconstruction.
In a situation where the rough camera pose of each captured image is known (for example an ARKit tracked frame, or a static array of cameras) I would love the ability to add an initial camera transform matrix to each PhotogrammetrySample.
Without assuming too much about how the underlying system works, I assume that the camera extrinsics and intrinsics are constantly being refined as the model is being reconstructed. In this situation I would not expect the input camera poses to be taken as absolute values.
But I think being able to give those initial matrix transform values would have several benefits:
Give an even better hint for the camera positions than just the gravity vector would alone.
Define object scale based on camera extrinsics, even if no depth data is available.
Predefine a coordinate space, origin and orientation for the capture.
Have a common and consistent origin point and orientation between scans with an identical (or similar) camera setup.
Without having tried a drone capture yet perhaps there is a way to do this with GPS data. But that feels like an unnecessary hack-a-round, and likely prone to conversion errors.
Which of these would work/be best practice?
sample.metadata = ["FocalLengthIn35mmFilm": "28mm"]
sample.metadata = ["FocalLengthIn35mmFilm": 28.0]
sample.metadata = ["kCGImagePropertyExifFocalLenIn35mmFilm": "28MM"]
Any images or PhotogrammetrySamples after 1000 will be rejected and ignored. This is regardless of image resolution, bit depth, and format.
This restriction is still present in macOS 12.0.1. Please remove this restriction.
This error is reported as soon as a PhotogrammetrySession starts processing.
libc++abi: terminating with uncaught exception of type std::runtime_error: Failed to access model resource path
This happens both with a custom application and the provided example command line demo.
The sample application and code does not seem to be available in the WWDC app or in the documentation along with other Object Capture samples. Where/when will this be released?