What do they mean by screen space text in ARKit

Question

When I went through the video for session 805, Creating Great AR Experiences, and they mention using screen space text when displaying text in AR. They use the example of their Measure app that works quite well.

Is this something that is already in ARKit 2 or do we have to roll our own? I know that when using SpriteKit the objects are always facing the user but there doesn't seem to be any way of doing this with SceneKit based AR.

The main thing I'm trying to figure out is how to put something like a SceneKit box onscreen and have dimension lines of some kind along three of the edges to show the size of the box. This would be a resizable box with updating dimensions.

It would be fantastic if Apple put Measure up as sample code, but short of that does anyone have any ideas where I should start looking? I'm thinking there is a boat load of geometry in my future that may or may not be fast enough to maintain 60fps to keep updating the euler angles of the labels based on the camera position.

Beta
iOS

2.9k

Posted by

Swan-HMH

Reply

Add a Comment

Answer 1

I was wondering that too. You can add a billboard constraint, but then the objects get smaller as you move away.

Posted by

LeonardRockstar

Add a Comment

Answer 2

Have you found the answer for this?

Posted by

jagsrajput21

Add a Comment

Answer 3

Screen Space is the 2D pixel coordinate system of your display. It's where you normally do things like laying out UIKit widgets in screen space, etc.

The alternative to Screen Space is a 3D coordinate space that relates to the 3D world tracking ARKit does or the 3D rendering engine you use to display virtual 3D content in AR. (In that case there are multiple 3D spaces to consider, like "absolute" world space vs camera-relative positioning.)

As mentioned in the WWDC18 talk Creating Great AR Experiences, screen space is a great way to display content that's inherently 2D (like text) in a way that remains readable regardless of the distance to or orientation of a real-world feature that content is associated with. (If you were to place that text label in world space, it'd get small and unreadable as you get far away.)

ARKit doesn't give you any automatic way to have 2D screen-space content follow a 3D world-space position, but making that happen yourself isn't especially difficult. Here's a general recipe for overlaying

UIView

s that follow 3D positions:

Add an
```
ARAnchor
```
at the real-world position where you want screen-space content to appear.
Use a per-frame callback
```
(renderer(_:didUpdate:for:)
```
or
```
renderer(_:updateAtTime:)
```
if you're using SceneKit with
```
ARSCNView
```
,
```
session(_:didUpdate:)
```
if you're building your own renderer) to get the current 3D position of that anchor.
Project the 3D position (
```
ARSCNView
```
```
projectPoint
```
,
```
ARCamera
```
```
projectPoint
```
) to get the 2D point where that 3D position appears on screen.
Set the position of your
```
UIView
```
to match that point.

For more advanced cases like how the Measure app makes its screen-space label parallel to the line between two 3D points, you'd need to project the two endpoints and use a function like

atan2

to get an angle for rotating the label.
ANSWER FROM:
https://stackoverflow.com/questions/51249236/what-is-mean-by-screen-space-in-arkit

Posted by

keynsianets@gmail.com

Add a Comment

What do they mean by screen space text in ARKit

Replies