[Reality Composer Pro] Is it possible to play video over a specific mesh like a material?

Question

PLAY4 OP

Created Nov ’24

Replies 3

Boosts 0

Participants 3

Hi!

I'm trying to play video on monitor 3D model like a material. I wanna know if it's possible work. I searched about it, but I couldn't get enough information. Thank you in advance.

Answered by Vision Pro Engineer in 812875022

Hi @PLAY4

Here's more detail on how you can get this to work by using a VideoMaterial.

Start by placing your video file somewhere inside of your Xcode project. In my case, I created a "Videos" folder in my project and placed a .mp4 file with the name "MyMovie" inside of it.

Next, define a RealityView that loads your video file and applies it to the surface of a model with a VideoMaterial, as shown below:

RealityView { content in
    // Create a URL that points to the movie file.
    if let url = Bundle.main.url(forResource: "MyMovie", withExtension: "mp4") {
        
        // Create an AVPlayer instance to control playback of that movie.
        let player = AVPlayer(url: url)

        // Instantiate and configure the video material.
        let material = VideoMaterial(avPlayer: player)

        // Create a new model entity using the video material.
        let modelEntity = ModelEntity(mesh: .generateBox(size: 1.0), materials: [material])
        modelEntity.position = [0, 1, -1]
        content.add(modelEntity)

        // Start playing the video.
        player.play()
    }
}

In this example, VideoMaterial is applied to a cube model positioned in front of the viewer, but you can apply it to the screen on your 3D monitor model instead.

Boost

Answer 1

JoonAhn OP

Nov ’24

We are currently developing a set of visionOS apps that detect and measure object surface geometries from MeshAnchor.

https://github.com/CurvSurf/FindSurface-visionOS

FindSurfaceST (Spatial Tap): Object surface detection by spatial tap
FindSurfaceRR (Responce-to-Request): autonomous object surface detection
FindSurfaceRT (Real-Time): Real-time object surface detection
FindSurfaceAD (Ads): rendering photos/videos on detected object surfaces. The corresponding iOS app is here; https://github.com/CurvSurf/FindSurface-SceneKit-ARDemo-iOS

The source code of the FindSurfaceAD app is planned to be released in December 2024. Photos/videos are planned to be selected in the Photos app.

Please keep GitHub CurvSurf running.

1

Answer 2

Vision Pro Engineer OP

Apple

Nov ’24

Hi @PLAY4

You can play a video on the surface of an entity with a VideoMaterial.

Let me know if you have any questions!

1

Answer 3

Vision Pro Engineer OP

Apple

Nov ’24

Accepted Answer

Recommended

Hi @PLAY4

Here's more detail on how you can get this to work by using a VideoMaterial.

Start by placing your video file somewhere inside of your Xcode project. In my case, I created a "Videos" folder in my project and placed a .mp4 file with the name "MyMovie" inside of it.

Next, define a RealityView that loads your video file and applies it to the surface of a model with a VideoMaterial, as shown below:

RealityView { content in
    // Create a URL that points to the movie file.
    if let url = Bundle.main.url(forResource: "MyMovie", withExtension: "mp4") {
        
        // Create an AVPlayer instance to control playback of that movie.
        let player = AVPlayer(url: url)

        // Instantiate and configure the video material.
        let material = VideoMaterial(avPlayer: player)

        // Create a new model entity using the video material.
        let modelEntity = ModelEntity(mesh: .generateBox(size: 1.0), materials: [material])
        modelEntity.position = [0, 1, -1]
        content.add(modelEntity)

        // Start playing the video.
        player.play()
    }
}

In this example, VideoMaterial is applied to a cube model positioned in front of the viewer, but you can apply it to the screen on your 3D monitor model instead.

1