Video textures are going to be revolutionary for the platform. I am curious what filetypes are supported? Is there any native support for transparency using alpha channels?
Video formats and alpha handling.
RealityKit can accept video from any source that AV Foundation supports. See the AVFileType documentation. The AVURLAsset class also has a function you can call to get a list of supported formats on your device/OS.
RealityKit does not support alpha channels from video at this time. If you would be interested in support, please submit feedback using Feedback Assistant.
RealityKit does not support alpha channels from video at this time. If you would be interested in support, please submit feedback using Feedback Assistant.
This is a work around I have been playing with. It isn't great though:
/// Creates A Video Player As An SCNGeometries Diffuse Contentsstatic func setupVideoOnNode(_ node: SCNNode, fromURL url: URL) {
//1. Create An SKVideoNode var videoPlayerNode: SKVideoNode!
//2. Create An AVPlayer With Our Video URL let videoPlayer = AVPlayer(url: url)
//3. Initialize The Video Node With Our Video Player videoPlayerNode = SKVideoNode(avPlayer: videoPlayer) videoPlayerNode.yScale = -1
//4. Create A SpriteKitScene & Position It let spriteKitScene = SKScene(size: CGSize(width: 1024, height: 768)) spriteKitScene.scaleMode = .aspectFit videoPlayerNode.position = CGPoint(x: spriteKitScene.size.width/2, y: spriteKitScene.size.height/2) videoPlayerNode.size = spriteKitScene.size spriteKitScene.backgroundColor = .clear
//5. Alpha transparency let effectNode = getAlphaEffectNode(videoPlayerNode: videoPlayerNode) spriteKitScene.addChild(effectNode) effectNode.addChild(videoPlayerNode)
//6. Set The Nodes Geoemtry Diffuse Contenets To Our SpriteKit Scene node.geometry?.firstMaterial?.diffuse.contents = spriteKitScene //7. Play The Video videoPlayerNode.play() videoPlayer.volume = 0 //8. Loop Video loopVideo(videoPlayer: videoPlayer, node: node)}
// MARK: - Add Alpha Transparencystatic func getAlphaEffectNode(videoPlayerNode: SKVideoNode) -> SKEffectNode { // Let's make it transparent, using an SKEffectNode, // since a shader cannot be applied to a SKVideoNode directly let effectNode = SKEffectNode() effectNode.shader = EffectNodeHelper.getAlphaShader() return effectNode
/// Creates A Video Player As An SCNGeometries Diffuse Contentsstatic func setupVideoOnNode(_ node: SCNNode, fromURL url: URL) {
//1. Create An SKVideoNode var videoPlayerNode: SKVideoNode!
//2. Create An AVPlayer With Our Video URL let videoPlayer = AVPlayer(url: url)
//3. Initialize The Video Node With Our Video Player videoPlayerNode = SKVideoNode(avPlayer: videoPlayer) videoPlayerNode.yScale = -1
//4. Create A SpriteKitScene & Position It let spriteKitScene = SKScene(size: CGSize(width: 1024, height: 768)) spriteKitScene.scaleMode = .aspectFit videoPlayerNode.position = CGPoint(x: spriteKitScene.size.width/2, y: spriteKitScene.size.height/2) videoPlayerNode.size = spriteKitScene.size spriteKitScene.backgroundColor = .clear
//5. Alpha transparency let effectNode = getAlphaEffectNode(videoPlayerNode: videoPlayerNode) spriteKitScene.addChild(effectNode) effectNode.addChild(videoPlayerNode)
//6. Set The Nodes Geoemtry Diffuse Contenets To Our SpriteKit Scene node.geometry?.firstMaterial?.diffuse.contents = spriteKitScene //7. Play The Video videoPlayerNode.play() videoPlayer.volume = 0 //8. Loop Video loopVideo(videoPlayer: videoPlayer, node: node)}
// MARK: - Add Alpha Transparencystatic func getAlphaEffectNode(videoPlayerNode: SKVideoNode) -> SKEffectNode { // Let's make it transparent, using an SKEffectNode, // since a shader cannot be applied to a SKVideoNode directly let effectNode = SKEffectNode() effectNode.shader = EffectNodeHelper.getAlphaShader() return effectNode
In the WWDC 2020 session What's new in RealityKit the bug/insect that is shown uses video materials for its wings that are clearly transparent? How was this done? Is the big/insect in the video using video alpha? Or was another way used for that effect?