Post not yet marked as solved
Hello guys,How can I build a Core ML that ranks a list with the images that are most similar to an inputed image?Clarifai.ai has this feature, but I wanted to do it in CoreMLthanks!
Post not yet marked as solved
Hello usd team,
The WWDC 10141 talk mentions documentation comparing RealityKit vs Storm, but I didn’t find it.
is it available already?
thanks!
Post not yet marked as solved
Great session!
Will the demo app source code be available?
tks
Post not yet marked as solved
Hello USD team,
Thanks for the great demo USD and Hydra/Storm demo!
Is it possible to build an app with USD Hydra/Storm -> SwiftUI interactions?
Ex: a car configurator that has:
a car as a USD file
animations desired on the USD file.
SwiftUI button to opens the doors, another button to play a sound, etc?
everything rendered in Hydra/Storm. as the demo app
Thanks.
Post not yet marked as solved
On Scenekit, using SCNShapewe can create SCN geometry from SwiftUI 2D shapes/beziers:https://developer.apple.com/documentation/scenekit/scnshape
Is there an equivalent in RealityKit?
Could we use the generate(from:) for that?https://developer.apple.com/documentation/realitykit/meshresource/3768520-generate
https://developer.apple.com/documentation/realitykit/meshresource/3768520-generate
Post not yet marked as solved
Hey guys,Goal: export morph targets DMaya -> Dae -> SceneKitAE, with morph targets controllersIn Maya everything works fine, morphs with sliders work as intendedWhen exported to FBX/DAE and opened on Xcode, the morphs targets are there, but it places the entire avatar, instead of scaling the specific node (nose, arm, etc).https://gyazo.com/6f36a90ce5292b85a6f7a21b9a8918f2How can I export from Maya to DAE, keeping morphs for each node? Or any other Maya -> Scenekit?Thanks!
Post not yet marked as solved
Using Scenekit UIViewRepresentable (code block 1 bellow), we could add a tap gesture recognizer hit test like
With the new SceneView for Scenekit/SwiftUI (code block 2), we add a tap gesture recognizer?
I found this API, but its not clear how to use it...
https://developer.apple.com/documentation/scenekit/sceneview/3607839-ontapgesture
import SwiftUI
import SceneKit
import UIKit
import QuartzCore
struct SceneView: UIViewRepresentable {
func makeUIView(context: Context) -> SCNView {
let view = SCNView(frame: .zero)
let scene = SCNScene(named: "ship")!
view.allowsCameraControl = true
view.scene = scene
// add a tap gesture recognizer
let tapGesture = UITapGestureRecognizer(target: self, action: #selector(handleTap(_:)))
	view.addGestureRecognizer(tapGesture)
return view
}
func handleTap(_ gestureRecognize: UIGestureRecognizer) {
// retrieve the SCNView
let view = SCNView(frame: .zero)
// check what nodes are tapped
let p = gestureRecognize.location(in: view)
let hitResults = view.hitTest(p, options: [:])
// check that we clicked on at least one object
if hitResults.count > 0 {
// retrieved the first clicked object
let result = hitResults[0]
// get material for selected geometry element
let material = result.node.geometry!.materials[(result.geometryIndex)]
// highlight it
SCNTransaction.begin()
SCNTransaction.animationDuration = 0.5
// on completion - unhighlight
SCNTransaction.completionBlock = {
SCNTransaction.begin()
SCNTransaction.animationDuration = 0.5
material.emission.contents = UIColor.black
SCNTransaction.commit()
}
material.emission.contents = UIColor.green
SCNTransaction.commit()
}
}
func updateUIView(_ view: SCNView, context: Context) {
}
}
import SwiftUI
import SceneKit
struct ContentView: View {
var scene = SCNScene(named: "ship.scn")
var cameraNode: SCNNode? {
scene?.rootNode.childNode(withName: "camera", recursively: false)
}
var body: some View {
SceneView(
scene: scene,
pointOfView: cameraNode,
options: []
)
.allowsHitTesting(/*@START_MENU_TOKEN@*/true/*@END_MENU_TOKEN@*/)
}
}
struct ContentView_Previews: PreviewProvider {
static var previews: some View {
ContentView()
}
}
I am trying to build scover "Ray Tracing with Metal" on iMac Pro 2017, but got this error:
2020-06-25 14:29:35.233058+0200 MTLRaytracingSample-macOS[3602:145531] Failed to set (contentViewController) user defined inspected property on (NSWindow): Ray tracing isn't supported on this device
Xcode 12
Big Sur 11.0 Beta (20A4299v)
iMac Pro 2017
3,2 GHz 8-Core Intel Xeon W
32 GB 2666 MHz DDR4
Radeon Pro Vega 56 8 GB
What is the required hardware configuration?
Post not yet marked as solved
Goal:
Rewrite CoreGraphics resizing function from iOS (UIImage) to MacOS (NSImage)
Problem:
Getting error on this line of code: let bytesPerPixel = cgImage.bitsPerPixel / bitsPerComponent
Error: Value of type '(UnsafeMutablePointer?, NSGraphicsContext?, [NSImageRep.HintKey : Any]?) - CGImage?' (aka '(OptionalUnsafeMutablePointer, Optional, OptionalDictionaryNSImageRep.HintKey, Any) - Optional') has no member 'bitsPerPixel'
Question: how can I replace "cgImage.bitsPerPixel" for something that works with NSImage?
iOS: UIImage
extension UIImage {
// Resizeing using CoreGraphics
func resize(to size:CGSize) - UIImage? {
let cgImage = self.cgImage!
let destWidth = Int(size.width)
let destHeight = Int(size.height)
let bitsPerComponent = 8
let bytesPerPixel = cgImage.bitsPerPixel / bitsPerComponent
let destBytesPerRow = destWidth * bytesPerPixel
let context = CGContext(data: nil,
width: destWidth,
height: destHeight,
bitsPerComponent: bitsPerComponent,
bytesPerRow: destBytesPerRow,
space: cgImage.colorSpace!,
bitmapInfo: cgImage.bitmapInfo.rawValue)!
context.interpolationQuality = .high
context.draw(cgImage, in: CGRect(origin: CGPoint.zero, size: size))
return context.makeImage().flatMap { UIImage(cgImage: $0) }
}
}
MacOS: NSImage
extension NSImage {
// Resizeing using CoreGraphics
func resize(to size:CGSize) - NSImage? {
let cgImage = self.cgImage
let destWidth = Int(size.width)
let destHeight = Int(size.height)
let bitsPerComponent = 8
let bytesPerPixel = cgImage.bitsPerPixel / bitsPerComponent
let destBytesPerRow = destWidth * bytesPerPixel
let context = CGContext(data: nil,
width: destWidth,
height: destHeight,
bitsPerComponent: bitsPerComponent,
bytesPerRow: destBytesPerRow,
space: cgImage.colorSpace!,
bitmapInfo: cgImage.bitmapInfo.rawValue)!
context.interpolationQuality = .high
context.draw(cgImage, in: CGRect(origin: CGPoint.zero, size: size))
return context.makeImage().flatMap { NSImage(cgImage: $0) }
}
}
Post not yet marked as solved
Goal:
export multiple images with .fileExporter
What I did:
Code 1 bellow works fine for MacOS, export multiple files.
Problem:
When I replace for UIImage, it exports only 1 image
Question:
How can I export multiple images using .fileExporter in iOS?
Code 1 bellow works fine for MacOS, export multiple files.
import SwiftUI
class AppContext: ObservableObject {
@Published var fileSaveDialogShown = false
}
@main
struct FocalApp: App {
@StateObject var appContext = AppContext()
var body: some Scene {
WindowGroup {
ContentView()
.environmentObject(self.appContext)
.fileExporter(
isPresented: $appContext.fileSaveDialogShown,
documents: [
ImageDocument(image: NSImage(named: "1")),
ImageDocument(image: NSImage(named: "2"))
],
contentType: .jpeg // Match this to your representation in ImageDocument
) { url in
print("Saved to", url) // [URL]
}
}
}
}
import SwiftUI
import UniformTypeIdentifiers
struct ImageDocument: FileDocument {
static var readableContentTypes: [UTType] { [.jpeg, .png, .tiff] }
var image: NSImage
init(image: NSImage?) {
self.image = image ?? NSImage()
}
init(configuration: ReadConfiguration) throws {
guard let data = configuration.file.regularFileContents,
let image = NSImage(data: data)
else {
throw CocoaError(.fileReadCorruptFile)
}
self.image = image
}
func fileWrapper(configuration: WriteConfiguration) throws - FileWrapper {
// You can replace tiff representation with what you want to export
return FileWrapper(regularFileWithContents: image.tiffRepresentation!)
}
}
struct ContentView: View {
@EnvironmentObject var appContext: AppContext
var body: some View {
VStack {
Button(action: {
appContext.fileSaveDialogShown.toggle()
}, label: {
Text("Button")
})
}
.frame(width: 200, height: 200)
}
}
When I replace for UIImage, it exports only 1 image
import SwiftUI
class AppContext: ObservableObject {
@Published var fileSaveDialogShown = false
}
@main
struct FocalApp: App {
@StateObject var appContext = AppContext()
var body: some Scene {
WindowGroup {
ContentView()
.environmentObject(self.appContext)
.fileExporter(
isPresented: $appContext.fileSaveDialogShown,
documents: [
ImageDocument(image: UIImage(named: "1")),
ImageDocument(image: UIImage(named: "2"))
],
contentType: .jpeg // Match this to your representation in ImageDocument
) { url in
print("Saved to", url) // [URL]
}
}
}
}
import SwiftUI
import UniformTypeIdentifiers
struct ImageDocument: FileDocument {
static var readableContentTypes: [UTType] { [.jpeg, .png, .tiff] }
var image: UIImage
init(image: UIImage?) {
self.image = image ?? UIImage()
}
init(configuration: ReadConfiguration) throws {
guard let data = configuration.file.regularFileContents,
let image = UIImage(data: data)
else {
throw CocoaError(.fileReadCorruptFile)
}
self.image = image
}
func fileWrapper(configuration: WriteConfiguration) throws - FileWrapper {
// You can replace tiff representation with what you want to export
return FileWrapper(regularFileWithContents: image.jpegData(compressionQuality: 1)!)
}
}
struct ContentView: View {
@EnvironmentObject var appContext: AppContext
var body: some View {
VStack {
Button(action: {
appContext.fileSaveDialogShown.toggle()
}, label: {
Text("Button")
})
}
.frame(width: 200, height: 200)
}
}
Post not yet marked as solved
Goal: export a group of images in SwiftUI
What I did: I am using the .fileExporter modifier, with the FileDocument struct. Also open to other approach, like . fileMover modifier for example.
Problem: When setting the FileDocument for multiple images struct I am getting am error on func fileWrapper (check code bellow).
Question: How can I export multiple images in SwiftUI (could be any method)?
//file exporter
		.fileExporter(isPresented: $exportFile, document: ImageDocument(
										
										
										image: UIImage(data: product.cover ?? Data())!,
										image2:	UIImage(data: product.cover2 ?? Data())!)
									
									,
					
contentType: .jpeg, onCompletion: { (result) in
						if case .success = result {
								
								print("Success")
						} else {
								print("Failure")
						}
				})
//export group of images
struct ImageDocument: FileDocument {
		
		static var readableContentTypes: [UTType] { [.jpeg] }
		var image: UIImage
		var image2: UIImage
		init(
				image: UIImage?,
				image2: UIImage?
		) {
				
				self.image = image ?? UIImage()
				self.image2 = image2 ?? UIImage()
		}
		
		init(configuration: ReadConfiguration) throws {
				guard let data = configuration.file.regularFileContents,
							let image = UIImage(data: data),
							let image2 = UIImage(data: data)
				else {
						throw CocoaError(.fileReadCorruptFile)
				}
				self.image = image
				self.image2 = image2
		}
		func fileWrapper(configuration: WriteConfiguration) throws -> FileWrapper {
				return FileWrapper(regularFileWithContents:
														image.jpegData(compressionQuality: 0.80)!,
														image2.jpegData(compressionQuality: 0.80)!//<----- getting an "extra argument error here
														)
		}
		
}
Post not yet marked as solved
I have added a material to my geometry of SCNNode and now I want to add another material to it and set it to blend mode 'multiply'.I tried a lot but unable to find a way to do this. If we blend the texture asmaterial.lightingModel = .physicallyBasedlet image = UIImage(named: "1.PNG")material.multiply.contents = imagematerial.multiply.contentsTransform = SCNMatrix4MakeScale(10, 10, 0)material.multiply.wrapT = .repeatmaterial.multiply.wrapS = .repeatmaterial.multiply.intensity = 1.0when set to “ physically based”. The multiply doesn’t work....any clue what is wrong?thanks!
Post not yet marked as solved
When building a SceneKit Project, Xcode cannot save .scn files and gives a warning: “The document “ship.scn” could not be saved. “
1- Create SceneKit project
2- Change anything on ship.scn, like background color for ex
3- build
4- Xcode says “The document “ship.scn” could not be saved.
Post not yet marked as solved
Goal: Convert FBX to USD (with Morph Targets)Tried:1 - Reality converterFBX to USD worked, but not Morph Targets.From what I researched, Reality converter is using USDPython 0.62, which doesnt support Morph targets.Would be great if apple update Reality converter to suppport 0.64 with Morph Targets.2- USDPython 0.64USDZconvert command for FBX to USD worked, but not Morph TargetsLooks like the script that supports FBX to USD is usdStageWithFbx.py (inside example folder). I have seen no documentation on how to run this script.Can someone from Apple help out with step by step instructions?Cheers to all!
Post not yet marked as solved
Is the Playground for Shape Edit code available?
manemanero.