Post

Replies

Boosts

Views

Activity

Handling user-initiated re-centering in group immersive space?
Hi, currently tinkering with a little shareplay app for the Vision Pro that allows people to facetime and shareplay to play with random 3d models (as well as move them around, which should sync the model positions for everyone in relative space). When the users start their facetime call, then open the immersive space to see the 3d models, the models load in properly in context of the group immersive space's coordinate system, and moving the models reflects the new positions real-time for each participant. The main issue comes if/when users use the digital crown to re-center their view. It appears to re-center the model and view, which is expected. However, it also seems to re-position the model/root entity to match the user's origin. Not sure if this is intentional or not, but this essentially makes it so that it "de-syncs" the model (so me moving the model next to someone does not reflect it 1:1 - it still moves properly, but the new "initial" position after re-centering makes it offset). Is there a potential solution or work-around for this such that re-centering the view doesn't de-sync the model/entity's position? Rough code for my RealityView component is below: RealityView { content, attachments in content.add(appModel.originEntity) appModel.originEntity.addChild(appModel.modelContainerEntity) appModel.setInitialModelPosition() configureGestures(forModel: appModel.modelContainerEntity) configureToolbarAttachment(content: content, attachments: attachments) } update: { content, _ in // I have modified the Apple provided gesture components to // send the app model the new positions/rotations // as well as broadcast the position/rotation to shareplay participants // When user re-centers view, it seems to also re-position the model // so that its origin is at the local user's origin, rather than // the original origin // Can we receive a notification that user has re-centered view? // Or some other work-around? appModel.modelContainerEntity.setPosition(appModel.modelState.position, relativeTo: nil) appModel.modelContainerEntity.setOrientation(.init(appModel.modelState.rotation3d), relativeTo: nil) } attachments: { Attachment(id: "customViewAttachment") { CustomView() } } .installGestures() Please let me know if anything wasn't clear or if more information is needed. Thanks!
4
0
340
Oct ’24
GroupSessionJournal attachment loading error on Vision Pro
Hi all, Currently working on a shareplay feature where users pull data from a remote source and are able to share it in a volumetric window with others in the facetime call. However, I am running into an issue where the group activity/session seems to be throwing an error on the recipient of the journal's attachment with the description of notSupported. As I understand it, we use GroupSessionJournal for larger pieces of data like images (like in the Drawing Together example) and in my case 3d models. The current flow goes as follows: User will launch the app and fetch a model from remote. User can start a shareplay instance in which the system captures the volumetric window for users to join and see. At this point, only the original user can see the model. The user can press a button to share this model with the other participants using /// modelData is serialized `Data` try await journal.add(modelData) In the group session configuration, I already have a task listening for for await attachments in journal.attachments { for attachment in attachments { ... } } This task attempts to load data via the following code: let modelData = try await attachment.load(Data.self) /// this is where the error is thrown: `notSupported` I expect the attachment.load(Data.self) call to properly deliver the model data, but instead I am receiving this error. I have also attempted to wrap the model data within an enclosing struct that has a name and data property and conform the enclosing struct to Transferable but that continued to throw the notSupported error. Is there something I'm doing wrong or is this simply a bug in the GroupSessionJournal? Please let me know if more information is required for debugging and resolution. Thanks!
0
1
546
Jun ’24
GroupSessionJournal loading attachments error
Hi all, I had previously posted this on the Media Technologies section but didn't receive any replies so thought I would try my luck here. Apologies if re-posting questions on the forum in a short-ish time span is against the rules, but would greatly appreciate some assistance with the following situation regarding GroupSessionJournal and loading attachments. I'm currently working on a shareplay feature that allows users to pull 3d models from icloud and view it via volumes/immersive space on the vision pro. Was able to get the sharing working with multiple windows recently so now all that's left is to be able to sync/share the model in the SharePlay session. As I understand it, we should generally use GroupSessionMessenger for commands and light data like model positioning/syncing properties. Whereas for bigger pieces of data (images/videos/models), we should send these through GroupSessionJournal which the group session manages and syncs it for all users in the call. I have a button to get the current user's model data and add it to the journal via /// modelData is type `Data` try await journal.add(modelData) I have also set up a task to observe/receive updates to the journal's attachments in when receiving a group session. for await groupSession in MyModelActivity.sessions() { ... tasks.insert { Task { for await attachments in journal.attachments { for attachment in attachments { do { let modelData = try await attachment.load(Data.self) // throws error here - `notSupported` let modelUrl = writeModelDataToTempDirectory(modelData: modelData) self.modelUrlToLoadForGroupSession = modelUrl } catch let error { print("Error: \(error)") } } } } } } Not quite sure why I'm running into an error being thrown when attempting to load the attachment data on the other devices, any thoughts? The documentation for add(_:) and load(_:) say that the attachment should conform to Transferable but Data.Type should already conform to Transferable
1
0
494
Jun ’24
GroupSessionJournal attachment loading error
Hello, currently working on a shareplay feature that allows users to pull 3d models from icloud and view it via volumes/immersive space on the vision pro. Was able to get the sharing working with multiple windows recently so now all that's left is to be able to sync/share the model in the SharePlay session. As I understand it, we should generally use GroupSessionMessenger for commands and light data like model positioning/syncing properties. Whereas for bigger pieces of data (images/videos/models), we should send these through GroupSessionJournal which the group session manages and syncs it for all users in the call. I have a button to get the current user's model data and add it to the journal via /// modelData is type `Data` try await journal.add(modelData) I have also set up a task to observe/receive updates to the journal's attachments in when receiving a group session. for await groupSession in MyModelActivity.sessions() { ... tasks.insert { Task { for await attachments in journal.attachments { for attachment in attachments { do { let modelData = try await attachment.load(Data.self) // throws error here - `notSupported` let modelUrl = writeModelDataToTempDirectory(modelData: modelData) self.modelUrlToLoadForGroupSession = modelUrl } catch let error { print("Error: \(error)") } } } } } } Not quite sure why I'm running into an error being thrown when attempting to load the attachment data on the other devices, any thoughts? The documentation for add(_:) and load(_:) say that the attachment should conform to Transferable but Data.Type should already conform to Transferable
1
0
655
Jun ’24
Multi-scene/window shareplay on visionOS
Hi all, been working with visionOS for a bit so far and am trying to develop a feature that allows users to shareplay and interact with a 3D model pulled from the cloud (icloud in this case, but may use a regular backend service in the future). Ideally, I would want to be able to click a custom button on a regular window that starts the group activity/shareplay with another person in the facetime call and opens the volumetric window with the model and can switch to an immersive space freely. TLDR/questions at the very end for reference point. I was able to get this working when only working with a single window group (i.e. a volumetric window group scene and an immersive space scene). However I am running into trouble getting shareplay to correctly grab the desired scene (or any scene at all) when I have multiple window group scenes defined. I have been referencing the following documentation in my attempts to implement this: https://developer.apple.com/documentation/groupactivities/sceneassociationbehavior https://developer.apple.com/documentation/groupactivities/adding-spatial-persona-support-to-an-activity https://developer.apple.com/documentation/groupactivities/defining-your-apps-shareplay-activities https://developer.apple.com/documentation/groupactivities/joining-and-managing-a-shared-activity No luck so far however. Additionally, here is a quick breakdown of what I've done so far to attempt implementation: Define group activity that contains static var activityIdentifier: String and var metadata: GroupActivityMetadata as well as conforms to GroupActivity. Provide methods to start shareplay via a startShareplay() method that instantiates the above group activity and switch awaits on activity.prepareForActivation() to activate the activity if case .activationPreferred. I have also provided a separate group activity registration method to start shareplay via airdrop as mentioned in the Building spatial SharePlay experiences developer video (timestamped), which does expose a group activity to the share context menu/ornament but does not indicate being shared afterwards. On app start, trigger a method to configure group sessions and provide listeners for sessions (including subscribers for active participants, session state, messages of the corresponding state type - in my case it's ModelState.self, journal attachments for potentially providing models that the other user may not have as we are getting models from cloud/backend, local participant states, etc). At the very end, call groupSession.join(). Add external activation handlers to the corresponding scenes in the scene declaration (as per this documentation on SceneAssociationBehavior using the handlesExternalEvents(matching:) scene modifier to open the scene when shareplay starts). I have also attempted using the view modifier handlesExternalEvents(preferring:allowing:) on views but also no luck. Both are being used with the corresponding activityIdentifier from the group activity and I've also tried passing a specific identifier while using the .content(_) sceneAssociationBehavior as well but no luck there either. I have noted that in this answer regarding shareplay in visionOS, the VP engineer notes that when the app receives the session, it should setup any necessary UI then join the session, but I would expect even if the UI isn't being set up via the other person's session that the person who started shareplay should still see the sharing ornament turn green on the corresponding window which doesn't seem to occur. In fact, none of the windows that are open even get the green sharing ornament (just keeps showing "Not shared"). TLDR: Added external events handling and standard group activity stuff to multi-window app. When using shareplay, no windows are indicated as being shared. My questions thus are: Am I incorrect in my usage of the scene/view modifiers for handlesExternalEvents to open and associate a specific windowgroup/scene with the group activity? In regards to opening a specific window when the group activity is activated, how do we pass any values if the window group requires it? i.e. if it's something like WindowGroup(id: ..., for: URL.self) { url in ... } Do I still need to provide UI setup in the session listener (for await session in MyActivity.sessions())? Is this just a simple openWindow? Past the initializing shareplay stuff above, what are the best practices for sharing 3d models that not all users in the session might have? Is it adding it as an attachment to GroupSessionJournal? Or should I pass the remote URL to everyone to download the model locally instead? Thanks for any help and apologies for the long post. Please let me know if there's any additional information I can provide to help resolve this.
3
0
916
Jun ’24