We are working with an app that uses the INPlayMediaIntent to allow users to select and play music using Siri.
In building out this feature, we have noticed that when selecting playlists to play, Siri will consistently leave out information from the intent that we are use to resolve the media to play in the app.
It seems that there is generally no rhyme or reason as to why some information is left out.
Walking through a couple test cases, here is the phrase and corresponding mediaSearch that we receive when testing:
"Hey Siri, play the playlist happy songs in the app " (this is a working example)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x114050780> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = happy songs;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
"Hey Siri, play the playlist my favorites in the app " (this fails with a null mediaName)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x114050600> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = <null>;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
"Hey Siri, play the playlist working out playlist in the app " (this fails as the term "playlist" is excluded)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x114050ae0> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = working out;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
"Hey Siri, play the playlist recently added in the app " (this fails with a null mediaName)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x1140507e0> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = <null>;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
Based on the above, Siri seems to ignore playlists named "Recently Added", "My Favorites", and playlists that have the word "playlist" in them such as "Working Out Playlist".
To rectify this, we attempted to set the INVocabulary for the playlist titles that a user has in the app, as suggested in this WWDC session: https://developer.apple.com/videos/play/wwdc2020/10060/
let vocabulary = INVocabulary.shared()
vocabulary.setVocabularyStrings(NSOrderedSet(array: [
"my favorites",
"recently added",
"working out playlist"
]), of: .mediaPlaylistTitle);
This seems to have no effect. We understand the note in https://developer.apple.com/documentation/sirikit/registering_custom_vocabulary_with_sirikit/ stating that "a few minutes" should be waited before testing custom vocabulary, but waiting upwards of 20 minutes and even restarting the device did not result in any of the custom vocabulary making a difference.
If these playlist names are set in AppIntentVocabulary.plist, "Recently Added" and "My Favorites" are able to be discovered as playlists, but the other failed test cases remain failing. The obvious shortcoming here is that these are not dynamic.
<key>ParameterVocabularies</key>
<array>
<dict>
<key>ParameterNames</key>
<array>
<string>INPlayMediaIntent.playlistTitle</string>
</array>
<key>ParameterVocabulary</key>
<array>
<dict>
<key>VocabularyItemIdentifier</key>
<string>working out playlist</string>
<key>VocabularyItemSynonyms</key>
<array>
<dict>
<key>VocabularyItemPhrase</key>
<string>working out playlist</string>
</dict>
</array>
</dict>
<dict>
<key>VocabularyItemIdentifier</key>
<string>recently added</string>
<key>VocabularyItemSynonyms</key>
<array>
<dict>
<key>VocabularyItemPhrase</key>
<string>recently added</string>
</dict>
</array>
</dict>
<dict>
<key>VocabularyItemIdentifier</key>
<string>my favorites</string>
<key>VocabularyItemSynonyms</key>
<array>
<dict>
<key>VocabularyItemPhrase</key>
<string>my favourites</string>
</dict>
<dict>
<key>VocabularyItemPhrase</key>
<string>my favorites</string>
</dict>
</array>
</dict>
</array>
</dict>
</array>
Given the above, our questions are as follows:
Is there documentation surrounding how Siri may pass along the mediaSearch in INPlayMediaIntent and how/why information may be left out?
Why does setting custom vocabulary with INVocabulary seem to have no effect, yet the same vocabulary in AppIntentVocabulary does have an effect?
Is the functionality we are experiencing to be expected, or should this be reported as a bug?
We've published the test app that we are using for debugging this functionality at this link: https://github.com/awojnowski/SiriTest
Post
Replies
Boosts
Views
Activity
I am working on integrating widget support into an existing app and have a question regarding the best practices for reloading a widget. For context, the existing app allows the user to add a playlist widget which will show the title, track count, and thumbnail associated with that playlist.
Within the app, playlists can of course be edited which will change the data that should be reflected on the widget. In addition, edits may happen over and over, in quick succession (ex. deleting multiple tracks, reordering tracks, etc).
My question is as follows: should the widget be reloaded every time an edit is made, or is there a way to mark a widget as needing to be reloaded?
Considering the use case where a user is editing a playlist and makes 10-20 changes in quick succession, it feels wasteful to reload a widget 10-20 times for each small change. However, the app doesn't necessarily know when the user is going to stop editing or terminate the app, so it's important that the widget is reloaded rather proactively.
In addition to the above question, is there any way to only reload a specific, single widget, rather than all widgets or all widgets of a certain kind?
Considering the use case where a user may have 3-4 playlist widgets on their homescreen, it also feels wasteful to reload all of them when only one of the playlists may have been edited. Yet, from my understanding of WidgetCenter, the only two functions for reloading a widget are reloadAllTimelines and reloadTimelines(ofKind:) which seem to only allow broad reloading.
I recently updated to Xcode 15.0 and seem to be encountering permissions related errors after using the Replace Container function to replace an app container. Similarly, editing the current run scheme and choosing a new App Data container fails with an identical error.
It seems like the Documents directory and other similar directories are blocked from being able to write. Here is an example of what appears in the log after replacing the container:
Failed to create directory /var/mobile/Containers/Data/Application/A3C32E7A-34F3-4C69-B037-478027F2A4AC/Library/Preferences because of 13.
Couldn't write values for keys (
"/google/measurement/app_instance_id"
) in CFPrefsPlistSource<0x280a82760> (Domain: com.google.gmp.measurement, User: kCFPreferencesCurrentUser, ByHost: No, Container: (null), Contents Need Refresh: No): Directory needed
os_unix.c:47395: (13) lstat(/private/var/mobile/Containers/Data/Application/A3C32E7A-34F3-4C69-B037-478027F2A4AC/Documents)
- Permission denied os_unix.c:47395: (13) lstat(/private/var/mobile/Containers/Data/Application/A3C32E7A-34F3-4C69-B037-478027F2A4AC/Documents/<redacted>.sqlite)
- Permission denied os_unix.c:46898: (13) statfs(/private/var/mobile/Containers/Data/Application/A3C32E7A-34F3-4C69-B037-478027F2A4AC/Documents/<redacted>.sqlite)
- Permission denied
Steps to reproduce:
Create a new app and replace -viewDidLoad: with the following. This will write a string to a new file and read that string:
NSFileManager * const fileManager = [NSFileManager defaultManager];
NSURL * const documentsDirectoryURL = [[fileManager URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask] lastObject];
if (!documentsDirectoryURL) {
[NSException raise:@"DocumentsDirectoryException" format:@"Application documents directory URL is nil"];
}
NSURL * const fileURL = [documentsDirectoryURL URLByAppendingPathComponent:@"test.txt"];
if ([fileManager fileExistsAtPath:[fileURL path]]) {
NSError *error;
if (![[NSFileManager defaultManager] removeItemAtPath:[fileURL path] error:&error]) {
[NSException raise:@"DocumentsDirectoryException" format:@"Encountered error removing file: %@",error];
}
}
NSString * const fileContents = [NSString stringWithFormat:@"Date: %@",[NSDate date]];
NSError *error;
if (![[fileContents dataUsingEncoding:NSUTF8StringEncoding] writeToURL:fileURL options:0 error:&error]) {
[NSException raise:@"DocumentsDirectoryException" format:@"Encountered error writing to file: %@",error];
}
NSString * const writtenFileContents = [[NSString alloc] initWithData:[NSData dataWithContentsOfURL:fileURL] encoding:NSUTF8StringEncoding];
NSLog(@"Written file contents: %@", writtenFileContents);
Run the app and observe the log message "Written file contents: ..."
Save the app container via Window > Devices & Simulators by clicking the app, clicking the "(...)" button, and then clicking "Download Container..."
Use the same procedure in step 3 but instead, click "Replace Container..." and choose the existing app container that was downloaded
Run the app again and observe that it fails with the following exception message:
*** Terminating app due to uncaught exception 'DocumentsDirectoryException', reason: 'Encountered error writing to file: Error Domain=NSCocoaErrorDomain Code=513 "You don’t have permission to save the file “test.txt” in the folder “Documents”." UserInfo={NSFilePath=/var/mobile/Containers/Data/Application/CF1E09C8-7BE2-4D87-8AD2-648AFBE038A5/Documents/test.txt, NSUnderlyingError=0x2831d4d50 {Error Domain=NSPOSIXErrorDomain Code=13 "Permission denied"}}'
Is anyone else encountering this issue and can confirm? Are my next steps to create a bug report? Is there a manner to escalating the bug report? This is a pretty core piece of functionality for QA and testing apps.