We are working with an app that uses the INPlayMediaIntent to allow users to select and play music using Siri.
In building out this feature, we have noticed that when selecting playlists to play, Siri will consistently leave out information from the intent that we are use to resolve the media to play in the app.
It seems that there is generally no rhyme or reason as to why some information is left out.
Walking through a couple test cases, here is the phrase and corresponding mediaSearch that we receive when testing:
- "Hey Siri, play the playlist happy songs in the app <app name>" (this is a working example)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x114050780> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = happy songs;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
- "Hey Siri, play the playlist my favorites in the app <app name>" (this fails with a null mediaName)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x114050600> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = <null>;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
- "Hey Siri, play the playlist working out playlist in the app <app name>" (this fails as the term "playlist" is excluded)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x114050ae0> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = working out;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
- "Hey Siri, play the playlist recently added in the app <app name>" (this fails with a null mediaName)
▿ Optional<INMediaSearch>
- some : <INMediaSearch: 0x1140507e0> {
reference = 0;
mediaType = 5;
sortOrder = 0;
albumName = <null>;
mediaName = <null>;
genreNames = (
);
artistName = <null>;
moodNames = (
);
releaseDate = <null>;
mediaIdentifier = <null>;
}
Based on the above, Siri seems to ignore playlists named "Recently Added", "My Favorites", and playlists that have the word "playlist" in them such as "Working Out Playlist".
To rectify this, we attempted to set the INVocabulary for the playlist titles that a user has in the app, as suggested in this WWDC session: https://developer.apple.com/videos/play/wwdc2020/10060/
let vocabulary = INVocabulary.shared()
vocabulary.setVocabularyStrings(NSOrderedSet(array: [
"my favorites",
"recently added",
"working out playlist"
]), of: .mediaPlaylistTitle);
This seems to have no effect. We understand the note in https://developer.apple.com/documentation/sirikit/registering_custom_vocabulary_with_sirikit/ stating that "a few minutes" should be waited before testing custom vocabulary, but waiting upwards of 20 minutes and even restarting the device did not result in any of the custom vocabulary making a difference.
If these playlist names are set in AppIntentVocabulary.plist
, "Recently Added" and "My Favorites" are able to be discovered as playlists, but the other failed test cases remain failing. The obvious shortcoming here is that these are not dynamic.
<key>ParameterVocabularies</key>
<array>
<dict>
<key>ParameterNames</key>
<array>
<string>INPlayMediaIntent.playlistTitle</string>
</array>
<key>ParameterVocabulary</key>
<array>
<dict>
<key>VocabularyItemIdentifier</key>
<string>working out playlist</string>
<key>VocabularyItemSynonyms</key>
<array>
<dict>
<key>VocabularyItemPhrase</key>
<string>working out playlist</string>
</dict>
</array>
</dict>
<dict>
<key>VocabularyItemIdentifier</key>
<string>recently added</string>
<key>VocabularyItemSynonyms</key>
<array>
<dict>
<key>VocabularyItemPhrase</key>
<string>recently added</string>
</dict>
</array>
</dict>
<dict>
<key>VocabularyItemIdentifier</key>
<string>my favorites</string>
<key>VocabularyItemSynonyms</key>
<array>
<dict>
<key>VocabularyItemPhrase</key>
<string>my favourites</string>
</dict>
<dict>
<key>VocabularyItemPhrase</key>
<string>my favorites</string>
</dict>
</array>
</dict>
</array>
</dict>
</array>
Given the above, our questions are as follows:
-
Is there documentation surrounding how Siri may pass along the mediaSearch in INPlayMediaIntent and how/why information may be left out?
-
Why does setting custom vocabulary with INVocabulary seem to have no effect, yet the same vocabulary in AppIntentVocabulary does have an effect?
-
Is the functionality we are experiencing to be expected, or should this be reported as a bug?
We've published the test app that we are using for debugging this functionality at this link: https://github.com/awojnowski/SiriTest