Simulate and send keyboard events inside the Mac OS X Sandbox

Hello, everyone.


My name is Matt, I'm the developer of Eternal Storms Software.


I recently released a freeware app: SiriMote, outside of the Mac App Store.


The reason I could not release it on the Mac App Store is that it uses CGEventPost to simulate keypresses (for example, when the play/pause button is pressed on the Siri Remote, the Play/Pause media key (on the F8 key) of the Mac's keyboard is pressed), and CGEventPost is ignored inside the sandbox.


So I was wondering - is there a way to do this inside the sandbox?


I guess an alternative would be using the Scripting Bridge, but then I would have to specifically communicate with particular apps. The nice thing about CGEventPost is that any app that responds to the media keys can be used with SiriMote.

I'd love a more open approach, like CGEventPost.


Any hints appreciated!


Thank you kindly,

Matt

Post not yet marked as solved Up vote post of m.gansrigler Down vote post of m.gansrigler
5.6k views
Posted by
m.gansrigler
Reply
Add a Comment

Apple Recommended

[did] something changers re sandboxing rules that's it allowed now?

I recently researched a very similar situation as part of a DTS incident, so I have concrete info to share on that front. I think it’s safe to extrapolate this to your issue, but lemme explain and you can make your own call.

In my case the developer was trying to get the Input Monitoring privilege. My immediate thought was that this is not available to sandboxed apps but, on researching it in detail, I discovered that’s not the case.

In recently releases [1] we’ve added user data protection (aka TCC) checks to various facilities, including Input Monitoring. As part of that we made it possible for a sandboxed app to gain such privileges. Starting with 10.15 a sandboxed app can use APIs like CGEventTap to monitor input events as long as the user approves that in Security > Security & Privacy > Privacy > Input Monitoring.

I think the same logic applies to event generation but I haven’t actually researched that privilege in detail. If you want me to do that, open a DTS tech support incident. Alternatively, you could just suck it and see.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Starting with 10.14 but, in this specific case, the Input Monitoring privilege was added in 10.15.

Posted by
eskimo
Post not yet marked as solved Up vote reply of eskimo Down vote reply of eskimo
Post marked as Apple Recommended
Add a Comment

Replies

So I was wondering - is there a way to do this inside the sandbox?

No.

Normally I’d recommend that you file an enhancement request for the features you want, although in this case you have to be a little careful. There’s no point filing an enhancement request for access to 

CGEventPost
because such access would allow your app to defeat the sandbox entirely. Thus your enhancement request should be focused on your high-level goal, simulating the media keys, which seems like a reasonable thing to do from within the sandbox.

Please post your bug number, just for the record.

Share and Enjoy
—
Quinn "The Eskimo!"
Apple Developer Relations, Developer Technical Support, Core OS/Hardware 

let myEmail = "eskimo" + "1" + "@apple.com"

DTS will close for the winter holidays at the end of business on Wed, 23 Dec 2015 and re-open on Mon, 4 Jan 2016.

Posted by
eskimo
Up vote reply of eskimo Down vote reply of eskimo
Post marked as solved
Add a Comment

filed an enhancement request:

rdar://24278614


thank you.

Posted by
m.gansrigler
Post not yet marked as solved Up vote reply of m.gansrigler Down vote reply of m.gansrigler
Add a Comment

Infact, you can call AppleScript in sandbox. tell key code/key down/key up/keystroke to "System Events" to simulate normal keys, and tell iTunes some other commands to simulate F7 to F9.


But bug exist at tell "System Events": key up cannot lift key which user is pressing. Thus, modifier may cannot be what you want, if making key-remapping tool.

Posted by
OpenThread
Up vote reply of OpenThread Down vote reply of OpenThread
Add a Comment

No there is. AppleScript can achive this.

Posted by
OpenThread
Up vote reply of OpenThread Down vote reply of OpenThread
Add a Comment

AppleScript can achive this.

Right. But the App Sandbox does not let you execute arbitrary AppleScripts. You can execute AppleScripts via the 

NSUserScriptTask
mechanism, but my understanding is that App Review expects apps to use that as a general user-configurable option, not simply as a way to bypass sandbox restrictions.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware 

let myEmail = "eskimo" + "1" + "@apple.com"
Posted by
eskimo
Post not yet marked as solved Up vote reply of eskimo Down vote reply of eskimo
Add a Comment

Hi Matt,


Did you get a response to your enhancement request? I don't have access to it.
I have a similar requirement, only I need to post key events for arrow keys instead of media keys.


Thanks,

Jeremy

Posted by
JeremyHX
Up vote reply of JeremyHX Down vote reply of JeremyHX
Add a Comment

Hi

I know its an old thread, but I'm trying to find an answer and this seems like a good place.

There is an app on Mac App Store, a new app, that simulates user input, and it seems it's doing it while being sandboxed. Is there some new APIs that were introduced, or something changers re sandboxing rules that's it allowed now?

https://apps.apple.com/us/app/remmo-remote-mouse-keyboard/id1598906173

I'm have a similar app and I have to distribute it like Matt outside of the store, and I wonder if there was a change that can allow me in.

Best regards

Posted by
Phoenix
Up vote reply of Phoenix Down vote reply of Phoenix
Add a Comment

[did] something changers re sandboxing rules that's it allowed now?

I recently researched a very similar situation as part of a DTS incident, so I have concrete info to share on that front. I think it’s safe to extrapolate this to your issue, but lemme explain and you can make your own call.

In my case the developer was trying to get the Input Monitoring privilege. My immediate thought was that this is not available to sandboxed apps but, on researching it in detail, I discovered that’s not the case.

In recently releases [1] we’ve added user data protection (aka TCC) checks to various facilities, including Input Monitoring. As part of that we made it possible for a sandboxed app to gain such privileges. Starting with 10.15 a sandboxed app can use APIs like CGEventTap to monitor input events as long as the user approves that in Security > Security & Privacy > Privacy > Input Monitoring.

I think the same logic applies to event generation but I haven’t actually researched that privilege in detail. If you want me to do that, open a DTS tech support incident. Alternatively, you could just suck it and see.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] Starting with 10.14 but, in this specific case, the Input Monitoring privilege was added in 10.15.

Posted by
eskimo
Post not yet marked as solved Up vote reply of eskimo Down vote reply of eskimo
Post marked as Apple Recommended
Add a Comment