URGENT!!! Need to reset system extensions from Recovery Mode

Hi guys,


I'm working on system extension and I was using SimpleFirewall Apple sample. That firewall was working, but then I tried to create another framework which would be used by system extension target and the UI target. When starting the app UI window showed up, but without Start/Stop button. Spinner was shown instead. Sample app usually does that when firewall config is enabled and it tries to resume. It didn'tdo anything so I closed the app or stopped it from Xcode. Then I found I'm completely cut off from the network.

Then I deleted the app in Debug folder and another older copy in Applications, which asked for admin credentials, because it was hopsting system extension. I was hoping this would remove/disable system exension and therefore firewall filtering.


I was however only using filtering on port 80 - HTTP connections, but nothing else. Not sure why entire network went down. I think system extension after rebuilding somehow invalidated system extension and it could not do the handle flow, so entire netwrok traffic froze.


Unfortunatelly it didn't help and after rebooting the machine it was unable to connect to the network, possibly because system extension with blocked sample firewall filtering is still active. However macOS requires connection to the network, because needs "Critical update". Of course I had this machine completely upto date. I think macOS might be trying to validate that the machine is not stolen or something so it simply doesn't activate and requires to connect to Apple servers, which with firewall filtering in system extension in some corrupted state is not possible.


I tried to run the machine without netwrok, but it does not allow it and I can only shut down the machine or setup network again.


I was trying to use systemextensionctl reset from terminal in recovery mode, but that is not available.

My machine is currently completely unusable.


I really need some help from guy who knows how to fix it, not some customer relations rep, who is going to walk me through basics or online resources.


I haven't done backup for a week or 2, because it takes hours, so I don't do it on daily basis.

Does anyone know how to recover my machine?


Thanks.

Replies

You have to use Recovery Mode to turn off SIP. Then, after rebooting normally, you can use systemextenionsctl to remove the system extension. Don't forget to go back into Recovery Mode and re-enable SIP.


A Time Machine backup should only take a couple of minutes. You may have more serious problems.
In the future, you should test things like system extensions in a VPN that can easily be reset to a factory-fresh configuration. You will still need another machine for doing live testing. If you can't afford multiple machines, find some other kind of software to develop.

I had SIP disabled, because I was developing system extensions and signing the app with development prov.profile. I also tried to enable SIP in hope kernel might pinpoint potentially offending kernel extension.

My machine is unable to boot up with SIP disabled and enabled, it makes no difference.

I also deleted app in Debug folder and another copy in /Applications to remove system extension. I actually did that while the machine was still in normal mode only with broken network, before reboot (before hitting "Critical update required" popup).


It looks like System Extensions can render mac totally unusable just like kernel extensions.


Yes, we do have virtual machines on VPN, these can be restored quicker than our machines, but that's not optimal either as you can easily cut off the network when filtering traffic, especially in early stages of development. Believe me, I wouldn't develop system extensions as an indie developer, trying to sell security apps on my own.

Sometimes you just want to test something small on your machine. In my case I was trying to figure out minimal set of entitlements (for example if system extension and app need App Group entitlement in order to communicate) and I also tried to create framework, which would be used by the app and system extension (to use common classes in both targets, like IPC commiunication, filter rules, etc).

System Extensions are supposed to run in user space and should be safe without the ability to break kernel (definitely not to the point of total recovery required).


It's also not clear to me if that framework needs to be integrate with "Do not embed", "Embed without signing" or "Embed and sign". At the time of the failure I was using "Do not embed" in both app and system extension. I'm thinking that perhaps extension trying to use the common framework was prevented by OS security and that's what may have caused total network block. No network connection could then pass through. I thought that simply deleting app with extension should automatically disable system extension and any configuration setup by extension using NEFilterConfiguration.


At least I have learned how to write a virus that can disable mac :-)

Anyway, I'm restoring the machine now, there does not appear to be a fix for this.

Sorry. That was either autocorrect or a typo. I meant "VM". Do not attempt to install any system modification, of any kind, on your development machine. That will scramble the system really nicely. And if you've got SIP disabled, then you can do an even more thorough job of scrambling.


Wherever you test in the future, you can only use systemextensionctl on a live, running system with SIP disabled.