Use a HTTP Proxy with WkWebView

In WKWebView, the client portion is out of process. If you are including an embedded proxy, that is a server. It doesn't matter if your client is in-process or out. If anything, it would work better out of process.

Also, Privoxy is GPL, making it strictly incompatible with the App Store as well as making your app GPL too. Plus it has very poor support for Apple platfoms. Maybe consider something like Polipo instead.

(Hi,

Thanks for the answer! I was getting nervous that this was a too nichè question after not finding help on StackOverflow...

I see your point about GPL, it is a valid one. However, I believe we have that sorted out, since our backend proxy-server is made with a docker image, which we are open-sourcing and making available at hub.docker.com. The idea of the app is just to redirect the webview traffic through a proxy-endpoint made available somewhere on the internet outside the app - in this case with our backend server. I will have to read further into this though to make sure this separation between app and backend is good enough for GPL. Thanks for pointing it out.

With regards to the WkWebView answer: The client portion is out of process indeed, but I thought we need to be able to access this part, in order to redirect the traffic of the webview through the proxy endpoint, since that happens out-of-process? It sounds like I might have misunderstood something here (or did not explain our use case properly).

NB.
The last comment from Suryakant on the original post made here https://stackoverflow.com/questions/44365986/is-it-possible-to-run-a-proxy-server-in-an-ios-app-to-be-called-from-wkwebview seems to be facing the same issue as me.

Again it is a comment made from before IOS 11 (and IOS 12 for that matter), where I hope something has been changed that I have just not been able to spot yet...

As said, we have tried without luck recently, but maybe we are missing something?

You’re not missing anything.

WKWebView

NSURLProtocol

UIWebView

WKWebView

Modern versions of

WKWebView

WKURLSchemeHandler

I encourage you to file an enhancement request (ER) describing your requirements in this space. For example:

• If you want to actually see all the requests à la

  WKURLSchemeHandler

  , you could file an ER against that.

• If all you need is the ability to set a custom proxy for the web view, that’d made a great ER because it has such limited scope.

If you do file any ERs, please post the number here, just for the record.

With regards VPN you wrote:

it seems that there is no split-tunneling options in iOS

What makes you say that? iOS can support split tunnels just fine.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

[1] That’s reflected by the fact that it doesn’t see all requests made by the view, for example, it does not see WebSocket requests.

Hi Eskimo,

Thanks for the response! Awesome with an official place to go to that can provide qualified answers on "niché" (read: very technical) subjects.

I filed an ER as you suggeste and it received the ID: #45622725. However, I can see it was closed due to being a duplicate of #20545691. I cannot view tickets I have not created myself, so I will put my faith in that they understood it correctly and matched it with a ticket with the same challenge.

Anyway, I'm very curious about the possibility of split-tunneling now, since you write it is possible. That would solve our headaches. I was of the belief that iOS VPNs could not be per-app (split-tunnel), unless it is on a supervised device? Since the app is meant to be distributed on the appstore, it would be nonsupervised devices... I was of the belief due to this apple-page: https://developer.apple.com/documentation/networkextension/netunnelprovidermanager

It states that: "The only way to configure Per-App VPN is by enrolling the device in a Mobile Device Management (MDM) system". :/

If it is possible to achieve this without a MDM system, then it would work wonders and we can go for this approach instead of the HTTP proxy (as we can route the traffic of just our app through the VPN and filter the traffic in the backend for ads and trackers).

NB. Sidenote, but as mentioned earlier, the problem with having the VPN be device-wide instead of per-app is that the traffic, instead of just being from casual browsing of politics and shopping in our app (where we want people to be anonymeous), will be for everything. It will cause the bandwidth to explode if the users start downloading app-updates through us and whatnot, which will force us to charge the same as a vpn (5-10x what we do now) just for the purpose of routing HTTP(S) traffic through a filter. 😟

I cannot view tickets I have not created myself, so I will put my faith in that they understood it correctly and matched it with a ticket with the same challenge.

I took a quick look at the original bug and it does accurately reflect the requirements in your bug. Yay!

Anyway, I'm very curious about the possibility of split-tunneling now, since you write it is possible. That would solve our headaches. I was of the belief that iOS VPNs could not be per-app (split-tunnel), unless it is on a supervised device? Since the app is meant to be distributed on the appstore, it would be nonsupervised devices... I was of the belief due to this [page, which] states that: "The only way to configure Per-App VPN is by enrolling the device in a Mobile Device Management (MDM) system". :/

You are mixing up your terms. Let’s see if I can make this clearer.

When dealing with the Network Extension framework there are three groups of devices that you need to consider:

• Normal devices

• Managed devices, where the device is being managed by an MDM server

• Supervised devices, a managed device that’s actually owned by the managing organisation

There’s a critical difference between managed devices and supervised devices. Some things require a managed device but the most intrusive things (like the on-device content filter) require a supervised device.

There are two types of VPN tunnel providers on iOS:

• Packet tunnel providers

• App proxy providers

These are distinguished by how they receive traffic from the network stack. Packet tunnel providers receive IP packets from the network stack, whereas app proxy providers receive TCP and UDP flows. The situation with UDP is complex, but for TCP a flow corresponds to a TCP connection, that is, the app proxy provider sees the actual data passing through the TCP connection not the packets being used to transfer that data.

App proxy providers are one type of per-app VPN, but they are not the only type.

A packet tunnel provider can operate in one of two modes:

• Destination IP, where the system passes packets to the provider based on their destination IP address

• Source app, where the system passes packets to the provider based on the source application

The latter is the second flavour of per-app VPN.

On iOS per-app VPN requires that the device be managed (not supervised) because configuring an app to use the VPN can only be done via MDM.

A packet tunnel provider in destination IP mode can set up the tunnel in two ways:

• A full tunnel claims the default route

• A split tunnel does not

For a split tunnel the packet tunnel provider must configure the tunnel with a set of networks [1] and the system will route packets to the tunnel if they are destined for an address on one of those networks.

So, coming back to your issue, per-app VPN would be great for your app if it weren’t for the managed device requirement. If you want to support normal devices, your only option is a packet tunnel provider in destination IP mode. It is possible to set this up with a split tunnel but that requires you to configure the networks that must go over the tunnel. For many VPN configurations that’s fine (for example, Apple owns the 17.0.0.0/24 network, so any address on that network should go over the tunnel) but I doubt it’ll work for you.

Share and Enjoy
—
Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

[1] For IPv4 each of these is represented by an address and subnet mask. For IPv6 you use an address and a prefix.

#45622725. However, I can see it was closed due to being a duplicate of #20545691

Any update to either of those requests? It's a little weird that we cannot intercept embedded WkWebView traffic. This forces apps to apply settings device wide (using a VPN configuration) leading to bad UX in some cases.

With recent iOS versions, it's possible to modify the DoH settings inside the app without affecting device-wide settings but of course this doesn't change the embedded WkWebView settings which to me seems like a bug. These features can still be supported transparently even if the wkwebview is out of process.

Any update to either of those requests?

AFAIK there’s still no way to intercept all network traffic being generated by a WKWebView )-:

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

any update? i found that you can use in WKURLSchemeHandler your cases.

1. add hook for webview

@implementation WKWebView (Hook)

+ (void)load {
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        Method origin = class_getClassMethod(self, @selector(handlesURLScheme:));
        Method hook = class_getClassMethod(self, @selector(cdz_handlesURLScheme:));
        method_exchangeImplementations(origin, hook);
    });
}

+ (BOOL)cdz_handlesURLScheme:(NSString *)urlScheme {
    if ([urlScheme isEqualToString:@"http"] || [urlScheme isEqualToString:@"https"]) {
        return NO;
    }
    return [self cdz_handlesURLScheme:urlScheme];
}

@end

2. set url schemehandler

extension WKWebViewConfiguration{
    class func proxyConifg() -> WKWebViewConfiguration{
        let config = WKWebViewConfiguration()
        let handler = HttpProxyHandler()
        config.setURLSchemeHandler(handler, forURLScheme: "http")
        config.setURLSchemeHandler(handler, forURLScheme: "https")
        return config
    }
}

3. write proxy logic in your WKURLSchemeHandler

func webView(_ webView: WKWebView, start urlSchemeTask: WKURLSchemeTask) {
    	let proxy_server = "YourProxyServer" // proxy server
        let proxy_port = 1234 // your port
        let hostKey = kCFNetworkProxiesHTTPProxy as String
        let portKey = kCFNetworkProxiesHTTPPort as String
        let proxyDict:[String:Any] = [kCFNetworkProxiesHTTPEnable as String: true, hostKey:proxy_server, portKey: proxy_port]
        let config = URLSessionConfiguration.ephemeral
        config.connectionProxyDictionary = proxyDict
    
        let defaultSession = URLSession(configuration: config)
        
        dataTask = defaultSession.dataTask(with: urlSchemeTask.request, completionHandler: {[weak urlSchemeTask] (data, response, error) in
            /// fix crash error                                                                            
            guard let urlSchemeTask = urlSchemeTask else {
                return
            }
            
            if let error = error {
                urlSchemeTask.didFailWithError(error)
            } else {
                if let response = response {
                    urlSchemeTask.didReceive(response)
                }
                
                if let data = data {
                    urlSchemeTask.didReceive(data)
                }
                urlSchemeTask.didFinish()
            }
        })
        dataTask?.resume()
}

:)

any update?

No )-:

i found that you can use in WKURLSchemeHandler your cases.

Do not swizzle methods on Apple classes. That is not the path to long-term binary compatibility.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

iOS 17 has new proxyConfigurations API

https://developer.apple.com/documentation/network/nwparameters/privacycontext/4156642-proxyconfigurations

Is this the approach to intercept WKWebView requests ?

Is this the approach to intercept WKWebView requests ?

That’s my understanding, yes.

Let us know how you get along with this. The inability to see all the network requests made by WKWebView has been a major stumbling block for a small but important subset of folks moving from UIWebView, so I’d like to know how this pans out in practice.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

I was able to confirm that on iOS 17, using proxyConfiguration with WkWebView does work. Here's a snippet from my working code,

import WebKit

class WebKitViewModel: ObservableObject {
    let webView: WKWebView
    @Published var urlString: String = "https://example.com"

    init() {
        webView = WKWebView(frame: .zero)
    }
    
    func loadUrl() {
        guard let url = URL(string: urlString) else {
                    return
                }
        var request = URLRequest(url: url)
        let endpoint = NWEndpoint.hostPort(host: "127.0.0.1", port: 9077)
        let proxyConfig = ProxyConfiguration.init(httpCONNECTProxy: endpoint)
        let websiteDataStore = WKWebsiteDataStore.default()
        websiteDataStore.proxyConfigurations = [proxyConfig]
        webView.configuration.websiteDataStore = websiteDataStore
        webView.load(request)
    }
}

It will even work with HTTPS requests, assuming you have a CA certificate for your proxy installed as a trusted root cert on your device and that the proxy's certs (dummy certs if doing MiTM) and the CA cert satisfy Apple's requirements for trusted certs on iOS 13+ (https://support.apple.com/en-us/HT210176).

One issue that I am still having is that I cannot add a header to every HTTP request coming from the WkWebView, which is crucial to my proxy's functionality. I am able to add a header to the initial URL request, but if that URL request results in additional requests (which is how every website works), the header is not added to the subsequent requests. But I guess this is a separate, known issue maybe, closely related to this thread, https://developer.apple.com/forums/thread/14462?

Hope this helps out other people who are interested in using a proxy with WebKit.