Intermittent 400 "Invalid Verb" error uploading image to server using background URLSession

Question

Created May ’21

Replies 3

Boosts 0

Views 1.1k

Participants 2

Hi,

I have an Enterprise iOS app that allows users to upload images to our Sharepoint Intranet.

The app simply does a PUT request from a file:// url of the JPG (from app's temporary dir) to the Sharepoint location using a background configuration URLSession.

Code Block Swiftvar request = URLRequest(url: url)
            request.httpMethod = "PUT"           
            var session: URLSession!           
            if let kUploadInBackground = appDelegate.getPref("kUploadInBackground") as? Bool
                , kUploadInBackground == true {
                session = appDelegate.backgroundSession
            } else {
                let config = URLSessionConfiguration.default             
                session = URLSession(configuration: config, delegate: UIApplication.shared.delegate as? URLSessionDelegate, delegateQueue: nil)
            }
            let task = session.uploadTask(with: request, fromFile: fileUrl)
            if let size = item.data?.count {
                task.countOfBytesClientExpectsToSend = Int64(size)
            }
            task.resume()

appDelegate.backgroundSession is defined (currently as I've been trying to tweak to resolve) as:

Code Block swiftvar _backgroundSessionConfig: URLSessionConfiguration!
    var backgroundSessionConfig: URLSessionConfiguration {
        get {
            objc_sync_enter(self)
            defer { objc_sync_exit(self) }
            guard let _backgroundSessionConfig = _backgroundSessionConfig else {
                self._backgroundSessionConfig = URLSessionConfiguration.background(withIdentifier: (Bundle.main.infoDictionary!["CFBundleIdentifier"] as! String))
           self._backgroundSessionConfig.timeoutIntervalForRequest = 300
                self._backgroundSessionConfig.waitsForConnectivity = true
                return self._backgroundSessionConfig!
            }
            return _backgroundSessionConfig
        }
        set {
            _backgroundSessionConfig = newValue
        }
    }
    var _backgroundSession: Foundation.URLSession?
    var backgroundSession: Foundation.URLSession {
        get {
            objc_sync_enter(self)
            defer { objc_sync_exit(self) }
            guard let _backgroundSession = _backgroundSession else {
                self._backgroundSession = Foundation.URLSession(configuration: backgroundSessionConfig, delegate: self, delegateQueue: nil)
                return self._backgroundSession!
            }
            return _backgroundSession
        }
        set {
            _backgroundSession = newValue
        }
    }
//    lazy var backgroundSession: Foundation.URLSession = {
//        let configuration = URLSessionConfiguration.background(withIdentifier: (Bundle.main.infoDictionary!["CFBundleIdentifier"] as! String))
//
//        configuration.httpMaximumConnectionsPerHost = 1 // PJR 180105
//
//        self._backgroundSession = Foundation.URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
//
//        return self._backgroundSession!
//    }()
//    lazy var backgroundSession: Foundation.URLSession = Foundation.URLSession(configuration: URLSessionConfiguration.background(withIdentifier: (Bundle.main.infoDictionary!["CFBundleIdentifier"] as! String)), delegate: self, delegateQueue: nil)

Most uploading is on our office WiFI.

Intermittently (maybe half the requests) fail (errors can occur with the device unlocked and app in foreground, no backgrounding) with the following response as received by func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data)

Code Block HTML<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Verb</h2>
<hr><p>HTTP Error 400. The request verb is invalid.</p>
</BODY></HTML>

Apparently that error is thrown by Window's HTTP.SYS layer before it even hands it off to the Sharepoint web site in IIS, and from its HTTPERR logs, it seems to claim that no HTTP method was passed in the request.

#Software: Microsoft HTTP API 2.0
#Version: 1.0
#Date: 2021-05-26 04:21:11
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri sc-status s-siteid s-reason s-queuename
2021-05-26 06:52:10 10.50.65.22 59212 10.50.51.11 443 - - - 400 - Verb -

If I change the code to simply use a non-background session everything works as expected. I can also confirm from logging that the authentication challenge handler is called by the background transfer.

In the console for the device (even with CFNETWORK_DIAGNOSTICS=3) I can't really see (maybe my inexperience at this level of debugging/troubleshooting though) what exact request is being sent to the server to cause the problem. I also tried rvictl & tcpdump however all I seem to see is the TLS packets which I haven't yet researched on how to try and inspect further.

Can anyone offer suggestions as to what I could look at? As I mentioned, a non-background URLSession has no issues, and I only get these issues (and only ~50% of the time) using a background URLSession (even though app is completely in foreground, although I understand a background system service).

Thanks
Peter

Boost

Answer 1

DTS Engineer OP

Apple

May ’21

CFNetwork diagnostics won’t help here. That only applies to the process where you’ve set the environment variable, and in the case of an NSURLSession background session the actual networking is done by a system process (nsurlsessiond).

Have you tried a debugging HTTP proxy?

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 2

iRyder OP

May ’21

OK the plot thickens somewhat. With a HTTP proxy enabled (I used Charles on the Mac when simulator debugging, and tried both Charles and Proxyman on an iOS device when doing a tethered debug session), then I experience no errors with the background uploads. As soon as I turn off the proxy I start getting the intermittent "400 - Invalid Proxy" errors. I don't understand how the man-in-the-middle proxy would be doing to help it work...maybe timing or something?

Here is the Charles log of the successful uploads. The "Errors" here are just the 401 - Unauthorised challenges, otherwise after auth the upload works:

Details of a successful image "PUT":

0

Answer 3

DTS Engineer OP

Apple

May ’21

With a HTTP proxy enabled … then I experience no errors with the background uploads.

Yeah, that’s not unheard of. A proxy will ‘normalise’ the HTTP request stream, and that can mask an error caused by a misunderstanding between the client and the server.

Unfortunately this means you’ll have to debug this on the server side. You need to be able to see the traffic on the ‘wire’ to decide whether this is an iOS problem or a server problem. Unfortunately, you’ve no way to do this on iOS [1] so either:

You need the equivalent of CFNetwork diagnostic logging on the server.
You need to get the server’s private key and use that undo the TLS.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

[1] For a background session. For a standard session you can use CFNetwork diagnostic logging but, as you’ve noted, this problem doesn’t show up in standard sessions.

0