How to stream upload recorder audio as multipart message to a http/2 endpoint?

Hi!


I am implementing the Alexa Voice Service in a native iOS app (swift 2.3) and i'm struggling with the http/2 implementation.


The AVS documentation states the following:

"The request sent to AVS is a multipart message: the first part a JSON-formatted object, the second part binary audio captured by the microphone. We encourage streaming (chunking) captured audio to the Alexa Voice Service to reduce latency; the stream should contain 10ms of captured audio per chunk (320 bytes)."


To be a bit more clear: I want to upload (stream) the audio to AVS while I'm still recording the user's voice.


I already succeeded in uploading a full sound sample and sending that to AVS as a whole. However, the streaming implemtation is what I need and I have no idea how to accomplish this.


Can somebody give me some guidance ?


Thank you!!!

Replies

I presume you’re asking about the networking side of this, not the audio recording side. I can help with the former but not the latter.

You can use

URLSession
to do a streamed upload by creating the upload task with
uploadTask(withStreamedRequest:)
. The session will then call the
urlSession(_:task:needNewBodyStream:)
delegate method to get an input stream for the upload body. It will then open and read (asynchronously) data from that stream, uploading it as it becomes available.

IMPORTANT It’s possible for the session to call your delegate method more than once. For example, if the request fails with an authentication error, the session may automatically send the request again, and that will need a new body stream.

You can set up this input stream in two ways:

  • As the input side of a stream pair (via

    Stream.getBoundStreams(withBufferSize:inputStream:outputStream:)
    ), whereupon writing data to the output side of the stream will make it appear on the input side, and hence cause it to get uploaded
  • As an

    InputStream
    subclass

IMPORTANT Remember that the session reads the data from the stream asynchronously. If you choose to implement the second option, you must support asynchronous reads properly.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi!

First of all, thank you so much for showing interest in the question! I posted this topic on several other forums but people just ignored me there..

Ì looked into the uploadTask(withStreamedRequest:) function you talked about in your reply. I grasp the concept of the delegate methods but one thing I could not understand was how the multipart message is going to work. I tried to make it work today but kept getting 400 responses from AVS.

Here is what I tried to do today:

To keep things simple, I pre-recorded a sound sample containing "hi" and used that to get some response out of AVS. This is the function I used to build the request body

func uploadSoundSample() -> Void {
        let configuration = URLSessionConfiguration.default
        let session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
        let request = NSMutableURLRequest(url: URL(string: "https:/
        let boundry = UUID().uuidString
        let bodyData = NSMutableData()
     
        bodyData.append("--\(boundry)\r\n".data(using: String.Encoding.utf8)!)
        bodyData.append("Content-Disposition: form-data; name=\"metadata\"\r\n".data(using: String.Encoding.utf8)!)
        bodyData.append("Content-Type: application/json; charset=UTF-8\r\n\r\n".data(using: String.Encoding.utf8)!)
        bodyData.append(getJSONData().data(using: String.Encoding.utf8)!)
        bodyData.append("\r\n".data(using: String.Encoding.utf8)!)
        bodyData.append("--\(boundry)\r\n".data(using: String.Encoding.utf8)!)
        bodyData.append("Content-Disposition: form-data; name=\"audio\"\r\n".data(using: String.Encoding.utf8)!)
        bodyData.append("Content-Type: application/octet-stream\r\n\r\n".data(using: String.Encoding.utf8)!)
     
        request.setValue("Bearer \(Settings.TOKEN)", forHTTPHeaderField: "authorization")
        let contentType = "multipart/form-data; boundary=\(boundry)"
        request.setValue(contentType, forHTTPHeaderField: "content-type")
        request.httpMethod = "POST"
        request.httpBody = bodyData as Data
        request.httpBodyStream = InputStream(url: URL(fileURLWithPath: Bundle.main.path(forResource: "hi", ofType: "wav")!))
        let task = session.uploadTask(withStreamedRequest: request as URLRequest)
        task.resume()
    }

    fileprivate func getJSONData() -> String {
        return "{\"context\":[{\"header\":{\"namespace\":\"Alerts\",\"name\":\"AlertsState\"},\"payload\":{\"allAlerts\":[],\"activeAlerts\":[]}},{\"header\":{\"namespace\":\"AudioPlayer\",\"name\":\"PlaybackState\"},\"payload\":{\"token\":\"\",\"offsetInMilliseconds\":0,\"playerActivity\":\"IDLE\"}},{\"header\":{\"namespace\":\"Speaker\",\"name\":\"VolumeState\"},\"payload\":{\"volume\":100,\"muted\":false}},{\"header\":{\"namespace\":\"SpeechSynthesizer\",\"name\":\"SpeechState\"},\"payload\":{\"token\":\"\",\"offsetInMilliseconds\":0,\"playerActivity\":\"FINISHED\"}}],\"event\":{\"header\":{\"namespace\":\"SpeechRecognizer\",\"name\":\"Recognize\",\"messageId\":\"messageId-123\",\"dialogRequestId\":\"dialogRequestId-321\"},\"payload\":{\"profile\":\"NEAR_FIELD\",\"format\":\"AUDIO_L16_RATE_16000_CHANNELS_1\"}}}"
    }

And these are the delegate methods I've implemented:

func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void) {
        let inputStream = InputStream(url: URL(fileURLWithPath: Bundle.main.path(forResource: "hi", ofType: "wav")!))
        completionHandler(inputStream)
    }
  
    func urlSession(_ session: URLSession, task: URLSessionTask, didSendBodyData bytesSent: Int64, totalBytesSent: Int64, totalBytesExpectedToSend: Int64) {
        print("body data sent")
    }
  
    func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {
        print("task done sending data: \(error)")
    }
    func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) {
        print("got data")
    }
  
    func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive response: URLResponse, completionHandler: @escaping (URLSession.ResponseDisposition) -> Void) {
        print("got response: \(response)")
    }

I guess I'm still doing a couple of things wrong but I have limited understanding of Swift networking in general 😉

I'm looking forward to your answer!

Greets from Belgium!

You’re definitely getting mixed up here. When you use

urlSession(_:task:needNewBodyStream:)
that stream has to yield the entire HTTP body. You’re trying to use two streams, one that contains the
multipart/form-data
preamble, which you set up via the
httpBodyStream
property, and another that contains the bulk data for that form, which you set up via
urlSession(_:task:needNewBodyStream:)
. That won’t work. You need one stream that yields the entire body.

For a simple test you could:

  1. Remove the

    httpBody
    and
    httpBodyStream
    setup code from
    uploadSoundSample()
  2. Implement

    urlSession(_:task:needNewBodyStream:)
  3. In that implementation:

    1. Generate the

      multipart/form-data
      preamble using the
      bodyData
      code you currently have in
      uploadSoundSample()
    2. Load

      hi.wav
      into another data value
    3. Concatenate the two data values

    4. Build a stream from the result

Once you have this basic test working you can then go on to investigate using an indefinite stream.

btw Given that you’re using Swift 3 you should skip NSMutableData and use a Swift

Data
value. You can make this mutable by declaring it using
var
.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi!

I modified the code to meet the requirements of your previous reply. I now get 200 responses from AVS! 🙂


However, AVS sends back data containing the binary audio to play in my app but I never receive the data in the

urlSession:didReceive data
function. Is this normal behaviour ?


Just for reference: here is the code I have now:


fileprivate let boundary = UUID().uuidString
   
    func uploadSoundSample() -> Void {
        let configuration = URLSessionConfiguration.default
        let session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
        let request = NSMutableURLRequest(url: URL(string: "https:/
        let contentType = "multipart/form-data; boundary=\(boundary)"
       
        request.setValue("Bearer \(Settings.TOKEN)", forHTTPHeaderField: "authorization")
        request.setValue(contentType, forHTTPHeaderField: "content-type")
        request.httpMethod = "POST"
       
        let task = session.uploadTask(withStreamedRequest: request as URLRequest)
        task.resume()
    }
func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void) {
        var bodyData = Data()
        do {
            let audioData = try Data(contentsOf: URL(fileURLWithPath: Bundle.main.path(forResource: "hi", ofType: "wav")!))
          
            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Disposition: form-data; name=\"metadata\"\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Type: application/json; charset=UTF-8\r\n\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append(getJSONData().data(using: String.Encoding.utf8)!)
            bodyData.append("\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Disposition: form-data; name=\"audio\"\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Type: application/octet-stream\r\n\r\n".data(using: String.Encoding.utf8)!)
          
            bodyData.append(audioData)
            bodyData.append("\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
        } catch let error {
            print("something went wrong: \(error)")
        }
        let inputStream = InputStream(data: bodyData)
        completionHandler(inputStream)
    }


Thanks!

However, AVS sends back data containing the binary audio to play in my app but I never receive the data in the

urlSession:didReceive
data function.

How did you determine that the server actually send you any data in that case?

Is this normal behaviour ?

I would expect you to get the ‘did receive data’ call with the body data of the response.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Well, before posting this thread, I already played around a bit with the

NSUrlSession
class. Back then, the AVS communication was implemented like this:
let uploadTask = self.session.uploadTaskWithRequest(request, fromData: bodyData){
(data:NSData?, response:NSURLResponse?, error:NSError?) -> Void in
     //handle response

}
uploadTask.resume()


When this function threw a response, the audio of AVS was contained in the data parameter of the function. That's why I expect to receive data in the current implementation with the delegate methods but that ain't happening.


I'm looking forward to your answer! 😉

So, to confirm, you've implemented the

urlSession(_:dataTask:didReceive:)
delegate callback and it’s not getting called, right?

Weird. I tried this out here in my office and, while I saw some weird stuff, I definitely get the response data back from the server.

Consider the test code pasted in at the end of this post. When I run

startIdentity()
, it prints this:
2017-05-02 10:01:35.288 xxsi[2506:145728] success, status: 200
2017-05-02 10:01:35.288 xxsi[2506:145728] response: {
    args =    {
    };
    data = "";
    files =    {
    };
    form =    {
        greetings = "Hello Cruel World!";
        salutations = "Goodbye Cruel World!";
    };
    headers =    {
        Accept = "*/*";
        "Accept-Encoding" = "gzip, deflate";
        "Accept-Language" = "en-us";
        Connection = close;
        "Content-Length" = 383;
        "Content-Type" = "multipart/form-data; boundary=Boundary-0DFADA5A-6C2F-4FD9-9E92-CCBB2A100278";
        Host = "httpbin.org";
        "User-Agent" = "xxsi/1.0 CFNetwork/811.4.18 Darwin/16.5.0";
    };
    json = "<null>";
    origin = "88.97.8.212";
    url = "https://httpbin.org/post";
}

As expected:

  • The server (

    httpbin.org
    ) has sent me some JSON that describes the
    POST
    request I sent it; I correctly received and printed that
  • The server has correctly parsed my form data (not the

    form
    element within the JSON)

Now look at when happens when I run

startChunked()
:
2017-05-02 10:01:40.811 xxsi[2506:145728] data chunk: <7b0a2020 …>
2017-05-02 10:01:40.811 xxsi[2506:145728] success, status: 200
2017-05-02 10:01:40.812 xxsi[2506:145728] response: {
    args =    {
    };
    data = "";
    files =    {
    };
    form =    {
    };
    headers =    {
        Accept = "*/*";
        "Accept-Encoding" = "gzip, deflate";
        "Accept-Language" = "en-us";
        Connection = close;
        "Content-Type" = "multipart/form-data; boundary=Boundary-0DFADA5A-6C2F-4FD9-9E92-CCBB2A100278";
        Host = "httpbin.org";
        "Transfer-Encoding" = Chunked;
        "User-Agent" = "xxsi/1.0 CFNetwork/811.4.18 Darwin/16.5.0";
    };
    json = "<null>";
    origin = "88.97.8.212";
    url = "https://httpbin.org/post";
}

There’s two things to note here:

  • As before, the server has sent me some JSON and I’ve been able to parse and print it

  • The server did not recognise my form data.

AFAICT I’ve done nothing wrong here. Looking at a packet trace of what I sent on the wire, the only difference between the two requests is the transfer encoding. The identity request uses no transfer encoding and the chunked request uses a transfer encoding of

Chunked
. This is both expected (NSURLSession doesn’t know the length of the body stream and thus has to use chunked encoding) and correct (HTTP 1.1 requires that the server support chunked encoding). However, it’s clear that
httpbin.org
is not handling this properly.

Coming back to your situation, I’m not sure what this means for you. To do a streamed upload you’re going to have to use chunked encoding. I presume that the server you’re talking to will accept that, but it’s hard to know for sure because I don’t control the server.

IMO your best way forward here is to:

  • Learn how to use packet tracing technologies so you can see exactly what’s happening on the wire

  • Find a working client (perhaps the server’s SDK includes some sample code) and compare its on-the-wire behaviour to that of your client’s

I discuss this idea in depth in my Debugging HTTP Server-Side Errors post.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"
import UIKit

class MainViewController : UITableViewController, URLSessionDataDelegate {

    var session: URLSession!
    var accumulated = Data()

    override func viewDidLoad() {
        super.viewDidLoad()
        let config = URLSessionConfiguration.default
        config.requestCachePolicy = .reloadIgnoringLocalCacheData
        self.session = URLSession(configuration: config, delegate: self, delegateQueue: OperationQueue.main)
    }

    override func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
        switch (indexPath.section, indexPath.row) {
            case (0, 0): self.startIdentity()
            case (0, 1): self.startChunked()
            default: fatalError()
        }
        self.tableView.deselectRow(at: indexPath, animated: true)
    }

    func startIdentity() {
        self.session.uploadTask(with: MainViewController.testRequest, from: MainViewController.testBody) { (data, response, error) in
            MainViewController.print(error: error, response: response, body: data)
        }.resume()
    }

    func startChunked() {
        self.accumulated.removeAll()
        self.session.uploadTask(withStreamedRequest: MainViewController.testRequest).resume()
    }

    func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void) {
        let bodyStream = InputStream(data: MainViewController.testBody)
        completionHandler(bodyStream)
    }

    func urlSession(_ session: URLSession, dataTask: URLSessionDataTask, didReceive data: Data) {
        NSLog("data chunk: %@", data as NSData)
        self.accumulated.append(data)
    }

    func urlSession(_ session: URLSession, task: URLSessionTask, didCompleteWithError error: Error?) {
        MainViewController.print(error: error, response: task.response, body: self.accumulated)
    }

    static let testRequest: URLRequest = {
        var result = URLRequest(url: URL(string: "http://httpbin.org/post")!)
        result.httpMethod = "POST"
        result.setValue("multipart/form-data; boundary=Boundary-0DFADA5A-6C2F-4FD9-9E92-CCBB2A100278", forHTTPHeaderField: "Content-Type")
        return result
    }()

    static let testBody: Data = {
        var result = Data()

        func add(_ string: String) {
            result.append(string.data(using: String.Encoding.utf8)!)
        }

        // empty preamble
        add("\r\n") 
        add("--Boundary-0DFADA5A-6C2F-4FD9-9E92-CCBB2A100278\r\n") 
        add("Content-Disposition: form-data; name=\"greetings\"\r\n") 
        add("Content-Type: text/plain; charset=UTF-8\r\n") 
        add("\r\n") 
        add("Hello Cruel World!") 
        add("\r\n") 
        add("--Boundary-0DFADA5A-6C2F-4FD9-9E92-CCBB2A100278\r\n") 
        add("Content-Disposition: form-data; name=\"salutations\"\r\n") 
        add("Content-Type: text/plain; charset=UTF-8\r\n") 
        add("\r\n") 
        add("Goodbye Cruel World!") 
        add("\r\n") 
        add("--Boundary-0DFADA5A-6C2F-4FD9-9E92-CCBB2A100278--\r\n") 
        add("\r\n") 
        //empty epilogue

        return result
    }()

    static func print(error: Error?, response: URLResponse?, body: Data?) {
        if let error = error as NSError? {
            NSLog("failed, error: %@ / %d", error.domain, error.code)
        } else {
            let response = response! as! HTTPURLResponse
            let body = body!

            NSLog("success, status: %d", response.statusCode)

            if let root = try? JSONSerialization.jsonObject(with: body, options: []) {
                NSLog("response: %@", "\(root)")
            } else {
                NSLog("response not parsed")
            }
        }
    }
}

Hi eskimo!


I searched for the cause of the

didReceivedata
function not being called and found out that I just forgot to call the completion handler in the
didreceiveresponse
function.. 😁 To be clear: I receive the data now, so that's working!


Now there is one part missing to make all of this work.. We are now sending prerecorded sound samples but how do we get to stream the sound of the voice recorder directly to the request. You mentioned it could be possible with something called a stream pair ?


How could I implement this in my current request code/configuration ?


PS. I really appreciate the amount of effort and time you spend on this thread!

I receive the data now, so that's working!

Yay!

… how do we get to stream the sound of the voice recorder directly to the request. You mentioned it could be possible with something called a stream pair ?

There are two approaches you can use:

  • Subclass

    InputStream
  • Stream pair

The stream pair approach is the simplest to described:

  1. Call

    Stream.getBoundStreams(withBufferSize:inputStream:outputStream:)
    to create a bound pair of streams
  2. Pass the input stream to the upload task

  3. Open the output stream

  4. Write data to it

That data will show up on the input stream, whereupon it’ll be read by the upload task and sent to the server.

Subclassing

InputStream
sounds pretty straightforward but it’s actually quite tricky because you have to deal with run loops. The basic idea is:
  1. Create a subclass of

    InputStream
    that returns your audio data
  2. Instantiate that and pass it to the upload task

The upload task will then open that stream, read data from it, and send it to the server. The complication here is that the upload task will read the stream asynchronously, by scheduling it on a run loop, and correctly implementing that is a bit tricky (in fact, it wasn’t even supported prior to iOS 7).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Currently i am recroding the audio data into a temp file and then fetching data from that file in form of NSData (while setting up request to upload).

Same as above


func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void) {
        var bodyData = Data()
        do {
            let audioData = try Data(contentsOf: URL(fileURLWithPath: Bundle.main.path(forResource: "hi", ofType: "wav")!))

            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Disposition: form-data; name=\"metadata\"\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Type: application/json; charset=UTF-8\r\n\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append(getJSONData().data(using: String.Encoding.utf8)!)
            bodyData.append("\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Disposition: form-data; name=\"audio\"\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Type: application/octet-stream\r\n\r\n".data(using: String.Encoding.utf8)!)

            bodyData.append(audioData)
            bodyData.append("\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
        } catch let error {
            print("something went wrong: \(error)")
        }
        let inputStream = InputStream(data: bodyData)
        completionHandler(inputStream)
    }


How can I write audio data directly to my request body also uploading it at the same time??

I am using AVAudioRecorder for recording audio into a temp file.


    let settings : [String : AnyObject] = [
            AVFormatIDKey: Int(kAudioFormatLinearPCM),
            AVSampleRateKey: samplerate,
            AVNumberOfChannelsKey: noOfChannel,
            AVEncoderBitRatePerChannelKey : bitRate,
            AVEncoderAudioQualityKey: AVAudioQuality.High.rawValue
        ]
      
        do {
            let url = NSURL(fileURLWithPath: tempFilename)
            self.mRecorder?.deleteRecording()
            self.mRecorder = try AVAudioRecorder(URL: url, settings: settings)
            self.mRecorder?.prepareToRecord()
            self.mRecorder?.record()

        } catch {
            throw RecorderSetUpError.Error("Something went wrong when audiorecorder is setup")
        }


Thanks and Regards.

I can’t help you with the audio side of this. I’m going to assume that you have some way to get chunks of audio in a format that your server will accept. If not, you’ll need to ask questions about that over in Media > AVFoundation (Audio).

On the networking front the basic strategy for uploading a stream of data is as follows:

  1. Create an upload that uses a streamed body by calling

    uploadTask(withStreamedRequest:)
    .
  2. Once you resume the request, URLSession will call

    urlSession(_:task:needNewBodyStream:)
    to get a body stream for the request.
  3. You have two choices for creating this body stream:

    • You can subclass

      InputStream
    • You can create a bound stream pair.

    I’ll discuss the details of each below.

  4. Your implementation of the

    urlSession(_:task:needNewBodyStream:)
    method must call the completion handler with that input stream.

Subclassing

InputStream
is simple in the synchronous case but your subclass must support async reads. This should be possible on recent systems but it’s quite tricky. I generally recommend the bound stream pair approach.

Creating a bound stream pair is a bit tricky from Swift because the API to do so,

getBoundStreams(withBufferSize:inputStream:outputStream:)
, uses a bunch of ‘out’ parameters. I generally wrap that API as follows:
extension Stream {
    func boundPair(bufferSize: Int) -> (inputStream: InputStream, outputStream: OutputStream) {
        var inStream: InputStream? = nil
        var outStream: OutputStream? = nil
        Stream.getBoundStreams(withBufferSize: bufferSize, inputStream: &inStream, outputStream: &outStream)
        return (inStream!, outStream!)
    }
}

Once you have a bound stream pair your can pass the input stream to the completion handler (step 4 above) and then you ‘own’ the output stream. You should run the stream asynchronously, writing data to it in chunks as that data gets recorded. Any data you write to the stream will be sent as a chunk of the HTTP body (that is, using the

chunked
Transfer Encoding).

The output stream here is no different from any other output stream, and there’s tonnes of examples of how to run an output stream asynchronously.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks @eskimo for details you have mentioned here.


Yes i am already recording and uploading to the server and getting 200 in response from my server. But its like uploading a pre-recorded audio data to the request, using similar code like this one -


    func uploadSoundSample() -> Void {
        let configuration = URLSessionConfiguration.default
        let session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
        let request = NSMutableURLRequest(url: URL(string: "https:/
        let contentType = "multipart/form-data; boundary=\(boundary)"

        request.setValue("Bearer \(Settings.TOKEN)", forHTTPHeaderField: "authorization")
        request.setValue(contentType, forHTTPHeaderField: "content-type")
        request.httpMethod = "POST"

        let task = session.uploadTask(withStreamedRequest: request as URLRequest)
        task.resume()
    }

and providing reuest body via NSInputStream in the delegate "needNewBodyStream". As you can see the audio data is coming from a file (already recorded with correct audio format).

func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void) {
        var bodyData = Data()
        do {
            let audioData = try Data(contentsOf: URL(fileURLWithPath: Bundle.main.path(forResource: "hi", ofType: "wav")!))

            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Disposition: form-data; name=\"metadata\"\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Type: application/json; charset=UTF-8\r\n\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append(getJSONData().data(using: String.Encoding.utf8)!)
            bodyData.append("\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Disposition: form-data; name=\"audio\"\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("Content-Type: application/octet-stream\r\n\r\n".data(using: String.Encoding.utf8)!) 

            bodyData.append(audioData)
            bodyData.append("\r\n".data(using: String.Encoding.utf8)!)
            bodyData.append("--\(boundary)\r\n".data(using: String.Encoding.utf8)!)
        } catch let error {
            print("something went wrong: \(error)")
        }
        let inputStream = InputStream(data: bodyData)
        completionHandler(inputStream)
    }

I’m going to assume that you have some way to get chunks of audio in a format that your server will accept :

So do you mean that now i should get the chunks in real time from the recorder and pass it to same inputStream in that delegate method??
or do you mean i should i write body(wiithout audio part) before - task.resume()
and when the delegate gets called then i should pass the audio part via inputstream (not sure if we can pass body in different streams i.e httpBody/HttpsBodyStream and rest in "needNewBodyStream" delegate via NSInputStream).


Also i had this one doubt, previously i was using "uploadTaskWithRequest" for same request and now this - "uploadTaskWithStreamedRequest" .
Although request and responses are working just fine, just wondering i was checking logs in my delegate method -


    func URLSession(session: NSURLSession, task: NSURLSessionTask, didSendBodyData bytesSent: Int64, totalBytesSent: Int64, totalBytesExpectedToSend: Int64) {
     
        print("bytesSent = \(bytesSent)")
        print("totalBytesSent = \(totalBytesSent)")
        print("totalBytesExpectedToSend = \(totalBytesExpectedToSend)")    
    }


which is almost same (in both cases - uploadTaskWithRequest and StreamRequest), still same chunks of data are being sent. Is it expected behaviour, or should there be some difference between the dataBytes sent over the network?


Thanks & Regards!!

Can you start a new thread for this? The indentation here is getting in the way of me reading your post.

Also, perhaps you can use this as an opportunity to describe the overall big picture of your project, because I’m not sure I’ve got a good handle on that right now.

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Thanks @eskimo


Sure will do that!!


Thanks and Regards.

Hi eskimo and vivekmp_ios.


Could you provide the link to the new thread? I'm trying to do something very similar, that is take microphone samples and send them via chunks. Thank you!