A non-destructive URL.lines?

Question

carlynorama OP

Created Feb ’23

Replies 2

Boosts 0

Participants 2

I am working on a Server-Sent-Event parser which has newlines as part of the protocol:

https://html.spec.whatwg.org/multipage/server-sent-events.html#event-stream-interpretation

If the line is empty (a blank line)
Dispatch the event, as defined below.

Unfortunately, when using:

let (asyncBytes, _) = try await self.session.bytes(from: sseStreamURL)

 for try await line in asyncBytes.lines {
    //parse line
 }

The iterator actually skips empty lines entirely, which seems destructive and therefore can't be the only way? Can it? With split there is a omittingEmptySubsequences: false, but I didn't find any trace of anything like that for .lines. I couldn't find the var definition in the Swift repo either, if anyone could point me at it.

I did write something that works for now, but I'd love if anyone had any comments on how to improve it / point me at any exiting language features that I missed.

extension AsyncSequence where Element == UInt8 {
    var allLines:AsyncThrowingStream<String, Error> {
        AsyncThrowingStream { continuation in
            Task {
                var buffer:[UInt8] = []
                var iterator = self.makeAsyncIterator()
                while let byte = try await iterator.next() {
                    // b/c 10 == \n
                    if byte != 10 { buffer.append(byte) } 
                    else {
                        if buffer.isEmpty { continuation.yield("") } 
                        else {
                            if let line = String(data: Data(buffer), encoding: .utf8) { continuation.yield(line) } 
                            else {
                                //Is there a different AsyncSequence error I could use? 
                                throw SSEError("allLines: Couldn't make string from [UInt8] chunk")
                            }
                            buffer = []
                        } 
                    }
                }
            }
        }
    }
}

usage:

for try await line in asyncBytes.allLines {
    //parse line
}

or of course also

var iterator = asyncBytes.allLines.makeAsyncIterator()
while let line = try await iterator.next() {
    //parse line
}

Boost

Answer 1

carlynorama OP

Feb ’23

Didn't want to leave that code up without the task closer:

extension AsyncSequence where Element == UInt8 {
    //Works.
    var allLines_v1:AsyncThrowingStream<String, Error> {
        AsyncThrowingStream { continuation in
            let bytesTask = Task {
                var accumulator:[UInt8] = []
                var iterator = self.makeAsyncIterator()
                while let byte = try await iterator.next() {
                    //10 == \n
                    if byte != 10 { accumulator.append(byte) }
                    else {
                        if accumulator.isEmpty { continuation.yield("") }
                        else {
                            if let line = String(data: Data(accumulator), encoding: .utf8) { continuation.yield(line) }
                            else { throw MastodonAPIError("allLines: Couldn't make string from [UInt8] chunk") }
                            accumulator = []
            }   }   }   }
            continuation.onTermination = { @Sendable _ in
                bytesTask.cancel()
    }   }   }

0

Answer 2

Regattaguru OP

Feb ’24

Thank you for this insight - I'm working on a DXF parser and blank lines are essential.

0