Memory leak in Swift's FileHandler.read

Question

_Michael_ OP

Created May ’21

Replies 5

Boosts 0

Views 1.8k

Participants 5

Hi,

I try to to read a file from Swift using class FileHandle and function

Code Block Swift@available(macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4, *)
public func read(upToCount count: Int) throws -> Data?

This works fine in principle. However, I'm faced with a memory leak. The data allocated in the returned Data buffer is not freed. Is this possibly a bug in the underlying Swift wrapper?

Here is my test function, which reads a file block by block (and otherwise does nothing with the data).

Code Block Swiftfunc readFullFile(filePath: String) throws {
    let blockSize = 32 * 1024
    guard let file = FileHandle(forReadingAtPath: filePath) else {
        fatalError("Failed to open file")
    }
    while (true) {
        guard let data = try file.read(upToCount: blockSize) else {
            break
        }
    }
}

If I call the function in a loop, and watch the program's memory consumption, it's obvious that the memory goes up in each loop by the size of the read file.

Can anybody confirm this behavior or knows how to fix it or if there's possibly already a bug issue opened at Apple?

My environment:

macOS 11.3.1 Big Sur
XCode 12.5

Best,
Michael

Boost

Answer 1

Claude31 OP

May ’21

You always read the same data, definitely. Your loop is just to check if memory use grows ?

What happens when you exit the func ? Is memory released ?

0

Answer 2

_Michael_ OP

May ’21

As a already wrote: I e.g. use this function in a loop. Memory is not released.

0

Answer 3

OOPer OP

May ’21

I e.g. use this function in a loop. Memory is not released.

Many of the intermediate data consumed in Apple's APIs are autoreleased.
Have you tried using autoreleasepool?

0

Answer 4

omarzl2 OP

Feb ’22

I had the same issue, the method read(upToCount:) was growing the memory usage, as suggested the autoreleasepool solved the issue, you could rewrite it like this:

func readFullFile(filePath: String) throws {
    let blockSize = 32 * 1024

    guard let file = FileHandle(forReadingAtPath: filePath) else {
        fatalError("Failed to open file")
    }

    while (true) {
        var data: Data?
        try autoreleasepool {
            data = try file.read(upToCount: blockSize)
        }
        guard let data = data else {
           break
        }
    }
}

0

Answer 5

vldalx OP

Mar ’23

I've also tried to read a large file using the approach above and faced the same issues with memory consumption. My implementation is similar to that one.

autoreleasepool did't solve the issues.

It appears the new async/await API consumes the same large amounts of memory

let reader = try FileHandle(forReadingFrom: url)
for try await line in reader.bytes.lines {
    &#x2F;&#x2F; some code processing `line`
}

0