Dangling pointer warnings

I have a bunch of recommendations for you:

• The Peril of the Ampersand (1)

• WWDC 2020 Session 10648 Unsafe Swift

• WWDC 2020 Session 10167 Safely manage pointers in Swift

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

(1) Sorry about the formatting here. It’s caused by a bug in the way that content was migrated from the old forums (r. 64523131). I will eventually fix it but it’s not possible right now (r. 65175233).

Quinn - thanks for the links. The presentations were very informative, and I understand more about the risks, rewards and philosophy of using pointers in Swift (though I’m still not entirely clear on why my pointer is dangling nor exactly what to do about it.)

I didn’t set out to use ‘unsafe’ pointers - I set out to get data from a file into a local struct. I rooted around in the dev docs and turned up the FileHandle object; overview: “You use file handle objects to access data associated with files, sockets, pipes, and devices. For files, you can read, write, and seek within the file. “ Yeah, that’s the ticket. The .seek method gets me to where I need to be to get at the data I want and the .readData(ofLength: ) method reads a specified number of bytes of data from the file specified URL and loads them into a Data buffer for me. How do I get the data from the Data buffer into my local variables? The .copyBytes function seemed like a good candidate! How do I specify *where* to copy the data? With a UnsafeMutablePointer<UInt8>!  And my mother always told me to “be safe”. (Interestingly enough, both the FileHandle “readFrom” and “seek” methods have been deprecated. (!?) I’ll worry about that later, I’m just hacking to see how I could satisfy a series of use cases at this point, and to get some hands-on experience with the tools and technology.) 

I admit, I’m a rank novice with developing in Swift using Xcode. But I’m a seasoned developer (laid down my first lines of Fortran code in 1970) with experience on many platforms using a variety of languages, including a long stint with C coding close to the metal back in the early to mid-80’s. I started developing for Macintosh in the late 80’s (using a framework called Splash) and worked on a commercial product for Newton in the late 90’s - I even gave a presentation on that app at WWDC c.1998. I retired some years ago but have really missed development work, so decided I would take a crack at developing an iOS app. I first approached the tools about 4 years ago, but life intervened and that goal got delayed until now. I’ve been at it for a couple of months (my Covid pastime) but have found the learning curve daunting. Maybe this dog is too old to learn new tricks.

I’m not even sure if the FileHandle fork was the right road to take for file I/O. The dev docs are a rich source for reference, but aren’t necessarily the best resource to get an idea about how to approach a specific problem. I’ve always found (working & simple) sample code to be the best jump start, but have had little luck finding any useful examples for doing file I/O - maybe I picked a bad topic to start with. I think I’ll park this experiment and move on to other frameworks, for now.

If anyone has experience with doing file access with binary files and would be able to offer some insight as to how you approached the problem I would be most grateful to hear from you!

This mod works and the compiler isn't warning me of dangling pointers. Safer??   

private func loadFileHeader( fileHandle: FileHandle, fileHeader: inout FileHeader ) {

        withUnsafeMutableBytes( of: &fileHeader.fileCode ) { pointer in
            let data = fileHandle.readData(ofLength: sizeOfInt32)
            data.copyBytes(to: pointer)
        }
        fileHeader.fileCode = fileHeader.fileCode.bigEndian

        SeekTo(fileHandle: fileHandle, offset: 24)
        withUnsafeMutableBytes( of: &fileHeader.fileLength ) { pointer in
            let data = fileHandle.readData(ofLength: sizeOfInt32)
            data.copyBytes(to: pointer)
        }
        fileHeader.fileLength = fileHeader.fileLength.bigEndian

//etc.

Safer??

Safer, certainly. Safe, probably not (-:

Is FileHeader defined in Swift? Or in C? This technique is only safe if FileHeader is defined in C because Swift does not provider any way to fix the layout of a struct.

Personally, I avoid this whole problem by reading files byte by byte. See this post for some code, and also my second post on that thread for more context.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

Quinn - Thanks for the post link. I got more out of studying that thread for 20 minutes than I got out of untold hours scratching around in the dev docs. Seriously.

The file header struct is defined in Swift. BUT… 

The header data in the file is composed of 9 UInt32 values followed by 8 Double values, or:

   9 ea. UInt32 * 4 bytes/UInt32 + 8 ea. Double * 8 bytes/Double = 36 bytes + 64 bytes = 100 bytes

I verified that this data is indeed packed into exactly 100 bytes by inspecting the file with a hex editor.

The corresponding Swift struct dutifully defines 9 UInt32 values followed by 8 Doubles. I printed out the MemoryLayout of the struct and got:

Code Block 
	size   = 104
	align  = 8
	stride = 104

The 104 makes perfect sense since the alignment is 8 (due to the doubles), requiring a padding of 4 bytes after the 9 (odd) UInt32 values to force the 8 byte alignment for the doubles. This is, I think, the reason for your caution RE the layout of the struct - I can’t just blast 100 bytes of disk data into address of the 104 byte struct or the doubles would all be misaligned. BUT - in the last snippet I sent I’m setting the pointer to the address of the target struct property, not the base address of the struct, then I copy the correct number of bytes for the target data type (i.e. 4 or 8) from the Data buffer to the struct property (then clean up the byte ordering, as needed, for correct endian order). This seems to address your concern about memory mapping of the struct v. the packed bytes in the disk image. Safe or Not Safe?

The other big takeaway from your post is I shouldn’t nickel & dime the disk input an int or double at a time, but should pull in large logical blocks at once (e.g. the whole file header in a single read) and then copy the bytes into the target structs one property at a time. The test data file I’m using is ~1.5 Mb so I’d tend to avoid loading the whole file up front. (But then I’m from the generation where memory was so preciou$ we skimped on carrying the ’19’ in the year ‘1967’, thus setting up the eventual Y2K panic! Who knew we'd still be around when the bill came due on that decision!)

I’ll also keep in mind that FileHandle.seek(toOffset:) and FileHandle.readData(ofLength:) are, among others, going away some time in the future when I get down to actually implementing this logic for real.

Thank you again for your help. You’ve got me going again so I can focus on the next hurdle.

Peace!

The file header struct is defined in Swift. BUT…

There is no but here. Swift currently provides no way to fix the layout of structs and thus relying on that layout is undefined behaviour. It might work with your current Swift compiler with your current settings with your current target platforms, but it could stop working if any of those vary. If you care about the long-term maintainability of your code, do not use this technique.

If you want to continue with this overall approach then your easiest option is to define the structure in C. That way you can rely on C’s struct layout rules (which are less strong than you might expect but you do at least get something).

However, I stand by the advice in that other thread. The best way to deal with this stuff is to separate your in-memory working representation from your on-disk representation. If you do that up front, you don’t have to scramble to fix things when, say, the size of Int changes, or you suddenly have to run your code on a big endian system, or whatever.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

OK - in this flavor I avoid any byte moves into the fileHeader struct at all by using a local variable in the getUInt32 method, and then use the returned variable to assign to the struct property. (I also tell the getUInt method which endian the value from the file is in and have it do the byte swaps if it's not the same as the current host's preferred endian order.)

Safe yet?

Code Block 
				 private func loadFileHeader(fileHandle: FileHandle, fileHeader: inout fileHeader) {
 
						 let data = fileHandle.readData(ofLength: FILE_HEADER_SIZE)
						 if data.count == FILE_HEADER_SIZE {
						
											fileHeader.fileCode = getUInt32( from: data, at: 0, endian: Endian.big )
											fileHeader.fileLength = getUInt32( from: data, at: 24, endian: Endian.big )
											fileHeader.version = getUInt32( from: data, at:28, endian: Endian.little )
								 /* etc. */
							 } else { 
						 /* not a valid file err */
							}
			 }
	
				 private func getUInt32(from: Data, at: Int, endian: Endian) -> UInt32 {
						 var unsignedInt32: UInt32 = 0
						 _ = withUnsafeMutableBytes( of: &unsignedInt32 )	{ pointer in
								 from.copyBytes(to: pointer, from: at..<at+4)
								if endian != getHostEndian() {
										 swapBytes(a: &pointer[0], b: &pointer[3])
										 swapBytes(a: &pointer[1], b: &pointer[2])
								 }
						 }
						 return unsignedInt32
				 }
				 private func swapBytes(a: inout UInt8, b: inout UInt8) {
						 let t = a
						 a = b
						 b = t
				 }
				 public enum Endian {
						 case big
						 case little
				 }
				 private func getHostEndian() -> Endian {
						 var value: UInt = 0x76543210
						 var endian: Endian = Endian.big
						 _ = withUnsafeBytes(of: &value) { pointer in
								 if pointer[0] == 0x10 {
										 endian = Endian.little
								 }
						 }
						 print("Endian on this machine is \(endian)")
						 return endian
				 }

Safe yet?

Yes.

The only thing I’d change here is how you’re dealing with endianness. You can get rid of the swapBytes(…) and getHostEndian methods by changing your code to use the endian initialisers. For example:

Code Block  
private func getUInt32(from: Data, at: Int, endian: Endian) -> UInt32 {
		 var unsignedInt32: UInt32 = 0
		 … do the `withUnsafeMutableBytes(…)` / `copyBytes(…)` dance …
		 switch endian {
		 case .big:		return UInt32(bigEndian:		unsignedInt32)
		 case .little: return UInt32(littleEndian: unsignedInt32)
		 }
}

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

Many thanks for all your help Quinn. I learned a lot struggling through this exercise. (Maybe this old dog can learn a few new tricks!)