8 Replies
      Latest reply: Jan 26, 2017 3:52 AM by eskimo RSS
      igorland Level 1 Level 1 (0 points)

        Hello. I am trying to figure out the best way to convert an array of bytes to IEEE 754 float with big endian.

         

        This is what I have for Integer:

         

        func bytesToInt(Bytes b:[UInt8]) -> Int
        {
                let bigEndianValue = b.withUnsafeBufferPointer {($0.baseAddress!.withMemoryRebound(to: UInt32.self, capacity: 1) { $0 })}.pointee
                let value = UInt32(bigEndian: bigEndianValue)
              
                return Int(value);
        }
        

         

        I am not sure whether something like this is correct:

         

        let bigEndianValue = b.withUnsafeBufferPointer {($0.baseAddress!.withMemoryRebound(to: Float.self, capacity: 1) { $0 })}.pointee
        

         

        Besides, how to make it a big endian IEEE float?

         

        Thank you so much!

        • Re: Array of bytes to IEEE 754 big endian
          QuinceyMorris Level 7 Level 7 (3,980 points)

          The problem with Float is that a byte-swapped bit sequence may not be a valid Float, which means you may not be able to pass the "big-endian" version around like you can with UInt32.

           

          My suggestion would be to treat the 32 bits as a big-endian UInt32 initially, convert it to native endianness using your existing UInt32 code, then use the memory-rebind function a second time to reinterpret that as a Float.

          • Re: Array of bytes to IEEE 754 big endian
            OOPer Level 7 Level 7 (3,545 points)

            `Float` does not have `init(bigEndian:)`, but has `init(bitPattern:)`.

             

            func bytesToFloat(bytes b: [UInt8]) -> Float {
                let bigEndianValue = b.withUnsafeBufferPointer {
                    $0.baseAddress!.withMemoryRebound(to: UInt32.self, capacity: 1) { $0.pointee }
                }
                let bitPattern = UInt32(bigEndian: bigEndianValue)
              
                return Float(bitPattern: bitPattern)
            }
            
            let value = bytesToFloat(bytes: [0x3F,0x80,0x00,0x00])
            print(value) //->1.0
            
              • Re: Array of bytes to IEEE 754 big endian
                igorland Level 1 Level 1 (0 points)

                Thank you, OOPer. Can I ask one more question?

                 

                I also tried to do the following code to convert an array of bytes to Integer using this code:

                 

                func bytesToInt(Bytes b:[UInt8]) -> Int
                {
                        let n = b.count
                        var resultTemp : UInt8 = 0;
                
                        for i in 0..<n
                        {
                            resultTemp = resultTemp<<8
                            resultTemp |= (b[i] & 0xFF);
                        }
                        return Int(resultTemp)
                }
                

                 

                In C++ and Java, it works well. But in Swift, it gives me an error:

                 

                fatal error: shift amount is larger than type size in bits


                in this line:


                resultTemp = resultTemp<<8
                

                 

                I wonder why...

                 

                Thank you!

                  • Re: Array of bytes to IEEE 754 big endian
                    eskimo Apple Staff Apple Staff (7,190 points)

                    In C++ and Java, it works well. But in Swift, it gives me an error:

                    C-based languages (and presumably Java) have the concept of integer promotion.  That feature has a bunch of complex rules, but one effect is that arithmetic is done in a default integer type if the inputs are smaller than that type.

                    Swift doesn’t have this concept.  Arithmetic is done based on the type of the inputs, and you have to explicitly convert any values of the wrong type.  This means that line 8 in your code is being done as a UInt8, and you can’t shift that left by 8 because it’s only 8 bits long.

                    You should, instead, do your work in the large type you’re looking for, explicitly promoting the small integer type when you read it.  For example:

                    func bytesToInt(Bytes b:[UInt8]) -> UInt32
                    { 
                        precondition(b.count <= 4)
                        let n = b.count 
                        var resultTemp : UInt32 = 0; 
                    
                        for i in 0..<n 
                        { 
                            resultTemp = resultTemp<<8 
                            resultTemp |= (UInt32(b[i]) & 0xFF); 
                        } 
                        return UInt32(resultTemp) 
                    }
                    

                    Note that I’ve changed your return type to UInt32.  In Swift Int is pointer sized, so can’t hold an unsigned 4-byte integer on 32-bit platforms (older iOS devices, all watchOS devices).

                    Share and Enjoy

                    Quinn “The Eskimo!”
                    Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                    let myEmail = "eskimo" + "1" + "@apple.com"

                      • Re: Array of bytes to IEEE 754 big endian
                        igorland Level 1 Level 1 (0 points)

                        Thank you very much, eskimo. The only modification I will make is to change it to UInt64, since the library I am creating is supposed to convert binary data to Integers using various number of bytes, ranging from one to eight.

                         

                        Here is another question, if I may. So, I am using the code that you suggested with a return type of UInt64. Is the following code safe for 64-bit and 32-bit devices?

                         

                        /*
                         * @discussion Converts an array of bytes to an integer.
                         * @param b Vector of bytes
                         * @return Integer represented by an array of bytes
                         */
                           
                        func bytesToInt(Bytes b:[UInt8]) -> UInt64
                        {
                                var resultTemp : UInt64 = 0;
                                let n = b.count
                           
                                for i in 0..<n
                                {
                                    resultTemp <<= 8
                                    resultTemp |= (UInt64(b[i]) & 0xFF);
                                }
                           
                                return UInt64(resultTemp)
                        }
                        
                        
                            /*
                             * Converts an array of bytes to an integer with the first bit used as a sign.
                             * @param b Array of bytes
                             * @return Integer represented by an array of bytes
                             */
                            func bytesToIntWithSign(Bytes b:[UInt8]) -> Int
                            {
                                var result = 0
                                let numberOfBits = b.count * 8;
                               
                                let intTemp = bytesToInt(Bytes: b);
                          
                                let sign = (intTemp) >> UInt64(numberOfBits-1);
                          
                                var intRevised = intTemp;
                                intRevised &= ~(1<<(UInt64(numberOfBits-1)));
                          
                                if (sign == 1)
                                {
                                    result = -(Int(intRevised));
                                }
                                else
                                {
                                    result = Int(intRevised);
                                }
                          
                                return result;
                         }
                        

                         

                        Best regards,

                         

                        Igor

                          • Re: Array of bytes to IEEE 754 big endian
                            eskimo Apple Staff Apple Staff (7,190 points)

                            bytesToInt(Bytes:) looks OK.  I’d recommend the following changes, but they’re all stylistic:

                            • The parameter label should be bytes not Bytes (note the case)

                            • Given the Swift naming conventions, I’d use something like uint64(from:)

                            • I’d add a precondition like the one I showed in my previous post; without it you’ll crash (sometimes) with a UInt64 overflow, which is much harder to debug

                            • Your for loop can use 0..<b.count; unlike C, Swift doesn’t re-evaluate the upper bound each time around the loop

                            • In fact, you could just iterate through the bytes directly (using for thisByte in b)

                            • You don’t need the & 0xFF because a UInt8 to UInt64 conversion won’t sign extend

                            • Your return statement can just return resultTemp because it’s already a UInt64


                            With regards bytesToIntWithSign(Bytes:), I’m not exactly sure what you’re trying do here.  The fundamental problem here occurs when you get an UInt64 whose value won’t fit into an Int, when Int is 32 bits.  AFAICT you’re trying to truncate to 32 bits, and I’m as a loss trying to understand why you’d want to do that.  For example, if I have a sequence like this:

                            7f ff ff ff 00 00 00 01
                            

                            I’d expect your code to produce a value of 1, which seems pretty illogical (I’d expect an error indicating overflow, or a clipped value of Int.max).

                            Regardless, the code as it current stands doesn’t work with the above sequence on 32-bit hardware.  numberOfBits is 64, so your masking is a no-op.

                            If you can explain what you want this code to do in the under- and overflow cases, I’d be happy to suggest some options.

                            Share and Enjoy

                            Quinn “The Eskimo!”
                            Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                            let myEmail = "eskimo" + "1" + "@apple.com"

                              • Re: Array of bytes to IEEE 754 big endian
                                igorland Level 1 Level 1 (0 points)

                                Thank you, eskimo. This was helpful. I have revised my code to ensure that any potential big numbers (like number of bytes in the binary file) are returned as Int64.

                                 

                                The only question remaining is when I want to extract part of my binary data using the following:

                                 

                                let rangeStart = currentFileByte // Int64 or UInt64
                                let rangeEnd = currentFileByte + sectionLength
                                let bytesTemp : [UInt8] = Array(bytes[rangeStart..<rangeEnd])
                                

                                 

                                It does not allow me to use Int64 for ranges ("Cannot subscript a value of type [UInt8] with an index of type 'CountableRange<Int64>"). So, do I absolutely need to cast it to Int with a possibility of overflow in 32-bit environment?

                                 

                                Thank you!

                                  • Re: Array of bytes to IEEE 754 big endian
                                    eskimo Apple Staff Apple Staff (7,190 points)

                                    So, do I absolutely need to cast it to Int

                                    Yes.

                                    with a possibility of overflow in 32-bit environment?

                                    Overflow isn’t the real problem here.  Remember that your bytes array has to fit in memory, so if you get a value that’s too big you were going to die on the array bounds check anyway.

                                    What is a problem is bounds checking.  When you get a value out of your binary format, you have to check that it’s within bounds.  It’s hard to offer concrete advice on that front without knowing more about how your file is structured.

                                    Share and Enjoy

                                    Quinn “The Eskimo!”
                                    Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                    let myEmail = "eskimo" + "1" + "@apple.com"