Incorrect behavior for CC_SHA256 in Release configuration (Swift)

The return value for method
Code Block
public func CC_SHA256(_ data: UnsafeRawPointer!, _ len: CC_LONG, _ md: UnsafeMutablePointer<UInt8>!) -> UnsafeMutablePointer<UInt8>!

comes as incorrect. For the same input data, different hash values are output containing lot of empty data. This only happens in release configuration.

The below code easily reproduces the issue when run in Release configuration. Release configuration is crucial here since it is not visible in Debug configuration. The issue is also visible when not run concurrently (in a single thread) but the frequency of occurrence is very less. The below code is not the usage but concurrent execution helps to create the issue easily

Code Block    
func sha256(data: Data) -> Data {
    var digestBytes = [UInt8](repeating: 0, count: Int(CC_SHA256_DIGEST_LENGTH))
    let buffer = CC_SHA256(Array(data), UInt32(data.count), &digestBytes)!
    return Data(bytes: buffer, count: Int(CC_SHA256_DIGEST_LENGTH))
}
func testHashMismatch() {
    let rawInput = "Hello World!" // can be any input
    let input = Data(rawInput.utf8)
    let inputHash = self.sha256(data: input)
    print("inputHash: \(inputHash.base64EncodedString())")
    DispatchQueue.global(qos: .background).async {
        while (true) {
            let computedHash = self.sha256(data: input)
            if computedHash != inputHash {
                print("Value Mis-match !!! \(computedHash.base64EncodedString())")
            }
        }
    }
    DispatchQueue.global(qos: .background).async {
        while (true) {
            let computedHash = self.sha256(data: input)
            if computedHash != inputHash {
                print("Value Mis-match !!! \(computedHash.base64EncodedString())")
            }
        }
    }
}


sample output:
Code Block
inputHash: f4OxZX/x/FO5LcGBSKHWXfwtSx+j1ncoSt3SABJtkGk=
Value Mis-match !!! f4OxZX/x/FO5LcGBAAAAAAAAAAAAAAAAAAAAAAAAAAA=
Value Mis-match !!! f4OxZX8AAAAAAAAAAAAAAPwtSx+j1ncoSt3SABJtkGk=
Value Mis-match !!! AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
Value Mis-match !!! AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=

iOS: 13.x, 14.x
Swift: 5.3

The loop can be run for some value around 100,000 iterations but the mismatch occurrence varies and first occurrence is seen to vary from tens to hundreds to thousands, ten thousands of execution.

However, if the passed in value is used in the sha256(:) method defined above, the issue is not seen and behavior is as expected.
Code Block
return Data(bytes: digestBytes, count: Int(CC_SHA256_DIGEST_LENGTH))

The only documentation for CC_SHA256 from Apple seems to be CC_SHA256 which states that return value should be same as the passed in pointer.

All routines return 1 except for the one-shot routines ( CC_SHA1(), etc.), which return the pointer passed in via the md parameter.

PS: This code has been working for a long time but started giving this issue few months back.

Would like to know if a bug needs to be reported and what is the recommendation for the usage?
Answered by DTS Engineer in 640824022
This is a very innovative variant on the problems I describe in The Peril of the Ampersand. The value returned by CC_SHA256 is the digest pointer, that it, it’s equal to the pointer that you pass in to the third parameter. When you call CC_SHA256 with an UInt8 array, Swift only guarantees that the pointer it passes in will persist for the lifetime of the call. So, on line 4, when you go to create a Data from that pointer, it’s invalid.

The fix is really simple. Ignore the result from CC_SHA256 and change line 4 to this:

Code Block
return Data(digestBytes)


Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"
Accepted Answer
This is a very innovative variant on the problems I describe in The Peril of the Ampersand. The value returned by CC_SHA256 is the digest pointer, that it, it’s equal to the pointer that you pass in to the third parameter. When you call CC_SHA256 with an UInt8 array, Swift only guarantees that the pointer it passes in will persist for the lifetime of the call. So, on line 4, when you go to create a Data from that pointer, it’s invalid.

The fix is really simple. Ignore the result from CC_SHA256 and change line 4 to this:

Code Block
return Data(digestBytes)


Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"
@eskimo
Thanks for the prompt response.
Is this called out in the documentation anywhere ? I think this is something which is definitely not obvious and needs an explicit call out.

Is this called out in the documentation anywhere ?

Which bit?

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

Which bit?

That the pointer returned only remains valid for the execution of the call


The best documentation that I’m aware of (other than my post :-) is that two WWDC talks the Swift team gave this year:
I think you could make a reasonable argument for this to be covered in more depth in the documentation (either The Swift Programming Language or Using Imported C Functions in Swift or both). If you agree, you should file a bug with your thoughts on this.

Please post your bug number, just for the record.

Having said that, I think the better long-term strategy is to make it harder to make these mistakes:
  • The Swift compiler has got a lot better about detecting these problems. Unfortunately it wasn’t able to detect your specific case )-:

  • We should replace and deprecate APIs based on unsafe pointers. In this case, we’ve done the replacement part (you can use Apple CryptoKit to calculate SHA-256 digests) but not the deprecation part.

  • Personally I think we should remove these automatic pointer conversions from the language because they make it too easy to do the wrong thing. Unfortunately that’d break source code compatibility, something the Swift team is reluctant to do. If you want to get involved in that discussion, hope on over to Swift Evolution.

Share and Enjoy

Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@apple.com"

Personally I think we should remove these automatic pointer conversions from the language because they make it too easy to do the wrong thing.

Totally agree! Yes better would be to move to Apple's CryptoKit but cannot as of now due to minimum iOS 13 support.

I have reported on the feedback assistant. Problem ID: FB8804161

Incorrect behavior for CC_SHA256 in Release configuration (Swift)
 
 
Q