C Compiler Error Buffer overlap causes Crash

When I copy an address in a buffer to another location in the same buffer the application crashes. It logs: "detected source and destination buffer overlap" EXC_BAD_INSTRUCTION (code=EXCI386_INVOP, subcode=0x0)


for example

char myBuffer[1024];


while myBuffer[0] == ' '

strcpy(myBuffer, myBuffer + 1); or alternately

strcpy(myBuffer, &myBuffer[1]);


This should strip leading spaces from a buffer of text. This is perfectly legitimate in all versions of C and has been OK in all versions of XCode and all C compilers since C was invented.


This is in XCode 8.3.3 (8E3004b)


Seriously Apple, you just broke hundreds of millions of lines of source code. Is there any way for me to fix this other than re-writing the text editing code in thousands of appplications and hundreds of millions of lines of source code?


Can I use some other compiler?


The Code Analyser doesn't find these Can I get it to?


Can I fix the build settings so the LLVM compiler won't break this?


C IS NOT Swift. Apple doesn't need to save me from myself. PLEASE HELP ME!

Accepted Reply

Sure, but that's not the point. There is a ***** programmer error in your example, because the code lies to the compiler. (The parameters promise that there are 5 bytes available to write into, starting from myBuffer+4, and this simply isn't true.)


The point is that the number of bytes in the source string doesn't affect the safety of writing into the destination. That means an attacker can't force a buffer overflow by altering the source string's length.


FWIW, there are other ways that strncpy can go wrong. For example, if the source string length is the same the size parameter, the destination will not be null terminated. In that sense, strncpy is a string-to-buffer copy, not a string-to-string copy.

Replies

More really bad news: The only compiler option is LLVM 8.1 so until Apple fixes this I'm screwed.

More bad news: Setting the Apple LLVM 8.1 - Code Generation, No Common Blocks flag to NO won't fix this.

Even more bad news: there are 8 differen Apple LLVM 8.1 - Language, C Language Dialect options. None of these will fix this.


It looks like I would have to un-install XCode 8.3.3 and start using an earlier version untill the brilliant engineers at Apple fix this. Can anyone tell me how to do this? Time Machine maybe?

A quick internet search indicates that you are mistaken. For example:


pubs.opengroup.org/onlinepubs/9699919799/functions/strcpy.html


"If copying takes place between objects that overlap, the behavior is undefined."

"If copying takes place between objects that overlap, the behavior is undefined."


C buffers aren't objects. This is not in Kernigan and Richie or any other book about C. I have worked with C compilers extensively since 1988. They all allow this including Think C, Code warrior and all previous versions of XCode before 8.3.3. I'll look at the ANCII and ISO standards but I think that this is bogus.

>> C buffers aren't objects.


In C lingo (that is, in the jargon of the C language spec), an "object" doesn't refer to objects in the sense used in object-oriented languages like Objective-C. Instead, it means something along the lines of "entity known to the compiler".


By all means, look into the standards. My guess is that the undefined behavior got standardized at some point (perhaps C99). It's quite possible that Think C and CodeWarrior had some other behavior or guarantees, since those languages likely used an earlier standardization, or were officially non-standard. It's also possible that the compiler in Xcode 8.3.3 doesn't do the copy any differently from earlier versions, but changed to just report the issue.


Note that "overlapping" strcpy cannot be safe without a run-time check for the nature of the overlap (to choose between a pointer-incrementing or pointer-decrementing copy, or something more complicated). In that case, the run-time must call strlen on both strings to find their length, which is a massive performance penalty. That in itself is enough to make the overlap check implausible.

"...Xcode 8.3.3 doesn't do the copy any differently from earlier versions, but changed to just report the issue."


No, XCode 8.3.3 crashes the App when it encounters this.

I looked at five books I have about C and none of them mentions the overlap issue. I searched for "ANSI/ISO C strcpy" and the on-line documentation is all over the map - some references say that the buffers can't overlap but others either don't mention it or even say that they can so be careful because compilers can't and don't check this. XCode's help documents this as an Objective C function and says that it was deprecated as of 10.6. XCode deprecated functions warnings are turned on in my project settings but they don't seem to actually work like they did in earlier versions. The quickest way for me to fix this is to replace all instances of strcpy() with something like this:


void myStrcpy(char *dest, char *source)

{

size_t sourceLen = strlen(source);

memcpy(dest, source, sourceLen);

* (dest + sourceLen) = '\0'; //null terminate

}


Interestingly strncpy() isn't mentioned as having the same restriction since obviously you could ***** up by carelessly using this function as well.

Well, there's this, which which purports to be the C11 spec and looks pretty official:


www.open-std.org/JTC1/SC22/WG14/www/docs/n1570.pdf


and says:


"7.24.2.3 The strcpy function

"The strcpy function copies the string pointed to by s2 (including the terminating null character) into the array pointed to by s1. If copying takes place between objects that overlap, the behavior is undefined."


Regarding the Apple deprecation, that's presumably because all C stdlib functions that don't have an explicit buffer length parameter are vulnerable to buffer overflow exploits. Functions like strncpy don't have that vulnerability.


Regarding your fix, given that there's a question about the behavior of overlapping source and destination buffers, you shouldn't use memcpy either, since it has the same undefined behavior as strcpy. The correct function to use is memmove, which is documented to behave correctly. (See 7.24.2.1 and 7.24.2.2 in the above document.)

You could certainly cause an array overflow with strncpy. For example:


char myBuffer[5] = "ABCD\0";


strncpy(myBuffer[4], myBuffer, 5);

Sure, but that's not the point. There is a ***** programmer error in your example, because the code lies to the compiler. (The parameters promise that there are 5 bytes available to write into, starting from myBuffer+4, and this simply isn't true.)


The point is that the number of bytes in the source string doesn't affect the safety of writing into the destination. That means an attacker can't force a buffer overflow by altering the source string's length.


FWIW, there are other ways that strncpy can go wrong. For example, if the source string length is the same the size parameter, the destination will not be null terminated. In that sense, strncpy is a string-to-buffer copy, not a string-to-string copy.

You could easily cause an array overflow with memmove and memcpy as well. Anyway I'll have to rewrite some of my code to make this new implementation of strncpy work. I'll mark your reply as correct.

HM....