Problems with std libraries

I have a major issue that is blocking development.


The game I am working on use an archive system for handling assets based on stream operators.


This code works perfectly on Windows and Linux but fails on Mac.


Originally the code had a single string streamer...


Archive& operator <<(Archive& ar, std::string& val)


When I tried to link the game I got the error


Undefined symbol: ge::operator<<(ge::Archive&, std::__1::basic_string<char, std::__1::char_traits<char="">, std::__1::allocator >)


I checked around and other people had had this issue.They all swapped from libc++ to libstdc++. I cannot do that


So I added ...


Archive& operator <<(Archive &ar, std::basic_string<char,std::char_traits,std::allocator>val)


Just to get it to link, hoping the runtime code would perform as expected.


It doesn't.


The basic_string version is the method that is called and the problem with that is it creates a new string instead of writing to the supplied string.


So you cannot load any strings.


I also have had problems with templates.


template<> TMap<std::string, soundbuffer=""> UrlResourceCache::map;


Won't link. I had to do ...


template<typename T> TMap<std::string, T*> UrlResourceCache<T>::map;


I am finding it harder and harder to work in c++ on Mac


Has anyone come up with a solution for the archive issue?

Accepted Reply

Well actually it does work.


I eventually found the problem was that one of the games libraries, when I added it to the game project, changed include paths for the core library. This library contains all the archive handling.


So some of the code was including the correct version of core, some was looking at the unconverted (win32) version.


When I corrected the include path and removed the basic_string version of the serialiser, everything worked.


However the bug in libc++ remains. Taking a copy of a string before serialisation is a very bad idea for lot's of reasons.


I don't care now though as I have working code and can progress. I would love to know why xcode changed include paths though, that's a weird one.

Replies

Are you sure your Archive library is compiling correctly on a Mac? Was it developed for a gcc environment? Does it use GCC extensions? Are you compiling with GCC extensions enabled in whatever version of C++ you are using?


How is TMap defined? How is soundbuffer defined?

Yes everything is correct the issue is that this code.


Archive& operator<<(Archive& ar, PlayerState& s)
{
  int ver = 1;
  ar << ver;

  ar << s.name << s.campaignState;

  return ar;
}


Should call


Archive& operator <<(Archive& ar, std::string& val)



But it doesn't. It calls

Archive& operator <<(Archive &ar, std::basic_string<char,std::char_traits,std::allocator>val)


This method creates a NEW std::string , loads the contents from disk, then throws it away.


This code has been working fine for years on everything except mac.

Does anyone know how to display the stack contents in xcode?


What I am thinking of doing is finding the pointer to the string I am supplying to << on the stack, and in the case of reading from disk, use this instead of the string "val"


Would mean inserting some assembler, but it's the only fix I can think of.

That’s backwards. You are using an output operator. The type you are supplying should be const. It sounds like you want an input operator >> instead.

No,


It's an old trick we have been doing for years. You override the << operator so you can use the same code to load AND save data.


That way you never get out of sync.


You have a single serialise function that you use for reading and writing


For example.


Archive& operator <<(Archive& ar, char* val)
{
  uint len = 0;
  if (ar.IsWriting())
  {
       len = sysStrLen(val);
       ar << len;
  }
  else
       len = Arctor(ar);
  ar.Serialize(val, len+1);
  return ar;
}

You are going to have to learn a new trick.

After a lot of debugging I have tied down exactly what is happening.


The method for a std::string is being ignored due to namespace issues in libc++.


Instead it takes a copy of the std:string as a std::basic_string (actually probably std::_1::basic_string), and send that to the basic_string method.


Obviously this is not what I want as I read the data into the basic_string which is then thrown away and my std::string is never modified

John, that is not an option.


The options are ..


  1. Find a solution
  2. Spend thousands of pounds changing thousands of lines of code
  3. Drop MAc and IOS and just not support them


I obviously want to go with option 1, but if it comes down to it I know which one the managers are going to pick.

It is your only option. Your code is wrong. It was wrong the first day you wrote it. People who write software for a living live within a system of rules for languages and libraries. If the rules didn't matter, then all of it would fall apart quite quickly.


1. The solution is obvious. Write syntactically correct code. The problem goes away.

2. If you write syntactically incorrect code, you are responsible the time and effort required to correct the code.

3. That is an idle and pointless threat. Nobody cares whether you drop the Mac or not. Go for it. Just don't try to tell your Mac customers that Apple is to blame, because that is not true.


For a little background...


Many years ago, at least 21 years, if not more, there were a number of early C++ compilers. There was no standard for the language before 1998, so it was a bit of a wild west. No one really understood how C++ was supposed to work back then. Compilers were not very advanced. People had bad habits from C. One of the things that caused people problems were references. They were considered pointers, but prettier. Programmers abhor ugliness everwhere that it provides value and embrace it when it helps to obfuscate the meaning of code. Therefore, early C++ programmers embraced the reference as a way introduce side effects and eliminate those ugly * and & characters. After a couple of years dealing with side effects, the idea of const-correctness was born.


Unfortunately, there was already a lot of bad example code posted on the internet. That stuff never goes away. It just gets plagiaraized for eternity. Certain compiler-writers who shall remain nameless (namely GNU) achieved a sizeable market with their free compilers and "flexible" approach to standards. Anything would compile on GCC. If it didn't work on Solaris, MPW, Metrowerks, VC++, then that was somebody else's problem.


I was one of those clueless young programmers who tried to use references too often. I learned the error of my ways. But I also experienced the unfortunate consequences of being the only one on the project who knew the value of const correctness while other people are depending on those side effects. And I have even taken advantage of certain constructs that were legal at the time, but were later "corrected" by the standards committees. So I definitely understand your perspective, but there is no option here. C++ I/O operations are not flexible. You simply can't do it that way.

Well actually it does work.


I eventually found the problem was that one of the games libraries, when I added it to the game project, changed include paths for the core library. This library contains all the archive handling.


So some of the code was including the correct version of core, some was looking at the unconverted (win32) version.


When I corrected the include path and removed the basic_string version of the serialiser, everything worked.


However the bug in libc++ remains. Taking a copy of a string before serialisation is a very bad idea for lot's of reasons.


I don't care now though as I have working code and can progress. I would love to know why xcode changed include paths though, that's a weird one.

The C++ IO operators are function templates when used with strings. Here is how they are defined:


template <class CharT, class Traits, class Allocator>

std::basic_ostream<CharT, Traits>&

operator<<(std::basic_ostream<CharT, Traits>& os,

const std::basic_string<CharT, Traits, Allocator>& str);


template <class CharT, class Traits, class Allocator>

std::basic_istream<CharT, Traits>&

operator>>(std::basic_istream<CharT, Traits>& is,

std::basic_string<CharT, Traits, Allocator>& str);


If you try to do something non-standard, there is no way to predict what the compiler is going to do in order to generate an implemetation of these functions to match the types you have provided. I don't know what your "Archive" types are or how they are constructed. But you should follow the general IO architecture of the standard. That means using operator << with a const reference for output and operator >> with a mutable reference for input. You could instead override the member functions in the iostream classes for classes other than strings. Don't forget the sentry on input.


Xcode follows the C++ standard. If your code is working now and you are happy with it, fine. But if you are making up your own standard, you can expect these kinds of problems. It isn't because Xcode "changed" anything. Xcode is always changing and evolving to match how the standard changes and evolves. If you go your own way, it will be your responsibility to support your custom IO routes and game engine.

Hi John,


This is a very old technique that people have used for ever, however your reply did hilight what was causing my problem.


In the broken version of the archive class I had missed the & in my string handler. This caused all the problems as I was changing the code in a different class than the compiler was importing, and the code it should have been using was correct. That was why it was so hard to find.


In your reply you point out what I was expecting.


template <class CharT, class Traits, class Allocator>

std::basic_ostream<CharT, Traits>&

operator<<(std::basic_ostream<CharT, Traits>& os,

const std::basic_string<CharT, Traits, Allocator> &str);


However what I got was effectively ....


template <class CharT, class Traits, class Allocator>

std::basic_ostream<CharT, Traits>&

operator<<(std::basic_ostream<CharT, Traits>& os,

const std::basic_string<CharT, Traits, Allocator> str);


Note the single very important difference. One character can really ruin your day.


So what I actually received in the code was a copy of the string, not the string itself.


Had I actually got what we both expected, then it wouldn't have been a major problem, though in the end I am glad I found out what was the root cause rather than just sticking a plaster on it.


The bottom line is I made a mistake, but libc++ also is wrong, It should have been passing a reference to the string, instead it passed a new string, and I don't know why.


Cheers


Paul