Deploying a Postgres client app: where is libpq.tbd?

Hi,


I'm writing a small Postgres SQL database client app in Objective C for macOS Sierra. Just to be clear: my app will be the SQL client that will allow you to type in SQL queries and see result sets by sending the queries to a separate Postgres server instance (which may be local or remote and has nothing to do with my app).


My question: if my app is installed on a fresh/clean Mac running macOS Sierra that has no version of Postgres or Xcode installed, where will the Postgres drivers be found? Are they built in to macOS Sierra? Will they be wrapped up inside my app when it is compiled and linked? I need to be able to distribute my app to non-developers who won't have either Xcode or Postgres installed on their Macs.


I'm adding libpq.tbd as a "Linked Frameworks and Libraries" to my project's General page, that's how I'm "including" the Postgres drivers in my app's Xcode project.


On my build machine, I have a version of Postgres server installed so that the libpq headers (.h files) are available at /usr/local/Cellar/postgresql/9.5.4/include and can be seen in Xcode's headers search path. However, those were installed by Homebrew, which isn't guaranteed to be on an end user's machine. By mousing over libpq.tbd in Xcode, I see that it is located at /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/lib/libpq.tbd . But that's inside the Xcode app, so where will it be on a Mac that doesn't have Xcode or Postgres installed but does have my app installed?


I guess I'm not sure what a .tbd file is and if libpq is being treated as a dynamic library or is being compiled in, wholesale and statically, into my app at link time.


Any ideas?


Apple?


Thanks!

Replies

OK, I see:


bash> ls -1l /usr/lib/libpq.*

-rwxr-xr-x 1 root wheel 144K Dec 2 2015 /usr/lib/libpq.5.6.dylib*

lrwxr-xr-x 1 root wheel 15B Nov 29 2015 /usr/lib/libpq.5.dylib@ -> libpq.5.6.dylib

lrwxr-xr-x 1 root wheel 15B Nov 29 2015 /usr/lib/libpq.dylib@ -> libpq.5.6.dylib


So does that mean that libpq.5.6.dylib (or some variant) will be installed on those Macs running macOS Sierra even if they don't have either Xcode or Postgres server/Postgres dev or buildtools installed on them? Please confirm. :-)


I thought /usr/lib was a "protected location" under SIP in 10.11 and later and I don't remember monkeying with /usr/lib myself.

Bump. Apple, any response? Thanks.

macOS uses

libpq
internally but that is not considered public API. You can tell by looking at the macOS 10.12 SDK, wherein you’ll find no
libpq
headers (specifically, there’s no
<libpq-fe.h>
).

Note It is odd, and misleading, that the macOS 10.12 SDK still includes the

libpq
stub library (
libpq.tbd
and friends). Please feel free to file a bug about that.

To use

libpq
you’ll have to build it from source and then include that built library in your app (either statically linked in, or as a
.dylib
that’s bundled inside your app). I don’t have specific advice on how to do that, but you’ll find it’s a common subject of blog posts and so on (typically in the context of more popular libraries, like OpenSSL).

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"

Hi Quinn,


Thanks so much for replying!


When you say "libpq ... is not considered public API" what do you mean? I can see /usr/lib/libpq.5.6.dylib on any base macOS install (independent of whether I have installed Xcode or postgres) so am I not allowed to use that dynamic libraryin /usr/lib? As a "unixy/linuxy" OS, isn't the tradition that I can use anything in /usr/lib? Especially wityh SIP on macOS, I can pretty much count on that dynamic lib being there right?


Well, what if I link against something in my /usr/local homebrew install of postgres, could I still, as an App Store Mac app, use /usr/lib/libpq.5.6.dylib? Or would that be a no-no?


I'll try to hunt down directions for compiling libpq as a static library, and post those here for others to find. But I understand it can be difficult to compile it some times due to SSL deps, etc.


Thanks again and take care!

I can see /usr/lib/libpq.5.6.dylib on any base macOS install …

Right. But the test for whether an API is considered public is threefold:

  • Does it have headers in the platform SDK? (A)

  • Does it have a stub library (that

    .tbd
    file) in the platform SDK? (B)
  • Does it have an implementation library on the platform? (C)

libpq
passes B and C but does not pass A.

Well, what if I link against something in my /usr/local homebrew install of postgres, could I still, as an App Store Mac app, use /usr/lib/libpq.5.6.dylib?

No. The correct solution is to:

  1. build the library from source, either as a dynamic or static library

  2. compile your app using the resulting headers

  3. link your app with the resulting library

  4. if you decided to use a dynamic library, bundle that dynamic library within your app (in

    Foo.app/Contents/Frameworks/
    )

By way of explanation, Apple uses many open source libraries on our platforms, and those libraries naturally get placed in

/usr/lib
. Whether we support those libraries as public API depends on a bunch of factors, but a really critical one is binary compatibility. If we give an open source library the public API ‘blessing’, that library has to be binary compatible from release to release. Without that, we end up in the unenviable place of having to either:
  • not update the library consistently

  • include

    /usr/lib/libFoo.1.dyib
    , then
    /usr/lib/libFoo.2.dyib
    , then
    /usr/lib/libFoo.3.dyib
    , and so on

Historically we’ve been a bit lax about this rule (which has got us into trouble, see below) but these days we’re much stricter.

The worst case scenario for this was OpenSSL. We included OpenSSL in early releases of macOS and added its headers to the macOS SDK. That has caused serious problems because OpenSSL does not provide binary compatibility. So the OpenSSL in

/usr/lib
is a really old version that we can’t update (modulo hand-crafted binary compatibility security updates).

Eventually we resolved this by:

  1. deprecating the OpenSSL in

    /usr/lib
  2. requesting that folks who need OpenSSL build it from source and bundle it within their app (per the above)

For libraries that have never been public API (like

libpq
), step 2 isn’t a request (-:

Share and Enjoy

Quinn “The Eskimo!”
Apple Developer Relations, Developer Technical Support, Core OS/Hardware

let myEmail = "eskimo" + "1" + "@apple.com"
This is now very relevant as a fresh install of Big Sur does not include /usr/lib/libpq.dylib so anything that previously linked via libpq.tbd will fail to run as it can't load the dynamic library.