24 Replies
      Latest reply on Apr 15, 2019 2:17 AM by eskimo
      Rulovic Level 1 Level 1 (0 points)

        I am working on an application written in Qt for macOS environment. In order to generate release build i have set up external server.

        I am seeing this:

        After generating build, if I download and install the application on Applications folder, when opening it, first I see the popup asking if I am sure to open app downloaded from Internet, so I click Open and then app dies.

        App works fine if I open it from CommandLine in Terminal. Maybe is due to the fact there I don't see the popup asking if I am sure to open it.

        But I am kind of lost because I don't know where I can check logs or the error I am having

        Could someone help me please ?

        Thanks is advance

        • Re: Qt application for macos not being launched
          john daniel Level 3 Level 3 (380 points)

          Most of those open-source ports have a funky bundle structure that does not conform to what Xcode produces. So Gatekeeper blocks the primary executable and prompts you to confirm. But then the secondary executable is being launched via some shell script and that does not invoke Gatekeeper properly.

           

          The solution is to avoid those open-source UI frameworks on macOS.

          • Re: Qt application for macos not being launched
            Pereirinha98 Level 1 Level 1 (0 points)

            Same issue over here

            • Re: Qt application for macos not being launched
              eskimo Apple Staff Apple Staff (11,355 points)

              So only solution is to drop Qt?

              No.  The “funky bundle structure” that john daniel refers to presents deployment challenges, but it is possible to use a non-standard bundle structure and be compatible with modern versions of macOS [1].

              You need to look to your tools vendor for support here.  They should be keeping abreast of the latest platform developments and updating their tools accordingly.  For example, Oracle seems to do a good job of this on the Java front.  And if your tools vendor is not keeping up with platform developments, that’s something you need to factor into your tooling choices.

              Now, if you want to fix this for yourself — or, perhaps, cooperate with other members of the community that rely on the same tools — then my #1 recommendation is that you look at how your built binary complies with the rules in the Nested Code section of Technote 2206 macOS Code Signing In Depth.  That is by far the most common cause of problems like this.

              If you find that your built binary isn’t following the nested code rules, you may be able to fix that using the technique outlined in this post.

              Share and Enjoy

              Quinn “The Eskimo!”
              Apple Developer Relations, Developer Technical Support, Core OS/Hardware
              let myEmail = "eskimo" + "1" + "@apple.com"

              [1] In most cases.  Some things, like self-modifying apps, are just not supported.

                • Re: Qt application for macos not being launched
                  Rulovic Level 1 Level 1 (0 points)

                  But there something I dont understand here. If problem with signature was the problem application would never run, right? not only first time.

                  Because after dying first time then I can launch it without any problem

                    • Re: Qt application for macos not being launched
                      eskimo Apple Staff Apple Staff (11,355 points)

                      Yes, that is weird.  Gatekeeper does a bunch of extra checks on the first launch of your app on first launch, and it’s possible that these checks are triggering this failure.  Normally I’d expect that this failure would cause Gatekeeper to re-run its checks, but it’s possible that the failure occurs after Gatekeeper has recorded a success.

                      Does the failure generate a crash report?  If so, post it here and I’ll see if I can learn more.

                      Regardless, my previous point stands: A common cause of weird code signing / Gatekeeper / notarisation failures is incorrectly nested code, and that’s the place where I recommend that you start this investigation.

                      Share and Enjoy

                      Quinn “The Eskimo!”
                      Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                      let myEmail = "eskimo" + "1" + "@apple.com"

                        • Re: Qt application for macos not being launched
                          Rulovic Level 1 Level 1 (0 points)

                          I have checked in Library/Logs/DiagnosticReports and there are no crash reports there.

                          Looking at console logs displayed I have seen this line "LSExceptions shared instance invalidated for timeout. " I dont even know if it is relevant. Any idea related to that ?

                          Thanks

                            • Re: Qt application for macos not being launched
                              john daniel Level 3 Level 3 (380 points)

                              I doubt your app is crashing. I also doubt you are going to find any meaningful information in the logs. And I'm not sure what you mean by "looking at console logs displayed". That suggests you might be unaware of significant changes to the macOS logging infrastructure over the past few years. You can't use Console.app like you would have years ago. It has to be up and running before you do your test. Otherwise, you have to use the new "log" command line tool as well as its predicate language for searching for relevent log entries in the past. It's not a trivial task.

                               

                              When doing any kind of testing on Gatekeeper or other system-level interactions, I strongly recommend testing in a VM. A decent VM will support snapshots, making it easy to roll back to an inital state and test again.

                               

                              I'm not going to take the same diplomatic approach as eskimo. Qt is just a dead end on the Mac - end of story. If you want to keep using it, fine. Please see my earlier suggestions on how to do an end-around of Gatekeeper.

                               

                              That code signing documentation was first written in 2008 and has not been updated in over two years. In particular, it doesn't say anything about Notarization. About six months ago, Apple announced that Notarization was going to be mandatory. The other day, Apple finally announced when Notarization was going to be required and a number of popular developers are freaking out over it today. Or, if nothing else, they are enjoying a noticeable uptick in blog traffic .

                               

                              But my point is that it is going to be difficult to beat your old Qt app into conformance with modern, and constantly evolving, security practices and infastructures. In my opinion, I don't think it is worth that level of effort. It would better to just identify the bottleneck, Gatekeeper, and change your processes to avoid it. That's an easy, 5-minute fix.

                               

                              But if you do plan to distribute this app publicly, then a more thorough review is in order. Stripping out Qt would be a high-cost, but low-risk solution. Beating code signing into submission would be a moderate-cost, moderate-risk solution. Which is best for you? What other value, if any, does Qt bring to you? What other value would an alternative, but more mainstream approach, bring?

                                • Re: Qt application for macos not being launched
                                  Rulovic Level 1 Level 1 (0 points)

                                  I am using Qt because the app is cross platfom and so far it is not up to me to drop it.

                                  I have more info, after downloading the application if i do ls -la on it it has extended attributes. Checking them i see that app is marked as com.apple.quarantine which is logical as it was downloaded.

                                  If I remove the quarantine with xattr -dr com.apple.quarantine <path> app runs with no problem.

                                  Could you help me to see what is going on ?

                                    • Re: Qt application for macos not being launched
                                      john daniel Level 3 Level 3 (380 points)

                                      Yes, I realize why people might use Qt to try to get a free cross-platform app. However, nothing is free. You pay the cost with problems like this. Many popular open-source Mac ports have similar problems. There have been many attempts to develop a cross-platform framework for app development over the past 3 decades. All of them have failed. Every. Single. One.

                                       

                                      I say just write a little shell script that downloads your app via curl and then immediately runs that xattr command. That will get you over this problem until the next time Qt causes your app to fail. That will happen in about two months.

                                       

                                      If you want, you can inspect the app bundle and compare what is different between your Qt app and a native app. Most likely, there is some stub, possibly native app wrapper that immediately launches the Qt version. More likely, it does that via a shell script instead of a native stub. It could have done a fork/exec, or maybe an XPC, or maybe just launch an embedded helper to bootstrap the Qt version. All of those would have likely made it through Gatekeeper unscathed. But I'm just guessing.

                                       

                                      Unfortunately, there is no easy answer. Maybe I'm wrong and you can figure out what is going on with Gatekeeper. Maybe you can get it working without any hacks. But I still expect it all to fail again in some novel, unexpected way in 10.15. Maybe you might be able to hack that and get it working. Maybe the Qt folks will do that after several months. Had you posted an innocent question about using Qt years ago, I would have told you the same thing. Those things work until they fail. And when they fail, either you have to fix them or the community has to fix them. The community isn't very big and they don't care much for Macs to begin with.

                                        • Re: Qt application for macos not being launched
                                          Rulovic Level 1 Level 1 (0 points)

                                          Is there any way to see the output of GateKeeper in order to find out where & why it fails ???

                                            • Re: Qt application for macos not being launched
                                              eskimo Apple Staff Apple Staff (11,355 points)

                                              Is there any way to see the output of GateKeeper in order to find out where & why it fails ???

                                              You can use the spctl tool for this.  In verbose mode that tool may give you some hints as to what’s going wrong, but it’s not necessarily easy to interpret.

                                              Note that TN2206 discusses this tool in a section entitled Checking Gatekeeper Conformance.  I strongly recommend that you read that entire doc in detail.  It contains a wealth of knowledge of about how to resolve issues with a “funky bundle structure”, knowledge uncovered by a colleague here in DTS who spent many years of toiling on the code signing coal face.

                                              Share and Enjoy

                                              Quinn “The Eskimo!”
                                              Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                              let myEmail = "eskimo" + "1" + "@apple.com"

                                                • Re: Qt application for macos not being launched
                                                  Rulovic Level 1 Level 1 (0 points)

                                                  have run the three tools recommended in the site and according to them signature is valid:

                                                  codesign --verify --deep --strict --verbose=2

                                                  check-signature

                                                  spctl -a -t exec -vv

                                                   

                                                  What else can I check ?

                                                    • Re: Qt application for macos not being launched
                                                      eskimo Apple Staff Apple Staff (11,355 points)

                                                      What else can I check ?

                                                      Nothing immediately springs to mind.  To offer further advice I’d need to take a look at the structure of your app, and that’s not something I can do here on DevForums.  My recommendation is that you open a DTS tech support incident and we can pick things up there.

                                                      Make sure to reference this DevForums thread, just for context.

                                                      Share and Enjoy

                                                      Quinn “The Eskimo!”
                                                      Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                                      let myEmail = "eskimo" + "1" + "@apple.com"

                                                  • Re: Qt application for macos not being launched
                                                    john daniel Level 3 Level 3 (380 points)

                                                    If that's the route you want to go, I can't give you better advice than what eskimo is saying. I can tell you that said Tech Note 2206 includes a link to the "SignatureCheck" tool that may provide more insight than spctl. I'm just guessing though. I haven't tried to run it.

                                                     

                                                    I also suggest that your first step would be to thoroughly study the bundle structure of your app and see exactly how it is launching.

                                                      • Re: Qt application for macos not being launched
                                                        Rulovic Level 1 Level 1 (0 points)

                                                        Yep I already run SignatureCheck and it passes the checks. This is why I was asking. Could be possible that problem is not in Gatekeeper ?

                                                          • Re: Qt application for macos not being launched
                                                            john daniel Level 3 Level 3 (380 points)

                                                            The problem was never Gatekeeper. The problem was always the Qt app.

                                                              • Re: Qt application for macos not being launched
                                                                Rulovic Level 1 Level 1 (0 points)

                                                                I was not saying that problem was in Gatekeeper. I just said that after running all signture checks I got all of them passed. So maybe problem is not related to signature (i meant that when I said gatekeeper, sorry).

                                                                I thought about gatekeeper because if I remove manually the quarantine app runs.

                                                                Basically my question was what else I could check to find the problem.

                                                                I have checked also the folder structure and it is the same that regular macOS app.
                                                                I am reading to Notarize the app. Could that be possible solution ??

                                                                Thanks

                                                                  • Re: Qt application for macos not being launched
                                                                    john daniel Level 3 Level 3 (380 points)

                                                                    I was just assuming there was something strange about the folder structure based on what I have seen from other popular open-source apps that have been ported to the Mac. If you are saying your folder structure is the same as other Mac apps, then I'm totally off-base. Follow eskimo's suggestion to open a DTS ticket.

                                                                      • Re: Qt application for macos not being launched
                                                                        Rulovic Level 1 Level 1 (0 points)

                                                                        Just in case. This is my folder structure:

                                                                        Inside .app I have Contents folder. Inside Contents:

                                                                        Folder called _CodeSignature

                                                                        Folder called Frameworks with dylib and framework files

                                                                        Info.plist file

                                                                        MacOS folder with the main executable file and another executable (tool needed)

                                                                        PkgInfo file

                                                                        Plugins folder

                                                                        Resources folder

                                                                         

                                                                        This organization is correct, isn't it?

                                                                          • Re: Qt application for macos not being launched
                                                                            john daniel Level 3 Level 3 (380 points)

                                                                            The issue is "and another executable (tool needed)". There should only be one executable in that folder. I'm not sure what you mean by "tool needed". The "CFBundleExecutable" in the Info.plist file identifies the executable.

                                                                             

                                                                            In theory, you can have anything you want in that folder as long as "CFBundleExecutable" specifies the executable. That executable can do anything it wants, including launching other executables to do handle the app's logic.

                                                                             

                                                                            But that is a theoretical world, whereas we live in a practical world. Recent and upcoming versions of macOS may not respect that logic. Gatekeeper, for example, may cause problems. I can't say definitively that it will cause problems, but when you deviate from the norm (i.e. what Apple normally tests against) you raise your risk of encountering problems like this.

                                                                             

                                                                            If you do encounter problems, you have a number of choices:

                                                                            1) File an Apple "radar" bug report and hope for a fix. If a fix does arrive, which is very unlikely, you should expect it to take 1-2 years. If you are extraordinarily lucky, Apple could fix a bug in 1-2 minor releases. But for this, you have to demonstrate a significant failure of expected functionality and someone at Apple has to find that problem, recognize it, and decide that they really screwed something up and should fix it straightaway.

                                                                            2) Dig into the details of app launching and Gatekeeper and idenfify exactly how this is failing. Ideally, if you do #1, Apple would really appreciate it if you did #2 as well and sent your results in your #1 radar submission.

                                                                            3) Dig into the details of app launching and Gatekeeper and identify exactly how this is failing. Make necessary changes in your app to correct the problem. Ideally, you should also do #1, and include your results in yoru radar report, so that other people don't encounter this problem in the future. Maybe you should also submit your changes to Qt so that they can fix their system in case Apple does nothing.

                                                                            4) Don't use Qt. Regardless of platform, it is a good idea to separate your app's core logic from your app's user interface. If nothing else, this will give you a command-line version of your app as well as a GUI version. And maybe you'll get a stand-along framework that could be used in other apps, or perhaps as part of a web service. And if you are doing this, it isn't too difficult to write a native Mac user interface that wraps that underlying code. And maybe, for good measure, you write an iOS app that uses the aforementioned web service.

                                                                             

                                                                            With a moderate amount of effort, you should be able to figure out and fix the problem. But you have a very good chance of having even more serious problems with 10.15 comes out. And then, when the expected Arm version of 10.16 arrives, your app is completely dead and maybe you are out of a job.

                                                                             

                                                                            Or, you could do a little bit more work, eliminate your dependency on Qt, make your underlying functionality more reliable and more testable, and deploy to two additional popular platforms (iOS and the web). And you will be the one responsible for it all.

                                                                            • Re: Qt application for macos not being launched
                                                                              eskimo Apple Staff Apple Staff (11,355 points)

                                                                              MacOS folder with the main executable file and another executable (tool needed)

                                                                              Contents/MacOS/ is the canonically correct place to put helper tools.  As john daniel mentions, you should make sure that CFBundleExecutable points to your main executable.

                                                                              Plugins folder

                                                                              Resources folder

                                                                              The presences of these directories is fine, but you need to make sure that the contents of these is structured correctly.

                                                                              Again, I recommend that you open a DTS tech support incident so that I can help you out on a one-to-one basis.

                                                                              Share and Enjoy

                                                                              Quinn “The Eskimo!”
                                                                              Apple Developer Relations, Developer Technical Support, Core OS/Hardware
                                                                              let myEmail = "eskimo" + "1" + "@apple.com"