External Testers can only test builds that have passed Apple’s Beta Review process (similar to the App Store Review process). Each new version of the App must go through this process, however subsequent builds for the same version will not require review once the initial review process succeeds.
Internal Testers do not have this limitation.
Since it’s not exactly obvious, you can trigger a TestFlight build to be submitted for the Beta App Review process by adding an external tester to the build. Doing so will immediately change the status to Waiting for Review, once Apple reviews the beta version of the app the status will change again and you’ll get an email saying it’s ready for testing, and the previously selected External Testers will be able to access the build.