I am also in the same boat. The page after you generate the Auth Key states to double click the .key file to add it to Keychain Access but as you mentioned, I only receive a .p8 file which will not open with Keychain Access.
See https://forums.developer.apple.com/thread/64073 about reading this with openSSL. Depending on your usecase the keychain actually won't do you that much good as you need to integrate this in your push service client implementation.
https://developer.apple.com/library/content/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Chapters/APNsProviderAPI.html#//apple_ref/doc/uid/TP40008194-CH101-SW1 has some information about actually integrating this (see the section about provider authentication tokens). The website jwt.io has some more information on how to actually construct a valid token and links to libraries generating these.
From what I gathered, basically you need to construct the listed JSON structures (with the current time as issuing date) and your key ID / team ID and sign it with the given private key. This then can go into the Authorization header of your APNS HTTP/2 requests as Bearer token. At least this is the theory for now ;-) right now my test messages all get stuck without getting any response (valid token or not), but that's a subject for a separate topic.