Not an expert, but seems like jwt.io doesn't use public key for token creation, but just for token verification.
In other words, you can insert anything (even keep it empty) into public key part on jwt.io and resulting token will be the same (and working for Appstore API).
But you can also generate public key from your private key. I used open SSL as hinted here https://www.baeldung.com/linux/public-key-from-private-key
Let's assume individual-ApiKey_2X9R4HXF34.p8 is file with private key downloaded from Developer Apple account and following command prints out public key
openssl pkey -in individual-ApiKey_2X9R4HXF34.p8 -pubout