I am experiencing the same issue on a 2015 MacBook Air - I haven't tried with my MacBook Pro yet. Updating my Apple TV now from 14.0.2. I will post if this fixes the issue, but I believe its Big Sur causing the problem. I too do not have Airplay in my Toolbar.
Hope they fix this soon, as I use airplay often.
-- Update
I don't know that updating the Apple TV helped. After the update however I tried this again from the video browser window (On Video Airplay Icon) where it showed two selections with Checkboxes. Clicking on the Apple TV selection didn't do anything, but clicking the Apple TV's checkbox did.
Its working now for me. Now I need to see if I can mirror my screen to the Apple TV.