I thought I am the only one who is facing this error, I confirm the issue is exist also in a very small isolated piece of code,
give AddOnly permission
try to obtain ReadWrite permissions
fetch albums (fail)
and even worse the photoLibraryDidChange never get called in this scenario,
close and reopen the app, everything working as expected.
(UIActivityViewController save image or video, UIImageWriteToSavedPhotosAlbum, PHPhotoLibrary performChanges, ...) all of them will cause the same exact issue when used to obtain AddOnly permission first then try to get a ReadWrite permission
It seems it is an iOS 14.0 bug, (and I also confirm it is still exist with iOS 14.0.1)