ugh, reposted below as an answer to fix formatting: https://developer.apple.com/forums/thread/675482?answerId=691564022#691564022

This worked for me but with a couple caveats... In macOS Big Sur the spotlight data folders are protected by SIP, and so Terminal needs to be granted "Full Disk Access" per this StackOverflow post: https://unix.stackexchange.com/questions/507063/cant-list-directory-even-as-root I believe that it is macOS's SIP (System Integrity Protection) that stops the du and lsprocesses from accessing the files/directories, even when running as root. SIP appears to be, since macOS Mojave, active for a number of locations under ~/Library, which means that only trusted applications can access them. I added my terminal application (iTerm2) to the list of applications that should be granted "Full Disk Access", and I did that in the "Security & Privacy" Systems Preferences settings, under the "Privacy" tab. This seems to have solved it for me. I say "seems" as I'm no macOS expert, but I believe this is what made me able to access everything under ~/Library. Apple recommends not turning off SIP globally on the system, for obvious reasons, and I don't see a real reason to do so either. After you have followed the instructions above then sudo mdutil -a -i off / sudo killall -KILL Spotlight spotlightd mds sudo rm -rf ~/Library/Metadata/CoreSpotlight/ sudo rm -Rf /System/Volums/Data/.Spotlight-V100 Then restart, you should be good to go! You can optionally then disable "Full Disk Access" for Terminal at this point if you are concerned about security.