coinspiranted post above worked for me but with a couple caveats...
In macOS Big Sur the spotlight data folders are protected by SIP, and so Terminal needs to be granted "Full Disk Access" per this StackOverflow post:
https://unix.stackexchange.com/questions/507063/cant-list-directory-even-as-root
I believe that it is macOS's SIP (System Integrity Protection) that stops the du and lsprocesses from accessing the files/directories, even when running as root. SIP appears to be, since macOS Mojave, active for a number of locations under ~/Library, which means that only trusted applications can access them.
I added my terminal application (iTerm2) to the list of applications that should be granted "Full Disk Access", and I did that in the "Security & Privacy" Systems Preferences settings, under the "Privacy" tab. This seems to have solved it for me. I say "seems" as I'm no macOS expert, but I believe this is what made me able to access everything under ~/Library.
Apple recommends not turning off SIP globally on the system, for obvious reasons, and I don't see a real reason to do so either.
After you have followed the instructions above then
sudo mdutil -a -i off /
sudo killall -KILL Spotlight spotlightd mds
sudo rm -rf ~/Library/Metadata/CoreSpotlight/
sudo rm -Rf /System/Volums/Data/.Spotlight-V100
Then restart, you should be good to go!
You can optionally then disable "Full Disk Access" for Terminal at this point if you are concerned about security.