Just wanna add to this discussion as I was having the same problem trying to create an aggregate device with my usb mixer/motu interface. Crackling/weird digital distortion. I'm on a Mac mini M1 running Mac OS 12.6.
The thing that fixes it for me is to add in my "Zoom Audio Device" into the aggregate Device. I have no idea, and it makes no sense (at least to me) but when I add that into the device, the distortion goes away and everything works as it should. Very weird!