Hello,
somehow, self-signed certificates don't work anymore (even though they could be trusted before and most likely adhere to the requirements).
I was able to provide MDM payload with a Step-CA root certificate, it's intermediate certificate and the signed certificate of the mailserver - it seems to be important that the chain of trust be completely working, including working DNS and alt name of the host.