Class Activation Maps with CoreML

Hi, I am trying to build a simple image classification app. I have a trained model using Keras and have it converted to an MLModel. Basing on the sample app in the CoreML documentation, I can use my model to classify a given image. But i would also like to visualize how model gives the classification. Is there a way I can show class activation maps along with the predictions in the app?

Thanks!