Hello, everyone, I have been testing tensorflow-metal in my 2020 Macbook Pro (M1) running macOS 12.0.1 by performing the inference of a pre-trained model on a known dataset. To my surprise, Tensorflow produces different (wrong) results when performing the inference using the Metal pluggable device GPU vs when performing it in the CPU. I might very well be doing something wrong, but my test program is fairly simple: #!/usr/bin/env python3 import pathlib import numpy as np import tensorflow as tf from tensorflow import keras def main(model_path, dataset_path): # Print some system info print('Tensorflow configuration:') print(f'\tVersion: {tf.__version__}') print('\tDevices usable by Tensorflow:') for device in tf.config.get_visible_devices(): print(f'\t\t{device}') # Load the model & the input data model = keras.models.load_model(model_path) matrix_data = np.genfromtxt(dataset_path) matrix_data = matrix_data.reshape([1, matrix_data.shape[0], matrix_data.shape[1]]) # Perform inference in CPU with tf.device('/CPU:0'): prediction = model.predict(matrix_data)[1] print('Model Evaluation on CPU') print(f'\tPrediction: {prediction[0, 0]}') # Perform inference in GPU with tf.device('/GPU:0'): prediction = model.predict(matrix_data)[1] print('Model Evaluation on GPU') print(f'\tPrediction: {prediction[0, 0]}') if __name__ == "__main__": main('model/model.h5', 'dataset/01.csv') The CPU path produces a result of 4.890502452850342 and this is coherent with the results I'm seeing in Ubuntu Linux using CPU & GPU (CUDA) based inference. The GPU code path results in a prediction of 3.1839447021484375, which is way off. I have set up a GitLab repo with all the resources required for replicating the problem here This is quite concerning for me, since the big difference in results is something that I was not expecting and -if confirmed- makes me not trust the results provided by the Metal backend. Am I doing something wrong? Is there any place where I can report this as a bug?