Possible tensorflow-metal performance regression with macOS 12 beta 2

Question

Simon Benvenutto OP

Created Jun ’21

Replies 2

Boosts 0

Participants 1

My benchmark is an extremely simple model for Fashion MNIST:

train_images = train_images / 255.0
test_images = test_images / 255.0

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=(28, 28)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(10)
])

model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])

model.fit(train_images, train_labels, epochs=10)

Yesterday this was running at 10 secs per epoch on my Mac mini M1 16 GB unified memory (and shockingly 2 secs lower than my 16" 8-core MBP with 5500M 8GB) but today that's jumped to 16 secs :(

I've tried re-installing Miniforge 3 both 4.10.1-5 and 4.10.2-0 with no difference. I've also installed the latest Xcode CLI tools.

Any thoughts?

(haven't tested X86 on the MBP yet)

Answered by Simon Benvenutto in 680677022

Having tried various option including installing beta 1 command line tools and then re-installing earlier miniforge release, I thought I'd download the restore image for beta 2, and do a revive against it in Apple Configurator. Timings for the above benchmark are now 1 second better than beta 1, at 9 secs per epoch :) so issue resolved (but not sure why).

Boost

Answer 1

Simon Benvenutto OP

Jun ’21

Timings on the MBP 16" have gone from 12 secs to 14 secs per epoch.

0

Answer 2

Simon Benvenutto OP

Jun ’21

Accepted Answer

Having tried various option including installing beta 1 command line tools and then re-installing earlier miniforge release, I thought I'd download the restore image for beta 2, and do a revive against it in Apple Configurator. Timings for the above benchmark are now 1 second better than beta 1, at 9 secs per epoch :) so issue resolved (but not sure why).

0