Huge memory leakage issue with tf.keras.models.predict()

Question

Created Aug ’22

Replies 19

Boosts 1

Views 6.8k

Participants 10

Comparison between MAC Studio M1 Ultra (20c, 64c, 128GB RAM) vs 2017 Intel i5 MBP (16GB RAM) for the subject matter i.e. memory leakage while using tf.keras.models.predict() for saved model on both machines:

MBP-2017: First prediction takes around 10MB and subsequent calls ~0-1MB

MACSTUDIO-2022: First prediction takes around 150MB and subsequent calls ~70-80MB.

After say 10000 such calls o predict(), while my MBP memory usage stays under 10GB, MACSTUDIO climbs to ~80GB (and counting up for higher number of calls).

Even using keras.backend.clear_session() after each call on MACSTUDIO did not help.

Can anyone having insight on TensorFlow-metal and/or MAC M1 machines help?

Thanks, Bapi

Boost

Answer 1

karbapi OP

Oct ’22

I don't see any improvement after upgrading to MacOS Ventura. Only difference is the step time has reduced to ~80ms vs ~140ms. Rest remains the same.

1

Answer 2

vedantvarshney OP

Nov ’22

I am facing the same issue. A huge memory leak (20GB+) quickly builds up when fitting a model under default settings. Only when the device is set to CPU does the leak not occur. This, however, is not a permanent solution - the GPUs are an important part of the value proposition of Apple silicon. Please address ASAP!

0

Answer 3

wangfmn OP

Feb ’23

I noticed the memory increase is mainly from swap memory. I compared keras.fit with different input size of image, the training memory of small size rarely increased but the larger one increased very much. Maybe it is related to macOS swap memory issue?

0

Answer 4

Najabir OP

May ’23

For me, the leakage problem on M2 was magically solved by:

import tensorflow as tf

tf.config.set_visible_devices([],&#039;GPU&#039;)

0