I am trying to get the AMD Radeon Pro 5700 XT GPU on my iMac 27" 2021 running Big Sur 11.4 to work with tensorflow-macos. If I disable eager execution I get an exception, if I don't, tensorflow-macos choses the CPU and not the GPU. Here's a simple example which shows the exception: import tensorflow as tf import tensorflow_hub as hub import tensorflow_text import numpy as np from sklearn.preprocessing import normalize from tensorflow.python.framework.ops import disable_eager_execution disable_eager_execution() m4 = hub.load("/Users/davidlaxer/Downloads/universal-sentence-encoder_4") english_sentences = ["dog", "Puppies are nice.", "I enjoy taking long walks along the beach with my dog."] r4 = np.array(m4(english_sentences)) print(r4) print(m4) type(r4) type(m4) --------------------------------------------------------------------------- NotImplementedError Traceback (most recent call last) <ipython-input-4-8b0ba0e4c28c> in <module> 1 m4 = hub.load("/Users/davidlaxer/Downloads/universal-sentence-encoder_4") 2 english_sentences = ["dog", "Puppies are nice.", "I enjoy taking long walks along the beach with my dog."] ----> 3 r4 = np.array(m4(english_sentences)) 4 print(r4) 5 print(m4) ~/anaconda3/envs/tensorflow_mac/lib/python3.8/site-packages/tensorflow/python/framework/ops.py in __array__(self) 850 851 def __array__(self): --> 852 raise NotImplementedError( 853 "Cannot convert a symbolic Tensor ({}) to a numpy array." 854 " This error may indicate that you're trying to pass a Tensor to" NotImplementedError: Cannot convert a symbolic Tensor (StatefulPartitionedCall_1:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported And commenting out disable_eager_execution(): from tensorflow.python.framework.ops import disable_eager_execution #disable_eager_execution() m4 = hub.load("/Users/davidlaxer/Downloads/universal-sentence-encoder_4") english_sentences = ["dog", "Puppies are nice.", "I enjoy taking long walks along the beach with my dog."] r4 = np.array(m4(english_sentences)) print(r4) print(m4) type(r4) type(m4) [-0.06334164 -0.01812314 0.03680531 ... -0.02809388 0.02786911 -0.04715428] [ 0.01975714 -0.02284616 0.04316505 ... -0.01376714 -0.00614742 -0.00124967] [-0.02169351 -0.003993 0.06716524 ... 0.05952153 0.02262796 0.03501643]] <tensorflow.python.saved_model.load.Loader._recreate_base_user_object.<locals>._UserObject object at 0x7fbb3892a9a0> [5]: tensorflow.python.saved_model.load.Loader._recreate_base_user_object.<locals>._UserObject The tensorflow group does not support this GPU and the tensorflow-mac repository is now read-only. https://github.com/tensorflow/tensorflow/issues/50353 https://github.com/apple/tensorflow_macos Any ideas?

I installed tensorflow-mac and tensorflow-metal on an iMac 2021 27" running Big Sur with an AMD Radeon Pro 5700 XT. I am running Python 3.8.5 (tensorflow-metal) (base) davidlaxer@x86_64-apple-darwin13 ~ % pip list Package                 Version             Location ----------------------- ------------------- ------------------------- ... tensorboard             2.5.0 tensorboard-data-server 0.6.1 tensorboard-plugin-wit  1.8.0 tensorflow              2.5.0 tensorflow-estimator    2.5.0 tensorflow-hub          0.12.0 tensorflow-macos        2.5.0 tensorflow-metal        0.1.1 tensorflow-text         2.5.0 When I try to import tensorflow I get this error:  % ipython Python 3.8.5 (default, Sep  4 2020, 02:22:02)  Type 'copyright', 'credits' or 'license' for more information IPython 7.24.1 -- An enhanced Interactive Python. Type '?' for help. In [1]: import tensorflow --------------------------------------------------------------------------- NotFoundError                             Traceback (most recent call last) <ipython-input-1-d6579f534729> in <module> ----> 1 import tensorflow ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/__init__.py in <module>     447     _plugin_dir = _os.path.join(_s, 'tensorflow-plugins')     448     if _os.path.exists(_plugin_dir): --> 449       _ll.load_library(_plugin_dir)     450       # Load Pluggable Device Library     451       _ll.load_pluggable_device_library(_plugin_dir) ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py in load_library(library_location)     152      153     for lib in kernel_libraries: --> 154       py_tf.TF_LoadLibrary(lib)     155      156   else: NotFoundError: dlopen(/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib, 6): Symbol not found: _TF_AssignUpdateVariable   Referenced from: /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow-plugins/libmetal_plugin.dylib   Expected in: flat namespace

I have installed tensorflow-macos and tensorflow-metal on Big Sur on a iMac 27" with AMD Radeon Pro 5700 XT. I am trying to run Keras code from Francios Challet's Deep Learning example: E.g Chapter 11-part04_sequence-to-Sequence https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/chapter11_part04_sequence-to-sequence-learning.ipynb seq2seq_rnn.compile( optimizer="rmsprop", loss="sparse_categorical_crossentropy", metrics=["accuracy"]) seq2seq_rnn.fit(train_ds, epochs=15, validation_data=val_ds) 2021-07-15 13:17:00.117869: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. 2021-07-15 13:17:01.403133: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at partitioned_function_ops.cc:114 : Invalid argument: No OpKernel was registered to support Op 'CudnnRNNV3' used by {{node cond_41/then/_0/cond/CudnnRNNV3}} with these attrs: [T=DT_FLOAT, input_mode="linear_input", direction="unidirectional", rnn_mode="gru", seed2=0, is_training=true, num_proj=0, time_major=false, seed=0, dropout=0] Registered devices: [CPU, GPU] Registered kernels: <no registered kernels> [[cond_41/then/_0/cond/CudnnRNNV3]] 2021-07-15 13:17:01.419061: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at partitioned_function_ops.cc:114 : Invalid argument: No OpKernel was registered to support Op 'CudnnRNNV3' used by {{node cond_41/then/_0/cond/CudnnRNNV3}} with these attrs: [time_major=false, dropout=0, seed=0, T=DT_FLOAT, input_mode="linear_input", direction="unidirectional", rnn_mode="gru", seed2=0, is_training=true, num_proj=0] Registered devices: [CPU, GPU] Registered kernels: <no registered kernels> [[cond_41/then/_0/cond/CudnnRNNV3]] --------------------------------------------------------------------------- InvalidArgumentError Traceback (most recent call last) /var/folders/3n/56fpv14n4wj0c1l1sb106pzw0000gn/T/ipykernel_94493/3093225856.py in <module> 3 loss="sparse_categorical_crossentropy", 4 metrics=["accuracy"]) ----> 5 seq2seq_rnn.fit(train_ds, epochs=15, validation_data=val_ds) ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing) 1181 _r=1): 1182 callbacks.on_train_batch_begin(step) -> 1183 tmp_logs = self.train_function(iterator) 1184 if data_handler.should_sync: 1185 context.async_wait() ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds) 887 888 with OptionalXlaContext(self._jit_compile): --> 889 result = self._call(*args, **kwds) 890 891 new_tracing_count = self.experimental_get_tracing_count() ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds) 948 # Lifting succeeded, so variables are initialized and we can run the 949 # stateless function. --> 950 return self._stateless_fn(*args, **kwds) 951 else: 952 _, _, _, filtered_flat_args = \ ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/function.py in __call__(self, *args, **kwargs) 3021 (graph_function, 3022 filtered_flat_args) = self._maybe_define_function(args, kwargs) -> 3023 return graph_function._call_flat( 3024 filtered_flat_args, captured_inputs=graph_function.captured_inputs) # pylint: disable=protected-access 3025 ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/function.py in _call_flat(self, args, captured_inputs, cancellation_manager) 1958 and executing_eagerly): 1959 # No tape is watching; skip to running the function. -> 1960 return self._build_call_outputs(self._inference_function.call( 1961 ctx, args, cancellation_manager=cancellation_manager)) 1962 forward_backward = self._select_forward_and_backward_functions( ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/function.py in call(self, ctx, args, cancellation_manager) 589 with _InterpolateFunctionError(self): 590 if cancellation_manager is None: --> 591 outputs = execute.execute( 592 str(self.signature.name), 593 num_outputs=self._num_outputs, ~/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 57 try: 58 ctx.ensure_initialized() ---> 59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, 60 inputs, attrs, num_outputs) 61 except core._NotOkStatusException as e: InvalidArgumentError: 2 root error(s) found. (0) Invalid argument: No OpKernel was registered to support Op 'CudnnRNNV3' used by {{node cond_41/then/_0/cond/CudnnRNNV3}} with these attrs: [T=DT_FLOAT, input_mode="linear_input", direction="unidirectional", rnn_mode="gru", seed2=0, is_training=true, num_proj=0, time_major=false, seed=0, dropout=0] Registered devices: [CPU, GPU] Registered kernels: <no registered kernels> [[cond_41/then/_0/cond/CudnnRNNV3]] [[model/bidirectional/backward_gru/PartitionedCall]] [[broadcast_weights_1/assert_broadcastable/is_valid_shape/else/_1/broadcast_weights_1/assert_broadcastable/is_valid_shape/has_valid_nonscalar_shape/then/_53/broadcast_weights_1/assert_broadcastable/is_valid_shape/has_valid_nonscalar_shape/has_invalid_dims/concat/_66]] (1) Invalid argument: No OpKernel was registered to support Op 'CudnnRNNV3' used by {{node cond_41/then/_0/cond/CudnnRNNV3}} with these attrs: [T=DT_FLOAT, input_mode="linear_input", direction="unidirectional", rnn_mode="gru", seed2=0, is_training=true, num_proj=0, time_major=false, seed=0, dropout=0] Registered devices: [CPU, GPU] Registered kernels: <no registered kernels> [[cond_41/then/_0/cond/CudnnRNNV3]] [[model/bidirectional/backward_gru/PartitionedCall]] 0 successful operations. 0 derived errors ignored. [Op:__inference_train_function_520769] Function call stack: train_function -> train_function

I am running tensorflow-macos and tensorflow-metal on Big Sur. I am getting this error: AttributeError: module 'tensorflow.keras' has no attribute 'utils_dataset_from_directory' https://github.com/keras-team/keras-io/issues/12 Can I install tf_nightly? Or does it conflict with tensorflow-macos? from tensorflow import keras from tensorflow.python.framework.ops import disable_eager_execution dataset = keras.utils_dataset_from_directory( "celeba_gan", label_mode=None, image_size=(64, 64), batch_size=32, smart_resize=True) --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /var/folders/3n/56fpv14n4wj0c1l1sb106pzw0000gn/T/ipykernel_41859/2519466253.py in <module> 1 from tensorflow import keras 2 from tensorflow.python.framework.ops import disable_eager_execution ----> 3 dataset = keras.utils_dataset_from_directory( 4 "celeba_gan", 5 label_mode=None, AttributeError: module 'tensorflow.keras' has no attribute 'utils_dataset_from_directory'

Details here: https://stackoverflow.com/questions/68551935/why-does-my-tensorflow-model-stop-training I can run the VariationalDeepSemantic Hashing model in an Anaconda python 3.85 virtual environment with tensorflow 2.5 on CPUs. If I run the same code in the tensorflow-metal virtual environment with python 3.82, accessing my AMD Radeon Pro 5700 XT GPU, the process stops training at epoch 5 on the 5200 batch.

I am trying to profile a tensorflow 2.5 model with tensorflow-macos and tensorflow-metal. I am getting this error: AttributeError: module 'tensorflow.compat.v1.profiler' has no attribute 'experimental' Here's a code snippet: import tensorflow as tf import numpy as np from utils import * tf.compat.v1.enable_v2_behavior() from tensorflow.python.framework.ops import disable_eager_execution disable_eager_execution() options = tf.profiler.experimental.ProfilerOptions(host_tracer_level = 3,                                                    python_tracer_level = 1,                                                    device_tracer_level = 1) tf.profiler.experimental.start('~/logdir', options=options) ... tf.profiler.experimental.stop() % pip list Package                    Version -------------------------- ------------------- absl-py                    0.12.0 anyio                      3.2.1 appnope                    0.1.2 argon2-cffi                20.1.0 astunparse                 1.6.3 async-generator            1.10 attrs                      21.2.0 Babel                      2.9.1 backcall                   0.2.0 bleach                     3.3.1 cachetools                 4.2.2 certifi                    2021.5.30 cffi                       1.14.6 charset-normalizer         2.0.1 cloudpickle                1.6.0 cycler                     0.10.0 Cython                     0.29.24 debugpy                    1.3.0 decorator                  5.0.9 defusedxml                 0.7.1 dill                       0.3.4 dm-tree                    0.1.6 dotmap                     1.3.23 entrypoints                0.3 flatbuffers                1.12 future                     0.18.2 gast                       0.4.0 gensim                     4.0.1 google-auth                1.32.1 google-auth-oauthlib       0.4.4 google-pasta               0.2.0 googleapis-common-protos   1.53.0 grpcio                     1.34.1 gviz-api                   1.9.0 gym                        0.18.3 h5py                       3.1.0 idna                       3.2 importlib-resources        5.2.0 ipykernel                  6.0.1 ipython                    7.25.0 ipython-genutils           0.2.0 ipywidgets                 7.6.3 jedi                       0.18.0 Jinja2                     3.0.1 json5                      0.9.6 jsonschema                 3.2.0 jupyter-client             6.1.12 jupyter-core               4.7.1 jupyter-server             1.9.0 jupyterlab                 3.0.16 jupyterlab-pygments        0.1.2 jupyterlab-server          2.6.1 jupyterlab-widgets         1.0.0 keras-nightly              2.5.0.dev2021032900 Keras-Preprocessing        1.1.2 kiwisolver                 1.3.1 Markdown                   3.3.4 MarkupSafe                 2.0.1 matplotlib                 3.4.2 matplotlib-inline          0.1.2 memory-profiler            0.58.0 mistune                    0.8.4 nbclassic                  0.3.1 nbclient                   0.5.3 nbconvert                  6.1.0 nbformat                   5.1.3 nest-asyncio               1.5.1 notebook                   6.4.0 numpy                      1.19.5 oauthlib                   3.1.1 opt-einsum                 3.3.0 packaging                  21.0 pandas                     1.3.0 pandocfilters              1.4.3 parso                      0.8.2 pexpect                    4.8.0 pickleshare                0.7.5 Pillow                     8.2.0 pip                        21.2.1 prometheus-client          0.11.0 promise                    2.3 prompt-toolkit             3.0.19 protobuf                   3.17.3 psutil                     5.8.0 ptyprocess                 0.7.0 pyasn1                     0.4.8 pyasn1-modules             0.2.8 pybind11                   2.6.2 pycparser                  2.20 pyglet                     1.5.15 Pygments                   2.9.0 pyparsing                  2.4.7 pyrsistent                 0.18.0 python-dateutil            2.8.2 pytz                       2021.1 pyzmq                      22.1.0 requests                   2.26.0 requests-oauthlib          1.3.0 requests-unixsocket        0.2.0 rsa                        4.7.2 scipy                      1.7.0 Send2Trash                 1.7.1 setuptools                 41.2.0 six                        1.15.0 smart-open                 5.1.0 sniffio                    1.2.0 tensorboard                2.5.0 tensorboard-data-server    0.6.1 tensorboard-plugin-profile 2.4.0 tensorboard-plugin-wit     1.8.0 tensorflow-datasets        4.3.0 tensorflow-estimator       2.5.0 tensorflow-hub             0.12.0 tensorflow-macos           2.5.0 tensorflow-metadata        1.1.0 tensorflow-metal           0.1.1 tensorflow-probability     0.13.0 termcolor                  1.1.0 terminado                  0.10.1 testpath                   0.5.0 tornado                    6.1 tqdm                       4.61.2 traitlets                  5.0.5 typing-extensions          3.7.4.3 urllib3                    1.26.6 wcwidth                    0.2.5 webencodings               0.5.1 websocket-client           1.1.0 Werkzeug                   2.0.1 wheel                      0.36.2 widgetsnbextension         3.5.1 wrapt                      1.12.1 zipp                       3.5.0

Will tensorflow_text be supported with Tensorflow_macos & Tensorflow_metal? E.g. https://github.com/tensorflow/text/issues/654

[I am running Top2vec on Big Sur 11.6 with tensorflow-macos and tensorflow-metal. Python crashed ... linkText Crashed Thread: 0 Dispatch queue: metal gpu stream Exception Type: EXC_CRASH (SIGABRT) Exception Codes: 0x0000000000000000, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY Application Specific Information: /System/Volumes/Data/SWE/macOS/BuildRoots/38cf1d983f/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MetalPerformanceShaders-124.6.1/MPSCore/Utility/MPSCommandBufferImageCache.mm:1386: failed assertion `Failed to allocate private MTLBuffer for size 421888000 Crash Log

I just recreated my tensorflow-metal virtual environment on OS X 11.6. Now, when I import tensorflow, I am getting: No supported GPU was found. My AMD GPU worked in the prior versions of OS X 11.6. Do the latest versions require OS X 12? % ipython Python 3.8.5 (default, Sep 4 2020, 02:22:02) Type 'copyright', 'credits' or 'license' for more information IPython 7.28.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: import tensorflow No supported GPU was found. % pip list Package Version ------------------------ --------- absl-py 0.12.0 anyio 3.3.2 appnope 0.1.2 argon2-cffi 21.1.0 astunparse 1.6.3 attrs 21.2.0 Babel 2.9.1 backcall 0.2.0 bleach 4.1.0 bokeh 2.3.3 cachetools 4.2.4 certifi 2021.5.30 cffi 1.14.6 charset-normalizer 2.0.6 clang 5.0 cloudpickle 2.0.0 cycler 0.10.0 Cython 0.29.24 debugpy 1.5.0 decorator 5.1.0 defusedxml 0.7.1 dill 0.3.4 distinctipy 1.1.5 dm-tree 0.1.6 entrypoints 0.3 flatbuffers 1.12 future 0.18.2 gast 0.4.0 gensim 3.8.3 google-auth 1.35.0 google-auth-oauthlib 0.4.6 google-pasta 0.2.0 googleapis-common-protos 1.53.0 grpcio 1.41.0 h5py 3.1.0 hdbscan 0.8.27 idna 3.2 importlib-resources 5.2.2 ipykernel 6.4.1 ipython 7.28.0 ipython-genutils 0.2.0 ipywidgets 7.6.5 jedi 0.18.0 Jinja2 3.0.2 joblib 1.1.0 json5 0.9.6 jsonschema 4.0.1 jupyter-client 7.0.6 jupyter-core 4.8.1 jupyter-server 1.11.1 jupyterlab 3.1.18 jupyterlab-pygments 0.1.2 jupyterlab-server 2.8.2 jupyterlab-widgets 1.0.2 keras 2.6.0 Keras-Preprocessing 1.1.2 kiwisolver 1.3.2 llvmlite 0.37.0 Markdown 3.3.4 MarkupSafe 2.0.1 matplotlib 3.4.3 matplotlib-inline 0.1.3 mistune 0.8.4 nbclassic 0.3.2 nbclient 0.5.4 nbconvert 6.2.0 nbformat 5.1.3 nest-asyncio 1.5.1 nmslib 2.1.1 notebook 6.4.4 numba 0.54.0 numpy 1.20.3 oauthlib 3.1.1 opt-einsum 3.3.0 packaging 21.0 pandas 1.3.3 pandocfilters 1.5.0 parso 0.8.2 pexpect 4.8.0 pickleshare 0.7.5 Pillow 8.3.2 pip 21.2.4 prometheus-client 0.11.0 promise 2.3 prompt-toolkit 3.0.20 protobuf 3.18.1 psutil 5.8.0 ptyprocess 0.7.0 pyasn1 0.4.8 pyasn1-modules 0.2.8 pybind11 2.6.1 pycparser 2.20 Pygments 2.10.0 pynndescent 0.5.4 pyparsing 2.4.7 pyrsistent 0.18.0 python-dateutil 2.8.2 pytz 2021.3 PyYAML 5.4.1 pyzmq 22.3.0 requests 2.26.0 requests-oauthlib 1.3.0 requests-unixsocket 0.2.0 rsa 4.7.2 scikit-learn 1.0 scipy 1.7.1 Send2Trash 1.8.0 setuptools 47.1.0 six 1.15.0 smart-open 5.2.1 sniffio 1.2.0 tabulate 0.8.9 tensorboard 2.6.0 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.0 tensorflow 2.6.0 tensorflow-consciousness 0.1 tensorflow-datasets 4.4.0 tensorflow-estimator 2.6.0 tensorflow-gan 2.1.0 tensorflow-hub 0.12.0 tensorflow-macos 2.6.0 tensorflow-metadata 1.2.0 tensorflow-metal 0.2.0 tensorflow-probability 0.14.1 tensorflow-similarity 0.13.45 tensorflow-text 2.6.0 termcolor 1.1.0 terminado 0.12.1 testpath 0.5.0 threadpoolctl 3.0.0 top2vec 1.0.26 tornado 6.1 tqdm 4.62.3 traitlets 5.1.0 typing-extensions 3.7.4.3 umap-learn 0.5.1 urllib3 1.26.7 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 1.2.1 Werkzeug 2.0.2 wheel 0.37.0 widgetsnbextension 3.5.1 wordcloud 1.8.1 wrapt 1.12.1 zipp 3.6.0

I am running tensorflow-macos and tensorflow-metal version 2.6 on Monterey Beta (21A5543b) on an iMac 27" 2021 with an AMD Radeon GPU. I got the following error training the model VariationalDeepSemanticHashing e.g. 2021-10-09 13:05:14.521286: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. 2021-10-09 13:05:27.092823: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. 2021-10-09 13:05:27.153 python[6315:1459657] -[MPSGraph adamUpdateWithLearningRateTensor:beta1Tensor:beta2Tensor:epsilonTensor:beta1PowerTensor:beta2PowerTensor:valuesTensor:momentumTensor:velocityTensor:gradientTensor:name:]: unrecognized selector sent to instance 0x600000eede10 [I 2021-10-09 13:05:28.157 ServerApp] AsyncIOLoopKernelRestarter: restarting kernel (1/5), keep random ports kernel d25e6066-74f7-4b4a-b5e7-b2911e7501d9 restarted https://github.com/unsuthee/VariationalDeepSemanticHashing/blob/master/Run_Experiment_Unsupervised.ipynb Here's the repository: https://github.com/unsuthee/VariationalDeepSemanticHashing

I measured a significant performance difference running the 'keras-io' example 'text_extraction_with_bert.ipynb' on Google Colab and my tensorflow_metal GPU (AMD Radeon Pro 5700 XT). Google Colab Pro w/TPU finished 3 epochs in 11 minutes, while tensorflow_metal ran for many hours for 1 epoch. So, I tried to profile the model in both environments. I was able to profile text_extraction_with_bert.ipynb on Google Colab Pro, but not on tensorflow_metal. My Mac has 128gb ... the OOM exception happened when the Python 3.8 process got to ~85GB. ResourceExhaustedError: 2 root error(s) found. (0) Resource exhausted: OOM when allocating tensor with shape[256,384,3072] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator Simple allocator [[node model/tf_bert_model/bert/encoder/layer_._6/intermediate/Gelu/add (defined at Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/transformers/models/bert/modeling_tf_bert.py:354) ]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode. [[model/tf_bert_model/bert/encoder/layer_._7/attention/output/dense/Tensordot/Prod/_632]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode. (1) Resource exhausted: OOM when allocating tensor with shape[256,384,3072] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator Simple allocator [[node model/tf_bert_model/bert/encoder/layer_._6/intermediate/Gelu/add (defined at Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/transformers/models/bert/modeling_tf_bert.py:354) ]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode. 0 successful operations. 0 derived errors ignored. [Op:__inference_train_function_24748] Function call stack: train_function -> train_function Here's the model: https://github.com/keras-team/keras-io/blob/master/examples/nlp/ipynb/text_extraction_with_bert.ipynb

In a tensorflow-metal virtual environment on OS X 12.1: tensorboard 2.6.0 tensorboard-data-server 0.6.1 tensorboard-plugin-profile 2.5.0 tensorboard-plugin-wit 1.8.0 tensorflow 2.6.0 tensorflow-addons 0.14.0 tensorflow-consciousness 0.1 tensorflow-datasets 4.4.0 tensorflow-estimator 2.7.0 tensorflow-gan 2.1.0 tensorflow-hub 0.12.0 tensorflow-io-gcs-filesystem 0.22.0 tensorflow-macos 2.7.0 tensorflow-metadata 1.2.0 tensorflow-metal 0.3.0 tensorflow-probability 0.14.1 tensorflow-similarity 0.13.45 tensorflow-text 2.7.3 Running the Top2vec model: https://github.com/ddangelov/Top2Vec import numpy as np import pandas as pd import json import os import ipywidgets as widgets from IPython.display import clear_output, display from top2vec import Top2Vec papers_prepared_df = pd.read_feather("/Users/davidlaxer/Downloads/archive/covid19_papers_processed.feather") top2vec_trained = Top2Vec(documents=papers_prepared_df.text.tolist(), embedding_model="universal-sentence-encoder", use_embedding_model_tokenizer=True, embedding_model_path="/Users/davidlaxer/Downloads/universal-sentence-encoder_4/", workers=4) 2021-12-20 06:30:52,188 - top2vec - INFO - Pre-processing documents for training /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead. warnings.warn(msg, category=FutureWarning) 2021-12-20 06:31:57,351 - top2vec - INFO - Loading universal-sentence-encoder model at /Users/davidlaxer/Downloads/universal-sentence-encoder_4 2021-12-20 06:31:57.488459: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-12-20 06:31:57.489288: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. 2021-12-20 06:31:57.489490: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>) Metal device set to: AMD Radeon Pro 5700 XT 2021-12-20 06:31:59.447260: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. 2021-12-20 06:32:00,841 - top2vec - INFO - Creating joint document/word embedding 2021-12-20 06:32:00.923838: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. Some resource has been exhausted. For example, this error might be raised if a per-user quota is exhausted, or perhaps the entire file system is out of space. @@__init__ 2 root error(s) found. (0) RESOURCE_EXHAUSTED: OOM when allocating tensor with shape[114389,320] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator Simple allocator [[{{node EncoderDNN/EmbeddingLookup/EmbeddingLookupUnique/GatherV2}}]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode. [[StatefulPartitionedCall/StatefulPartitionedCall/EncoderDNN/EmbeddingLookup/EmbeddingLookupUnique/Reshape_1/_188]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode. (1) RESOURCE_EXHAUSTED: OOM when allocating tensor with shape[114389,320] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator Simple allocator [[{{node EncoderDNN/EmbeddingLookup/EmbeddingLookupUnique/GatherV2}}]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode. ... I tried adjusting the batchsize (e.g - 500, 100, 50, 10, 5).

I am getting a segmentation violation in Tensorflow running the Top2vec model. https://github.com/ddangelov/Top2Vec Here's my code snippet: import sys import faulthandler import numpy as np import pandas as pd import json import os import ipywidgets as widgets from IPython.display import clear_output, display from top2vec import Top2Vec faulthandler.enable(file=sys.stderr, all_threads=True) def filter_short(papers_df): papers_df["token_counts"] = papers_df["text"].str.split().map(len) papers_df = papers_df[papers_df.token_counts > 50].reset_index(drop=True) papers_df.drop('token_counts', axis=1, inplace=True) return papers_df papers_prepared_df = pd.read_feather("/Users/davidlaxer/Downloads/archive/covid19_papers_processed.feather") papers_feathered_filtered_df = filter_short(papers_prepared_df) top2vec_trained = Top2Vec(documents=papers_feathered_filtered_df.text.tolist(), embedding_model="universal-sentence-encoder", use_embedding_model_tokenizer=True, embedding_model_path="/Users/davidlaxer/Downloads/universal-sentence-encoder_4/", workers=4) Here's the stack trace: /Users/davidlaxer/tensorflow-metal/bin/python /Users/davidlaxer/Top2Vec/notebooks/test1.py 2021-12-24 08:50:04,782 - top2vec - INFO - Pre-processing documents for training /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead. warnings.warn(msg, category=FutureWarning) 2021-12-24 08:51:37,701 - top2vec - INFO - Loading universal-sentence-encoder model at /Users/davidlaxer/Downloads/universal-sentence-encoder_4/ 2021-12-24 08:51:37.837087: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-12-24 08:51:37.838170: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. 2021-12-24 08:51:37.838380: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>) Metal device set to: AMD Radeon Pro 5700 XT systemMemory: 128.00 GB maxCacheSize: 7.99 GB 2021-12-24 08:51:39.764518: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. 2021-12-24 08:51:41,157 - top2vec - INFO - Creating joint document/word embedding 2021-12-24 08:51:41.241496: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. Fatal Python error: Segmentation fault Thread 0x000000010c32a600 (most recent call first): File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 58 in quick_execute File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 598 in call File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1959 in _call_flat File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 3130 in __call__ File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 949 in _call File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 910 in __call__ File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 150 in error_handler File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/tensorflow/python/saved_model/load.py", line 701 in _call_attribute File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/top2vec/Top2Vec.py", line 538 in _embed_documents File "/Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages/top2vec/Top2Vec.py", line 344 in __init__ File "/Users/davidlaxer/Top2Vec/notebooks/test1.py", line 24 in <module> Process finished with exit code 139 (interrupted by signal 11: SIGSEGV) This is likely related to: https://github.com/ddangelov/Top2Vec/issues/232 However, SIGSEGV indicates some other problem(s). % python --version Python 3.8.5 % pip show tensorflow_macos Name: tensorflow-macos Version: 2.7.0 Summary: TensorFlow is an open source machine learning framework for everyone. Home-page: https://www.tensorflow.org/ Author: Google Inc. Author-email: packages@tensorflow.org License: Apache 2.0 Location: /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages Requires: absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, keras-preprocessing, libclang, numpy, opt-einsum, protobuf, six, tensorboard, tensorflow-estimator, tensorflow-io-gcs-filesystem, termcolor, typing-extensions, wheel, wrapt Required-by: % pip show tensorflow_metal Name: tensorflow-metal Version: 0.3.0 Summary: TensorFlow acceleration for Mac GPUs. Home-page: https://developer.apple.com/metal/tensorflow-plugin/ Author: Author-email: License: MIT License. Copyright © 2020-2021 Apple Inc. All rights reserved. Location: /Users/davidlaxer/tensorflow-metal/lib/python3.8/site-packages Requires: six, wheel Required-by:

I am using a 2022 Intel based iMac 27" with an AMD Radeon Pro GPU running OS X Monterey 12.3 Beta. I am working with Francois Chollet's GAN example: https://github.com/fchollet/deep-learning-with-python-notebooks/blob/master/chapter12_part05_gans.ipynb Here is my tensorflow-metal environment: Package Version Editable project location ---------------------------- ---------- ---------------------------- ... Send2Trash 1.8.0 sentence-transformers 2.1.0 sentencepiece 0.1.96 setuptools 47.1.0 shap 0.40.0 six 1.15.0 slicer 0.0.7 smart-open 5.2.1 sniffio 1.2.0 soupsieve 2.3.1 sympy 1.9 tables 3.6.1 tabulate 0.8.9 tbb 2021.5.0 tbb-devel 2021.5.0 tenacity 8.0.1 tensorboard 2.6.0 tensorboard-data-server 0.6.1 tensorboard-plugin-profile 2.5.0 tensorboard-plugin-wit 1.8.0 tensorflow 2.6.0 tensorflow-addons 0.14.0 tensorflow-consciousness 0.1 tensorflow-datasets 4.4.0 tensorflow-estimator 2.7.0 tensorflow-gan 2.1.0 tensorflow-hub 0.12.0 tensorflow-io-gcs-filesystem 0.22.0 tensorflow-macos 2.7.0 tensorflow-metadata 1.2.0 tensorflow-metal 0.3.0 tensorflow-probability 0.14.1 tensorflow-similarity 0.13.45 tensorflow-text 2.7.3 termcolor 1.1.0 terminado 0.12.1 testpath 0.5.0 threadpoolctl 3.0.0 tokenizers 0.10.3 toml 0.10.2 top2vec 1.0.26 torch 1.10.1 torchvision 0.11.2 tornado 6.1 tqdm 4.62.3 traitlets 5.1.0 transformers 4.11.3 typeguard 2.13.0 typing-extensions 3.7.4.3 umap-learn 0.5.1 urllib3 1.26.7 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 1.2.1 Werkzeug 2.0.2 wheel 0.37.0 widgetsnbextension 4.0.0b0 wordcloud 1.8.1 wrapt 1.12.1 xgboost 1.5.1 zipp 3.6.0 (tensorflow-metal) (base) davidlaxer@x86_64-apple-darwin13 ~ % The Keras Adam optimizer crashes tensorflow. The SGD optimzer runs but the GAN generates a single color block image. The Adagrad optimizer runs but also generates a one color block image. Here is output: To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2022-02-04 07:36:35.810367: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. 2022-02-04 07:36:35.810606: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>) Metal device set to: AMD Radeon Pro 5700 XT 2022-02-04 07:36:38.833846: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled. Example output image: The code ran for 4 days and completed with no errors, however, the images were always single color blocks. I tried running with the Adam optimizer on the CPU but it was too slow.

I am running a recurrent neural network example on an iMac 27" with an AMD Radeon Pro 5700 XT on OS X 12.3. The code runs, the GPU was initialized the GPU is not active. Each epoch takes: 819/819 [==============================] - 32417s 40s/step - loss: 121.7538 - mae: 8.9641 - val_loss: 100.3145 - val_mae: 8.0313 % python chapter-10.py ['"Date Time"', '"p (mbar)"', '"T (degC)"', '"Tpot (K)"', '"Tdew (degC)"', '"rh (%)"', '"VPmax (mbar)"', '"VPact (mbar)"', '"VPdef (mbar)"', '"sh (g/kg)"', '"H2OC (mmol/mol)"', '"rho (g/m**3)"', '"wv (m/s)"', '"max. wv (m/s)"', '"wd (deg)"'] 420451 num_train_samples: 210225 num_val_samples: 105112 num_test_samples: 105114 2022-03-28 18:28:59.988516: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: SSE4.2 AVX AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. Metal device set to: AMD Radeon Pro 5700 XT systemMemory: 128.00 GB maxCacheSize: 7.99 GB 2022-03-28 18:28:59.989242: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support. 2022-03-28 18:28:59.989616: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>) WARNING:tensorflow:Layer lstm will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU. Epoch 1/50 2022-03-28 18:29:02.342296: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled. 819/819 [==============================] - ETA: 0s - loss: 121.7538 - mae: 8.9641 2022-03-29 03:21:02.092397: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled. 819/819 [==============================] - 32417s 40s/step - loss: 121.7538 - mae: 8.9641 - val_loss: 100.3145 - val_mae: 8.0313 Epoch 2/50 381/819 [============>.................] - ETA: 4:50:31 - loss: 93.7597 - mae: 7.6880 from tensorflow import keras from tensorflow.keras import layers num_features = 14 steps = 120 import os fname = os.path.join("jena_climate_2009_2016.csv") with open(fname) as f: data = f.read() lines = data.split("\n") header = lines[0].split(",") lines = lines[1:] print(header) print(len(lines)) import numpy as np temperature = np.zeros((len(lines),)) raw_data = np.zeros((len(lines), len(header) - 1)) for i, line in enumerate(lines): values = [float(x) for x in line.split(",")[1:]] temperature[i] = values[1] raw_data[i, :] = values[:] sampling_rate = 6 sequence_length = 120 delay = sampling_rate * (sequence_length + 24 - 1) batch_size = 256 num_train_samples = int(0.5 * len(raw_data)) num_val_samples = int(0.25 * len(raw_data)) num_test_samples = len(raw_data) - num_train_samples - num_val_samples print("num_train_samples:", num_train_samples) print("num_val_samples:", num_val_samples) print("num_test_samples:", num_test_samples) train_dataset = keras.utils.timeseries_dataset_from_array( raw_data[:-delay], targets=temperature[delay:], sampling_rate=sampling_rate, sequence_length=sequence_length, shuffle=True, batch_size=batch_size, start_index=0, end_index=num_train_samples) val_dataset = keras.utils.timeseries_dataset_from_array( raw_data[:-delay], targets=temperature[delay:], sampling_rate=sampling_rate, sequence_length=sequence_length, shuffle=True, batch_size=batch_size, start_index=num_train_samples, end_index=num_train_samples + num_val_samples) test_dataset = keras.utils.timeseries_dataset_from_array( raw_data[:-delay], targets=temperature[delay:], sampling_rate=sampling_rate, sequence_length=sequence_length, shuffle=True, batch_size=batch_size, start_index=num_train_samples + num_val_samples) inputs = keras.Input(shape=(steps, num_features)) x = layers.SimpleRNN(16, return_sequences=True)(inputs) x = layers.SimpleRNN(16, return_sequences=True)(x) outputs = layers.SimpleRNN(16)(x) inputs = keras.Input(shape=(sequence_length, raw_data.shape[-1])) x = layers.LSTM(32, recurrent_dropout=0.25)(inputs) x = layers.Dropout(0.5)(x) outputs = layers.Dense(1)(x) model = keras.Model(inputs, outputs) callbacks = [ keras.callbacks.ModelCheckpoint("jena_lstm_dropout.keras", save_best_only=True) ] model.compile(optimizer="rmsprop", loss="mse", metrics=["mae"]) history = model.fit(train_dataset, epochs=50, validation_data=val_dataset, callbacks=callbacks) Any idea why the GPU is not active? How does this code example run on an M1 Ultra?