Alguem me pode indicar se os devolopers que estão no espaço da união europeia, possam aceder aos serviços apple intelligence ? Obrigado

We use MLModel in our app, which uses two file formats: mlmodel and mlpackage. We find that when the model is released, models using mlmodel format have a certain probability of crashing. And these crashes account for the majority (over 85%) in the iOS 16.x system. Here is the crash stack: Exception Type: SIGTRAP Exception Codes: TRAP_BRKPT at 0x1b48e855c Crashed Thread: 5 Thread 5 Crashed: 0 libdispatch.dylib 0x00000001b48e855c _dispatch_semaphore_dispose.cold.1 + 40 1 libdispatch.dylib 0x00000001b48b2b28 _dispatch_semaphore_signal_slow 2 libdispatch.dylib 0x00000001b48b0e58 _dispatch_dispose + 208 3 AppleNeuralEngine 0x00000001ef07b51c -[_ANEProgramForEvaluation .cxx_destruct] + 32 4 libobjc.A.dylib 0x00000001a67ed4a4 object_cxxDestructFromClass(objc_object*, objc_class*) + 116 5 libobjc.A.dylib 0x00000001a67f221c objc_destructInstance + 80 6 libobjc.A.dylib 0x00000001a67fb9d0 _objc_rootDealloc + 80 7 AppleNeuralEngine 0x00000001ef079e04 -[_ANEProgramForEvaluation dealloc] + 72 8 AppleNeuralEngine 0x00000001ef07ca70 -[_ANEModel .cxx_destruct] + 44 9 libobjc.A.dylib 0x00000001a67ed4a4 object_cxxDestructFromClass(objc_object*, objc_class*) + 116 10 libobjc.A.dylib 0x00000001a67f221c objc_destructInstance + 80 11 libobjc.A.dylib 0x00000001a67fb9d0 _objc_rootDealloc + 80 12 AppleNeuralEngine 0x00000001ef07bd7c -[_ANEModel dealloc] + 136 13 CoreFoundation 0x00000001ad4563cc cow_cleanup + 168 14 CoreFoundation 0x00000001ad49044c -[__NSDictionaryM dealloc] + 148 15 Espresso 0x00000001bb19c7a4 Espresso::ANERuntimeEngine::compiler::reset() + 1340 16 Espresso 0x00000001bb19cac8 Espresso::ANERuntimeEngine::compiler::~compiler() + 108 17 Espresso 0x00000001bacd69e4 std::__1::__shared_weak_count::__release_shared() + 84 18 Espresso 0x00000001ba944d00 std::__1::__hash_table<std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>, std::__1::__unordered_map_hasher<Espresso::platform, std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>, std::__1::hash<Espresso::platform>, std::__1::equal_to<Espresso::platform>, true>, std::__1::__unordered_map_equal<Espresso::platform, std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>, std::__1::equal_to<Espresso::platform>, std::__1::hash<Espresso::platform>, true>, std::__1::allocator<std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>>>::__deallocate_node(std::__1::__hash_node_base<std::__1::__hash_node<std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>, void*>*>*) + 40 19 Espresso 0x00000001ba8ea640 std::__1::__hash_table<std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>, std::__1::__unordered_map_hasher<Espresso::platform, std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>, std::__1::hash<Espresso::platform>, std::__1::equal_to<Espresso::platform>, true>, std::__1::__unordered_map_equal<Espresso::platform, std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>, std::__1::equal_to<Espresso::platform>, std::__1::hash<Espresso::platform>, true>, std::__1::allocator<std::__1::__hash_value_type<Espresso::platform, std::__1::shared_ptr<Espresso::net_compiler>>>>::~__hash_table() + 28 20 Espresso 0x00000001ba8e5750 Espresso::net::~net() + 396 21 Espresso 0x00000001bacd69e4 std::__1::__shared_weak_count::__release_shared() + 84 22 Espresso 0x00000001bad750e4 std::__1::__vector_base<std::__1::shared_ptr<Espresso::net>, std::__1::allocator<std::__1::shared_ptr<Espresso::net>>>::clear() + 52 23 Espresso 0x00000001ba902448 std::__1::__vector_base<std::__1::shared_ptr<Espresso::net>, std::__1::allocator<std::__1::shared_ptr<Espresso::net>>>::~__vector_base() + 36 24 Espresso 0x00000001ba8ed99c std::__1::unique_ptr<EspressoLight::espresso_plan::priv_t, std::__1::default_delete<EspressoLight::espresso_plan::priv_t>>::reset(EspressoLight::espresso_plan::priv_t*) + 188 25 Espresso 0x00000001ba95b7fc EspressoLight::espresso_plan::~espresso_plan() + 72 26 Espresso 0x00000001ba902078 EspressoLight::espresso_plan::~espresso_plan() + 16 27 Espresso 0x00000001ba8e690c espresso_plan_destroy + 372 28 CoreML 0x00000001c48c45cc -[MLNeuralNetworkEngine _deallocContextAndPlan] + 40 29 CoreML 0x00000001c48c43bc -[MLNeuralNetworkEngine dealloc] + 40 30 libobjc.A.dylib 0x00000001a67ed4a4 object_cxxDestructFromClass(objc_object*, objc_class*) + 116 31 libobjc.A.dylib 0x00000001a67f221c objc_destructInstance + 80 32 libobjc.A.dylib 0x00000001a67fb9d0 _objc_rootDealloc + 80 ~~~~ Our code that release the MLModel object ~~~~ Moreover, we use a synchronization mechanism to ensure that the release of the MLModel and the data processing of the model (by calling [model predictionFromFeatures]) do not occur simultaneously. What could be the possible causes of the problem, and how can we prevent it from happening? Any advice would be appreciated.

I want to get depth map that when camera zoom in or zoom out or switch to telephoto. I have got the depth map using ARkit that provide depth map that the colored RGB image from the wide-range camera and the depth ratings from the LiDAR scanner are fused together. Now I want to switch camera to telephoto and hope to get new depth map.

FB:FB16079804 Hello, I've made the FastAI's Cat vs Dog model into model that distinguishes lemons from limes and it all works fine in a notebook. I am now looking to transform this model into Core ML for my iOS app using TorchScript and Apple official guidelines for coremltools. Model converts but I cannot see the Preview Tab in. Xcode. Have anyone of you tried to convert to Core ML? I guess my input types are not matching with coremltools expectations for preview but I am stuck . Here is my code. import torch import coremltools as ct from fastai.vision.all import * import json from torchvision import transforms # Load your Fastai model (replace with your actual path) learn = load_learner('lemonmodel.pkl') # Example input image (you can use any image from your dataset) input_image = PILImage.create('example.jpg') # Preprocess the image (assuming you used these transforms during training) to_tensor = transforms.ToTensor() normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]) input_tensor = to_tensor(input_image) input_tensor = normalize(input_tensor) # Apply normalization # Add a batch dimension input_tensor = input_tensor.unsqueeze(0) # Ensure float32 type input_tensor = input_tensor.float() # Trace the model trace = torch.jit.trace(learn.model, input_tensor) # Define the Core ML input type (considering your model's input shape) _input = ct.ImageType( name="input_1", shape=input_tensor.shape, bias=[-0.485/0.229, -0.456/0.224, -0.406/0.225], scale=1./(255*0.226) ) # Convert the model to Core ML format mlmodel = ct.convert( trace, inputs=[_input], minimum_deployment_target=ct.target.iOS14 # Optional, set deployment target ) # Set model type as 'imageClassifier' for the Preview tab mlmodel.type = 'imageClassifier' # Correct structure for preview parameters** (assuming two classes: 'lemon' and 'lime') labels_json = { "imageClassifier": { "labels": ["lemon", "lime"], "input": { "shape": list(input_tensor.shape), # Provide the actual input shape "mean": [0.485, 0.456, 0.406], # Match normalization mean "std": [0.229, 0.224, 0.225] # Match normalization std }, "output": { "shape": [1, 2] # Output shape for your model (2 classes) } } } # Setting up the metadata with correct 'preview' params mlmodel.user_defined_metadata['com.apple.coreml.model.preview.params'] = json.dumps(labels_json) # Save the model as .mlmodel mlmodel.save("LemonClassifierGemini.mlmodel") mlmodel = ct.convert( trace, inputs=[_input], minimum_deployment_target=ct.target.iOS14 # Optional, set deployment target ) # Set model type as 'imageClassifier' for the Preview tab** mlmodel.type = 'imageClassifier' # Correct structure for preview parameters** (assuming two classes: 'lemon' and 'lime') labels_json = { "imageClassifier": { "labels": ["lemon", "lime"], "input": { "shape": list(input_tensor.shape), # Provide the actual input shape "mean": [0.485, 0.456, 0.406], # Match normalization mean "std": [0.229, 0.224, 0.225] # Match normalization std }, "output": { "shape": [1, 2] # Output shape for your model (2 classes) } } } # Setting up the metadata with correct 'preview' params** mlmodel.user_defined_metadata['com.apple.coreml.model.preview.params'] = json.dumps(labels_json) # Save the model as .mlmodel mlmodel.save("LemonClassifierGemini.mlmodel") My model is : Input batch shape: torch.Size([32, 3, 192, 192]) Labels batch shape: torch.Size([32]) Validation Loss: None, Validation Metric: None Predictions shape: torch.Size([63, 2]) Targets shape: torch.Size([63]) Code for the model : searches = 'lemon','lime' path = Path('lemon_or_not') for o in searches: dest = (path/o) dest.mkdir(exist_ok=True, parents=True) download_images(dest, urls=search_images(f'{o} photo')) time.sleep(5) resize_images(path/o, max_size=400, dest=path/o) dls = DataBlock( blocks=(ImageBlock, CategoryBlock), get_items=get_image_files, splitter=RandomSplitter(valid_pct=0.2, seed=42), get_y=parent_label, item_tfms=[Resize(192, method='squish')] ).dataloaders(path, bs=32) dls.show_batch(max_n=6) learn = vision_learner(dls, resnet18, metrics=error_rate) learn.fine_tune(3) is_lemon,_,probs = learn.predict(PILImage.create('lemon.jpg')) print(f"This is a: {is_lemon}.") print(f"Probability it's a lemon: {probs[0]:.4f}") This is a: lemon. Probability it's a lemon: 1.0000 learn.export('lemonmodel.pkl') I am stuck to why it doest show the Preview Tab.

I used the multifunction models feature introduced in iOS 18 to merge three VAE Encoder models with different resolutions into a single model. However, loading this merged model on iOS causes a crash with the error EXC_BAD_ACCESS (code=1, address=0x0). In contrast, merging VAE Decoder models using the same method does not result in crashes. Additionally, merging only two VAE Decoder models with different resolutions also leads to a crash when loaded on iOS. As for the Stable Diffusion Unet model, merging two or even three models does not cause any crashes, and it successfully generates images as expected. I use the following code to load the model: let config = MLModelConfiguration() config.computeUnits = .cpuAndNeuralEngine config.functionName = "test" try MLModel(contentsOf: url, configuration: config)

When using the imagePlaygroundSheet modifier in SwiftUI, the system presets an image playground in a fixed size. Especially on macOS, this modal is rather small and doesn't utilize the available space efficiently. Is there a way to make the modal bigger, or allow the user to resize the dialog? I tried presentationDetents, but this would need to be applied to the content of the sheet, which is provided by the system... I guess this question applies to other system-provided sheets like the photo picker as well.

I am using the depthAnything v2 provided by Apple on the developer website. On my iPhone 15 Pro, if I choose all or cpuAndNeuralEngine, it will stuck in loading models. let config = MLModelConfiguration() config.computeUnits = .cpuAndGPU//normal when not using neuralEngine. let model = try await DepthModel.load(configuration: config) with following error: E5RT encountered an STL exception. msg = MILCompilerForANE error: failed to compile ANE model using ANEF. Error=无法与帮助程序通信。. E5RT: MILCompilerForANE error: failed to compile ANE model using ANEF. Error=无法与帮助程序通信。 (11)

Hi everyone, I'm working on an iOS app that uses VisionKit and I'm exploring the .visualLookUp feature. Specifically, I want to extract the detailed information that Visual Look Up provides after identifying an object in an image (e.g., if the object is a flower, retrieve its name; if it’s a clothing tag, get the tag's content).

I'm working on a cross-platform AI app. It is a CMake project. The inference part should be built as a library separately on Windows and MacOS. On MacOS it should be built with objective-c and CoreML. Here's my step roughly: Create a XCode Project for CoreML inference and build it as static library. Models are compiled to ".mlmodelc", and codes are compile to binary ".a" lib. Create a CMake Project for the app, and use the ".a" lib built by XCode. Run the App. I initialize the CoreML model like this(just for demostration): #include "det.h" // the model header generated by xcode auto url = [[NSURL alloc] initFileURLWithPath:[NSString stringWithFormat:@"%@/%@", dir, @"det.mlmodelc"]]; auto model = [[det alloc] initWithContentsOfURL:url error:&error]; // no error The url is valid, and the initialization doesn't report any error. However, when I tried to do inference using codes like this: auto cvPixelBuffer = createCVPixelBuffer(960, 960); // util function auto preds = [model predictionFromImage:cvPixelBuffer error:NULL]; The output preds will be null and I got these errors: 2024-12-10 14:52:37.678201+0800 望言OCR[50204:5615023] [e5rt] E5RT encountered unknown exception. 2024-12-10 14:52:37.678237+0800 望言OCR[50204:5615023] [coreml] E5RT: E5RT encountered an unknown exception. (11) 2024-12-10 14:52:37.870739+0800 望言OCR[50204:5615023] H11ANEDevice::H11ANEDeviceOpen kH11ANEUserClientCommand_DeviceOpen call failed result=0xe00002e2 2024-12-10 14:52:37.870758+0800 望言OCR[50204:5615023] Device Open failed - status=0xe00002e2 2024-12-10 14:52:37.870760+0800 望言OCR[50204:5615023] (Single-ANE System) Critical Error: Could not open the only H11ANE device 2024-12-10 14:52:37.870769+0800 望言OCR[50204:5615023] H11ANEDeviceOpen failed: 0x17 2024-12-10 14:52:37.870845+0800 望言OCR[50204:5615023] H11ANEDevice::H11ANEDeviceOpen kH11ANEUserClientCommand_DeviceOpen call failed result=0xe00002e2 2024-12-10 14:52:37.870848+0800 望言OCR[50204:5615023] Device Open failed - status=0xe00002e2 2024-12-10 14:52:37.870849+0800 望言OCR[50204:5615023] (Single-ANE System) Critical Error: Could not open the only H11ANE device 2024-12-10 14:52:37.870853+0800 望言OCR[50204:5615023] H11ANEDeviceOpen failed: 0x17 2024-12-10 14:52:37.870857+0800 望言OCR[50204:5615023] [common] start: ANEDeviceOpen() failed : ret=23 : It seems that CoreML failed to find ANE device. Is there anything need to be done before we use a CoreML Model as a library in a CMake or other non-XCode project? By the way, codes like above will work on an XCode Native App with CoreML (I tested this before) . So I guess I missed some environment initializations in my non-XCode project?

Hey Chat, I'm researching personality analysis using LLMs, and I'm curious about whether Apple’s AI can be allowed access to your messages, Instagram DMs, and similar communications to perform a personality analysis based on your writing style. If anyone has insights on this, I would greatly appreciate your input. Thx a ton

I’m trying to use a Decimal as a @Property in my AppEntity, but using the following code shows me a compiler error. I’m using Xcode 16.1. The documentation notes the following: You can use the @Parameter property wrapper with common Swift and Foundation types: Primitives such as Bool, Int, Double, String, Duration, Date, Decimal, Measurement, and URL. Collections such as Array and Set. Make sure the collection’s elements are of a type that’s compatible with IntentParameter. Everything works fine for other primitives as bools, strings and integers. How do I use the Decimal though? Code struct MyEntity: AppEntity { var id: UUID @Property(title: "Amount") var amount: Decimal // … } Compiler Error This error appears at the line of the @Property definition: Generic class 'EntityProperty' requires that 'Decimal' conform to '_IntentValue'

I'm currently trying to add support for Image Playground to our apps. It seems that it's not working in an app that is "Designed for iPad" and runs on a Mac. The modal just shows a spinner and the following is logged to console: Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> Private sandbox for com.apple.GenerativePlaygroundApp.remoteUIExtension : <none> GP extension could not be loaded: Extension (platform: 2) could not be found (in update) dealloc Query controller [C32BA176-6A3E-465D-B3C5-0F8D91068B89] ImagePlaygroundViewController.isAvailable returns true, however. In a "real" Mac Catalyst app, it's working. Just not when the app is actually an iPad app. Is this a bug?

My app was rejected because of this error below but I cannot find any documentation on a key related to Image Playground. My app is set to minimum of 18.2 already. Rejection Message: The UIRequiredDeviceCapabilities key in the Info.plist is set in such a way that the app will not install on iPhone running iOS 18.1.1 Next Steps To resolve this issue, check the UIRequiredDeviceCapabilities key to verify that it contains only the attributes required for the app features or the attributes that must not be present on the device. Attributes specified by a dictionary should be set to true if they are required and false if they must not be present on the device. Resources Learn more about the UIRequiredDeviceCapabilities key.

I cannot find the hardware requirements for Image Playground documented anywhere. I'm also not sure if they are identical to devices that support Apple Intelligence. On the App Store, the only requirement listed for Image Playground is iOS 18.2. Not knowing the requirements is an issue because I need to be able to clearly state the requirements for the feature in my app description. Also, I'm sure my mother's current iPad is too old, but I'm not sure what models support it if I were to buy her a new one.

The OpenXLA PJRT plugin used by JAX is missing features, such as 64-bit floating point. Is there some way to track the progress of these features?

Hello all, I'm working on a project that involves listening to a person speak off of a script and I want to stop then restart the recognitionTask between sections so I don't run afoul of keeping the recognitionTask running for longer than it needs to. Also, I'd like to be able to flush the current input between sections so the input from the previous section doesn't roll over into the next one. This is based on the sample code for SFSpeechRecognizer so there's a chance I might be misunderstanding something. private func restartRecording() { let inputNode = audioEngine.inputNode audioEngine.stop() inputNode.removeTap(onBus: 0) recognitionRequest?.endAudio() recordingStarted = false recognitionTask?.cancel() do { try startRecording() } catch { print("Oopsie.") } } Here's my code. When I run it, the recognition task doesn't restart. Any ideas?

I'm trying to determine the best practice for handling if Image Playground is available but not installed or simply not supported. If ImagePlaygroundViewController.isAvailable is true, I will just display a button to start an Image Playground session. If it is false, does that mean ImagePlayground is supported but not installed? If it's supported and not installed, instead of a button to launch it, I want to display something like "Enable Apple Intelligence in Settings" or, better yet, a button that opens the Intelligence settings. Is that possible? But if it is on a system that doesn't support it, of course, I don't want to instruct the user to enable it. How can I determine if a device cannot install Image Playground? I read that Apple Intelligence requires iPhone 15 Pro, iPhone 15 Pro Max, and all iPhone 16 models, and no mention of the M1 iPad Pro, yet Image Playground runs on my M1 iPad Pro. What are the hardware requirements for Image Playground?

Submited as : FB16052050 I am looking to adopt Machine Learning in a more granular manner, going beyond just using pre-built Metal, Core ML, or Create ML approaches. Specifically, I want to train models using Open Python PyTorch libraries, as these offer greater flexibility compared to Apple's native tools. However, these PyTorch APIs are primarily optimised for NVIDIA GPUs (or TPUs), not Apple's M3 or Apple Neural Engine (ANE). My goal is to train the models locally without resorting to cloud-based solutions for training or inference, and to then convert the models into Core ML format for deployment on Apple hardware. This would allow me to leverage Apple's hardware acceleration (via ANE, Metal, and MPS) while maintaining control over the training process in PyTorch. I want to know: What are my options for training models in PyTorch on local hardware (Apple M3 or equivalent), and how can I ensure that the PyTorch model can eventually be converted to Core ML without losing flexibility in model training and customisation? How can I perform training in PyTorch and avoid being restricted to inference-only workflows as Core ML typically allows? Is it possible to use the training capabilities of PyTorch and still get the performance benefits of Apple's hardware for both training and inference? What are the best practices or tools to ensure that my training pipeline in PyTorch is compatible with Apple's hardware constraints and optimised for local execution? I'm seeking a practical, cloud-free approach on Apple Hardware only that allows me to train models in PyTorch (keeping control over the training process) while ensuring that they can be deployed efficiently using Core ML on Apple hardware.

I was installing TensorFlow metal in the environment called "arm64_tf'" in anaconda using command line "python -m pip install tensorflow-metal" in terminal and it shows : ERROR: Could not find a version that satisfies the requirement tensorflow-metal (from versions: none) ERROR: No matching distribution found for tensorflow-metal I have already tried using " conda install -c anaconda libffi" but it still doesn't work is there a solution ? Thanks apologies for my bad English

Hi, I'm working with vision framework to detect barcodes. I tested both ean13 and data matrix detection and both are working fine except for the QuadrilateralProviding values in the returned BarcodeObservation. TopLeft, topRight, bottomRight and bottomLeft coordinates are rotated 90° counter clockwise (physical bottom left of data Matrix, the corner of the "L" is returned as the topLeft point in observation). The same behaviour is happening with EAN13 Barcode. Did someone else experienced the same issue with orientation? Is it normal behaviour or should we expect a fix in next releases of the Vision Framework?