Detect issues like logic failures, UI problems, and performance regressions by running tests on your app.

Posts under Testing tag

109 Posts
Sort by:
Post not yet marked as solved
0 Replies
337 Views
I'm starting to set up a testing environment for macOS applications, and I want to be able to rollback to different versions of the OS (all machines can be scrubbed, so I don't worry about preserving user data). I noticed that I can download the latest versions of Ventura and Sonoma from the Mac App Store, but not past versions of Ventura and Sonoma Is there a way to download earlier versions of these macOSes (or maybe even earlier ones like Monterey)? Alternatively, if I regularly start downloading macOS installers from the Mac App Store and maintain my own local copies so I can use these to rollback to these versions, do I need to download a version for each hardware? For example, I currently have an M1 Mac mini, an M2 Mac mini, and an M1 Mac Studio. Do I need to download different installers for each of these hardware platforms?
Posted Last updated
.
Post not yet marked as solved
2 Replies
1.4k Views
The new test report, with the automatic video recording and scrubber, is great. I'm setting up different configurations for different languages to improve localization testing, but I was wondering if it was possible to make the simulator device type part of the configuration. For example, I'd like to have a single test plan with an "iPhone 14" test plan, an "iPad Air" test plan, etc. Then I would just press Cmd-U, and Xcode would run through each device in sequence, leaving me with videos of each test run that I could review in the test report. Is that possible?
Posted Last updated
.
Post not yet marked as solved
1 Replies
478 Views
I have noticed that, as of late, the iOS simulator has been incredibly slow. Even simple (and perhaps silly) actions like changing the wallpaper take an unreasonably long time, to the point where it is no longer clear whether the process has hanged. I tried to change the wallpaper to a photo of mine, but it simply would not appear. Even booting up takes several seconds, and home screen animations are laggy. I've given up on testing my build for now. I have tried with an iPhone Xr and an iPhone 15 Pro simulator - both running iOS 17.0. Both are extremely laggy. This is on an M1 MacBook Air running the stable version of macOS Sonoma 14.0. with Xcode version 15.0.1 and Simulator version 15.0.1. I have checked the Activity Monitor for CPU and RAM. There is nothing abnormal. I would appreciate any suggestions.
Posted
by Lightmage.
Last updated
.
Post not yet marked as solved
0 Replies
567 Views
I'm encountering an intermittent issue while trying to run safaridriver on macOS Sonoma. Here are the details of the problem: I ran sudo /usr/bin/safaridriver -p0 --enable in the beginning. After that when I run this multiple times /usr/bin/safaridriver -p0, I sometimes receive an "Operation not permitted" error, but not consistently. This issue seems to occur intermittently. I've checked the sudo logs, and I see the following error message: kernel: (Sandbox) Sandbox: com.apple.WebDriver.HTTPService(2049) deny(1) user-preference-write com.apple.WebDriver.HTTPService cfprefsd: (CoreFoundation) [com.apple.defaults:cfprefsd] rejecting write of key(s) MobileDeviceRemoteXPCEnabled in { com.apple.WebDriver.HTTPService, nimish, kCFPreferencesAnyHost, /Users/nimish/Library/Preferences/com.apple.WebDriver.HTTPService.plist, managed: 0 } from process 2049 (com.apple.WebDriver.HTTPService) because setting these preferences requires user-preference-write or file-write-data sandbox access cfprefsd: (CoreFoundation) [com.apple.defaults:cfprefsd] Couldn't open parent path due to [2: No such file or directory kernel: (Sandbox) Sandbox: com.apple.WebDriver.HTTPService(2049) deny(1) network-bind local:*:7055 com.apple.WebDriver.HTTPService: (WebDriver) [com.apple.WebDriver:WebService] Error starting HTTP server listening on localhost:0: Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted" UserInfo={NSLocalizedDescription=Operation not permitted, NSLocalizedFailureReason=Error in bind() function} com.apple.WebDriver.HTTPService: (WebDriver) [com.apple.WebDriver:XPCService] Client connection invalidated for some reason I am getting this on Mac Os Sonoma Can anybody please help on this?
Posted Last updated
.
Post not yet marked as solved
1 Replies
372 Views
I have an animation app that runs fine on all iPhones I've tested but one. Menus show up differently, Tool bars appear differently. And worst of all, the image skews when painting with the fill tool. How can I find out what makes these things different on one iPhone than on others? The phone in question is an iPhone 13 Pro with plenty of memory available.
Posted
by KennyB.
Last updated
.
Post not yet marked as solved
1 Replies
401 Views
In order to load your data into the production environment in Apple CloudKit, follow the steps below: Deploy Your Schema: Initially, deploy your schema to the production environment using the recommended approach provided by Apple. Prepare for TestFlight: Retain all the codes and views utilized previously for uploading data to the development environment. Deploy your app to TestFlight. TestFlight Approval and Data Transition: Upon receiving approval and ensuring the functionality on TestFlight, the same codes and views that were used to upload data to the CloudKit records in the development environment will now point to the production environment. Consequently, all the data will be uploaded to the production environment, facilitating a smooth transition. Post this process, your app on TestFlight will interact with the data in the production environment of CloudKit, ensuring consistency and enabling effective testing. This method ensures a seamless transition of data from the development environment to the production environment, aligning with the operational flow of CloudKit and TestFlight.
Posted Last updated
.
Post not yet marked as solved
0 Replies
494 Views
Hello, I am looking for help with using TestPlans in my ci tool. My workspace consists of my main project and 2 frameworks, that are linked to my main project. I am using fastlane to run the test plan associated with my main app's scheme, which has my main project's test suite and my 2 frameworks' test suites. When using fastlane locally, it seems to build every test suite and run tests accordingly, but on my CI tool (I am using CircleCI), it just runs my main project's test suite. Can I get some help on this? I have tried to add a failing test to my CI tool and the tests still pass. Any help is much appreciated!
Posted
by kawaxman.
Last updated
.
Post not yet marked as solved
0 Replies
344 Views
I've developed an app utilizing substantial data uploaded to a CloudKit record in the development environment. Having deployed the Record Type schema to production and released a TestFlight version for testing, how can I transfer the data from the development to the production environment, or upload the data directly to the production environment? This is crucial as TestFlight testers can only access data from the production environment.
Posted Last updated
.
Post not yet marked as solved
0 Replies
266 Views
TestFlight on external testing requires a newer version of the ios update if according to the program parameters it is 16.4 and on the device it is 16.7?? Test app prevents download due to an error: To test this app, please update your device software to the latest version.
Posted Last updated
.
Post not yet marked as solved
1 Replies
489 Views
In my iOS app, When the API calls failed due to poor network, the app will display 'No network error screen'. I have to check if this error screen is displayed correctly or not in case of poor network, in automation testing using simulator. But I couldn't find any possibility to turn off internet manually in simulator and check whether the 'No network' screen displays. I installed 'Network Link Conditioner' in macOS and tried using it on simulator. But, in simulator settings -> developer , there is no section called "Network Link Conditioner" to handle this in simulator. So, is there any other solution to handle this network case in simulator ?
Posted Last updated
.
Post not yet marked as solved
7 Replies
2.8k Views
We are having an issue when trying to view the the coveage report in our "merged result bundle" Running xccov view --report --only-targets merged.xcresult results in: 09:52:54 Error: Error Domain=XCCovErrorDomain Code=0 "Failed to load coverage archive in scheme action '(null)' in result bundle" UserInfo={NSLocalizedDescription=Failed to load coverage archive in scheme action '(null)' in result bundle, NSUnderlyingError=0x7f8ff8714f30 {Error Domain=NSCocoaErrorDomain Code=260 "The file “Metadata.plist” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/tmp/action.xccovarchive/Metadata.plist, NSUnderlyingError=0x7f8ff8714930 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}} Checking the man page for xccov I was inspired by the idea of extracting the coverage report from the result bundle to try to understand or work around our error. The man xccov page states that If xccov is passed a result bundle directly, it will extract the report or archive implicitly, as part of the action being requested. Alternatively, the raw report/archive files can be extracted from the result bundle using xcresulttool(1) and subsequently passed to xccov. Checking man xcresulttool and trying out some stuff randomly I was not able to learn how a person might: Given an xcresult bundle use xcresulttool to create/extract an xccovreport bundle.
Posted Last updated
.
Post not yet marked as solved
0 Replies
483 Views
I can't work out how to deal with the dialog presented when an app first tries to access a photo library: This screenshot shows the dialog still presented after the app has quit, which suggests to me that it's a system-level rather than app-level dialog; but the handler for addUIInterruptionMonitor(withDescription:handler:) isn't called (whereas it is for both the "Allow MyApp to use your location?" and "MyApp Would Like to Send You Notifications" dialogs). But nor does it appear to be an app-level dialog under XCUIApplication. How can I handle this dialog in my UI testing? Thanks, Hamish
Posted
by hatfinch.
Last updated
.
Post not yet marked as solved
2 Replies
852 Views
Hi friends, I have a problem suddenly by archiving my app in XCode 14.3 I'm getting this errors: Showing All Errors Only Prepare build note: Building targets in dependency order error: Multiple commands produce '/Users/razvanu/Library/Developer/Xcode/DerivedData/Runner-aeiwqtydtcicmbcjpyzoxtfwhzig/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/GoogleUtilities.framework' note: Target 'GoogleUtilities-00567490' (project 'Pods') has create directory command with output '/Users/razvanu/Library/Developer/Xcode/DerivedData/Runner-aeiwqtydtcicmbcjpyzoxtfwhzig/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/GoogleUtilities.framework' note: Target 'GoogleUtilities-54e75ca4' (project 'Pods') has create directory command with output '/Users/razvanu/Library/Developer/Xcode/DerivedData/Runner-aeiwqtydtcicmbcjpyzoxtfwhzig/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/GoogleUtilities.framework' Multiple commands produce '/Users/razvanu/Library/Developer/Xcode/DerivedData/Runner-aeiwqtydtcicmbcjpyzoxtfwhzig/Build/Intermediates.noindex/ArchiveIntermediates/Runner/IntermediateBuildFilesPath/UninstalledProducts/iphoneos/GoogleUtilities.framework' I tried all the solutions that I found, nothing works for me :( please guide me how to fix this issue, thanks in advance!
Posted Last updated
.
Post not yet marked as solved
5 Replies
2.6k Views
Sometimes when running UI tests, the tests fail on keyboard input because the "Speed up your typing" keyboard tutorial overlay obscures the keys. This happens randomly, but sometimes repeatedly. Is there a way to turn this off? Checking for this tutorial before any keyboard input would significantly complicate tests just to check for one edge case...
Posted
by Aurelian.
Last updated
.
Post not yet marked as solved
3 Replies
1.3k Views
Hi everyone Premise: it works with Xcode 14.2 I have a problem with fastlane and slather, it gives me an error that it cannot find the "code coverage folder" Here are some details Fastlane file slather( build_directory: "./DerivedData", scheme: "MyApp", workspace: "MyApp.xcworkspace", proj: "./MyApp.xcodeproj", output_directory: "./DerivedData/testResult", cobertura_xml: true, verbose: true ) Error: Exit status of command 'slather coverage --cobertura-xml --build-directory ./DerivedData --output-directory ./DerivedData/test-output/slatherOutput --ignore Pods/\* --ignore fastlane/\* --ignore build/\* --ignore DerivedData/\* --verbose --scheme ProjectName --workspace ProjectName.xcworkspace ./ProjectName.xcodeproj' was 1 instead of 0. Slathering... /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/project.rb:239:in `profdata_coverage_dir': No coverage directory found. (StandardError) from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/project.rb:517:in `find_binary_files' from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/project.rb:336:in `configure' from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/lib/slather/command/coverage_command.rb:59:in `execute' from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/command.rb:66:in `run' from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/subcommand/execution.rb:18:in `execute' from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/command.rb:66:in `run' from /usr/local/lib/ruby/gems/3.0.0/gems/clamp-1.3.2/lib/clamp/command.rb:140:in `run' from /usr/local/lib/ruby/gems/3.0.0/gems/slather-2.7.4/bin/slather:17:in `<top (required)>' from /usr/local/lib/ruby/gems/3.0.0/bin/slather:25:in `load' from /usr/local/lib/ruby/gems/3.0.0/bin/slather:25:in `<main>' No coverage directory found. Are you sure your project is generating coverage? Make sure you enable code coverage in the Test section of your Xcode scheme. Did you specify your Xcode scheme? (--scheme or 'scheme' in .slather.yml) If you're using a workspace, did you specify it? (--workspace or 'workspace' in .slather.yml) If you use a different Xcode configuration, did you specify it? (--configuration or 'configuration' in .slather.yml) Any ideas in order to fix this issue? Thanks
Posted
by n3o999.
Last updated
.
Post not yet marked as solved
0 Replies
256 Views
Hi all, I'm looking for a way to get test coverage for the Swift APIs that don't have explicit implementations (e.g., properties, enum cases, type names, protocols, etc). Specifically, to understand whether they are used during execution or not. It looks like Xcode 14 skips all mentioned Swift constructs in the coverage report both in Xcode UI and in the .xccov file produced from .xcresult. Is there any way to get it?
Posted
by ylapitsky.
Last updated
.
Post not yet marked as solved
2 Replies
1.6k Views
I'm having problems with UI tests on an M1 runner using Xcode 14.3.1. In my project, I need to build using the Rosetta simulator because I have a dependency that is not updated to support the arm simulator. This dependency relies on OpenCV." This is the command that I use when I want to run UI tests on CI: xcodebuild ARCHS=x86_64 clean test -scheme UITests -configuration Debug -destination 'id=50E9C593-9A40-4E4D-B58D-1DE437CFBE3A' -parallel-testing-enabled YES -parallel-testing-worker-count 3 -maximum-concurrent-test-simulator-destinations 4 -test-iterations 3 -retry-tests-on-failure And this code run only with 1 clone of simulator. Anyone know why?
Posted
by smajekp.
Last updated
.
Post not yet marked as solved
0 Replies
368 Views
Based on Apple, you can only register up to 100 devices. And you can only remove devices at the beginning of a membership renewal cycle. However in testflight, you can create up to 10,000 external testers. AFAIK, you need to register your device in order to create the provisioning profile for you to be able to install the app from testflight on your device. So question is, do external testers need to have their devices registered, to be able to install the app on their phone? Or is it not necessary to register their devices? If they need to register their devices, then what good would it do to be able to create 10,000 external testers since only 100 of them can install the app?
Posted
by resting.
Last updated
.
Post not yet marked as solved
6 Replies
2.8k Views
On an intel mac, my XCUITest test cases which is included swipe actions works perfectly but it doesn't work on my m1 mac. I tried running simulator with Rosetta but it doesnt work out it still gives an error when I run my cases because it clicks instead of swipe. Does anyone have a solution for swiping problem on the m1?
Posted
by beyz.
Last updated
.