Incredible huge .xcresult is generated by Xcode 13.2.1 in case of failure

Question

Created Jan ’22

Replies 9

Boosts 3

Views 4.5k

Participants 9

We use .xcresult files on our CI and recently noticed that size of file significantly increased. So previously we had files ~ 50 MB but now they are ~ 300 MB.

After investigation of .xcresult contents we found that it contains macOS Logs - test-session-systemlogs-2022.01.05_13-13-07-+0000

In testmanagerd .log we see in a last line: Moved logarchive from /var/tmp/test-session-systemlogs-2022.01.05_13-13-07-+0000.logarchive to ../Staging/1_Test/Diagnostics/My Mac_4203018E-580F-C1B5-9525-B745CECA79EB/test-session-systemlogs-2022.01.05_13-13-07-+0000.logarchive

Issue is reproducible on a new project. Steps:

File -> New -> Project
Select macOS -> App
Enable include test
For simplicity disable(comment) all the UITests except ResultSampleUITests::testExample
Add XCTAssertTrue(false) after app.launch()
Run tests from console and specify path to resultBundlePath

xcodebuild -project ResultSample.xcodeproj -scheme ResultSample -resultBundlePath ~/Downloads/2/all-tests -test-timeouts-enabled YES test
7. When tests are finished navigate to resultBundlePath and check size of all-tests.xcresult.

It has size 51.6 MB for me.

If change XCTAssertTrue(false) to XCTAssertTrue(true) then size of all-tests.xcresult is decreased to 31 KB.

Any idea how to disable/delete macOS diagnostic logs or decrease a size of .xcresult

Answered by Apple Designer in 726552022

You can stop XCTest from collecting test diagnostics when it encounters a failure or error in 1 of 2 ways:

From within the test plan. You can set the Collect test diagnostics on failure setting to Never
Or from the Command-line, with the flag -collect-test-diagnostics never

The command line setting overrides the test plan setting, so opposing values have a deterministic behavior. For example, if you set the test plan to Always but the commandline to Never, then test diagnostics are not collected.

Boost

Answer 1

myurik2 OP

Jan ’22

Opened Feedback Assistance - FB9835135

0

Answer 2

DTS Engineer OP

Apple

Jan ’22

Opened Feedback Assistance - FB9835135

Thanks for that.

Any idea how to disable/delete macOS diagnostic logs

This looks like it’s capturing a copy of the system log. You can clear that log using:

% sudo log erase

I don’t know if there’s a way to disable log archive collection within XCTest. It seems like this is something that should be configurable in your test plan, but I can’t see any obvious knobs to twiddle there.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 3

myurik2 OP

Jan ’22

@eskimo, thank you for reply.

I tried sudo log erase but it decreases a size of xcresult on about 20%. XCResult still has a ton of system logs attached.

Unfortunately, I haven't found options where to disable logs in test plan or test target. Also xcodebuild -help doesn't have options to disable log.

I see following in diagnostic logs:

13:18:54.350 xcodebuild[1380:26502] Initiating control session with IDE capabilities: <XCTCapabilities: 0x7fa80b480e70>: {
}
13:18:54.351 xcodebuild[1380:26502] XPC connection for control was invalidated.
13:18:54.363 xcodebuild[1380:26502] Got reply to control session initiation request (result:error): <XCTCapabilities: 0x7fa80b2ad320>: {
    "authorize test session capability" = 1;
    "initiate daemon control session capability" = 4;
    "initiate test runner session capability" = 2;
    "report crashes capability" = 1;
    "request log archive capability" = 1;
    "request spindump capability" = 2;
}: (null)

I tried to dig what's XCTCapabilities but didn't find enough info. Looks like it's some private API. I wonder is any way to configure these capabilities. request log archive capability looks very promising.

0

Answer 4

DTS Engineer OP

Apple

Jan ’22

it decreases a size of xcresult on about 20%.

So, it’s helps, but not enough.

I wonder is any way to configure these capabilities.

I don’t know. I had a cursory look at this last week and couldn’t find anything that’d be obviously helpful. The next step is to open a DTS tech support incident and see if our tool specialist can help you out.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 5

siejkowski_proton OP

Jan ’22

I've encountered the same issue, it severely impacts my ability to store the test results for introspection.

As the temporary measure I'm no longer keeping .xcresult bundle, but processing it with https://github.com/XCTestHTMLReport/XCTestHTMLReport into an HTML file.

Hoping for a solution coming from tool specialists or an ability to customise logs archiving behaviour in next Xcode version.

0

Answer 6

myurik2 OP

Jan ’22

I don’t know. I had a cursory look at this last week and couldn’t find anything that’d be obviously helpful. The next step is to open a > DTS tech support incident and see if our tool specialist can help you out.

Not sure If I want to submit DTS since I have only one ticket left. Will it be credited back for this report?

As the temporary measure I'm no longer keeping .xcresult bundle, but processing it with https://github.com/XCTestHTMLReport/XCTestHTMLReport into an HTML file.

Good choice. Our workaround: save critical data outside .xcresult and delete .xcresult if it greater than 10MB

0

Answer 7

DTS Engineer OP

Apple

Jan ’22

Will it be credited back for this report?

Standard DTS policy is that we bill an incident if the DTS engineer spends a significant amount of time on it. Seeing as that’s the whole point of this exercise, it’s likely you’ll be billed.

Share and Enjoy
—
Quinn “The Eskimo!” @ Developer Technical Support @ Apple
let myEmail = "eskimo" + "1" + "@" + "apple.com"

0

Answer 8

benrudhart OP

Feb ’22

Wondering if you were able to solve this as we're facing a similar behavior of .xcresult files getting huge. Weird thing on our side is that we have several projects in our workspace and only one is affected.

1

Answer 9

Apple Designer OP

Apple

Sep ’22

Accepted Answer

You can stop XCTest from collecting test diagnostics when it encounters a failure or error in 1 of 2 ways:

From within the test plan. You can set the Collect test diagnostics on failure setting to Never
Or from the Command-line, with the flag -collect-test-diagnostics never

The command line setting overrides the test plan setting, so opposing values have a deterministic behavior. For example, if you set the test plan to Always but the commandline to Never, then test diagnostics are not collected.

0