Cross compilation incorrectly triggered, resulting in 40-85% build time regression

[MahdiBM]

Is it reproducible with SwiftPM command-line tools: swift build, swift test, swift package etc?

• Confirmed reproduction steps with SwiftPM CLI. The description text must include reproduction steps with either of command-line SwiftPM commands, swift build, swift test, swift package etc.

Description

The original issue is Significant build regressions on swift:6.0-noble compared to 5.10-noble.
The problem described below has the biggest impact in build regressions mentioned in the issue above, but it doesn't cover everything that the issue above was filed for. All those other problems seem to have already been addressed.

It appears when compiling on GHA ubuntu containers and swift images, Swift 6.0's SwiftPM compiles everything twice compared to Swift 5.10's.
The extra builds seem to have -tool added in the object names.

This issue's impact seems to have been multiplied by this other related issue I think(?): #8078
When the double builds are triggered, SwiftPM double builds almost everything instead of at least just double building only what's actually needed in our macros.

Expected behavior

How Swift 5.10 behaves and does not double-compile.

Actual behavior

Double compiles that result in 40% bigger build times in Swift 6.0 compared to 5.10 in smaller machines (that majority of users use, e.g. default GHA machines), or 85% if you use -j.

Steps to reproduce

This issue already took long enough to figure out. I'll see if i find time / mood to put an easy-to-use reproduction project together.

Swift Package Manager version/commit hash

5.10 RELEASE, 6.0.3 RELEASE

Swift & OS version (output of swift --version ; uname -a)

5.10 RELEASE, 6.0.3 RELEASE

[finagolfin]

Without a repro, nobody will ever likely look at this. 😿

[MahdiBM]

To be honest I don't think my chances of someone looking at this are that high anyway. I know the SwiftPM folks are busy so I'm not saying they are being "lazy" or something 😅 but yeah that still doesn't change the fact that I don't think chances of someone looking into this issue is low.
That's why I'll try to investigate it later myself and see if I can figure out the root of the issue.

[finagolfin]

Yes, given that you can't share your source, your best bet is to build a small repro package that has some of the same characteristics of your repo and see if it has the same problem. If you can reproduce it that way and share that instead, then somebody here would have something to look into.

[dschaefer2]

At this point no. That was an intentional design decision to ensure the -tools targets were built for host while allowing the regular targets to be cross compiled.

All eyes are now on the Swift Build integration which will do those things very differently.

[MahdiBM]

@dschaefer2 I'm looking forward to using SWBuild, but at the same time I'm not hopeful enough to expect it to properly work until Swift 6.2 or after that.

We're somewhat lucky to be able to afford AWS machines to run GitHub Actions on at work, so this is not even a big set back for us anymore, but the current server side swift build situation is bad enough that a bunch of builders can't even build SSS projects (DigitalOcean's for example, IIRC) and a bunch more only build the project if it's not too big as to make the linker run out of ram.
This regression made the situation even worse. We started getting a decent amount of new reports about Heroku failing to build projects with Swift 6.
I'd rather we/I try to fix this for now instead of waiting for SWBuild, assuming it doesn't require a tremendous amount of work.

[dschaefer2]

it doesn't require a tremendous amount of work.

That would be the problem. It's pretty fundamental to the old native build system design that the builds are broken up between plug-in tools builds and the builds that use the outputs of those tools. This enabled us to use SwiftPM for cross compilation builds. The difficulty to manage that with that old system is why we really needed something better.

[finagolfin]

OK, so it sounds like the fundamental issue here is using build plugins forces duplicate building of all dependencies with Swift 6.0 even when natively compiling? If so, maybe one of you can update the title to say that. If there's no plans to fix that now, at least we can verify that the new swift-build backend fixes it later.

@dschaefer2, any plans to get swift-build in for the upcoming 6.1 release, or will it need more time to bake with 6.2?

[dschaefer2]

Given 6.1 has branched and the auto merger has been turned off from main, and we just started working on Swift Build integration with SwiftPM, no it won't be in 6.1. Lots of work left to do. We'll have to see how it plays out. We really want to make sure we don't have regressions when we switch over.

[MahdiBM]

it sounds like the fundamental issue here is using build plugins forces duplicate building of all dependencies with Swift 6.0 even when natively compiling

TBH I haven't pin pointed the exact issue so not sure if that's correct. It might be macro related.

[MahdiBM]

build plugins forces duplicate building

We're not using "Build" plugins, but we do use the OpenAPI Generator command plugin.

[finagolfin]

I don't think it's macros, as I wasn't able to reproduce #8078 with macros even when cross-compiling with Swift 6.0. Can you try removing that plugin, or whatever other plugins you use too, and see if it makes a difference?

[MahdiBM]

Should be possible but it's in the swift-openapi-generator package itself so will take some work to remove that out of the whole app to test if it makes a difference.

[MahdiBM]

@dschaefer2 I thought I should ask: Are there any workarounds in the meantime?

[dschaefer2]

Not that I know of, sorry. We've heard many complaints. @rauhul has an even tougher case with swift-mmio which ends up building swift-syntax at least 3 times with macros, plug-in tools, and his run-time. We are motivated to fix this but right now it's all hands on deck to make sure replatforming onto Swift Build is successful.