[Unix] Go back to checking the runtime resource path for swiftrt.o if -sysroot unspecified#1822
[Unix] Go back to checking the runtime resource path for swiftrt.o if -sysroot unspecified#1822finagolfin wants to merge 2 commits intoswiftlang:mainfrom
-sysroot unspecified#1822Conversation
|
Ready for review along with the required linked frontend pull, which passed the linux CI when tested with this pull (this pull has no effect on Darwin and Windows). Comments should mostly go there, since this is just a consequence of that pull. |
|
@swift-ci please test |
|
Alright, this is ready to go, a partial revert of #1667, independent of the linked changes in swiftlang/swift#79621. This pull alone passed CI, along with a full Windows toolchain build. I'd like to get this into 6.1 also next. Some background: the prior change checked the As for the original cross-compilation model referenced in that doc, which is still the one commonly used, this change I'm reverting often breaks builds because @artemcm, please review. @compnerd, let us know what you think. |
|
Ping @artemcm, mind approving this? I'd like to get it into 6.1 next and I don't think Saleem will respond. Ever since I argued against this change in the linked pull last year, I have occasionally raised the problems that this caused, eg swiftlang/swift#76381, and got no response. I think they were probably experimenting with new SDK layouts at the time, tried this hack, then moved on to other things. Let's fix this now, otherwise it is going to start breaking 6.1 SDK bundles and toolchain builds on Unix, ie the two main scenarios where an |
|
@etcwilde, mind reviewing this? I suspect I'm not going to get a response from those above, since they approved the original pull that broke this, and I'd like to get this fixed before it finally ships in 6.1. |
etcwilde
left a comment
etcwilde
left a comment
There was a problem hiding this comment.
swiftrt.o is associated with the SDK content though so that the sections are set up correctly for the implementation of swift_addNewDSOImage coming from the stdlib.
During the hybrid phase, this means that we should be grabbing swiftrt from the same place that we found libswiftCore (either via the sdk or resource-dir), though for expedience I'm fine with not hooking that exact logic up today.
As a compromise, can we continue to search the SDK first, and if it's not found there, fall back on the resource-dir?
|
"SDK" can mean different things under the different cross-compilation models, so I try to be more specific. Most compilation does not set the
The problem with this previous Unix change for most cross-compilation SDK bundles is that they use the old model of As for searching both, the problem is it might find a |
|
Ping @etcwilde or @artemcm, I think this should be merged before the 6.1 release, as SDK bundles and building the Swift toolchain itself with Swift 6.1 is going to start breaking on non-Darwin Unix otherwise, as shown in swiftlang/swift#76381. I get these runtime resources from an external |
|
@shahmishal, could you take a look at this, as it's been a couple weeks and I haven't really been able to get a review here? I tried building the 6.1 compiler with a prebuilt 6.1 on linux to see if the change this is reverting broke anything and it didn't, as the compiler build is careful to use clang for linking Swift executables instead of swift-driver, but the moment that changes, this previous change that I'm reverting will break the compiler build. It has already broken cross-compiling with SDKs on my Android CI with the I'm asking you since you're the review manager for linux and I'd like to get this into 6.1 next. |
|
@al45tair, I know that this will affect the static linux SDK, do you have feedback?
This is true. But we are also moving away from the model where the SDK content is shoved in the compiler resource directory. @compnerd, you were driving the push to split the system C sysroot flag from the SDK flag. What is missing in |
Essentially nothing has been done to make that move, other than this change looking for a single file alone, which means currently under your new model, the 6.1 compiler always looks in two different directories for Swift runtime resources like
Yes, the invocation in swiftlang/swift#76381 is using the current |
|
Sigh, with all the delays in reviewing this, 6.1 has now shipped with this regression, so we'll have to fix it in a patch release. |
|
@swift-ci test |
|
@swift-ci test windows |
|
Forced another Windows CI run because the prior one was not registering with github, which means it would have been useless for the merge. |
|
@etcwilde and @compnerd, this previous change I'm reverting broke compiling Unix SDK bundles using external platform C/C++ SDKs with the Can we go ahead and get this in now? |
|
This is the final blocker for being able to have the Swift Android SDK use a completely external NDK sysroot without any gruesome hacks. Is there hope for getting this PR in? Or, at least, are there any objections or concerns that we can assuage? There were no responses from @al45tair or @compnerd to @etcwilde's inquiry about whether this might affect the existing SDKs — perhaps they could take another look and offer their thoughts? |
|
@swift-ci test |
|
This issue is breaking tests on the community Android CI: @compnerd, if you think you need this change being reverted for the Android SDKs in your Windows toolchain, can you apply this patch and let us know what error you're seeing? @etcwilde, there is a bug in this new But by hacking this in explicitly here, he has broken the build of those who do specify both It may be possible to reconcile all this with additional info, as I just asked from Saleem, but we've gotten silence on the matter since it was merged last summer in #1667, despite my repeatedly raising the issue since then. If someone needs that change, they should be able to explain why. If they cannot or just aren't using these Android SDKs anymore, we should merge this and go back to the way things were. Pinging @shahmishal too, as this is breaking the upcoming Android SDK bundle with an external NDK, and this pull has received no real review for the last four months. |
|
@nkcsgexi, could you take a look at this simple pull? It does essentially the same as the workaround suggested by Artem and others here, but I haven't been able to get them to review it. |
|
@jakepetroules, no response from potential reviewers on this simple pull for the last month, since I switched it up to follow the approach you and others put forth for Marc's similar pull earlier. Would you take a look? |
|
@owenv, haven't been able to get review from the others who proposed a similar approach months ago, could you take a look at this pull? |
|
Our CI is currently experiencing some trouble due to changes in main. Once that is restored, I think that we should be able to try a run with this. I still don't think that this is the right approach - we should be splitting out the resource dir and the SDK content and this is going backwards. |
So far, zero work has been done on splitting out compiler-specific resource directory content and those modules/libraries that belong in a target-specific SDK, so that the Swift compiler expects them in different places: that is a separate concern. This pull merely addresses the work you did to add the C |
|
I started a build with this PR on the BCNY CI. |
With these changes, the build failed to build the Android SDK with the following: It goes on for quite a bit. I am not certain whether the lack of path separators are an artifact of the display output or the source of the issue. |
|
Huh, that's very strange: is there a full build log we can look at? This pull passed all official CI last week and changes nothing for your |
|
I do not think you can access our logs, but here is the build. The relevant failing job has this CMake invocation: And here is the swiftc invocation that eventually fails: |
|
Huh, that build log appears to download a prebuilt Swift compiler from somewhere else, whereas you will have to apply this swift-driver pull to a fresh build of the Swift compiler itself, then use that freshly-built, modified toolchain to either cross-compile an Android SDK or cross-compile a package for Android, in order to test this pull. In any case, it looks like you're having a CMake path separator issue with Foundation object files, which this pull cannot cause, so even if this pull was applied properly, probably something else went wrong with that test run, eg some upstream issue. |
Our build runs across multiple jobs, each of them builds a part of the toolchain/SDK. In this case, the compiler comes out of the
The only difference between the build I linked and the build on |
Does it also build the swift-driver later? I ask because the mac/unix
This change only affects the path to a single stdlib object file and it tries to keep everything the same when I think you should run this through your CI again, and make sure any other outside patches aren't being inadvertently applied also. |
|
@finagolfin I rebased your branch in my fork and started a new build based on that. I'll let you know how it goes. There are no other changes to the repository, other than the |
@finagolfin This passed after I did that. I believe your branch is too far behind, so it was probably missing a few fixes from the main branch. |
|
Alright, this passed all CI again once rebased, just as it did in November, before winter break. @compnerd, this small measure fixes the semantic break from how the Whatever plans you and Evan may have that "we should be splitting out the resource dir and the SDK content," that work has not been done and the More importantly, the SDK bundle JSON format has no support for passing in a Let's get this in and fix this for the upcoming 6.3 release, and you can keep working on the new |
|
@shahmishal, not getting a meaningful response here: can you ask Xi Ge or Doug or somebody who knows these cross-compilation matters well to judge the best path forward? Otherwise, this cross-compilation has been broken for all Unix platforms, including linux, since Swift 6.1 last year, which I'd like to get fixed before the upcoming 6.3 release. We have been working around this long-standing bug by @marcprux sticking a bunch of symlinks in to the Android SDK bundle, among other measures, but the workaround in this swift-driver pull is a better way to separate the currently popular |
finagolfin
requested review from
DougGregor,
artemcm,
cachemeifyoucan,
compnerd and
owenv
as code owners
|
Simply rebased this one-line pull @swift-ci test |
finagolfin
changed the title
-sysroot unspecified
5 participants
This is made possible by the swift-frontend now setting this instead in swiftlang/swift#79621. More changes incoming...