I'm trying to create a distributable xcframework with module stability enabled.

To test if module stability is functioning correctly (using Xcode 13.3), I installed a old swift toolchain for xcode that I downloaded from Swift.org (swift-5.1.3-RELEASE-osx.pkg) to try and build my project using it and see if the generated framework can be used in a project that might be using a newer toolchain. However the build fails with a couple of errors, including from NSBundle.h, NSURLSession.h and one that went Could not build module 'Foundation'.

I tried creating a sample project with just one swift file and one simple function and tried to build it using the Swift 5.1.3 toolchain to see if code/ settings in my project is causing a issue. But that failed with the exact same error. Am I missing some settings or going about doing this the wrong way?