None of these resulted in build bustage using our default configuration,
but apparently ccache builds were broken due to a bytes/text issue. Wish
the original reporter had mentioned he was using ccache, but he saved me
a lot of busywork. Changes appear safe across whole range of Python
versions we support.
Forgot to test packaging on Linux this whole time.
Also, kept forgetting to commit these random changes to mozbuild
files that just refer to GNU/kFreeBSD consistently as kFreeBSD.
Not because it's the right thing to call it, but because it's
what we have in the enums that works consistently right now.
And ripping out support entirely would be out of scope.
Python no longer provides any version-related utilities in stdlib.
Did a thorough grep and made absolutely sure to replace every last
use. This was the work I was avoiding during my first port...
Ref: PEP 632, BZ 1777414, BZ 1858065, BZ 1766085
Got everything working under older Python 3, to deal with fewer issues at once.
Now the following had to be done:
- time.clock had to be replaced with time.process_time
- Drive letters now need explicit normalization because they are uppercase
by default (was the reason why I had to turn everything into a relpath before).
- Virtualenv works, but needs a couple extra modules because of new includes
in newer Python 3 versions.
Most notably, these tweaks were needed to fix ./mach build faster.
Also chose to set NSDISTMODE=copy by default for Windows, since
wildcards will be symlinks otherwise. This will always fail
because our binaries have the symlinked MSVC DLLs in their import
tables, and Windows loader silently fails in this situation.
For strings/bytes issue, used a similar strategy to what I saw in our old
layout code, having a flag set to be false by default as it's passed from
function to function, eventually being activated and doing something at
a specific point in the loop if it's ever true. The datetime stuff is
simply because in Python 3, integer comparisons were failing so I had
to make both sides be an integer Unix timestamp.
More small tweaks mostly related to decoding/encoding, some not even related to
failures but are more anxiously done in anticipation of something going wrong.
Even though these are called commits, I often revise little things several
times because I am not sure what the most robust approach would be.
The issue was essentially that the fourth field was being stripped due to the
way Python 3 handles text. I solved the problem originally with a "pad" hack to
add the extra space back in, since that actually gets interpreted as a reference
to the current directory. The eventual solution mostly involved using . instead of
a blank space to refer to the current directory. Don't know if the other stuff
was needed, may remove later if it causes any issues.
Most notable changes are the ugly stuff I had to do in FileAvoidWrite and
everything that interacts with it, which is where I expect a lot future pain
and revision to enter the picture from, if I have to revisit this part of the
work again.
Not really sure how to categorize these, except they are either
language-related cleanups, small bug fixes, or are careful to decode
things using the preferred OS encoding where reasonable. A lot of UTF-8 only
worked by luck because it only tried to decode ASCII bytes, or had good
replacements for bad chars due to explicit error handling.
I'm basically just commiting the last state before I update gyp, because
I can't really proceed without updating it, and that's a big enough step I want
it to have its own commit... and also don't want it mixed in with other stuff.
2to3 . --fix dict -wpn
Py2 dict was a list by default, the iterator versions
were switched around to be the default.
This lets you have the old behavior. Mechanical.
Ref: PEP 3106
https://peps.python.org/pep-3106/
Switch to using "create" instead of "makehybrid" when creating the disk image.
This fixes bogus extended attributes which interfere with the code signature.
Finally add any -bin or dylibs in the Resources folder since --deep skips that folder.
Added --with-macbundle-idenity configure option to set a codesign identity.
If no identity is set or cross-compiling from Linux no codesigning will be
done. Currently doing a full deep bundle v2 sign, instead of limited v1.
To make a long story short, there's an old flag called NSDISTMODE that
was never added to old-configure so it could be passed down through the
build system to all the places it needs to go nowadays if used in your
.mozconfig, but which still sort of works when set as an environment
variable. If you leave it unset, it uses relative symlinks. However, it
has two other modes. One of them is "copy" and the other is
"absolute_symlink." Copy simply copies the files into the directory, and
absolute_symlink attempts to use absolute symlinks instead of relative
ones.
I've been wondering for a while now if there was a way to make the
shared library files in `dist/bin` that we use `./mach run` against
*not* be relative symlinks, and this seems to be that elusive technique.
It seems to be a part of the institutional memory that was all but lost
shortly after Netscape went under. You mostly see a few references to it
in NSS, NSPR, and the Makefiles in the `config` directory. And also
there is one reference in a Makefile in the application directory, which
seems to explains why application executables themselves usually aren't
symlinks:
/platform/libs/nspr/src/pr/src/md/unix/Makefile.in#76
/platform/libs/nss/src/coreconf/UNIX.mk#34
/palemoon/app/Makefile.in#30
/platform/config/config.mk#396
My patch essentially revives NSDISTMODE and makes it work as intended
again, more or less. Some parts of the work are loosely inspired by this
bug that was never finished upstream, showing that Mozilla only
rediscovered it earlier this year while trying to disable symlinks in
dist/bin for WSL, as far as I can tell.
https://bugzilla.mozilla.org/show_bug.cgi?id=1699855
Basilisk-Dev
Jeremy Andrews
Brian Smith
FranklinDM
Moonchild
Matt A. Tobin
Matt A. Tobin