This fixes the original Python wrapper. So basically, the Python developers
have a poor grasp of how Windows handles anything outside of UTF-16 and
the W APIs, and barely understand ANSI codepages, let alone console codepages.
Both MSYS1 and Python 3 seem to treat Windows NT like Windows 9x way too often.
Not the kind of 1990s nostalgia trip we had in mind, but it's the one we got.
Reviewed-on: https://repo.palemoon.org/MoonchildProductions/UXP/pulls/3039
I reviewed and tested this PR on Fedora 43 two days ago. Gave it an r+. Was admittedly hoping Travis (our main Linux guy) would confirm my results, but Basilisk-Dev already tested Oracle Linux and I already tested Fedora, so that is likely good enough. Looked at the patch visually and see nothing that could go wrong on Windows or any other platform, since all the changes appear confined to Linux and WebRTC code paths that we don't use for Windows or Mac. Looks correct, and should be low-risk even if there are issues.
**Justification for merge into master on own initiative:**
1. I have enough Linux experience to confirm the patch shouldn't break our official Linux builds.
2. Patch mostly touches WebRTC related stuff in Basilisk-Dev's area of concern, so he is the most qualified to sign off on such changes.
3. The patch has been tested twice.
4. MC is a bit preoccupied with the server move.
5. Travis also appears to have been busy.
6. 10 days have passed since PR was first submitted.
If master breaks or anything unusual happens as a result of this merge, I am prepared to accept the blame and consequences.
Sincerely,
Jeremy Andrews (athenian200)
None of these resulted in build bustage using our default configuration,
but apparently ccache builds were broken due to a bytes/text issue. Wish
the original reporter had mentioned he was using ccache, but he saved me
a lot of busywork. Changes appear safe across whole range of Python
versions we support.
I felt like this was a cheat when I did it in my hacky build, and
seriously thought ChatGPT was just suggesting this because it was too
lazy to figure out how to fix virtualenv. I learned several things:
- The approach virtualenv 15.x uses to simulate a Python environment
does not work after Python 3.10. So we'd need a newer virtualenv
anyway, which has different quirks. I got it to "work" on Python 3.11...
but only by copying (almost) the whole stdlib and a bunch of DLL files to
bootstrap it. Which is no longer a virtualenv, it's... just an env.
- VirtualenvManager is Mozilla's code, not a part of virtualenv itself.
Which means everything we need is now contained within our own code,
no third-party module like virtualenv required.
- Our build system doesn't depend on virtualenv quirks at all. In
a pinch, if you copy all the .pth files it generates into the
system's site-packages directory (after turning them all into
absolute paths), it would even run under system Python. It really
only "needs" virtualenv to find its own modules because of how it
was designed.
- I've tested venv extensively across a wide range of Python 3 versions.
not a single one has any issues or behaves in a noticably different
fashion from a build based on a repaired virtualenv extended up to
Python 3.10, which means no regressions were introduced by the switch.
- Python's devs want the venv route over the way virtualenv did things.
PEP 405 was very critical of virtualenv's design, and PEP 587 created a
a new configuration mechanism precisely so no application would have to
do what virtualenv did to get isolation without copying
the whole interpreter into another directory.
- Mozilla also did something similar in BZ 1766497, so they clearly
thought venv had enough features for their build system.
The cgi module was removed from Python 3.14. html module has almost
the exact same thing, and this was apparently used to format html
rather than cgi anyway, so it's probably more correct.
Ref: PEP 594
https://peps.python.org/pep-0594/#cgi
Python no longer provides any version-related utilities in stdlib.
Did a thorough grep and made absolutely sure to replace every last
use. This was the work I was avoiding during my first port...
Ref: PEP 632, BZ 1777414, BZ 1858065, BZ 1766085
More small tweaks mostly related to decoding/encoding, some not even related to
failures but are more anxiously done in anticipation of something going wrong.
Even though these are called commits, I often revise little things several
times because I am not sure what the most robust approach would be.
Most notable changes are the ugly stuff I had to do in FileAvoidWrite and
everything that interacts with it, which is where I expect a lot future pain
and revision to enter the picture from, if I have to revisit this part of the
work again.
Not really sure how to categorize these, except they are either
language-related cleanups, small bug fixes, or are careful to decode
things using the preferred OS encoding where reasonable. A lot of UTF-8 only
worked by luck because it only tried to decode ASCII bytes, or had good
replacements for bad chars due to explicit error handling.
I'm basically just commiting the last state before I update gyp, because
I can't really proceed without updating it, and that's a big enough step I want
it to have its own commit... and also don't want it mixed in with other stuff.
This is apparently the one place we can't use UTF-8 on Windows, we absolutely
need CP1252 here. Mozilla originally avoided a lot of these encoding issues by
treating everything as bytes. That trick won't fly in Python 3, so we now
have to deal with text streams and face Windows and Unicode issues.
Ref: BZ 1585702
In decorators.py, the original logic fails because cls.__init__ is now a
function instead of a method in Python 3.
In string_version.py, the issue is that str.__init__(self, args) is no longer a
valid way of referencing the parent class of the function, the no-argument
super() can do that in Python 3 properly by using the __class__ cell variable.
Ref: PEP 3135
https://peps.python.org/pep-3135/
It seems to have moved out of the body of the function,
into the class definition. Hit this a couple times...
2to3 . --fix metaclass -wpn
Ref: PEP 3115
https://peps.python.org/pep-3115/
2to3 . --fix tuple_params -wpn
They actually had a fixer for this?! Anyway... yeah,
Py2 allowed tuple parameter unpacking. Py3 doesn't.
However, the adaptations are fairly mechanical.
Ref: PEP 3113
https://peps.python.org/pep-3113/
2to3 . --fix input -wpn
2to3 . --fix raw_input -wpn
Py2 input is the new Py3 input wrapped in eval tags.
The new input is the old raw_input.
This lets you have the old behavior. Mechanical.
Ref: PEP 3111
https://peps.python.org/pep-3111/
2to3 . --fix except -wpn
Python 3 exception syntax is different. Comma-based syntax is gone,
much like print syntax is changed.
This lets you have the old behavior. Mechanical.
Ref: PEP 3110
https://peps.python.org/pep-3110/
2to3 . --fix raise -wpn
Python 3 raise syntax is different. Three-argument raise is
deprecated, much like print syntax is changed.
This lets you have the old behavior. Mechanical.
Ref: PEP 3109
https://peps.python.org/pep-3109/
2to3 . --fix dict -wpn
Py2 dict was a list by default, the iterator versions
were switched around to be the default.
This lets you have the old behavior. Mechanical.
Ref: PEP 3106
https://peps.python.org/pep-3106/
Vim control lines were re-introduced or not entirely cleaned up.
This nukes them again.
Removing from modules, netwerk, security, storage, testing, toolkit, and
a few scattered misc files. More to come.
This change modifies the mozdebug module to include a special case for Visual Studio when looking for its path.
This also removes the workaround implemented in dcb027a718 which can break debugging in certain installations of Visual Studio 2015.
Basilisk-Dev
Jeremy Andrews
Andy
trav90
Moonchild
Matheus MarinhoandMoonchild
FranklinDM