@glyph @sethmlarson @freakboy3742 @jacob @sgillies Given how much of Python’s ecosystem nowadays depends on Rust, I don’t consider that a valid argument anymore. As Jacob wrote in the initial post, it’s a thing now wherever we like it or not. uv is neither the first nor the last and it shouldn’t be held against it. I’m pretty sure Hatch has a Rust component nowadays too.
@hynek @glyph I don't think anyone is objecting to Rust as a reasonable technical choice for this kind of tooling. But we do have to realistic about it shrinking the potential future maintainers pool by a lot which is a downside to weigh against the benefits. And with paid contributors right now is kind of masks the downside in a way that many of us who have been on the receiving end of this cycle before find concerning.
@coderanger @hynek 100%, and I am definitely not complaining when I use a project that's adopted uv and ruff and everything is almost implausibly fast. it's not really a *bottleneck* in my other projects, but I certainly don't mind the focus on performance that comes along with the choice of Rust.
I certainly don't object to Rust *qua* Rust, I depend on mountains of it in Cryptography and I wish I depended on more. but that's a different element of the stack with a very different context.
@coderanger @hynek but to acknowledge the validity of elements of your critique here as well — it's not like everyone with passing familiarity with Python is jumping at the chance to contribute meaningfully to the packaging ecosystem either. The language barrier is one barrier among dozens and it may not even be the most significant one, but it is the newest one.
@glyph @coderanger I don’t think y’all quite grok what uv makes so special due to your seniority. The speed is really cool, but the reason Rust is elemental is that it’s one compiled blob that can be used to bootstrap and maintain a Python development environment. A blob that will never break because someone upgraded Homebrew, ran pip install or any other creative way people found to fuck up their installations. Python has shown to be a terrible tech to maintain Python.
Edited 75d ago
@glyph @coderanger And Ruff is a separate topic entirely to me; a topic that I see a lot more critical. But we’ve been BEGGING corporations to spend money to fix Python packaging and now we’re turning up our noses. I find that highly irritating. We can’t expect this to be fixed by individuals making beer money on GitHub Sponsors.
@glyph @coderanger @hynek The performance is only one of the carrots involved tho. There's also the benefits of being a single executable outside of Python itself, consolidation, good design and probably something else I'm forgetting.
I've argued that what we've needed in Python packaging is agreement on a single tool to just focus on, and if it ends up being because we have one project that wins because it got 100x the resourcing of the other projects... that's fine?
@hynek @coderanger you are conflating two things here. We've been begging corporations to *give* money to the community, to invest in infrastructure. Which, to their credit, they have! The developer in residence, the PyPI team, Fastly's CDN stuff. Nobody's handwringing that Fastly is going to turn off their service, even though that *is* risk, because there's no alternative. But this is a VC investment. Which *could* be great, if we knew what Astral's revenue model was
@glyph @coderanger Yeah the times of “give us money, BUT NOT LIKE THAT” live in the ZIRP past. TBH I personally find the Fastly situation a lot more distressing because as you said there’s not alternative and unlike uv it’s not MIT/Apache-licensed.
@glyph @hynek @coderanger We've been asking corporations to donate money, and they're... you know... not keen on giving away money like that.
We've gotten increased funding around this stuff because the PSF/Python Core Devs/PyPI/pick-the-group have shown that it's not the same as throwing money into a fire pit (and someone within the political structure of those corporations successfully advocated for throwing money anyway as a bit of a gamble, at the start).
Edited 76d ago
Indeed, I see nothing about their revenue model. If it's another Redis "It's free (for now)!" we don't need it.
@glyph @hynek @coderanger I do 100% share the hesitance around the VC funded corporation. We have had multiple examples of how this plays out in the PyData ecosystem (and the wider industry) with VC-backed companies changing terms/licenses/enforcement.
Even if the underlying codebase is MIT/Apache, it's not a clear fork situation because of a slow rot/increasing papercuts. Usually, founders (and, sometimes hired C-suite) are smart enough to not trigger full migrations off their thing.
@pradyunsg @glyph @coderanger I think this is one of the most under appreciated lessons lately: corporations are much more likely to throw significant money if they know what they get. Sure, the PSF supporting conferences around the globe are good for them too long-term, but it’s too abstract for a balance sheet. But saying you pay X for Y and they want Y, works.
@hynek @glyph @coderanger Yea, this has been a good lesson to learn.
It's also why I'm not particularly concerned about a corporate takeover of Core Python today -- the in-group is aware of the risks and the funded pieces are targetted in specific ways to avoid overreach via those roles.
I think it helps that almost everyone involved is also at-least-mildly skeptical that corporations will do the right thing, when given the choice between that and more control. :)
@pradyunsg @glyph @hynek @coderanger As someone very close to both ecosystems, I can’t stress enough how important it is to not only get funding but to build lasting governance structures that can handle other stakeholders wanting to participate. Python did that well IMO with the PSF, the packaging ecosystem around it… not so much. There is still time though!
@jezdez @pradyunsg @glyph @coderanger Having suffered through the packaging open space at PyCon US 2023 I have exactly 0 hope that will happen. The best we can hope for is the PSF taking over.
@hynek @pradyunsg @glyph @coderanger I’m not sure what you mean, do you mean the summit? I don’t recall an open space last year TBH.
@jezdez @pradyunsg @glyph @coderanger Yes, summit, sorry.
@hynek @pradyunsg @glyph @coderanger Shrug, I felt this year was more effective, and led to some good conversations that acknowledged the difficulties of coordinating, had many new faces and also had Astral folks join, among other corpo stakeholders. The User Success WG idea was born which has now been established: https://github.com/psf/user-success-wg
@hynek @glyph @coderanger this. I legit don’t know what PyPi would do if fastly decided to pull their funding. I think it’s basically an existential threat that could end python package hosting as we know it. How do you even come back from that?
In comparison, a few packaging developers may need to learn rust in order to improve things... from a pretty good starting point. I don’t really see the comparison, or at least if I do, it’s not in favor of fastly.
@offby1 @hynek @glyph @coderanger The PSF would ask other cloud providers (Amazon, Google Cloud, etc) for emergency help to offset bandwidth costs.
Financially, they'd run out of cash in 30 to 60 days to fund it. Maybe less if the provider tried to tell them, they were 60 or 90 days behind, which other providers have done with other orgs.
They are already pretty minimally staffed, so I worry more about the personnel shift, which I suspect they would get enough volunteers to pull off.
@offby1 @hynek @glyph @coderanger When I was on the board and the Treasurer I brought this up several times a year. At the time, people feared bringing it up to our CDN out of fear they might re-review the arrangement, which, to me, was a really naive way of viewing it.
I'm happy/ecstatic that attitude changed, and they have a five-year deal now. That tells me the organization grew the fuck up, which has always been the bigger institutional threat.
@webology
Extremely this. Doesn't hurt that Fastly got a very good open source person heading their developer relations team this year
@offby1 @hynek @glyph @coderanger
@chrisjrn @webology @offby1 @hynek @coderanger re: how do you come back from that… I actually worry less about this. It is an immediate, obvious problem that can be solved pretty straightforwardly, and any corporate sponsor stepping into the gap to cover those bandwidth costs can expect an immediate and significant PR benefit. Picking up long-term engineer salary obligations for a tool that mostly kinda works right now is a much harder sell on every level
@chrisjrn @webology @offby1 @hynek @coderanger also given the immediate brand damage that Fastly would sustain if they pulled their support, I believe the folks that would advocate internally to keep the program running will not have difficulty doing so. It’s probably not *fair* that the reward for years of donations is getting blamed for the immediate crisis in the aftermath of the well running dry, but so it goes.
@glyph
In a vacuum, your argument makes sense. I think you've seen enough second order effects of things the PSF has done to know that there would be significant second order effects to deal with under time pressure.
@webology @offby1 @hynek @coderanger
@chrisjrn @webology @offby1 @hynek @coderanger I stand by the fundamentals of my analysis but I somewhat intentionally glossed over the 7 or so people who would sustain permanent life-altering psychological trauma and probably exit the industry after this and the thousands who would be engulfed in all-consuming Internet Drama for weeks or months, leading to hundreds of millions of dollars of lost productivity across the industry
@glyph
Stop assuming rational well-informed actors, Glyph
@webology @offby1 @hynek @coderanger
@glyph @chrisjrn @webology @offby1 @hynek @coderanger FWIW, I think there's absolutely a way that this can be a graceful changeover -- it's not necessary that moving away from Fastly would be disruptive to PyPI users. It's not a given that they'd pull support without sufficient notice such that it requires a panicked/immediate response.
They could just be like "Hey, in 6 months, we're gonna stop providing this for free" -- there'll be work to do but it can end up being fine?
@pradyunsg @glyph @chrisjrn @webology @hynek @coderanger sure, but a successful outcome still hinges on corporate generosity. Astral dropping uv or enshittifying it leaves us in control of our own destiny.
@pradyunsg @glyph @chrisjrn @webology @offby1 @hynek @coderanger
In fact, it is to my understanding based on the below clip of Fastly's sponsor spot announcement at PyCon US this year that there is a contract of some kind that commits to 5 years of support, which is fantastic.
@offby1 @pradyunsg @glyph @chrisjrn @webology @hynek @coderanger
Not for nothing, but we've been very careful about not tying anything tied to one corporate org into PyPI's external footprint. So yes, we use Fastly, but we could switch without it impacting users.
This is basically the opposite of client side tools, where migrations take 10+ years.
@offby1 @pradyunsg @glyph @chrisjrn @webology @hynek @coderanger
Just take a look at how long it took to break the "lock in" that setuptools had, and afaict uv doesn't _appear_ to be engaging the standards processes (I could have missed it!), which feels kinda very old Microsoft, "Embrace, Extend, Extinguish".
@dstufft @pradyunsg @glyph @chrisjrn @webology @hynek @coderanger in fairness to Astral, the packaging standards process has broken many a spirit over the years. I'm not surprised they'd consider just doing the thing and letting the userbase sort out whether it makes sense.
@offby1 @dstufft @pradyunsg @glyph @chrisjrn @webology @hynek @coderanger
Astral folk are very much engaging in packaging PEP discussions on d.p.o. For example, Charlie is one of the most frequent posters on the recent lockfiles topic:
https://discuss.python.org/t/pep-751-lock-files-again/59173/239 They were also at the packaging summit this year at PyCon US.
@chrisjrn @glyph @webology @offby1 @coderanger Classic Glyph
@hynek @chrisjrn @webology @offby1 @coderanger it me, “rational actor — will respond to incentives for food”
@hynek @coderanger I feel like somewhere a Homebrew maintainer is muttering "challenge accepted"
@hynek @glyph @coderanger This is such an underemphasized point. My team is responsible for the dev tooling at my employer, and I’m *so* delighted about uv, even as I’m also very skeptical about their business model.
@hynek @glyph @coderanger hm, never thought of this argument but it very much checks out
@hynek @glyph @coderanger as a user of micromamba, and now pixi, which did that for the conda world, I agree 100% with this. It is so much easier to setup an env from the ground up. Teaching is much easier, maintaining scientific environments consistent across multiple machines became much easier, everything is easier in that form. Single binary, no dependencies, quick to bootstrap anything python related with few commands and consistent instructions across multiple OSes.
Edited 75d ago
@hynek @glyph @coderanger I will probably be unable to make the time to read the licensing position but as long as it’s forkable, it should be fine, no?
There is quite some overlap in the Venn diagram of Python and Rust programmers and in the best case the tooling opens more channels between the two communities.
@hynek @glyph @coderanger I signed up for Mastodon just to post this 😅
I spent a significant amount of time making it so that Hatch also provides a similar experience such that it's the only tool you need to download manually. Installation takes 1-3 seconds and I don't think I can make it any faster, technically.
Example run locally and on GitHub Actions from here https://github.com/tox-dev/platformdirs/actions/runs/10758318125/job/29833485691:
@hynek @glyph @coderanger mojo and Julia can also create small binaries. There is a path to those languages from a rpython-like statically typed dialect.
@ofek @glyph @coderanger yeah I mention PyApp in some side-thread WRT Hatch has (indirectly) Rust inside too, to unjank the Python-ness :D
@tintvrtkovic @hynek @glyph @coderanger Agreed, this is the killer advantage of *any* compiled language over one with a runtime. And to make it worse, Python is far from being the best runtime based language for app deployment :-(
@pfmoore @tintvrtkovic @glyph @coderanger The fact that now I can give anyone a Python project and uv and tell them to “just run `uv run X` in the project folder” and it will take about everything including installing Python is nothing short of magic. There’s a short line to running Zip files.
@pfmoore @tintvrtkovic @hynek @coderanger interesting that "compiled" and "with a runtime" are the opposite ends of the spectrum here :). (I get what you're saying, but it is _possible_ to do this with Python, and a real shame that "rewrite in a different language" is actually an easier path than just using existing tooling)
@hynek @tintvrtkovic @glyph @coderanger Agreed. But to be fair, that's the way hatch and pdm are going as well. And a lot of uv's speed comes from language-agnostic tricks like aggressive caching. All of this could have been done in existing projects. It's funding and full time employees that has made the huge difference.