Being new to python, the astral stuff is such a relief.
I don’t think experienced python folks realize how much the flexible tooling slows people down, and creates friction for adopters. Setting up my project I tried 3 environment managers, 2 type checkers, 3 formatters/linters, 3 packagers/dependancy/project managers.
I know this is kinda the n+1 issue where astral is just adding another. But it feels like a complete and well designed stack, not a box of parts encouraging me to build my own. Reminds me a bit of go tooling.
I am a convert. I’ll happily jump on their type checker and packager when ready.
>Setting up my project I tried 3 environment managers, 2 type checkers, 3 formatters/linters, 3 packagers/dependancy/project managers.
I've been using Python for right about 20 years now. I've been through various prior iterations of the "standard" packaging tools. But I've never found myself wanting a type checker, formatter or linter; and aside from a brief period with Poetry (in which I used barely any of its functionality), I've had no interest in dependency management tools, either. What I have found myself wanting is fixes for the problems in Pip.
Different people work on different kinds of projects, and approach the language differently, and have radically different needs as a result.
I’ve been writing Python for 25 years and I love love love ruff and uv and friends.
I used to use any standard editor but these days I use PyCharm or Visual Studio (not Code) otherwise, I just use Sublime Text or any adjacent reasonable editor.
Elementary OS has the nicest editor that other Distros have not snatched up yet not sure why. Its fast, and fits the bill where KATE and Sublime Text are for me.
I started with python dependency management just executing pip install and thinking wow that's cool. No envs, no linters or formatters.
The point is, if you are learning to code, you can skip nearly all the ecosystem tools. Their need will arise when the time comes.
I primarily work on typescript projects, little bit of go. Never enjoyed working with python until I found uv. Glad to see folks rally around astral's tools.
> I know this is kinda the n+1 issue where astral is just adding another.
My bugbear with that XKCD is that even if it's true, it's also how progress is made. By this point I'm so annoyed whenever anyone links it that I kinda just wish Randall would take it down.
Yeah, and that comic is about standards anyways, not tools, so it's doubly annoying when it's linked out of context
I'm a little sad that Ruff took off as a re-implementation of a whole bunch of work that was already done, rather than as a project to improve the work that was already done.
It was nice to be able to write little extra linters or flake8 plugins in the language I was linting. Plus decades of effort had gone into making those plugins pretty great, finding the right separation of linting codes so that the right things could be linted/ignored per codebase.
I understand why they did it, and they have done it remarkably well, but "rewrite it from scratch" is almost never the best answer and part of me wonders if this result could have been even better achieved another way.
I'm also sad but from a different perspective.
If something can be rewritten from scratch fairly quickly and ends up being much faster, it makes you wonder what we (the general “we”) might have done “wrong” with the “decades of effort” before.
To be clear: there was nothing wrong with the old toolchains (hence the quotes). And of course, Ruff/uv can do what they do because they build on all that earlier knowledge. My point is just that sometimes it's easier to start over than to fix the old, and that fact itself feels kind of sad to me.
> If something can be rewritten from scratch fairly quickly and ends up being much faster, it makes you wonder what we (the general “we”) might have done “wrong” with the “decades of effort” before.
I think the impact of Rust (and Go) shouldn't be underestimated here. Prior to these languages if you wanted fast runtime you were either using C or C++ with all their associated footguns, tricky build systems, and poor support for exposing and consuming libraries. Or you were using something like Java or C# which meant managing another runtime just for tools, which is especially bad for something like UV which manages runtimes - imagine if you were using a python tool to manage your Java versions and a Java tool to manage you Python versions, it would be a mess!
With both Go and Rust you have fast languages without memory safety issues that compile to a single, easily-installable binary and have a broad ecosystem of libraries available that mean that half the time you don't even need to write any difficult code and can just glue some modules together, and the other half of the time you only need to write the code you actually want to write and not all the support code for general pupose stuff like reading config files, file watching, etc.
> there was nothing wrong with the old toolchains (hence the quotes). And of course, Ruff/uv can do what they do because they build on all that earlier knowledge
I don't think that's true. I think the old toolchains really were bad, partly because the Python community was uniquely resistant to building on what worked in other ecosystems, and uv in particular largely succeeds because it's the first tool in this space to finally just ignore the nonsense that had somehow become accepted wisdom in Python-land and apply a little bit of outside experience.
> I think the old toolchains really were bad, partly because the Python community was uniquely resistant to building on what worked in other ecosystems
I mostly attribute it to backwards compatibility concerns (which I in turn attribute to the traumatic 3.x migration). PyPI continued to accept egg uploads until August 2023 (https://packaging.python.org/en/latest/discussions/package-f...). The easy_install program, along the direct command-line use of `setup.py` (as opposed to having Setuptools invoke it behind the scenes), have been deprecated since October 2021 (https://setuptools.pypa.io/en/latest/history.html#v58-3-0); but not only are they still supported, Setuptools can't even remove support for `setup.py test` (as they tried to in 72.0 and immediately reverted - https://github.com/pypa/setuptools/issues/4519) without causing major disruption to the ecosystem. One of the packages affected was Requests, which: a) is one of the most downloaded packages on PyPI; b) is pure Python (aside from dependencies) with nothing particular complicated about its metadata (I see no reason it couldn't be done in pyproject.toml and/or setup.cfg); c) wasn't even reliant on that interface to run tests (they use Tox).
(For that matter: Requests has been maintained by the PSF since mid-2019 (https://www.reddit.com/r/Python/comments/cgtp87/seems_like_r...) and the problem could have easily been avoided once the deprecation was announced, but nobody did anything about it. The project still defines most of its metadata in `setup.py`; `pyproject.toml` exists but is only used for pytest and isort config, while `setup.cfg` is used for flake8 config and some requirements metadata that's completely redundant with `setup.py`.)
A lot of it also just has to do with lack of attention and focus. https://peps.python.org/pep-0427/ (defining the wheel format) was proposed in September 2012, and accepted in February 2013. But Setuptools itself wasn't available as a wheel until November 2013 (https://setuptools.pypa.io/en/latest/history.html#id1646), and in 2017 there were still reports of people having outdated versions of Pip and not being able to install Setuptools from that wheel (https://setuptools.pypa.io/en/latest/history.html#v34-0-0). Setuptools relied on a separate package to actually make wheels until July of last year (https://setuptools.pypa.io/en/latest/history.html#v70-1-0) - an effort which was initially proposed in June 2018 (https://github.com/pypa/setuptools/issues/1386). It also took years to notice that Pip and Setuptools had separate implementations for the code that understands the "tags" in wheel filenames and factor it out into `packaging` (https://github.com/pypa/packaging/pull/156).
> My point is just that sometimes it's easier to start over than to fix the old, and that fact itself feels kind of sad to me.
Sad, but also liberating. There are surely more projects out there that would benefit from breaking with conventional wisdom to start over.
I am quite grateful that they rewrote so many tools that I had been using. Upgrades were super painful with them split into a dozen separate packages, I routinely ran into incompatibilities and had to pin to specific versions to deal with transitive dependencies.
Given the state of python packaging and tooling, I'd say that consolidating the tooling is a big win in and of itself, and with the tremendous speed wins on top...
Rewrite from scratch is exactly what the Python ecosystem needs more of.
The strong push to always contribute to the existing projects rather than starting new ones in the Python community is what caused the previous subpar state of tooling. I think Astral is also picking a good middle-ground where they still aim to build as much as possible on existing Python standards (and help contributing to new ones), so it's still building on what has been done in the past.
There was probably a lot of engineering that went into those designs and that took time.
Having a clean process to rewrite from is what made it so fast. They knew exactly what outcome they wanted and they had access to the internal implementations on how to do it.
All that effort was not wasted at all.
Whenever Ruff comes up, I reflexively go and check https://github.com/astral-sh/ruff/issues/970, since Pylint is the one tool whose performance I find to be egregiously bad. Looks like Ruff is still very far from being able to replace it.
> this result could have been even better achieved another way.
Don't you have "decades of effort" to resolve this wonder?
Your argument seems ideological. There is no chance they could have improved Flake8 to be as good as Ruff is.
Folks said the same thing to me about grep 8.5 years ago when I released ripgrep.
FWIW, I think the reason I'm conflicted is probably a similar reason to why you made a separate new thing. Overcoming the inertia and ways of doing things, or proposing widespread changes, often doesn't go down well with existing maintainers – for very valid reasons. I probably wouldn't want to go into the grep project and suggest rearchitecting it, and I can see why the Ruff developers didn't want to go into projects like Flake8 and Pylint to do the same.
But that doesn't stop me from feeling that there were things lost in this process and wishing for a better outcome.
Wait... Did you misunderstand this comment? Or are you saying grep caught up with ripgrep now?
I'm agreeing with them. Some folks told me I should have improved grep instead of building my own thing.
And you think you've done it?
Yeah it's somewhat ideological, I think open source software is better for society when built as a community project than built and controlled by a VC funded company.
I don't think you could get flake8 to be as fast as Ruff, but I think you could improve it to be within an order of magnitude, and honestly that's plenty good enough. There are a lot of low hanging fruit, particularly around caching. I'd push back on there being no chance of being "as good as" Ruff, because "good" is not just speed. Ruff is _not complete_, if you want all the features and linters, you still need Flake8, or you have to sacrifice some. It's also not publicly extensible, and not written in Python, both of which are fair choices but I think deviate from the ideal scenario here.
Blame Guido. Until recently when he was bought out by Microsoft, he had been the primary blocker of higher Python performance. There were a bunch of attempts at adding a JIT to Python but Guido was more interested in splitting hairs over syntax than any real heavy lifting. Python could have been as performant as LuaJIT or V8 but their dictator ruined it. Python need more Mike Palls.
I'm very impressed by the recent developer experience improvements in the python ecosystem. Between ruff, uv, and https://github.com/gauge-sh/tach we'll be able to keep our django monolith going for a long time.
Any opinions about the current state of the art type checker?
I'm very happy with pyright. Most bug reports are fixed within a week and new peps/features are added very rapidly usually before pep is accepted (under experimental flag). Enough that I ended up dropping pylint and consider pyright enough for lint purposes as well. The most valuable lints for my work require good multi-file/semantic analysis and pylint had various false positives.
Main tradeoff is this only works if your codebase/core dependencies are typed. For a while that was not true and we used pylint + pyright. Eventually most of our code is typed and we added type stubs for our main untyped dependencies.
edit: Also on pylint, it did work well mostly. tensorflow was main library that created most false positives. Other thing I found awkward was occasionally pylint produces non-deterministic lints on my codebase.
I have always used MyPy but I have noticed some large open source projects adopt Pyright as of late. The folks behind ruff and uv are currently working on a type checker as well but haven't heard when they plan on releasing it.
Pyright is pretty good. It's easy to setup, and has first-class VSCode support, if that's what your team uses.
I’m used to running command line tools on fast machines, but the first time I ran ruff on my codebase I was blown away. My codebase isn’t massive but most (non-rust) tools just take a while to run. It might be less than a second of startup overhead, but it’s noticeable. Ruff is so fast you wonder if it even did anything. It reminded me how fast computers actually are.
At this point I think even people trapped for two years on the International Space Station have heard about the Astral toolchain.
Any more info about "Astral" the org?
I've used ruff/uv for sure, but I never pay attention to Astral who is behind it.
VC funded organization. Not sure what their business model is yet.
Astral toolchain?
Python development tools produced by Astral (https://github.com/astral-sh), primarily uv and ruff. GP is a complaint about how often people submit links to these projects. The resulting discussion tends not to produce any new insight and the most common points IMX are "it's fast because it's written in Rust" (which is fairly specious logic - most problems with existing native Python tools are due to algorithmic problems such as poorly designed caching) or "it avoids bootstrapping problems because it's not written in Python" (it's completely possible to avoid those problems for everything except actually provisioning the Python interpreter, which some people apparently do see as a killer feature).
Maybe, I use uv and rust has been my primary language for several years. I have never heard of astral though.
Astral is the organization making uv. It's right there in the GitHub URL.
How do you read me comment and take away that I still dont know what Astral is? You acted like everyone had heard of it. My point is not even all the users of uv have.
>How do you read me comment and take away that I still dont know what Astral is?
From the comment I read, quoted directly:
>I have never heard of astral though.
If you meant "I only learned about Astral thanks to this post", then I pointed out how you might have found out by yourself before.
> How do you read me comment and take away that I still dont know what Astral is?
The comment where you literally say "I have never heard of astral"? Gee, I wonder.
It really is as fast as it claims. I sometimes intentionally add something it will complain about just to make sure it’s still working.
I do the same thing, and I keep doing it regardless of how many times I see it working properly.
I don't care one bit about py-land, but it's always nice to see a rust project swoope in and save the day.
Language processing like compiling or linting is just one of the many aspect where rust can really play out it's awesome strength.
How speedy is the rust-language tooling itself these days? I remember wishing for a 'optimize nothing' or even 'just interpret' mode. Compile times noticably contributing to the feedback loop are a serious killjoy.
A small boon to it not being in python — it's now decoupled from your python toolchain.
There is no "coupling" inherent to native Python tools. They can generally be installed in a separate virtual environment from the one they operate on.
For example, with Pip you simply pass the `--python` option to have it install for a different environment - you can create your venvs `--without-pip`, and share a copy of Pip across all of them. If you use Pipx, you can expose and use Pipx's vendored Pip for this, as I described in a recent blog post (https://zahlman.github.io/posts/2025/01/07/python-packaging-...).
Twine and Build don't care about your project's environment at all - Twine uploads a specified file, and Build by default creates a new environment for the build (and even withough build isolation, it only requires your build environment to contain the build backend, and the build environment doesn't have to be the one you use for the rest of development). Setuptools, similarly, either gets installed by the build frontend into an isolated build environment, or can reside in a separate dedicated build environment. It doesn't actually operate on your project's environment - it only operates on your project's source tree.
That's a lot of words compared to "Download binary from server. Run it"
Sometimes it takes a lot of words to debunk a misconception. What you initially said didn't have anything to do with setup effort, but also there is quite little setup effort actually described here.
I've been using an amalgamation of pyenv, pip-tools, black, isort, etc. for projects and just gave uv and ruff a try. Wow, it really is fast! Skeptical of anything VC-backed but I'll be keeping my eye on it.
Some previous discussions:
For Django projects, what's fast that will cover Python, HTML, CSS and JavaScript files? Ruff only does Python?
If you already have tooling to cover each separately, why would it be better for it to be a single tool?
Prettier can do HTML CSS and JS but I don't think there's any one tool that can do it all. (It would be interesting if there were something like a tree sitter for auto formatting)
I recently learned that tree-sitter does not have a goal of accurately representing the whole source input, and thus as it currently stands could not be used for that purpose
Now, whether a tree-sitter-esque ecosystem that is designed for source modeling could catch on like t-s did is a more interesting question
So much marketing coolaid/bullshit.
I'm not that fan of Ruff because to me it doesn't make any sense to add a rust dependency to "python" and it blows my mind that people are so keen to accept the ugly formatting inherited from Black just because "it gives consistency to the team, so I don't have to think, and we don't have to discuss code style"...
But all of that personal opinion set aside, what triggers my initial statement is that so many persons are so excited to run to use Ruff because... "It is so fast"... when I'm quite sure most of them would never notice any real speed difference with their modest codebase.
In almost all the codebases I have seen, proprietary and OSS, pylint or any other linter was quasi-instant. There are probably a few huge codebases that would see a speed benefit of a few seconds to use Ruff, but their owner would probably better have a look at their code to understand how their reached such a monster!
> quite sure most of them would never notice any real speed difference with their modest codebase
On the contrary, within only a year or so of coding by a small team, this ecosystem takes single digit seconds or less, while the traditional tools are taking minutes and more.
Particularly in analytics or scientific computing we've seen minutes versus under a second.
Having the tools not require additional/parallel Python dependency management is a plus.
Note that we watched these tools for a long time, and jumped in when rye converged with uv. It was promising to see fewer ways to do it.
// More on that here: https://news.ycombinator.com/item?id=41309072
I don't know what kinds of codebases you've worked with, but I can tell you that pylint is so far from instant, it became the longest running CI job in multiple reasonably sized codebases I've worked with. Think tens of minutes. Other linters were not much better, until ruff came along. But that's far from the only advantage that ruff brings.
There are other issues with what you said, but the biggest one is: you have some strongly worded criticism for a project that has set a new bar for usability and consistency in Python code quality tooling. These tools are developed by humans like you and distributed to you for free with no obligation to use them. No matter how I look at your comment, I don't see how it's helping.
I'm confused: why are you linting code in CI, rather than as a precommit hook?
Pre-commit hooks are great in small, focused codebases with small, homogeneous teams. In large monorepos with lots of different teams committing, it's impossible to guarantee any kind of consistency in which pre-commit hooks get run, so you need CI to actually enforce the consistency or you'll spend all your time chasing (accidental) violations.
... am I the only one who figures that linting would logically be a very low priority in those circumstances?
Because devs can disable precommit hooks much more easily than they can work around CO.
I see precommit hooks as where you avoid the low-hanging fruit, like “is this code actually parsable?”
Well, for one, I use Jujutsu, where commits happen every time you run jj status and traditional notions of pre-commit hooks don't really apply. But also, I think (as a matter of principle) nothing should get in the way of performing commits or amends.
I have issues with some of Black’s formatting decisions but I’ve also suffered from inconsistent formatting and there is no question in my mind that consistent formatting that I find ugly sometimes is 1000% better than the alternative. And after so many years of dealing with it it’s so refreshing to just “give up” and let the formatter win.
Same here. I appreciate that Black annoyed everyone on our team about the same amount but in different ways. From the instant we added it, stupid style arguments completely disappeared.
Know what I care about more than Black making my own code look less pretty? It making my coworker’s code look less horrid. (And funnily, I’m certain everyone on the team thinks exactly that same thing.)
> I appreciate that Black annoyed everyone on our team about the same amount but in different ways.
If it did affect people equally, it would be great. Unfortunately, spaces for indentation is an accessibility issue and the Black maintainers are hostile to making this configurable. Normally I am in agreement about minimising configurability, but this isn’t a matter of taste, it is making the Python ecosystem more difficult for some disabled people to participate in.
https://github.com/psf/black/issues/2798
Fortunately, Ruff makes this configurable, so you don’t have to choose between autoformatting and accessibility any more.
Personally, I think excellence is a virtue in tooling in and of itself. The Python world just hasn't experienced this kind of excellence before.
I like that it's fast, and it is noticeably faster for even moderately sized codebases.
But the main thing I like about ruff is that it is a single tool with consistent configuration, as opposed to a combination of several different tools for formatting and linting that each have their own special way of configuring them and marking exceptions to lint rules.
Doesn’t Black mostly stick to PEP-8 style? What rules do you consider ugly?