> It seems to imply that a vast amount of it is not browser-based,
Given the number of CI pipelines I've seen which download package X and then install it, and do this every time the pipeline is run... I'm not surprised
Yes, one should be using some artifact caching solution. But I don't think any of them are truly seamless, they all involve extra busywork in each pipeline to actually turn them on, and it isn't surprising a lot of people don't. Or else, people do set it up for some things in the pipeline (e.g. APT/RPM, Maven, Gradle, PyPI, NPM, etc) but then there's some random other thing it needs which just gets pulled in with wget or curl.
Disk cache is so ungodly slow in Bitbucket it’s faster to not only download, but rebuild artifacts in a lot of cases. I pay for time used not bandwidth, so I optimize for that :(
This is mind blowing. if such a huge majority uses their hosted CA bundle, that makes curl a very attractive target. All under the control of a single individual (I am not questioning his integrity, just saying too much reliance on a single project/individual). We have seen such examples in past (e.g openssl)