11 ms·

Urllib3 in 2022

197 points by SethMLarson 4y ago

samwillis 4y ago
Interesting to note nearly half of the $26,615 raised was a single donation from Spotify ($12,900). They made the donation as it's the fourth most used Python library by Spotify (noted in the link to Spotify, I wander what the first three are?). It would be good to see more large companies making these sorts of donations. It's not like Spotify is unique in its usage of Urllib.
- SethMLarson 4y ago
  Yeah the amount of financial support we receive is mostly dependent on individual organizations. For example in 2021 we didn't receive any large donations, only support from Tidelift.
- H8crilA 4y ago
  FWIW $20k is approximately the total monthly cost of an average engineer at Spotify. I am appaled at how those fundamental libraries do not get the >= 8 digit support that they deserve. Surely some MBA people can turn this around. FFmpeg burns more in CPU cost every second across major platforms than all those donations amount to.
  - robertlagrant 4y ago
    Tidelift exists. I don't know why it's not more widely used.
    - hobofan 4y ago
      Because it is and has always been intransparent about earnings structure (e.g. what percentage of money of a Tidelift subscription is paid out to maintainers), and thus has never achieved critical mass among maintainers. It also seems like they have mostly pivoted away towards software supply-chain analytics.
  - beernet 4y ago
    Oh well, if you want money you shouldn't opt for open source in the first place...
    - wiseowise 4y ago
      That’s not the point.
    - H8crilA 4y ago
      Yes of course, but there's value in this approach for everyone. Look at Linux for example, there's a real benefit for various orgs in creating and funding a foundation to further the kernel project. It's not that I think these libraries should get something like a fair market value for their product, just a rather small (for a multibillion corp) donation plan. It's worth it for the PR / virtue signalling alone.
      - VLM 4y ago
        Donations can be very expensive to a large company. The labor cost is often huge. If you don't prevent fraud at the $500 level you'll have two problems: Lots of repeated fraud at the $500 level, and people trying to get away with fraud at the $5000, $50000, all the way up to FTX levels of fraud. So you need multiple people in different depts to agree to fund something while other people watch over to make sure the people being funded are not the spouses or shell companies of the approvers. Meanwhile there's paperwork costs to make sure the donation paperwork gets to accounting so accounting gets their tax deduction. At a "really big company" the labor cost of paying an invoice might be $2000. Now does the average programmer get more than $2K of annual value out of urllib3, given the alternatives (which could involve up to architecting around needing it?) Before you laugh and say "no way" to the $2K figure, imagine spending half of an hour long team meeting of ten software devs getting paid $100/hr but overhead etc costs the company $150/hr for them to attend that meeting, that's $750 of labor costs just to have the dev team possibly agree to support urllib3. Then its got to go up the chain to someone with signing approval, someone has to chase the paperwork to make sure it happens, frankly $2K is on the very low side of the cost of getting an invoice paid. Commercial non-free software is "put up with" where its not that some rando piece of required software costs $50 thus the cost of the company including labor and overhead is $2050 and they're happy to pay it, its more similar to if you don't dot all the Is and cross all the Ts that software will cost $10M in some crazy lawsuit, and $2050 is cheaper than $10M, so they pay up or they'll get a huge lawsuit or more likely someone will get fired after the audit, and I'd rather pay $2050 for something worth $50 if it prevents me from getting fired and better yet its not my $2050. But nobody never got fired for not paying for urllib3, so nobody's paying urllib3. Companies will pay for support and I could see a large company paying $10K/yr for a written guarantee if a dev at that company runs into a use problem or bug with urllib3 then an actual dev from the project will help them up to twenty hours per year or similar, even if no one asks for help. In contrast feel good donations are very expensive so they don't happen often, so trying to run a business off feel good donations doesn't work too well..
  - SethMLarson 4y ago
    I think it's less to do with a small number of companies making big donations like this, it'd be much nicer to have many organizations making even small contributions based on usage/complexity of the project. This is what Tidelift is trying to do, the biggest barrier to more projects getting the amount of funding that we do from them is the number of businesses wanting that sort of relationship with their open source dependencies. The amount we receive from Tidelift is not typical for most projects since we're a medium complexity project that is a part of almost every Python project dependency list.
  - jmbwell 4y ago
    > Surely some MBA people can turn this around. Surely some MBA people are the ones driving currently, so you're probably not wrong, but you and I might share different outlooks on anything turning around. I believe we're expected to be grateful that there was any donation at all.
  - thn-gap 4y ago
    That's still $20k above what they have to pay for the right of usage of the library. What donation amount above would you consider reasonable? Or how would you budget Spotify's donation accross all open source projects that they use? You can dual-license to avoid comercial exploitation. Why then set an expectation on how much should be donated for complete open source projects?
    - gnramires 4y ago
      Well, I think it depends on the project itself. If they are asking for money, then they probably need money! See: https://liberapay.org https://liberapay.org (and also patreon.com has a number of projects) The idea is that people that can give would give, and those that can't won't. And who can give can also choose how much they think the project is valuable and how much maintenance it needs. Hopefully that would be covering the needs of the developers at a fair salary. It would be nice if open source projects were more transparent about this as well: "I need X/month to reach a fair salary and good maintenance, and could expand, hire devs and add functionality for Y/month." (milestones in patreon partially fulfill that role). The problem is when there are others that rely on the work as well. How much should each give, such that the system would be fair? From a theoretical perspective, I think the money should be always coming from the source with the lowest marginal counterfactual return. That is, the organizations who have funds going to investors and other orgs with the lowest ROI should instead divert those funds to [something else], where [something else] is OSS in this case, until an equilibrium is reached and everywhere is operating at equal and optimal ROI. In practice things are not so easy. And this analysis (and financial analysis) tends to only value economic ROI, where we should be thinking of social ROI as well as environmental ROI. But to give a rule of thumb, for now, I think it would be reasonable to pay (a) if it is less than a % of the (internal) project cost: a % of the budget of the OSS project, depending on how valuable it is internally (say, 50% if their budget is low, or just 1% if their budget is high); (b) a % of the (internal) project cost otherwise (e.g. at most 5% of project cost). Ideally there would be some kind of system or framework to streamline this sort of evaluation and allocation to the devs. I actually think a whole complementary economic system (i.e. enhancing capitalism or socialism) should be developed around this idea. We should be paying whomever is providing value to society, according to their needs to provide this value (and also give them a good life of course! -- and provide a reward/incentive to do valuable things, even ones that don't currently have an incentive). So some kind of organizations, that could be tied to companies or governments too, would be responsible for evaluating on a reasonably objective basis which projects need money and then allocating it (sort of as an outsourcing of the resource allocation job to specialized entities). But meanwhile individual and voluntary giving is basically that without outsourcing. I try to do this personally through Effective Altruism (which is essentially just that: give effectively) and giving what I can to Open Source and other impactful causes.
  - eptcyka 4y ago
    Spotify doesn't pay silicon valley wages in Sweden.
  - UntrimmedDisk 4y ago
    Spotify did an episode of their Development podcast that touched on this recently. I thought it was quite interesting. https://open.spotify.com/episode/0erUH7oqqbW5HDUjcnK6cb?si=qO3_PyW-TimP9royOzrdXQ https://open.spotify.com/episode/0erUH7oqqbW5HDUjcnK6cb?si=q...
- dec0dedab0de 4y ago
  It would be good to see more large companies making these sorts of donations. It's not like Spotify is unique in its usage of Urllib. That would be nice, but I think it would be even better if these companies allowed their employees to contribute. And were open about their policies. Maybe there should be a ranking site where employees can report anonymously about company policies. And companies can respond with their official policies. The ranking could be something like this 0 - can't contribute at all 1 - can contribute in spare time with approval 2 - can contribute in spare time without approval 3 - can contribute on company time with approval 4- can contribute on company time without approval.
  - rebolek 4y ago
    0, 1 and 2 - companies pay me for my time but when they don't pay, it's none of their business what I do, so what's the point?
    - comex 4y ago
      If you’re an employee (not contractor) in California, your employer can specify in the employment contract that they own IP rights to anything you make even in your own time, as long as it “relate[s] [..] to the employer’s business, or actual or demonstrably anticipated research or development of the employer”. Many tech companies do so, including large tech companies whose business relates to just about everything. YMMV if you’re in another jurisdiction.
      - rebolek 4y ago
        Wow, interesting and horrifying.
    - ModernMech 4y ago
      A lot of companies have language in their employment contracts that amounts to them owning all of your ideas, even if they are thought of and worked on outside of company time. Under such a contract, you would need to get the company’s blessing to work on some open source project even in your spare time.
- CydeWeys 4y ago
  I'm at Google and I wish each engineer got some kind of budget (like the peer bonus budget) that could be used to distribute funds to third-party libraries. My project is itself open source, does not run in the Google prod environment, and thus is heavily reliant on other open source libraries. I wish I had, say, $1,000/year to personally distribute amongst our dependencies. I think what's holding this back is how logistically/legally complicated it is, especially considering how many of the donations are going to be international and might have tax consequences requiring issuance of a 1099. It's easy to donate to non-profits through work (we have an annual amount we can give, plus matching funds), but for donations to individual developers, the logistics are nigh impossible.
  - dmix 4y ago
    Maybe an internal voting system, where every SWE gets 1 vote, but they submit a couples projects (as a fallback mechanism), so at the end of the yr all donations get bunched up & paid out together. Then if the money can't go to the first project for whatever legal reason it goes down to the next one of their list. It's still a complicated system so it'd would take serious buy in. Not to mention the drama donations generate internally and publicly. But it's a noble goal. $1000 per engineer grant might still be easier and less fragile
    - SethMLarson 4y ago
      This is part of what Spotify did to determine urllib3 as an award recipient: https://engineering.atspotify.com/2022/04/announcing-the-spotify-foss-fund/ https://engineering.atspotify.com/2022/04/announcing-the-spo...
    - sandyarmstrong 4y ago
      This is what Microsoft does, basically. Each month, $10,000 is donated to some project. Anyone who has contributed to any open source repo during that month is eligible to vote. There's a good nomination system, too. The write-ups for each candidate are done by employees.
      - 4y ago
- comfypotato 4y ago
  Is it not a safe assumption that 1 and 2 are numpy and pandas? I would have thought your question should be phrased “I wonder what the third”. (Just in case this comes across as snarky, it’s a legitimate question.)
  - bornfreddy 4y ago
    I don't know about the third, but requests is probably tied with urllib3 - or does anyone use urllib3 directly these days?
  - Fatnino 4y ago
    I think boto3 would be at the very top.
- armini 4y ago
  If other large companies are considering donating we'd love to help with thanks.dev https://youtu.be/WX3_-cuDXgA https://youtu.be/WX3_-cuDXgA
remram 4y ago
Urllib 3 2? I am confused about the version numbers now, wasn't urllib3 the next version of urllib/urllib2?
- SethMLarson 4y ago
  The name isn't straightforward, is it? :) The project name "urllib3" is an unfortunate joke on "urllib" and "urllib2" in the standard library, since it's made to supersede both of those modules in functionality. The original author of the package likely didn't know it'd grow to be the most downloaded package at the time!
  - hackmiester 4y ago
    Thank you for the concise explanation. I am an only-occasional Python programmer and did not really understand what was going on with the naming.
- mminer237 4y ago
  https://stackoverflow.com/a/63233379/204381 https://stackoverflow.com/a/63233379/204381 urllib2 was a standard library replacement for urllib in Python 2. Python 3 further replaced this with a new base urllib. urllib3 is a completely separate, unofficial, alternative library.
- gjm11 4y ago
  Coming soon: urllib 3.2 gen 1 2x2.
  - jraph 4y ago
    Who in their right mind would ever go for such a broken version scheme though. Anything as universal as urllib3 should not need so many details. You should be able to take any latest version of urllib3 and have it all. Anything more complicated will lose many devs for sure. Universal, that's what the U in urllib is for, right? If you make specific derivatives, it's not universal anymore. Something fundamental is lost. Full backward compatibility is all we need. Keep it simple please.
tyingq 4y ago
Note that the popular "requests" module depends on urllib3. This space is a little messy for python. The old urllib ships with python, but it doesn't support some normal expected things like HTTP/1.1 pipelined requests. It adds a "Connection: close" header to each request.
- dec0dedab0de 4y ago
  I think both of them should be in the standard library at this point. The API has been fairly stable for over 5 years.
  - hotfixguru 4y ago
    Web and release cycles don’t always go hand in hand, though.
  - Frotag 4y ago
    stdlib is where things go to die. Backwards compatibility, limits on dependencies, release cycles, etc all make development a pain and discourage potential contributions. Or so I've read. Here's one discussion about it in the requests repo https://github.com/psf/requests/issues/2424 https://github.com/psf/requests/issues/2424
    - tyingq 4y ago
      Stdlib may not be the answer, but the current setup does make urllib3 suffer in anonymity...since for many it's hidden under the requests module.
    - dec0dedab0de 4y ago
      I read that at the time, but that was 7 years ago. Requests has been pretty much backward compatible for that whole time. They could still have active developments available in pypi for people who need the newer features, but python has been dropping minor releases about once a year, that should be plenty fast for new features in requests.
    - bodyfour 4y ago
deleted 4y ago
[deleted]
md8 4y ago
While I appreciate Indeed sponsoring, but why $420? Are you kidding Indeed? Your entire business is build around web scraping and using libraries to pull data from other sources. Common, you can do better!
- bornfreddy 4y ago
  Picking on one of the few companies that did contribute something will not help the cause. If more of the companies did likewise there would be real money in FOSS instead of peanuts. Thank you Indeed (and others on the list, especially Spotify). Edit: toned down a bit.
- richbell 4y ago
  In many cases it is difficult to get funding for ad-hoc donations such as this, let alone the authorization to do it on behalf of the company. I wouldn't be surprised if that effort was driven by a single employee.
  - md8 4y ago
    Well, thanks to the employee whoever it was
  - nerdponx 4y ago
    Some companies give employees an annual budget to be directed to charitable donations.
    - richbell 4y ago
      I've found that companies that do offer that often restrict it to an established list of charities, which is understandable but means you cannot donate to a random guy in Ukraine maintaining a vital open source project.
- renewiltord 4y ago
  This is exactly why it's better to not participate. When you participate and people see your name they'll give you grief. This is the Copenhagen Interpretation of Morality.
VLM 4y ago
If 1.63% of PyPi is worth $26615/yr, the implication is all of PyPi added together would only be worth $1.6M, which seems low. Its interesting to compare to other industries. "In the old days" the way to make money during a gold rush was not to try to mine gold after the good stuff was already dug up, but to sell shovels to wanna-be miners. Apparently, software is nearly the opposite of that, LOL.
- Moissanite 4y ago
  > the way to make money during a gold rush was not to try to mine gold after the good stuff was already dug up, but to sell shovels to wanna-be miners See also: cloud computing
nunez 4y ago
Insane that urllib3 only raised $26k, given how it's used EVERYWHERE.
wodenokoto 4y ago
I’ve been writing a library to interact with an api and used the built in urllib. Is that a mistake? Should I switch to urllib3?
- extasia 4y ago
  If it works and the interfaces are suitable for your needs I wouldn't bother adding an external dependency. It's something to be aware of should the above change I suppose:)
jdorfman 4y ago
Thank you for posting this Seth! I will be adding Urllib3 to our (Sourcegraph's) Open Collective budget asap.
sigmonsays 4y ago
just wait till you see urllib4
- extasia 4y ago
  What's new?

Urllib3 in 2022

197 points by SethMLarson 4y ago

Interesting to note nearly half of the $26,615 raised was a single donation from Spotify ($12,900). They made the donation as it's the fourth most used Python library by Spotify (noted in the link to Spotify, I wander what the first three are?). It would be good to see more large companies making these sorts of donations. It's not like Spotify is unique in its usage of Urllib.

SethMLarson 4y ago
Yeah the amount of financial support we receive is mostly dependent on individual organizations. For example in 2021 we didn't receive any large donations, only support from Tidelift.
H8crilA 4y ago
FWIW $20k is approximately the total monthly cost of an average engineer at Spotify. I am appaled at how those fundamental libraries do not get the >= 8 digit support that they deserve. Surely some MBA people can turn this around. FFmpeg burns more in CPU cost every second across major platforms than all those donations amount to.
- robertlagrant 4y ago
  Tidelift exists. I don't know why it's not more widely used.
  - hobofan 4y ago
    Because it is and has always been intransparent about earnings structure (e.g. what percentage of money of a Tidelift subscription is paid out to maintainers), and thus has never achieved critical mass among maintainers. It also seems like they have mostly pivoted away towards software supply-chain analytics.
- beernet 4y ago
  Oh well, if you want money you shouldn't opt for open source in the first place...
  - wiseowise 4y ago
    That’s not the point.
  - H8crilA 4y ago
    Yes of course, but there's value in this approach for everyone. Look at Linux for example, there's a real benefit for various orgs in creating and funding a foundation to further the kernel project. It's not that I think these libraries should get something like a fair market value for their product, just a rather small (for a multibillion corp) donation plan. It's worth it for the PR / virtue signalling alone.
    - VLM 4y ago
      Donations can be very expensive to a large company. The labor cost is often huge. If you don't prevent fraud at the $500 level you'll have two problems: Lots of repeated fraud at the $500 level, and people trying to get away with fraud at the $5000, $50000, all the way up to FTX levels of fraud. So you need multiple people in different depts to agree to fund something while other people watch over to make sure the people being funded are not the spouses or shell companies of the approvers. Meanwhile there's paperwork costs to make sure the donation paperwork gets to accounting so accounting gets their tax deduction. At a "really big company" the labor cost of paying an invoice might be $2000. Now does the average programmer get more than $2K of annual value out of urllib3, given the alternatives (which could involve up to architecting around needing it?) Before you laugh and say "no way" to the $2K figure, imagine spending half of an hour long team meeting of ten software devs getting paid $100/hr but overhead etc costs the company $150/hr for them to attend that meeting, that's $750 of labor costs just to have the dev team possibly agree to support urllib3. Then its got to go up the chain to someone with signing approval, someone has to chase the paperwork to make sure it happens, frankly $2K is on the very low side of the cost of getting an invoice paid. Commercial non-free software is "put up with" where its not that some rando piece of required software costs $50 thus the cost of the company including labor and overhead is $2050 and they're happy to pay it, its more similar to if you don't dot all the Is and cross all the Ts that software will cost $10M in some crazy lawsuit, and $2050 is cheaper than $10M, so they pay up or they'll get a huge lawsuit or more likely someone will get fired after the audit, and I'd rather pay $2050 for something worth $50 if it prevents me from getting fired and better yet its not my $2050. But nobody never got fired for not paying for urllib3, so nobody's paying urllib3. Companies will pay for support and I could see a large company paying $10K/yr for a written guarantee if a dev at that company runs into a use problem or bug with urllib3 then an actual dev from the project will help them up to twenty hours per year or similar, even if no one asks for help. In contrast feel good donations are very expensive so they don't happen often, so trying to run a business off feel good donations doesn't work too well..
- SethMLarson 4y ago
  I think it's less to do with a small number of companies making big donations like this, it'd be much nicer to have many organizations making even small contributions based on usage/complexity of the project. This is what Tidelift is trying to do, the biggest barrier to more projects getting the amount of funding that we do from them is the number of businesses wanting that sort of relationship with their open source dependencies. The amount we receive from Tidelift is not typical for most projects since we're a medium complexity project that is a part of almost every Python project dependency list.
- jmbwell 4y ago
  > Surely some MBA people can turn this around. Surely some MBA people are the ones driving currently, so you're probably not wrong, but you and I might share different outlooks on anything turning around. I believe we're expected to be grateful that there was any donation at all.
- thn-gap 4y ago
  That's still $20k above what they have to pay for the right of usage of the library. What donation amount above would you consider reasonable? Or how would you budget Spotify's donation accross all open source projects that they use? You can dual-license to avoid comercial exploitation. Why then set an expectation on how much should be donated for complete open source projects?
  - gnramires 4y ago
    Well, I think it depends on the project itself. If they are asking for money, then they probably need money! See: https://liberapay.org https://liberapay.org (and also patreon.com has a number of projects) The idea is that people that can give would give, and those that can't won't. And who can give can also choose how much they think the project is valuable and how much maintenance it needs. Hopefully that would be covering the needs of the developers at a fair salary. It would be nice if open source projects were more transparent about this as well: "I need X/month to reach a fair salary and good maintenance, and could expand, hire devs and add functionality for Y/month." (milestones in patreon partially fulfill that role). The problem is when there are others that rely on the work as well. How much should each give, such that the system would be fair? From a theoretical perspective, I think the money should be always coming from the source with the lowest marginal counterfactual return. That is, the organizations who have funds going to investors and other orgs with the lowest ROI should instead divert those funds to [something else], where [something else] is OSS in this case, until an equilibrium is reached and everywhere is operating at equal and optimal ROI. In practice things are not so easy. And this analysis (and financial analysis) tends to only value economic ROI, where we should be thinking of social ROI as well as environmental ROI. But to give a rule of thumb, for now, I think it would be reasonable to pay (a) if it is less than a % of the (internal) project cost: a % of the budget of the OSS project, depending on how valuable it is internally (say, 50% if their budget is low, or just 1% if their budget is high); (b) a % of the (internal) project cost otherwise (e.g. at most 5% of project cost). Ideally there would be some kind of system or framework to streamline this sort of evaluation and allocation to the devs. I actually think a whole complementary economic system (i.e. enhancing capitalism or socialism) should be developed around this idea. We should be paying whomever is providing value to society, according to their needs to provide this value (and also give them a good life of course! -- and provide a reward/incentive to do valuable things, even ones that don't currently have an incentive). So some kind of organizations, that could be tied to companies or governments too, would be responsible for evaluating on a reasonably objective basis which projects need money and then allocating it (sort of as an outsourcing of the resource allocation job to specialized entities). But meanwhile individual and voluntary giving is basically that without outsourcing. I try to do this personally through Effective Altruism (which is essentially just that: give effectively) and giving what I can to Open Source and other impactful causes.
- eptcyka 4y ago
  Spotify doesn't pay silicon valley wages in Sweden.
- UntrimmedDisk 4y ago
  Spotify did an episode of their Development podcast that touched on this recently. I thought it was quite interesting. https://open.spotify.com/episode/0erUH7oqqbW5HDUjcnK6cb?si=qO3_PyW-TimP9royOzrdXQ https://open.spotify.com/episode/0erUH7oqqbW5HDUjcnK6cb?si=q...
dec0dedab0de 4y ago
It would be good to see more large companies making these sorts of donations. It's not like Spotify is unique in its usage of Urllib. That would be nice, but I think it would be even better if these companies allowed their employees to contribute. And were open about their policies. Maybe there should be a ranking site where employees can report anonymously about company policies. And companies can respond with their official policies. The ranking could be something like this 0 - can't contribute at all 1 - can contribute in spare time with approval 2 - can contribute in spare time without approval 3 - can contribute on company time with approval 4- can contribute on company time without approval.
- rebolek 4y ago
  0, 1 and 2 - companies pay me for my time but when they don't pay, it's none of their business what I do, so what's the point?
  - comex 4y ago
    If you’re an employee (not contractor) in California, your employer can specify in the employment contract that they own IP rights to anything you make even in your own time, as long as it “relate[s] [..] to the employer’s business, or actual or demonstrably anticipated research or development of the employer”. Many tech companies do so, including large tech companies whose business relates to just about everything. YMMV if you’re in another jurisdiction.
    - rebolek 4y ago
      Wow, interesting and horrifying.
  - ModernMech 4y ago
    A lot of companies have language in their employment contracts that amounts to them owning all of your ideas, even if they are thought of and worked on outside of company time. Under such a contract, you would need to get the company’s blessing to work on some open source project even in your spare time.
CydeWeys 4y ago
I'm at Google and I wish each engineer got some kind of budget (like the peer bonus budget) that could be used to distribute funds to third-party libraries. My project is itself open source, does not run in the Google prod environment, and thus is heavily reliant on other open source libraries. I wish I had, say, $1,000/year to personally distribute amongst our dependencies. I think what's holding this back is how logistically/legally complicated it is, especially considering how many of the donations are going to be international and might have tax consequences requiring issuance of a 1099. It's easy to donate to non-profits through work (we have an annual amount we can give, plus matching funds), but for donations to individual developers, the logistics are nigh impossible.
- dmix 4y ago
  Maybe an internal voting system, where every SWE gets 1 vote, but they submit a couples projects (as a fallback mechanism), so at the end of the yr all donations get bunched up & paid out together. Then if the money can't go to the first project for whatever legal reason it goes down to the next one of their list. It's still a complicated system so it'd would take serious buy in. Not to mention the drama donations generate internally and publicly. But it's a noble goal. $1000 per engineer grant might still be easier and less fragile
  - SethMLarson 4y ago
    This is part of what Spotify did to determine urllib3 as an award recipient: https://engineering.atspotify.com/2022/04/announcing-the-spotify-foss-fund/ https://engineering.atspotify.com/2022/04/announcing-the-spo...
  - sandyarmstrong 4y ago
    This is what Microsoft does, basically. Each month, $10,000 is donated to some project. Anyone who has contributed to any open source repo during that month is eligible to vote. There's a good nomination system, too. The write-ups for each candidate are done by employees.
    - 4y ago
comfypotato 4y ago
Is it not a safe assumption that 1 and 2 are numpy and pandas? I would have thought your question should be phrased “I wonder what the third”. (Just in case this comes across as snarky, it’s a legitimate question.)
- bornfreddy 4y ago
  I don't know about the third, but requests is probably tied with urllib3 - or does anyone use urllib3 directly these days?
- Fatnino 4y ago
  I think boto3 would be at the very top.
armini 4y ago
If other large companies are considering donating we'd love to help with thanks.dev https://youtu.be/WX3_-cuDXgA https://youtu.be/WX3_-cuDXgA

remram 4y ago

Urllib 3 2? I am confused about the version numbers now, wasn't urllib3 the next version of urllib/urllib2?

SethMLarson 4y ago
The name isn't straightforward, is it? :) The project name "urllib3" is an unfortunate joke on "urllib" and "urllib2" in the standard library, since it's made to supersede both of those modules in functionality. The original author of the package likely didn't know it'd grow to be the most downloaded package at the time!
- hackmiester 4y ago
  Thank you for the concise explanation. I am an only-occasional Python programmer and did not really understand what was going on with the naming.
mminer237 4y ago
https://stackoverflow.com/a/63233379/204381 https://stackoverflow.com/a/63233379/204381 urllib2 was a standard library replacement for urllib in Python 2. Python 3 further replaced this with a new base urllib. urllib3 is a completely separate, unofficial, alternative library.
gjm11 4y ago
Coming soon: urllib 3.2 gen 1 2x2.
- jraph 4y ago
  Who in their right mind would ever go for such a broken version scheme though. Anything as universal as urllib3 should not need so many details. You should be able to take any latest version of urllib3 and have it all. Anything more complicated will lose many devs for sure. Universal, that's what the U in urllib is for, right? If you make specific derivatives, it's not universal anymore. Something fundamental is lost. Full backward compatibility is all we need. Keep it simple please.

tyingq 4y ago

Note that the popular "requests" module depends on urllib3. This space is a little messy for python. The old urllib ships with python, but it doesn't support some normal expected things like HTTP/1.1 pipelined requests. It adds a "Connection: close" header to each request.

dec0dedab0de 4y ago
I think both of them should be in the standard library at this point. The API has been fairly stable for over 5 years.
- hotfixguru 4y ago
  Web and release cycles don’t always go hand in hand, though.
- Frotag 4y ago
  stdlib is where things go to die. Backwards compatibility, limits on dependencies, release cycles, etc all make development a pain and discourage potential contributions. Or so I've read. Here's one discussion about it in the requests repo https://github.com/psf/requests/issues/2424 https://github.com/psf/requests/issues/2424
  - tyingq 4y ago
    Stdlib may not be the answer, but the current setup does make urllib3 suffer in anonymity...since for many it's hidden under the requests module.
  - dec0dedab0de 4y ago
    I read that at the time, but that was 7 years ago. Requests has been pretty much backward compatible for that whole time. They could still have active developments available in pypi for people who need the newer features, but python has been dropping minor releases about once a year, that should be plenty fast for new features in requests.
  - bodyfour 4y ago

deleted 4y ago

[deleted]

md8 4y ago

While I appreciate Indeed sponsoring, but why $420? Are you kidding Indeed? Your entire business is build around web scraping and using libraries to pull data from other sources. Common, you can do better!

bornfreddy 4y ago
Picking on one of the few companies that did contribute something will not help the cause. If more of the companies did likewise there would be real money in FOSS instead of peanuts. Thank you Indeed (and others on the list, especially Spotify). Edit: toned down a bit.
richbell 4y ago
In many cases it is difficult to get funding for ad-hoc donations such as this, let alone the authorization to do it on behalf of the company. I wouldn't be surprised if that effort was driven by a single employee.
- md8 4y ago
  Well, thanks to the employee whoever it was
- nerdponx 4y ago
  Some companies give employees an annual budget to be directed to charitable donations.
  - richbell 4y ago
    I've found that companies that do offer that often restrict it to an established list of charities, which is understandable but means you cannot donate to a random guy in Ukraine maintaining a vital open source project.
renewiltord 4y ago
This is exactly why it's better to not participate. When you participate and people see your name they'll give you grief. This is the Copenhagen Interpretation of Morality.

VLM 4y ago

If 1.63% of PyPi is worth $26615/yr, the implication is all of PyPi added together would only be worth $1.6M, which seems low. Its interesting to compare to other industries. "In the old days" the way to make money during a gold rush was not to try to mine gold after the good stuff was already dug up, but to sell shovels to wanna-be miners. Apparently, software is nearly the opposite of that, LOL.

Moissanite 4y ago
> the way to make money during a gold rush was not to try to mine gold after the good stuff was already dug up, but to sell shovels to wanna-be miners See also: cloud computing

nunez 4y ago

Insane that urllib3 only raised $26k, given how it's used EVERYWHERE.

wodenokoto 4y ago

I’ve been writing a library to interact with an api and used the built in urllib. Is that a mistake? Should I switch to urllib3?

extasia 4y ago
If it works and the interfaces are suitable for your needs I wouldn't bother adding an external dependency. It's something to be aware of should the above change I suppose:)

jdorfman 4y ago

Thank you for posting this Seth! I will be adding Urllib3 to our (Sourcegraph's) Open Collective budget asap.

sigmonsays 4y ago

just wait till you see urllib4

extasia 4y ago
What's new?