18 ms·

Fetch API has landed into Node.js

679 points by yamafaktory 5y ago

easton 5y ago
For those of us unfamiliar, how long until stuff comes out of experimental?
- inglor 5y ago
  In general "when we feel it's ready". For some features I've worked recently (like `flatMap` and friends for streams) I think it'll take about a year. For `fetch` it will take us around a year or two probably. I wrote more details on the PR itself but I (and others) basically feel strongly that it should be as close to compliant as possible (Deno have a nice list which they happily shared with us of how they diverge and that seems reasonable). Basically there is a bunch of stuff that should happen first: - We need to run (and pass) the web platform tests. - We want to go over the spec (again) and check compliance and add more tests and fix bugs. - We want community feedback on the implementation and to improve the DX to be better (while not sacrificing spec compliance). - We want better support for stuff like `File` in core. - Web streams need to go out of experimental first. This was a _long_ road that had many stepping stones (like EventTarget, AbortController, Blob, ReadableStream etc). It's important to get it right. Worth mentioning the fact it's experimental doesn't mean it's not safe to use in simple code in this case - it just means if you make your code behave in a way that diverges from the specification we may change our implementation to match the specification and break your code. Anything like `await fetch('./someUrl').then(x => x.text())` should be fine.
- deleted 5y ago
  [deleted]
jitl 5y ago
There’s an NPM package containing this implementation called ‘undici’ that you can install in previous versions of Node; the package.json claims support back to 12.x. https://www.npmjs.com/package/undici https://www.npmjs.com/package/undici However the fetch implementation itself is documented as unstable and only supports Node 16.x. I believe this is because it is not quite spec compliant yet, so there is latitude for breaking changes that make it more spec compliant.
inglor 5y ago
Hey, Node core person here (and the person who triggered the land) - we're super excited for this and would love help and feedback. This is still experimental and we'd love to hear from the community what you'd like to see.
- nefitty 5y ago
  Hey this one change is going to positively impact my day to day life. Just wanted to let you know. Thanks for sharing your hard work and time.
  - inglor 5y ago
    I'm happy to hear that but I had a very small part in it all! The person who took it through the finish line (and deserves the most props here IMO) is Michael https://github.com/targos https://github.com/targos The people who worked most on Undici are Matteo https://github.com/mcollina https://github.com/mcollina and Robert https://github.com/ronag https://github.com/ronag The person to work most on fetch in Undici is Ethan https://github.com/Ethan-Arrowood https://github.com/Ethan-Arrowood Also worth calling our James whose work on web streams in Node as well as events/cancellation helped drive a bunch of this. I can name maybe 50 people who worked on this effort overall at some capacity (I am one of them - spending maybe 40 years on APIs to help enable this). Note the job isn't entirely done and help is very much appreciated!
    - mccorrinall 5y ago
      What does undici mean?
      - Eriks 5y ago
        eleven
      - paulfitz 5y ago
        https://undici.nodejs.org/ https://undici.nodejs.org/ "Undici means eleven in Italian. 1.1 -> 11 -> Eleven -> Undici. It is also a Stranger Things reference."
pjc50 5y ago
Brief explanation?
- forty 5y ago
  Fetch is a web JS API to do HTTP requests. NodeJS has its own API to do HTTP requests. This add supports for the fetch API. I think the only benefit it that it makes it easier to share code between front end and back end.
  - giffarage 5y ago
    It makes it easier to learn and develop both frontend and backend.
  - Allstar 5y ago
    For more on Fetch, check out MDN: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
  - 9dev 5y ago
    This is an understatement. With fetch available in node, we have a single, standardised, promise-based HTTP client Interface available on all (more or less common) JavaScript platforms. This is great news for an ecosystem traditionally extremely fragmented and may lead the way to more consolidation in the JS library space.
    - forty 5y ago
      NodeJS API are traditionally not promised based (although nowadays some do offer a promisified option), so I'm not especially excited to have a promise-based API :) It's a great news for people doing both front end and backend I guess, but for people doing NodeJS développement it's just one more option to do http request (which I guess is going to be very similar to the module node-fetch that have existed for a while, without special consolidation effect)
      - 5y ago
krossitalk 5y ago
As someone browser focused this surprised me. I've been using fetch() for years now.
- zamadatix 5y ago
  Not every browser API is an instant candidates to be added to Node's core. If there isn't a big performance hit from doing it as a user library or lack of functionality to do it as such in the first place then it's rare it'll be brought into core right away, if it will at all. It's nice to see the really popular ones make it in though, makes for fewer dependencies when doing browser compatible projects and also allows one to be lazy and not learn the Node way of doing it. Maybe we'll see websocket go into core some day too.
  - herpderperator 5y ago
    This is what frustrates me so much about javascript. Everything is all over the place.
    - deleted 5y ago
      [deleted]
    - zamadatix 5y ago
      Eh, I mean I get it JS lacks a real standard library but at the same time you don't see HN fill up with people saying C is all over the place because when they went to implement an HTTP call glib doesn't mirror libuv or C++ is all over the place because POCO doesn't mirror Boost and so on. The "all over the place"-ness tends to come more from people who act like changing to a newer library is the only way to code something new not from the lack of JS forcing a single API be used across every app regardless of type.
- DrFell 5y ago
  Intercommunicating backend services isn't new, but having an HTTP API call another one from the backend is a case that used to be more normally accomplished by calling both APIs from the frontend. The slow ooze of browser-oriented thinking onto the backend took time. Existing backend engineers, who thought it a harebrained scheme, had to eventually wear down, and give up.
mcraiha 5y ago
Why it took so long? e.g. Deno had fetch support for ages
- inglor 5y ago
  Basically because fetch isn't a great API for servers it took a while to get consensus on actually landing it for the interoprability/simple API value. Then it took a while to get consensus it's fine to do even if we can't implement the standard fully and diverge from it on stuff like CORS (like Deno does). Then there were a bunch of work adding APIs like `EventTarget` and `AbortSignal` to Node.js which are quirky'ish web APIs. As a side note just to show positive collaboration: that work made me make maybe 10 PRs fixing things in EventTarget in Deno - and Deno helped Node a bunch now when landing fetch. Then web streams and blobs and a few other things as well as the undici work in the background. Then undici-fetch by Ethan, merge into Undici and then the PR by Michael - and it's still not done :) Finally
  - dstroot 5y ago
    First, this is awesome and congratulations! Second this sounds like a lot of work! Can you comment why was it so hard to add fetch to node, yet packages like “node-fetch” have existed for some time and seem to implement fetch rather easily? Only asking from a curiosity perspective.
    - inglor 5y ago
      node-fetch (which is great btw!) implements fetch "reasonably" as in stuff like `fetch('./foo').then(x => x.json())` works but it's very far from a "real" fetch. Some people argued users would still like it and there was an attempt[1] by Myles but at the end of the day - the changes were pretty big. For example: a response body in fetch is a web stream (a whole new stream type) and to cancel you use an AbortController (web type) which is an EventTarget (another web type). `node-fetch` uses Node streams instead which is very reasonable but not something a platform can "get away" with while still calling it fetch. Here is a comment I wrote about it in the tracker during the PR review process https://github.com/nodejs/node/pull/41749#issuecomment-1025720476 https://github.com/nodejs/node/pull/41749#issuecomment-10257... [1](https://github.com/nodejs/node/pull/27979 https://github.com/nodejs/node/pull/27979)
thyrox 5y ago
My biggest issue with node when I was working on it briefly was I couldn't do the 'import' statements like wepback. Are they supported too now? I'm not a web developer so I had a very hard time understanding why there are so many different type of imports in JavaScript like require, import, umd, amd, etc and which one works in browser and which one works in node? Also why do so many libraries have this strange 5 line header code for this umd, amd, business. Is that to make their packages work with nodejs? Does anyone who knows enough JavaScript point me in the right direction about it. I find all this very confusing.
- mcaruso 5y ago
  Node supports ES Modules (the import statements you mention) natively now: https://nodejs.org/api/esm.html https://nodejs.org/api/esm.html
- inglor 5y ago
  Those have been supported for several years now :) You need to tell Node to use the format by either naming your file `.mjs` or setting `"type": "module"` in your `package.json` file. > Also why do so many libraries have this strange 5 line header code for this umd, amd, business. Is that to make their packages work with nodejs? That's just old for "adapt the module system" from the old days and libraries just didn't bother updating. I still use amd in my job at Microsoft though for some things so I guess it's not useless :)
  - mschuetz 5y ago
    > either naming your file `.mjs` But why? This makes it much harder to ensure that identical code works in browser and nodejs, to the point where I sometimes just can't use nodejs. And I don't use package.json, so having to name files ".mjs" is a really painfull restriction.
    - inglor 5y ago
      Browsers will happily accept `.mjs` files they don't actually care about file extensions only about `Content-Type` and type=module on the script tag :]
songzme 5y ago
sorry if this is a stupid question, but how do I try this out? I ran node with the experimental flag on a file that calls fetch: node server.js --experimental-fetch Got 'fetch is not defined' reference error.
- inglor 5y ago
  First: not a stupid question - the release cadence, multiple release lines and other minutia of a project like Node are not something I'd expect users to be intimately familiar with! This _just landed today_. You can get it by building from master: ``` # there is more details in building.md git clone https://github.com/nodejs/node https://github.com/nodejs/node cd node # may need to pass --openssl-no-asm ./configure make -j12 ./out/Release/node --experimental-fetch ``` Otherwise - wait a bit for the next v17 release to land per the normal release cycle :)
  - songzme 5y ago
    oh I see so when node v17.4.1 is released, I would be able to run node server.js --experimental-fetch
    - inglor 5y ago
      v17.5 but yeah
  - huksley 5y ago
    Just slightly related to this - I wish I could ask GitHub to send me an email when specific release are available for the repo!
    - mattdeboard 5y ago
      Sounds like maybe an "if this, then that" kind of thing https://ifttt.com/ https://ifttt.com/
    - fs111 5y ago
      You can subscribe to only releases on github.
Meic 5y ago
Undici seems to be an HTTP 1.1 client. Is there any work or timescale to provide HTTP 2 support?
- inglor 5y ago
  Not yet but it's planned to eventually happen.
- bricss 5y ago
  check https://www.npmjs.com/package/rekwest https://www.npmjs.com/package/rekwest
itsbits 5y ago
Which NodeJS versions will this be landed to?
- inglor 5y ago
  It's behind a flag so that it can land in the next v16 semver minor - so soon'ish but keep in mind it _is_ experimental.
ezekg 5y ago
One thing I hate about fetch() is that you can't manually follow redirects. Using { redirect: 'manual' } doesn't expose the Location header of the redirect, so it's essentially useless. I know that node-fetch fixed this issue, so I hope the official Node fetch() does not have the same problem.
- inglor 5y ago
  We are not, this is a place the server-side implementation is going to diverge and this was discussed in the PR: https://github.com/nodejs/node/pull/41749#issuecomment-1025473227 https://github.com/nodejs/node/pull/41749#issuecomment-10254...
  - ezekg 5y ago
    This is really, really great to hear. Thanks for all the hard work!
tekstar 5y ago
Can you set more than one cookie in the request and read more than one cookie in the response? Was shocked when I learnt that's not supported by the spec..
- styfle 5y ago
  There's a relevant spec issue you can follow here https://github.com/whatwg/fetch/issues/973 https://github.com/whatwg/fetch/issues/973
bachmeier 5y ago
Good that some people are happy about this, but it really is pointless to post a link to "lib: add fetch" followed by code diffs. HN should have a requirement that there be an actual post, rather than a discussion starter for those in a particular community. All you'd have to do is type in a brief text post with an explanation and then link to the code diffs for those interested.
- inglor 5y ago
  There will be a blog post on this in the Node.js blog and official media communication. This _just_ landed today and I suspect the HN crowd are more in the "get news early" camp and not "wait for the official press release" camp.
sealthedeal 5y ago
Ive used Fetch in the past, great lib. I'm curious why Node core has decided to make this an official part of Node? From one perspective there could be a hard argument that this should never be a part of Node as it is an external library and not a building block of the language? Almost feels like feature creep for the core language? I know that's not the intent, but wanting to understand the logic behind this decision :)
- prollings 5y ago
  They're talking about JavaScript's fetch API. It's not an external library.
  - zamadatix 5y ago
    It ships browsers but not because it's a JS standard (ECMA) rather a web standard (WHATWG). Almost everything in JS is an external library as it doesn't have a true stdlib to go with the language just a core language and implementation specific APIs like browser Fetch or Node sockets. Hopefully WebSockets (W3C/IETF) will get embedded in Node's core one day as well.
- wruza 5y ago
  I second this, since fetch is a wrapper around xhr (though doesn’t have to be because node is a non-restricted runtime), and it diverged already for redirect semantics. It’s news, but is it big?
  - samtho 5y ago
    > It’s news, but is it big? As someone who has been working almost exclusively on Node.js applications for years now, this is big. Front-end developers have, for years, enjoyed this wrapper around the clunky XHR API. It is simple, intuitive, and it comes _free_ in the browser environment (and was easily shimmable before it was standard). Additionally, the depreciation of the popular request[0] a while ago, there has been a gap in The Right Way to do http requests without going into the low-level API that ‘http(s)’ provides. Fetch is promised-based (enabling seamless async/await) and, most importantly, standardized. Basically, fetch provides a meaningful abstraction atop each platforms’ implementation of http-request-doing further unifying server-side and client-side patterns and idioms. [0]: https://www.npmjs.com/package/request https://www.npmjs.com/package/request
esamatti 5y ago
Does this contain `request.formData()` and `FormData`? https://developer.mozilla.org/en-US/docs/Web/API/Request/formData https://developer.mozilla.org/en-US/docs/Web/API/Request/for... https://developer.mozilla.org/en-US/docs/Web/API/FormData https://developer.mozilla.org/en-US/docs/Web/API/FormData
- inglor 5y ago
  > `request.formData()` Yes. > FormData Yes, but not in that PR since `FormData` needs to behave differently as part of the platform but there is intent to support it before moving from experimental.
mariogintili 5y ago
I love that the most popular language's core lib now brings a trivial bit of code available in every other platform and its a celebrated super achievement software, why do you suck now?
- jmull 5y ago
  I don't think you understand this. e.g., here's your "trivial" API's spec. Then there are the intentional deviations for CORS, e.g., which there is no spec for. https://fetch.spec.whatwg.org https://fetch.spec.whatwg.org Not to mention the Fetch API isn't available on every other platform... maybe you're thinking of the general ability to make HTTP requests, but nodejs had that from the start.
- feupan 5y ago
  It’s not trivial and it’s not groundbreaking. Node has had an HTTP client since its inception, this is just a different API. It’s “great” just because it’s a decent abstraction that’s also available in browsers. The issue in the JavaScript ecosystem for this kind of stuff is that it runs on two very different environments and they need time to find solutions that work for all the parties involved.
bricss 5y ago
Try adv fetch-alike with HTTP2 support https://www.npmjs.com/package/rekwest https://www.npmjs.com/package/rekwest
chx 5y ago
Good for NodeJS However, I feel the fetch API is completely botched because it lacks timeout support. I have cobbled some together for https://github.com/Backblaze/gists/pull/8 https://github.com/Backblaze/gists/pull/8 but gosh. I really hope all that is actually not necessary :/
- nikeee 5y ago
  You can use an AbortController for that. Also, keep in mind that the linked solution does not abort the web request. It only aborts waiting for the response in the JS code. The browser does not close the connection.
nfriedly 5y ago
I love fetch, except for the way that it combines headers. If more than one of the same header is set (such as multiple set-cookie headers), it combines them all into a single comma-separated string. I know this is allowed in the HTTP spec, and it's probably even a sensible default. But the fetch spec doesn't allow any access to the raw headers, so there's no straightforward way to get the original uncombined headers back. Set-Cookie headers, in particular, often contain commas in their expiration dates, so just splitting around `, ` will lead to problems. I took a look through the source of this new fetch API, and it seems to have inherited that wart: https://github.com/nodejs/undici/blob/2dd3437e20c5a3cc466226c544f048351baffc78/lib/fetch/headers.js#L103 https://github.com/nodejs/undici/blob/2dd3437e20c5a3cc466226... I've argued with the authors of the fetch spec about this before, and ultimately settled on using https://www.npmjs.com/package/set-cookie-parser#user-content-splitcookiesstringcombinedsetcookieheader https://www.npmjs.com/package/set-cookie-parser#user-content... to work around this flaw. (For clarity, I published the package, but chrusart wrote that method - https://github.com/nfriedly/set-cookie-parser/pull/19 https://github.com/nfriedly/set-cookie-parser/pull/19)
- styfle 5y ago
  There's a relevant spec issue here https://github.com/whatwg/fetch/issues/973 https://github.com/whatwg/fetch/issues/973
  - nfriedly 5y ago
    Oh, good. The first comment there references my closed issue from 2017, but it's nice to see this being given some more consideration.
outside1234 5y ago
Thank god - no more node-fetch shims for node.js!
stereocodes 5y ago
Wow I can't believe how light years ahead Deno is than Node. You want imports you want fetch you want TS, use Deno.
- andyfleming 5y ago
  Why can't you use TypeScript with node? It's not built in, but it's easy enough to use by building or running it with ts-node. Also, once you are using TypeScript, you have import syntax support. Deno has chosen a different path, but there are definite trade-offs.
  - stereocodes 5y ago
    Yup the point I'm making is its built in. no hoops no babel. I find myself gravitating towards Deno before Node because with node I know I'll have to do some hoop jumping to make it work.
- ecf 5y ago
  Seriously. I maintain an open-source Node.js CLI utility and was shocked when import/export syntax is only achievable if you use a transpiler like Babel. Such overkill for something that’s been in the JS spec for nearly a decade.
  - ash_gti 5y ago
    Node does support ESM now by default (and has supported it for a while behind a flag), https://nodejs.org/api/esm.html https://nodejs.org/api/esm.html
- Scarbutt 5y ago
  Yes, and the go light years back in ecosystem ;)
  - stereocodes 5y ago
    sort of. There's only been one instance for me that this has caused any issues... I ended up just writing my own version of the missing tool. Deno has almost parity with node at the moment though https://deno.land/std@0.123.0/node https://deno.land/std@0.123.0/node.
saos 5y ago
Noicee
joelbondurant1 5y ago
quickthrower2 5y ago
Funny how my brain works, I thought it was there already, but I must have got so used to having it as a dependency I forgot that it was a dependency not built in. To be fair I don't "Node" all the time! It's on and off.
stevage 5y ago
Out of curiosity, what took so long? It's a small API that has been stable for a long time? I'm probably missing something.
- zamadatix 5y ago
  Same reason the WebSocket API isn't supported, Node already had lower level APIs and browser APIs moved 1:1 don't necessarily make sense. In this case it's a slightly extended/non-standard Fetch and enough of the "it doesn't need to be in core it can be a normal library" folks thought it'd be useful enough. And someone went through the work of actually making the native implementation.
  - digianarchist 5y ago
    I actually think both the Websocket API and fetch make sense. There's nothing more frustrating than adding a bunch of checks to a SvelteKit project to see if you're on the server or not.
    - zamadatix 5y ago
      I wouldn't be surprised if WebSocket made it into core eventually tbh, the above are just examples of why such things haven't immediately. If WebSocket does it'll probably have more modifications/deviations than say Fetch did here though as well as an entire server side half. `ws` is well loved but there are 2 optional native binary addons which greatly increase performance, that alone makes a good enough case for it to end up in core - just a matter of someone finding it interesting enough to make a solid PR to do so instead of just using the `ws` library as is.
pseudosavant 5y ago
Every Node.js dev should be thanking the Deno creators. It has forced them off of their "this is how Node does it" complacency. Most improvements I notice and care about on Node these days are all things that Deno already supports (fetch, ES modules, async std lib uses promises, etc). The Web Platform APIs aren't perfect, but they are better, and more thoroughly thought out than Node's.
- inglor 5y ago
  What is "this is how Node does it" complacency you speak of? Node has been talking about Fetch since at least 2018 _before_ Ryan even announced Deno. This is true for promises, ESModules etc. Ryan _attended those meetings_ in the Node collaborator summit. Don't get me wrong I am thankful for Deno (and a contributor!) but: - It's a lot easier to make changes when the project is new and you don't have a lot of users (which is great for Deno) whereas Node has a whole (huge) ecosystem to consider on every change. - It's a lot easier to make changes with venture capital and funding to hire a bunch of people to work on it full time vs. a project with a "regular" MIT license where the code is owned by the community and not a company. Those are not complaints about Deno, I like Deno (and not just Ryan, Lucas, Kit, Ben and the other people are nice and helpful and the project is really nice). It's just to explain very little of Node's innovation is (at the moment) driven by Deno.
  - inglor 5y ago
    @dang I can't seem to be able to edit this is there any way to fix the formatting by any chance?
siquick 5y ago
Any reason to use fetch if I’ve been happy with axios for years?
adam_arthur 5y ago
Awesome work! This was one of the main remaining pain points I had with node.

679 points by yamafaktory 5y ago

easton 5y ago

For those of us unfamiliar, how long until stuff comes out of experimental?

inglor 5y ago
In general "when we feel it's ready". For some features I've worked recently (like `flatMap` and friends for streams) I think it'll take about a year. For `fetch` it will take us around a year or two probably. I wrote more details on the PR itself but I (and others) basically feel strongly that it should be as close to compliant as possible (Deno have a nice list which they happily shared with us of how they diverge and that seems reasonable). Basically there is a bunch of stuff that should happen first: - We need to run (and pass) the web platform tests. - We want to go over the spec (again) and check compliance and add more tests and fix bugs. - We want community feedback on the implementation and to improve the DX to be better (while not sacrificing spec compliance). - We want better support for stuff like `File` in core. - Web streams need to go out of experimental first. This was a _long_ road that had many stepping stones (like EventTarget, AbortController, Blob, ReadableStream etc). It's important to get it right. Worth mentioning the fact it's experimental doesn't mean it's not safe to use in simple code in this case - it just means if you make your code behave in a way that diverges from the specification we may change our implementation to match the specification and break your code. Anything like `await fetch('./someUrl').then(x => x.text())` should be fine.
deleted 5y ago
[deleted]

jitl 5y ago

There’s an NPM package containing this implementation called ‘undici’ that you can install in previous versions of Node; the package.json claims support back to 12.x. https://www.npmjs.com/package/undici https://www.npmjs.com/package/undici However the fetch implementation itself is documented as unstable and only supports Node 16.x. I believe this is because it is not quite spec compliant yet, so there is latitude for breaking changes that make it more spec compliant.

inglor 5y ago

Hey, Node core person here (and the person who triggered the land) - we're super excited for this and would love help and feedback. This is still experimental and we'd love to hear from the community what you'd like to see.

nefitty 5y ago
Hey this one change is going to positively impact my day to day life. Just wanted to let you know. Thanks for sharing your hard work and time.
- inglor 5y ago
  I'm happy to hear that but I had a very small part in it all! The person who took it through the finish line (and deserves the most props here IMO) is Michael https://github.com/targos https://github.com/targos The people who worked most on Undici are Matteo https://github.com/mcollina https://github.com/mcollina and Robert https://github.com/ronag https://github.com/ronag The person to work most on fetch in Undici is Ethan https://github.com/Ethan-Arrowood https://github.com/Ethan-Arrowood Also worth calling our James whose work on web streams in Node as well as events/cancellation helped drive a bunch of this. I can name maybe 50 people who worked on this effort overall at some capacity (I am one of them - spending maybe 40 years on APIs to help enable this). Note the job isn't entirely done and help is very much appreciated!
  - mccorrinall 5y ago
    What does undici mean?
    - Eriks 5y ago
      eleven
    - paulfitz 5y ago
      https://undici.nodejs.org/ https://undici.nodejs.org/ "Undici means eleven in Italian. 1.1 -> 11 -> Eleven -> Undici. It is also a Stranger Things reference."

pjc50 5y ago

Brief explanation?

forty 5y ago
Fetch is a web JS API to do HTTP requests. NodeJS has its own API to do HTTP requests. This add supports for the fetch API. I think the only benefit it that it makes it easier to share code between front end and back end.
- giffarage 5y ago
  It makes it easier to learn and develop both frontend and backend.
- Allstar 5y ago
  For more on Fetch, check out MDN: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
- 9dev 5y ago
  This is an understatement. With fetch available in node, we have a single, standardised, promise-based HTTP client Interface available on all (more or less common) JavaScript platforms. This is great news for an ecosystem traditionally extremely fragmented and may lead the way to more consolidation in the JS library space.
  - forty 5y ago
    NodeJS API are traditionally not promised based (although nowadays some do offer a promisified option), so I'm not especially excited to have a promise-based API :) It's a great news for people doing both front end and backend I guess, but for people doing NodeJS développement it's just one more option to do http request (which I guess is going to be very similar to the module node-fetch that have existed for a while, without special consolidation effect)
    - 5y ago

krossitalk 5y ago

As someone browser focused this surprised me. I've been using fetch() for years now.

zamadatix 5y ago
Not every browser API is an instant candidates to be added to Node's core. If there isn't a big performance hit from doing it as a user library or lack of functionality to do it as such in the first place then it's rare it'll be brought into core right away, if it will at all. It's nice to see the really popular ones make it in though, makes for fewer dependencies when doing browser compatible projects and also allows one to be lazy and not learn the Node way of doing it. Maybe we'll see websocket go into core some day too.
- herpderperator 5y ago
  This is what frustrates me so much about javascript. Everything is all over the place.
  - deleted 5y ago
    [deleted]
  - zamadatix 5y ago
    Eh, I mean I get it JS lacks a real standard library but at the same time you don't see HN fill up with people saying C is all over the place because when they went to implement an HTTP call glib doesn't mirror libuv or C++ is all over the place because POCO doesn't mirror Boost and so on. The "all over the place"-ness tends to come more from people who act like changing to a newer library is the only way to code something new not from the lack of JS forcing a single API be used across every app regardless of type.
DrFell 5y ago
Intercommunicating backend services isn't new, but having an HTTP API call another one from the backend is a case that used to be more normally accomplished by calling both APIs from the frontend. The slow ooze of browser-oriented thinking onto the backend took time. Existing backend engineers, who thought it a harebrained scheme, had to eventually wear down, and give up.

mcraiha 5y ago

Why it took so long? e.g. Deno had fetch support for ages

inglor 5y ago
Basically because fetch isn't a great API for servers it took a while to get consensus on actually landing it for the interoprability/simple API value. Then it took a while to get consensus it's fine to do even if we can't implement the standard fully and diverge from it on stuff like CORS (like Deno does). Then there were a bunch of work adding APIs like `EventTarget` and `AbortSignal` to Node.js which are quirky'ish web APIs. As a side note just to show positive collaboration: that work made me make maybe 10 PRs fixing things in EventTarget in Deno - and Deno helped Node a bunch now when landing fetch. Then web streams and blobs and a few other things as well as the undici work in the background. Then undici-fetch by Ethan, merge into Undici and then the PR by Michael - and it's still not done :) Finally
- dstroot 5y ago
  First, this is awesome and congratulations! Second this sounds like a lot of work! Can you comment why was it so hard to add fetch to node, yet packages like “node-fetch” have existed for some time and seem to implement fetch rather easily? Only asking from a curiosity perspective.
  - inglor 5y ago
    node-fetch (which is great btw!) implements fetch "reasonably" as in stuff like `fetch('./foo').then(x => x.json())` works but it's very far from a "real" fetch. Some people argued users would still like it and there was an attempt[1] by Myles but at the end of the day - the changes were pretty big. For example: a response body in fetch is a web stream (a whole new stream type) and to cancel you use an AbortController (web type) which is an EventTarget (another web type). `node-fetch` uses Node streams instead which is very reasonable but not something a platform can "get away" with while still calling it fetch. Here is a comment I wrote about it in the tracker during the PR review process https://github.com/nodejs/node/pull/41749#issuecomment-1025720476 https://github.com/nodejs/node/pull/41749#issuecomment-10257... [1](https://github.com/nodejs/node/pull/27979 https://github.com/nodejs/node/pull/27979)

thyrox 5y ago

My biggest issue with node when I was working on it briefly was I couldn't do the 'import' statements like wepback. Are they supported too now? I'm not a web developer so I had a very hard time understanding why there are so many different type of imports in JavaScript like require, import, umd, amd, etc and which one works in browser and which one works in node? Also why do so many libraries have this strange 5 line header code for this umd, amd, business. Is that to make their packages work with nodejs? Does anyone who knows enough JavaScript point me in the right direction about it. I find all this very confusing.

mcaruso 5y ago
Node supports ES Modules (the import statements you mention) natively now: https://nodejs.org/api/esm.html https://nodejs.org/api/esm.html
inglor 5y ago
Those have been supported for several years now :) You need to tell Node to use the format by either naming your file `.mjs` or setting `"type": "module"` in your `package.json` file. > Also why do so many libraries have this strange 5 line header code for this umd, amd, business. Is that to make their packages work with nodejs? That's just old for "adapt the module system" from the old days and libraries just didn't bother updating. I still use amd in my job at Microsoft though for some things so I guess it's not useless :)
- mschuetz 5y ago
  > either naming your file `.mjs` But why? This makes it much harder to ensure that identical code works in browser and nodejs, to the point where I sometimes just can't use nodejs. And I don't use package.json, so having to name files ".mjs" is a really painfull restriction.
  - inglor 5y ago
    Browsers will happily accept `.mjs` files they don't actually care about file extensions only about `Content-Type` and type=module on the script tag :]

songzme 5y ago

sorry if this is a stupid question, but how do I try this out? I ran node with the experimental flag on a file that calls fetch: node server.js --experimental-fetch Got 'fetch is not defined' reference error.

inglor 5y ago
First: not a stupid question - the release cadence, multiple release lines and other minutia of a project like Node are not something I'd expect users to be intimately familiar with! This _just landed today_. You can get it by building from master: ``` # there is more details in building.md git clone https://github.com/nodejs/node https://github.com/nodejs/node cd node # may need to pass --openssl-no-asm ./configure make -j12 ./out/Release/node --experimental-fetch ``` Otherwise - wait a bit for the next v17 release to land per the normal release cycle :)
- songzme 5y ago
  oh I see so when node v17.4.1 is released, I would be able to run node server.js --experimental-fetch
  - inglor 5y ago
    v17.5 but yeah
- huksley 5y ago
  Just slightly related to this - I wish I could ask GitHub to send me an email when specific release are available for the repo!
  - mattdeboard 5y ago
    Sounds like maybe an "if this, then that" kind of thing https://ifttt.com/ https://ifttt.com/
  - fs111 5y ago
    You can subscribe to only releases on github.

Meic 5y ago

Undici seems to be an HTTP 1.1 client. Is there any work or timescale to provide HTTP 2 support?

inglor 5y ago
Not yet but it's planned to eventually happen.
bricss 5y ago
check https://www.npmjs.com/package/rekwest https://www.npmjs.com/package/rekwest

itsbits 5y ago

Which NodeJS versions will this be landed to?

inglor 5y ago
It's behind a flag so that it can land in the next v16 semver minor - so soon'ish but keep in mind it _is_ experimental.

ezekg 5y ago

One thing I hate about fetch() is that you can't manually follow redirects. Using { redirect: 'manual' } doesn't expose the Location header of the redirect, so it's essentially useless. I know that node-fetch fixed this issue, so I hope the official Node fetch() does not have the same problem.

inglor 5y ago
We are not, this is a place the server-side implementation is going to diverge and this was discussed in the PR: https://github.com/nodejs/node/pull/41749#issuecomment-1025473227 https://github.com/nodejs/node/pull/41749#issuecomment-10254...
- ezekg 5y ago
  This is really, really great to hear. Thanks for all the hard work!

tekstar 5y ago

Can you set more than one cookie in the request and read more than one cookie in the response? Was shocked when I learnt that's not supported by the spec..

styfle 5y ago
There's a relevant spec issue you can follow here https://github.com/whatwg/fetch/issues/973 https://github.com/whatwg/fetch/issues/973

bachmeier 5y ago

Good that some people are happy about this, but it really is pointless to post a link to "lib: add fetch" followed by code diffs. HN should have a requirement that there be an actual post, rather than a discussion starter for those in a particular community. All you'd have to do is type in a brief text post with an explanation and then link to the code diffs for those interested.

inglor 5y ago
There will be a blog post on this in the Node.js blog and official media communication. This _just_ landed today and I suspect the HN crowd are more in the "get news early" camp and not "wait for the official press release" camp.

sealthedeal 5y ago

Ive used Fetch in the past, great lib. I'm curious why Node core has decided to make this an official part of Node? From one perspective there could be a hard argument that this should never be a part of Node as it is an external library and not a building block of the language? Almost feels like feature creep for the core language? I know that's not the intent, but wanting to understand the logic behind this decision :)

prollings 5y ago
They're talking about JavaScript's fetch API. It's not an external library.
- zamadatix 5y ago
  It ships browsers but not because it's a JS standard (ECMA) rather a web standard (WHATWG). Almost everything in JS is an external library as it doesn't have a true stdlib to go with the language just a core language and implementation specific APIs like browser Fetch or Node sockets. Hopefully WebSockets (W3C/IETF) will get embedded in Node's core one day as well.
wruza 5y ago
I second this, since fetch is a wrapper around xhr (though doesn’t have to be because node is a non-restricted runtime), and it diverged already for redirect semantics. It’s news, but is it big?
- samtho 5y ago
  > It’s news, but is it big? As someone who has been working almost exclusively on Node.js applications for years now, this is big. Front-end developers have, for years, enjoyed this wrapper around the clunky XHR API. It is simple, intuitive, and it comes _free_ in the browser environment (and was easily shimmable before it was standard). Additionally, the depreciation of the popular request[0] a while ago, there has been a gap in The Right Way to do http requests without going into the low-level API that ‘http(s)’ provides. Fetch is promised-based (enabling seamless async/await) and, most importantly, standardized. Basically, fetch provides a meaningful abstraction atop each platforms’ implementation of http-request-doing further unifying server-side and client-side patterns and idioms. [0]: https://www.npmjs.com/package/request https://www.npmjs.com/package/request

esamatti 5y ago

Does this contain `request.formData()` and `FormData`? https://developer.mozilla.org/en-US/docs/Web/API/Request/formData https://developer.mozilla.org/en-US/docs/Web/API/Request/for... https://developer.mozilla.org/en-US/docs/Web/API/FormData https://developer.mozilla.org/en-US/docs/Web/API/FormData

inglor 5y ago
> `request.formData()` Yes. > FormData Yes, but not in that PR since `FormData` needs to behave differently as part of the platform but there is intent to support it before moving from experimental.

mariogintili 5y ago

I love that the most popular language's core lib now brings a trivial bit of code available in every other platform and its a celebrated super achievement software, why do you suck now?

jmull 5y ago
I don't think you understand this. e.g., here's your "trivial" API's spec. Then there are the intentional deviations for CORS, e.g., which there is no spec for. https://fetch.spec.whatwg.org https://fetch.spec.whatwg.org Not to mention the Fetch API isn't available on every other platform... maybe you're thinking of the general ability to make HTTP requests, but nodejs had that from the start.
feupan 5y ago
It’s not trivial and it’s not groundbreaking. Node has had an HTTP client since its inception, this is just a different API. It’s “great” just because it’s a decent abstraction that’s also available in browsers. The issue in the JavaScript ecosystem for this kind of stuff is that it runs on two very different environments and they need time to find solutions that work for all the parties involved.

bricss 5y ago

Try adv fetch-alike with HTTP2 support https://www.npmjs.com/package/rekwest https://www.npmjs.com/package/rekwest

chx 5y ago

Good for NodeJS However, I feel the fetch API is completely botched because it lacks timeout support. I have cobbled some together for https://github.com/Backblaze/gists/pull/8 https://github.com/Backblaze/gists/pull/8 but gosh. I really hope all that is actually not necessary :/

nikeee 5y ago
You can use an AbortController for that. Also, keep in mind that the linked solution does not abort the web request. It only aborts waiting for the response in the JS code. The browser does not close the connection.

nfriedly 5y ago

I love fetch, except for the way that it combines headers. If more than one of the same header is set (such as multiple set-cookie headers), it combines them all into a single comma-separated string. I know this is allowed in the HTTP spec, and it's probably even a sensible default. But the fetch spec doesn't allow any access to the raw headers, so there's no straightforward way to get the original uncombined headers back. Set-Cookie headers, in particular, often contain commas in their expiration dates, so just splitting around `, ` will lead to problems. I took a look through the source of this new fetch API, and it seems to have inherited that wart: https://github.com/nodejs/undici/blob/2dd3437e20c5a3cc466226c544f048351baffc78/lib/fetch/headers.js#L103 https://github.com/nodejs/undici/blob/2dd3437e20c5a3cc466226... I've argued with the authors of the fetch spec about this before, and ultimately settled on using https://www.npmjs.com/package/set-cookie-parser#user-content-splitcookiesstringcombinedsetcookieheader https://www.npmjs.com/package/set-cookie-parser#user-content... to work around this flaw. (For clarity, I published the package, but chrusart wrote that method - https://github.com/nfriedly/set-cookie-parser/pull/19 https://github.com/nfriedly/set-cookie-parser/pull/19)

styfle 5y ago
There's a relevant spec issue here https://github.com/whatwg/fetch/issues/973 https://github.com/whatwg/fetch/issues/973
- nfriedly 5y ago
  Oh, good. The first comment there references my closed issue from 2017, but it's nice to see this being given some more consideration.

outside1234 5y ago

Thank god - no more node-fetch shims for node.js!

stereocodes 5y ago

Wow I can't believe how light years ahead Deno is than Node. You want imports you want fetch you want TS, use Deno.

andyfleming 5y ago
Why can't you use TypeScript with node? It's not built in, but it's easy enough to use by building or running it with ts-node. Also, once you are using TypeScript, you have import syntax support. Deno has chosen a different path, but there are definite trade-offs.
- stereocodes 5y ago
  Yup the point I'm making is its built in. no hoops no babel. I find myself gravitating towards Deno before Node because with node I know I'll have to do some hoop jumping to make it work.
ecf 5y ago
Seriously. I maintain an open-source Node.js CLI utility and was shocked when import/export syntax is only achievable if you use a transpiler like Babel. Such overkill for something that’s been in the JS spec for nearly a decade.
- ash_gti 5y ago
  Node does support ESM now by default (and has supported it for a while behind a flag), https://nodejs.org/api/esm.html https://nodejs.org/api/esm.html
Scarbutt 5y ago
Yes, and the go light years back in ecosystem ;)
- stereocodes 5y ago
  sort of. There's only been one instance for me that this has caused any issues... I ended up just writing my own version of the missing tool. Deno has almost parity with node at the moment though https://deno.land/std@0.123.0/node https://deno.land/std@0.123.0/node.

saos 5y ago

Noicee

joelbondurant1 5y ago

quickthrower2 5y ago

Funny how my brain works, I thought it was there already, but I must have got so used to having it as a dependency I forgot that it was a dependency not built in. To be fair I don't "Node" all the time! It's on and off.

stevage 5y ago

Out of curiosity, what took so long? It's a small API that has been stable for a long time? I'm probably missing something.

zamadatix 5y ago
Same reason the WebSocket API isn't supported, Node already had lower level APIs and browser APIs moved 1:1 don't necessarily make sense. In this case it's a slightly extended/non-standard Fetch and enough of the "it doesn't need to be in core it can be a normal library" folks thought it'd be useful enough. And someone went through the work of actually making the native implementation.
- digianarchist 5y ago
  I actually think both the Websocket API and fetch make sense. There's nothing more frustrating than adding a bunch of checks to a SvelteKit project to see if you're on the server or not.
  - zamadatix 5y ago
    I wouldn't be surprised if WebSocket made it into core eventually tbh, the above are just examples of why such things haven't immediately. If WebSocket does it'll probably have more modifications/deviations than say Fetch did here though as well as an entire server side half. `ws` is well loved but there are 2 optional native binary addons which greatly increase performance, that alone makes a good enough case for it to end up in core - just a matter of someone finding it interesting enough to make a solid PR to do so instead of just using the `ws` library as is.

pseudosavant 5y ago

Every Node.js dev should be thanking the Deno creators. It has forced them off of their "this is how Node does it" complacency. Most improvements I notice and care about on Node these days are all things that Deno already supports (fetch, ES modules, async std lib uses promises, etc). The Web Platform APIs aren't perfect, but they are better, and more thoroughly thought out than Node's.

inglor 5y ago
What is "this is how Node does it" complacency you speak of? Node has been talking about Fetch since at least 2018 _before_ Ryan even announced Deno. This is true for promises, ESModules etc. Ryan _attended those meetings_ in the Node collaborator summit. Don't get me wrong I am thankful for Deno (and a contributor!) but: - It's a lot easier to make changes when the project is new and you don't have a lot of users (which is great for Deno) whereas Node has a whole (huge) ecosystem to consider on every change. - It's a lot easier to make changes with venture capital and funding to hire a bunch of people to work on it full time vs. a project with a "regular" MIT license where the code is owned by the community and not a company. Those are not complaints about Deno, I like Deno (and not just Ryan, Lucas, Kit, Ben and the other people are nice and helpful and the project is really nice). It's just to explain very little of Node's innovation is (at the moment) driven by Deno.
- inglor 5y ago
  @dang I can't seem to be able to edit this is there any way to fix the formatting by any chance?

siquick 5y ago

Any reason to use fetch if I’ve been happy with axios for years?

adam_arthur 5y ago

Awesome work! This was one of the main remaining pain points I had with node.