17 ms·
Hey, Node core person here (and the person who triggered the land) - we're super excited for this and would love help and feedback. This is still experimental
by inglor 5y ago
Hey, Node core person here (and the person who triggered the land) - we're super excited for this and would love help and feedback.
This is still experimental and we'd love to hear from the community what you'd like to see.
- nefitty 5y agoHey this one change is going to positively impact my day to day life. Just wanted to let you know. Thanks for sharing your hard work and time.
- inglor 5y agoI'm happy to hear that but I had a very small part in it all! The person who took it through the finish line (and deserves the most props here IMO) is Michael https://github.com/targos https://github.com/targos The people who worked most on Undici are Matteo https://github.com/mcollina https://github.com/mcollina and Robert https://github.com/ronag https://github.com/ronag The person to work most on fetch in Undici is Ethan https://github.com/Ethan-Arrowood https://github.com/Ethan-Arrowood Also worth calling our James whose work on web streams in Node as well as events/cancellation helped drive a bunch of this. I can name maybe 50 people who worked on this effort overall at some capacity (I am one of them - spending maybe 40 years on APIs to help enable this). Note the job isn't entirely done and help is very much appreciated!
- mccorrinall 5y agoWhat does undici mean?
- Eriks 5y agoeleven
- paulfitz 5y agohttps://undici.nodejs.org/ https://undici.nodejs.org/ "Undici means eleven in Italian. 1.1 -> 11 -> Eleven -> Undici. It is also a Stranger Things reference."
- _ZeD_ 5y ago"undici" means "eleven" in italian. But I don't know the rationale behind the name
- epolanski 5y agohttp 1.1
- kichimi 5y agoI do have a question that's only tangibly related to fetch, I was wondering what Nodes stance is on the seemingly increasing gap between the NodeJS and browser implementations of the JavaScript engine. It's still very common to see Node modules that use require(), for example, but would otherwise work flawlessly in a browser where there is no require() support (without using shims and other libraries).
- inglor 5y agoI don't speak for the project a whole :) Node has supported ESModules (`import`) for a while and you can use it (either by naming your files `.mjs` or setting `type: module` in your package.json. There is a solid interop story between require/import and you can mix if you want. `require` is going to be supported "forever" so code doesn't break and both module systems work. As for the general question: Node is committed to supporting modern JavaScript and to not allow such a gap to be created. If you see a place Node doesn't do a great job at that please tell us!
- arcatek 5y agoHowever note that ESM in Node comes with drawbacks that prevent end-users from relying them in various situations. Those will be mostly solved once loaders become stable, but until then it's still advised to ship packages as both CJS and ESM.
- rattray 5y agoHow do you do both?
- inglor 5y agoIt's very easy to re-export ESM (import) as CJS (require) and vice-versa. The main issue is that ESM by default For example to use `require` inside ESModules you would do: ```mjs import { createRequire } from 'module'; const require = createRequire(import.meta.url); require('./whatever-in-cjs'); ``` There is a reason this isn't "by default" though since ESM doesn't "silently" interop with CJS to not make writing universal code harder.
- staticelf 5y agoQuick question, judging from the commit alone this seems like a rather small change. I assume there's more to it though. I just wonder how come features like this that kind of seems obvious to include in the ecosystem takes quite some time to land? I understand the reality is more complex perhaps, so I am genuinely curious. I hope you realize no disrespect, this will greatly improve the daily work for me since I use node at work every day.
- oefrha 5y agoAdding 8k lines is hardly a small change.
- inglor 5y agoThe commit adds undici (another Node.js project at https://github.com/nodejs/undici https://github.com/nodejs/undici ) as a dependency and exposes its `fetch` - the code changes you see are probably just adding a flag :) > I just wonder how come features like this that kind of seems obvious to include in the ecosystem takes quite some time to land? I answered that below (check it out) but note how expensive adding a bad API is vs. asking people for one more `npm install` :) There is more discussion in https://github.com/nodejs/node/issues/19393 https://github.com/nodejs/node/issues/19393 and in https://docs.google.com/document/d/1tn_-0S_FG_sla81wFohi8Sc8YI5PJiTopEzSA7UaLKM/edit?usp=sharing https://docs.google.com/document/d/1tn_-0S_FG_sla81wFohi8Sc8... a discussion from 2018 we had on it
- staticelf 5y agoOk cool and thanks for the clarification, I imagined it was more to the topic than I realized. Last question I have is will this be included in the next version of the LTS version or how do experimental features usually progress for someone who are unfamiliar? Can't wait until I can use fetch in my codebase without any additional dependencies!
- Narretz 5y agoTo add, unidici is an http client. Node already has http / https, but unidici is much faster and probably has other improvements, too. They could have built a fetch interface on top of http, but they probably saw an opportunity to make more drastic improvements.
- tbarbugli 5y agoyes, yes, yes. This was a huge pain for us. We develop an SDK that needs to work on Node, browsers and React Native. I am super happy to hear about this :)
- rglover 5y agoThis is awesome, thank you. Excited to take it for a spin.
- zebracanevra 5y agoIs there any chance to break spec and allow manual redirect handling? the fetch API makes a lot of sense in a browser, but imo this is a pretty crucial feature that undici's implementation lacks. [0] Deno decided to break spec [1][2] so the following code works fine: fetch('https://httpbin.org/status/302', {redirect: 'manual'}) .then(res => console.log(res.status, res.headers)) In undici this will succeed but with res.status 0 and no headers, as per spec. You aren't allowed to see the content of a redirect, like in a browser. [0] https://github.com/nodejs/undici/issues/1072 https://github.com/nodejs/undici/issues/1072 [1] https://github.com/denoland/deno/pull/8353 https://github.com/denoland/deno/pull/8353 [2] https://github.com/denoland/deno/issues/4389 https://github.com/denoland/deno/issues/4389
- nailer 5y agoA better idea would be to replace / improve the fetch spec. Handle redirects, set JSON as the default request content type, encode URI components for users, decode response bodies with JSON headers as JS objects, etc. Right now most developers either write their own library to do these things on top of fetch or another HTTP client or use a third party library from npm. A new standard would allow us to make http requests out of the box comparable to high level HTTP clients like superagent, axios etc. that have been in use for the last decade, since before fetch was conceived, with whatever benefits fetch provides (I think it's cancellable now?).
- inglor 5y agoA new standard would need buy-in from browsers to actually be a standard. Browsers likely don't have a lot of incentive to make spec changes only Node is interested in (like making the whole spec more complicated from their point of view because of Node's (or Deno's) different security model). The level of collaboration and good-faith we've been getting from spec bodies like WHATWG is very high as it is and we really don't want to push or abuse it.
- andrewl-hn 5y agoDoes it have to be a buy-in, though? You could agree on server-specific extensions to fetch and codify them in the same spec (so it doesn't get lost otherwise). Browser vendors wouldn't need to implement it but they will still keep an eye on it when working on future browser apis.
- neves 5y agoWhere can I find an explanation about what is Fetch and why it should be in Node.js?
- neves 5y agoIt looks like it is a new standard: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API I thought it was a Node popular library.
- fiddlerwoaroof 5y agoIt’s “new” as of 6 years ago or so.
- inglor 5y agoThis is fetch: https://github.com/whatwg/fetch https://github.com/whatwg/fetch https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API Here is discussion about it in Node with some (pretty old) discussion https://docs.google.com/document/d/1tn_-0S_FG_sla81wFohi8Sc8YI5PJiTopEzSA7UaLKM/edit?usp=sharing https://docs.google.com/document/d/1tn_-0S_FG_sla81wFohi8Sc8...
- golergka 5y agoFetch is a popular browser api to download and request stuff from the internet. Having the same api both in browser and in node allows easier re-use of the same application code or libraries on frontend and backend.
- tehbeard 5y agofetch is a "modern" (ES6 era, so not brand new) API for making HTTP requests, to replace XMLHttpRequest (the older method that IE pioneered). It's promise based (thus easier to integrate into code than older callback styles), and designed to give some better options around CORS and handling responses. > Why it should be in node? There is a push to adopt some certain "web apis" (APIs that emerged in web browsers) to increase compatibility, reusability and reduce cognitive load when working with both backend and frontend javascript. Having a common low level call like fetch() means that libraries that are built atop it (SDKs to talk to services, or apply common middleware like JSON:API, oAuth etc) can be shared between node/deno and browsers.
- gk1256 5y agoI wish if Node had a way to enforce any case in its requests.
- mnutt 5y agoDo you have any sense of the expected performance of web streams as compared to the existing node.js streams?
- adityapatadia 5y agoThanks a ton for this. Fetch API is elegant and it working on node natively will so many edge cases we need to handle currently. Thanks a ton, please know that this will positively change our life and tens of thousands of other developers worldwide.
- franciscop 5y agoReally happy to see fetch() merged into core! Looking at the PR, what are those WASM blobs in Undici and how is it that Node.js accepts a random very large compiled blob instead of asking for the source file + compilation step? https://github.com/nodejs/node/commit/6ec225392675c92b102d3caad02ee3a157c9d1b7#diff-f516ab824a7722da938a4c7c851520d39731ddeb4f7198dff4e932c5d4f8fdf7R975 https://github.com/nodejs/node/commit/6ec225392675c92b102d3c...
- inglor 5y agoThose wasm blobs are Node's own llhttp https://github.com/nodejs/llhttp https://github.com/nodejs/llhttp in wasm to speed up HTTP parsing. The project itself added as a dependency is also by Node https://github.com/nodejs/undici https://github.com/nodejs/undici . The question is totally legitimate but please assume core doesn't make "load random binary" level kind of goofs :)
- franciscop 5y agoSorry def didn't mean in a bad way! I just wanted to ask/note that in the PR, and with the very little context I have, it seems like an arbitrary code blob so I was surprised, I'm 100% it'll be well documented in e.g. undici itself, and prob in other places.
- krono 5y agoExcept that's exactly what Corepack does, just silently and opaquely on the user's system. At least, the moment it switches from opt-in to opt-out. Its unverified binaries are just as good as random binaries, and silently replacing any pre-existing yarn/pnpm/whathaveyou without checking if it was perhaps a custom build is not exactly predictable either. Other than Corepack, though, I think you people are doing an amazing job!
- andrew_ 5y agopretty great. I've been using undici for a while now and very happy with the speed and experience.
- can16358p 5y agoGreat work! Curious to check but quick question: does it support upload progress? If not, is it considered?
- inglor 5y agoYou can upload a stream and monitor that for upload progress - so sure. If you upload a string for example you would have to take an extra step to get upload progress (make it into a stream and monitor that). There is additional (unrelated) talk in the fetch standard itself to provide this sort of functionality as part of `fetch` which would save the middle step.
- ivanb 5y agoIs there support for timeouts? It's the main reason I use https://github.com/sindresorhus/got https://github.com/sindresorhus/got
- ignoramous 5y agoHi there, > ...we'd love to hear from the community what you'd like to see. In our tests, fetch on Deno was wayy faster than undici fetch that now has been merged into nodejs. I couldn't figure out why that was case, but forwarding requests over node's http2 client (instead of undici fetch) then had comparable (but not as fast) performance as Deno (presumably because our hand rolled impl lacked connection pooling).
- inglor 5y agoThis is very new and the implementation _just_ landed today so it makes sense Deno would be faster with an API they have been working on for years. I am confident that Node's implementation will _eventually_ have comparable performance. That said: please do open an issue in the undici repo at https://nodejs.org/node/undici https://nodejs.org/node/undici so this gets tracked.
- mstade 5y agoThat URL gives me a 404 – should it be https://github.com/nodejs/undici https://github.com/nodejs/undici ?
- ignoramous 5y agoThanks. Filed: https://github.com/nodejs/undici/issues/1203 https://github.com/nodejs/undici/issues/1203
- thysultan 5y agoDeno's is probably written in rust via the tokio rust crate. This version of fetch in NodeJS is written in JavaScript. It would probably get upgraded to C++ in due time once it's mature enough.
- asdfman123 5y agoThank you for making "fetch" happen
- erichocean 5y agounderrated comment
- mitchellgoffpc 5y agoHuge thanks to the node team for adding this, I’ve been wanting fetch in node for years now! Installing node-fetch for every project was getting kind of old haha
- joshxyz 5y agoDoes it mean we have a built-in FormData in Node.js core for multi-part requests?
- eurasiantiger 5y agoSadly, no.
- eurasiantiger 5y agoFormData and Blob/File support?