5 ms·
Efficient data transfer through zero copy in Java
- rheide 15y agoIt's a nice idea, but I wonder about the use cases. How often would a Java webserver serve a file from disk without modifying it along the way? If performance is an issue then you'd serve the really static files with nginx (or a similar alternative), and every other file (jsp templates etc.) would be read in memory before being sent on. (Note that I'm not questioning the technique, I just don't entirely agree on the way the article portrays the benefit of this method in webservers.)
- jbri 15y agoOften you're serving big chunks of static content with a little bit of dynamic content sprinkled in. You can effectively concatenate stuff by just writing both to the same channel - which means you can get the zero-copy performance benefits for the static parts of your data if you know exactly how much static data there is before you need to write out something dynamic.
- virmundi 15y agoBut how could we use this in a servlet container environment? Perhaps we'd wrap the response with a special wrapper to take advantage of this? I'm not sure. I can see the use if one would recreate a server from the ground up. But that would be a rather large task.
- teraflop 15y agoTomcat has supported zero-copy file transfers for years -- since before this article was written, in fact. http://tomcat.apache.org/tomcat-6.0-doc/aio.html#Asynchronous_writes http://tomcat.apache.org/tomcat-6.0-doc/aio.html#Asynchronou...
- zaptheimpaler 15y agoI agree to a certain extent. If static content is a big issue, you should be using CDNs running barebone server applications anyways. At the same time, I can imagine plenty of companies dont have the resources or expertise to do these kinds of things.In that case this looks pretty helpful.
- rkalla 15y agoFor a specific example I just implemented recently: a database utilizing MappedByteBuffer's to the main DB file that can transferTo the record requested as-is to the requesting client. The use-cases are certainly specific, but when you need them, it's great.
- fauigerzigerk 15y agoI can think of quite a few use cases: Ebedding a webserver in a Java application. Writing distributed infrastructure software like Hadoop or some kind of messaging system. Anything that caches stuff on a disk to pass it on later. Database systems that replicate transaction logs. Backup and storage systems. I don't think nginx or web servers in general are the last word ever to be spoken in terms of networking software.
- Rauchg 15y agoYou might want to read this interesting (albeit slightly outdated) post about leveraging `sendfile` in Node.JS by Peter Griess: http://blog.std.in/2010/09/09/using-sendfile-with-nodejs/ http://blog.std.in/2010/09/09/using-sendfile-with-nodejs/
- mikeknoop 15y agoIs this still best practice? What's the more "up to date" method?
- justincormack 15y agosplice and tee are the newer kernel interfaces, sendfile is now implemented using them.
- ntoshev 15y agoApparently these primitives haven't found their way to the Node.js land yet... and sendfile is still undocumented.
- justincormack 15y agoNot really surprising given the cross platform focus in Node now.