3 ms·

I think FDWs are not more used because they're not easy to get into. The best link/example I found was https://github.com/beargiles/passwd-fdw https://github.c

I think FDWs are not more used because they're not easy to get into. The best link/example I found was https://github.com/beargiles/passwd-fdw https://github.com/beargiles/passwd-fdw and it's quite easy to follow the code and understand all the moving parts. Once you've written a FDW you'll see them everywhere. In fact the same author wrote https://github.com/beargiles/zip https://github.com/beargiles/zip file-fdw and https://github.com/beargiles/tarfile-fdw https://github.com/beargiles/tarfile-fdw If you (still?) need inspiration and want to see what already exists: https://wiki.postgresql.org/wiki/Foreign_data_wrappers https://wiki.postgresql.org/wiki/Foreign_data_wrappers Simpler, using a 'generic' file-based FDW shipping with pg's sources: https://aaronparecki.com/2015/02/19/8/monitoring-cpu-memory-usage-from-postgres https://aaronparecki.com/2015/02/19/8/monitoring-cpu-memory-... There's a python wrapper to get your feet wet (or prototype an idea) : https://github.com/Segfault-Inc/Multicorn https://github.com/Segfault-Inc/Multicorn (though I'm not sure how maintained this is). The only annoying part is that you're plugging your code to an interface that might (and has sometimes) broken between releases of PG. So kind of the same fun as maintaining a gcc plugin... By the way, anyone has any idea on the licensing terms/issues of PG FDWs and PG extensions in general?

chatmasta 6y ago
FDWs are a powerful feature. We’ve done a lot of work to make scaffolding them easier, if you use sgr. One of our earlier blog posts includes an example of making an FDW for the HN API and packaging it as a Splitgraph mount handler. [0] There’s also this great post about using FDWs for parallel query execution, by David Grier at Swarm64. [1] They seem to be doing cool things with FDWs too. Personally I think proxies are a really powerful abstraction in general. Cloudflare and Twilio are two examples of companies built around proxies. [0] https://www.splitgraph.com/blog/foreign-data-wrappers https://www.splitgraph.com/blog/foreign-data-wrappers [1] https://swarm64.com/post/scaling-elastic-postgres-cluster/ https://swarm64.com/post/scaling-elastic-postgres-cluster/
- touisteur 6y ago
  Thanks for both links! I'm not sure where you're going with this product, but I like the idea of proxies and your idea of a DDN, and I wish you the best. I'm trying to 'bind them all' at my job right now and Postgres is very inspiring with its extension success stories, and all the FDW work happening. And once you can bind to C, you can mostly chose your language :-)

chatmasta 6y ago

FDWs are a powerful feature. We’ve done a lot of work to make scaffolding them easier, if you use sgr. One of our earlier blog posts includes an example of making an FDW for the HN API and packaging it as a Splitgraph mount handler. [0] There’s also this great post about using FDWs for parallel query execution, by David Grier at Swarm64. [1] They seem to be doing cool things with FDWs too. Personally I think proxies are a really powerful abstraction in general. Cloudflare and Twilio are two examples of companies built around proxies. [0] https://www.splitgraph.com/blog/foreign-data-wrappers https://www.splitgraph.com/blog/foreign-data-wrappers [1] https://swarm64.com/post/scaling-elastic-postgres-cluster/ https://swarm64.com/post/scaling-elastic-postgres-cluster/

touisteur 6y ago
Thanks for both links! I'm not sure where you're going with this product, but I like the idea of proxies and your idea of a DDN, and I wish you the best. I'm trying to 'bind them all' at my job right now and Postgres is very inspiring with its extension success stories, and all the FDW work happening. And once you can bind to C, you can mostly chose your language :-)