6 ms·

The architecture and software behind Pipes

66 points by onli 7y ago

hermitcrab 7y ago
If you want desktop software in a similar vein, check out our recently released: https://www.easydatatransform.com https://www.easydatatransform.com
- jdc 7y ago
  I wish the JACK audio node graph editors were this polished!
- nicedicerice 7y ago
  thanks, man!
- dangoor 7y ago
  Does this pull data from the internet? That seems like a fundamental feature of Pipes.
  - hermitcrab 7y ago
    Currently import is from files (xls/xlsx/csv/tsv) and the clipboard. We might also add import from the Internet in future, if there is enough demand.
    - dangoor 7y ago
      Cool! I think there are a lot of ease-of-use aspects to this being a desktop app instead of a webapp requiring deployment.
      - hermitcrab 7y ago
        I think there are a lot of advantages to a local app: 1. richer interface 2. lower latency 3. no need to share sensitive data with a third party 1 & 2 are less of an issue than they used to be, but 3 is still a major issue.
        dangoor 7y ago
        That's all true. I think the advantages to a server-based app are 1. available from any device, 2. able to operate on your behalf at any time There's a reasonable place for both. Getting data and operating on it is a pretty fundamental need!
        hermitcrab 7y ago
        Yes, a web-based solution is going to be better for some use cases. But if you have a load of sensitive data files to wrangle then a desktop app is hard to beat.
srean 7y ago
I don't think pypes is maintained anymore but I was quite stoked about it when it was released. Coroutine based flow-based-programming pushes some pleasant buttons of mine https://pyvideo.org/pycon-us-2011/pycon-2011--large-scale-data-conditioning--amp--p.html https://pyvideo.org/pycon-us-2011/pycon-2011--large-scale-da...
greenie_beans 7y ago
Neat, thanks for the write up
dsl 7y ago
https://refinery.io/ https://refinery.io/ is a similar service and YC alum.
lincpa 7y ago
The other pipe architecture: The Pure Function Pipeline Data Flow https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow

66 points by onli 7y ago

If you want desktop software in a similar vein, check out our recently released: https://www.easydatatransform.com https://www.easydatatransform.com

jdc 7y ago
I wish the JACK audio node graph editors were this polished!
nicedicerice 7y ago
thanks, man!
dangoor 7y ago
Does this pull data from the internet? That seems like a fundamental feature of Pipes.
- hermitcrab 7y ago
  Currently import is from files (xls/xlsx/csv/tsv) and the clipboard. We might also add import from the Internet in future, if there is enough demand.
  - dangoor 7y ago
    Cool! I think there are a lot of ease-of-use aspects to this being a desktop app instead of a webapp requiring deployment.
    - hermitcrab 7y ago
      I think there are a lot of advantages to a local app: 1. richer interface 2. lower latency 3. no need to share sensitive data with a third party 1 & 2 are less of an issue than they used to be, but 3 is still a major issue.
      - dangoor 7y ago
        That's all true. I think the advantages to a server-based app are 1. available from any device, 2. able to operate on your behalf at any time There's a reasonable place for both. Getting data and operating on it is a pretty fundamental need!
        hermitcrab 7y ago
        Yes, a web-based solution is going to be better for some use cases. But if you have a load of sensitive data files to wrangle then a desktop app is hard to beat.

srean 7y ago

I don't think pypes is maintained anymore but I was quite stoked about it when it was released. Coroutine based flow-based-programming pushes some pleasant buttons of mine https://pyvideo.org/pycon-us-2011/pycon-2011--large-scale-data-conditioning--amp--p.html https://pyvideo.org/pycon-us-2011/pycon-2011--large-scale-da...

greenie_beans 7y ago

Neat, thanks for the write up

dsl 7y ago

https://refinery.io/ https://refinery.io/ is a similar service and YC alum.

lincpa 7y ago

The other pipe architecture: The Pure Function Pipeline Data Flow https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow