8 ms·
Windmill.dev
- jzig 5y agoIt reminds me of a simpler version of Apache NiFi (https://nifi.apache.org/ https://nifi.apache.org/)
- recov 5y agoSo powerful and so ugly
- aftbit 5y agoLooks really interesting! We have an "eng-b_team" channel which is a place for all of the non-technical folks at my company to collaborate on scripts, SQL queries, etc. They would love to use something like this. How do we set up a self-hosted instance to try it out?
- eatonphil 5y agohttps://github.com/windmill-labs/windmill#how-to-self-host https://github.com/windmill-labs/windmill#how-to-self-host Looks like those instructions don't exist yet.
- rubenfiszel 5y agoThe Dockerfile mostly work by itself, the only thing you will lack is how to generate the super-admin. Sorry I didn't expect to have any announcement today but promised it will be filled by EOD :)
- thealistra 5y agoThis could really use some better explaination with examples. I am a dev for 15 years and I have no idea what it does after a minute of looking at it
- eatonphil 5y agoIt makes sense to me (but maybe because I'm building something similar). You write code and they host it for you and provide a GUI around your code. It's like a crazier (in a good way) version of Kibana or Grafana or Metabase. In Kibana/Grafana/Metabase they provide a hosting platform for queries -- any queries within your internal eng team. This provides a hosting platform for scripts. Whether or not that is exactly what they are trying to do, it's what I'm trying to do. What I wish I'd had at many prior companies. I can't find them now while trying to search but it also reminds me of some tools I've seen that generate a GUI from a description of a tool's command line flags.
- mritchie712 5y agoStreamlit maybe?
- eatonphil 5y agoYeah that looks similar! I think I've seen open-source terminal GUI (TUI) wrappers too.
- scandox 5y agoThey host "scripts" and "apps" that you can execute against "resources" (i.e. databases etc...). So basically a lightweight, online environment for running code without any deployment? I suppose an example might be marketing has a DB of prospective customers and this would allow devs to quickly write scripts to clean it, apps to allow people to add to it and automated tasks to trigger events (eg. an email) based on certain metrics etc... I guess it means being able to rapidly build fairly throwaway systems for internal use.
- puchatek 5y agoSo... Self-hosted AWS lambdas (but only for python)?
- omarhaneef 5y ago
- jzig 5y agoAnyone know of anything like this but in JS/TS?
- rubenfiszel 5y agoIt will support JS/TS in about a week
- eatonphil 5y agoI build a somewhat similar app, DataStation [0], that is in JavaScript and Go. It supports scripting in Python, Julia, R, JavaScript, Ruby, etc. The server version of it exists and I run it myself but that process is not documented yet. (Most people use it as a desktop app today.) [0] https://github.com/multiprocessio/datastation https://github.com/multiprocessio/datastation
- anyfactor 5y agoThis is very interesting. I was actually looking for a cleaner way to write SQL and Python code. I find writing multistring SQL query on Python code messy so looking forward to giving DataStation a shot. Couple of question- - Are you building this as a pure Open Source project? I couldn't find the pricing section, and you have built it as an electron project. I am curious to know about your pricing strategy. - Have you considered incorporating dbt or dbt like syntax query?
- eatonphil 5y ago> - Are you building this as a pure Open Source project? I couldn't find the pricing section, and you have built it as an electron project. I am curious to know about your pricing strategy. Long-term plan is that everything currently open source will always be open source (and there's quite a lot I believe). I'd like to release a hosted (paid) SaaS version for teams and an enterprise version also aimed at teams. But the desktop app that exists now will always be free and open source. And I have no plans to stop adding new databases, file sources, and scripting languages as needed to the open source core. > Have you considered incorporating dbt or dbt like syntax query? I think it's already pretty close to this. You can import data within SQL queries (works with SQLite, MySQL, and PostgreSQL) using the `DM_getPanel` syntax. For everything else you can use Jinja-style templates [0]. This allows you to, for example, run a query to select IDs based on some filter in a PostgreSQL database and then pass those IDs to an Elasticsearch filter in another panel -- without writing any code. [0] https://datastation.multiprocess.io/docs/latest/Panels/Macros.html https://datastation.multiprocess.io/docs/latest/Panels/Macro...
- rubenfiszel 5y agoHello, author here. I did not plan to have a wide announcement today tbh. This is the repo: https://github.com/windmill-labs/windmill https://github.com/windmill-labs/windmill I have my YC interview in 10 minutes and wanted to OSS it before. Agree it lacks a lot of polishing, but that will be done in about 2 weeks :) JS/TS through Deno is coming in a week, I am currently working on a draft pull request for it. Really happy to answer all the questions after my interview :) (I host the demo on a small hetzner node, please be gentle to it, I will scale it up asap :)) PS: Since it's HN: The backend is all in Rust and the feature I am most proud of is automatic parameters and dependency inference for locking the dependency versions and generating the frontend of your app by analyzing the scripts using simple but surprisingly good heuristics :)
- poofsoda 5y agoGood luck with the interview!!
- rubenfiszel 5y agoUpdate: I GOT IN!!!
- bm-rf 5y agoCongrats! We currently use rundeck to ssh into servers to run python automations. This looks to be a much more elegant solution.
- fernandotakai 5y agocongrats!
- wantlotsofcurry 5y agoCongrats!!!
- jamesmcintyre 5y agoNow you must share a detailed account of your interview! Congrats!
- 5y ago
- ryukafalz 5y agoThis looks pretty neat! Once the FOSS self-hostable version is out I might try it out for my local hackerspace, it seems like it might be useful for those little things that are too small to warrant standing up infrastructure at our small scale.
- rubenfiszel 5y agoIf you're up to it, I can send you an email when it's ready to try, likely tomorrow. The Dockerfile is mostly working as is, you just need to setup the DB, pass the connection info and do some manual superadmin rows insertion. I will automate the process asap.
- maneesh 5y agoI would love access! just tried to deploy and it threw me a docker authorization error.
- rubenfiszel 5y agoYou can always login with your github account for now: https://alpha.windmill.dev https://alpha.windmill.dev I am not doing anything fancy for building the dockerfile: https://github.com/windmill-labs/windmill/blob/main/.github/workflows/docker-image.yml#L31 https://github.com/windmill-labs/windmill/blob/main/.github/... Running the docker on the other hand require to pass a few env variables, among them the connection info to a PG instance. It also require to setup an user by editing directly the database and hashing by hand your password which is super inconvenient. I am working right now on making the process easier, did not expect self-host users so early.
- Aeolun 5y agoIf your stuff is marketed as OSS and you get on HN, the first question would definitely be “where’s the docker file/image”. At least, that’s always my first reaction when seeing something cool like this.
- wmichelin 5y ago`import *` in the screenshot. just poking fun, I understand that by no means should you wait for perfection before shipping. Congrats on getting to the front page!
- rubenfiszel 5y agoActually I wanted to point out that there is a pyright LSP server interacting with Monaco that would highlight those kinds of error. It is not that unintentional!
- ohgodplsno 5y ago>UI is autogenerated from your scripts.Gain velocity, reduce tech debt. "Reduce tech debt by using this tool that might or might not be maintained in a few years to build a critical part of your business workflow" is a bold take I didn't think I'd see this year, but here we are.
- rubenfiszel 5y agoAgreed that is why it is primordial for this kind of core tools to: - make it OSS - build a community around it that can continue without the startup being around if needed I have done the first, the second is my aim.
- DaltonCoffee 5y agoI'm installing this asap tmrw, along with a couple others mentioned in this thread. Tools I didn't know I needed to know ;) Thanks so much for releasing this useful and time saving software, you da man! Can't wait to see where this goes and I'd like to be part of the community.
- abeppu 5y agoGiven the name, and premise of creating flows of python scripts ... is this intended to make people think of Airflow? A lot of what we see in the demo video (on a scheduled basis run a sequence of steps which gather data, produce some summary statistics, share it to slack, parameters which can be defined when a run is triggered or from prior steps, etc etc) is very familiar to users of existing workflow systems. The UI to create/author flows seems slick, but I also think that for important flows, teams will want to be able to review changes to flows, make cross-cutting updates to several flows when a popular script or connector is updated, etc, in which case being able to view/edit/diff flows as code is quite valuable.
- rubenfiszel 5y agoDemo is a bit old (2 months ago), I need to reshoot it. It has some resemblances with Airflow but one aim is to make it accessible to less technical users that can treat modules like blackbox. Similar to Zapier but the code is inspectable and editable at any point. At the core, it is a scheduler/orchestrator of script execution, so it's similar to Airflow in that sense, but the product is really not meant for the same usage. I picked the name a bit randomly because it is retro and one of the first automation humans have created. I did not think too much about it but now I love it :)
- gigatexal 5y agoPricing info?
- rubenfiszel 5y agoIt's OSS so free if you self-host, otherwise I have a pricing plan on the landing page but it I focused so far only on delivering a beta and not on billing. Hence, right now there is no quota on anything and it is all free for the hosted version. I will be delighted if it provides value to you, get adoption, and I will figure out how to monetize later.
- acgourley 5y agoI'd use it for basic process automation, airtable data IO, triggering marketing campaigns or slack bots, etc. It's a heavy weight feature request but if I could lean on it in the way I lean on Prefect.io now for spinning up cloud workers on demand I'd use it for a lot more.
- olah_1 5y agoI believe that Anvil is a competitor: https://anvil.works/ https://anvil.works/
- tomatowurst 5y agovery cool, did not know there was a market for python web products. constantly being gaslighted by frontend/backend.js crowd.
- debarshri 5y agoThis looks very similar to airplane.dev, is that intended?
- raviparikh 5y agoAgreed, it does look very similar to what we're building at https://www.airplane.dev/ https://www.airplane.dev/ However I've come across a few other similar products as well over the past year we've been building Airplane, and frequently things appear superficially similar but there's a difference in focus/use case/target customer/etc. For example, Airplane is focused on support, customer success or ops use cases (e.g. "I wrote a script to delete a user record / reset a 2fa token / etc, I can use Airplane to let the customer support team have access in a safe way"). Whereas I've seen other tools in this space target more of a "programmable Zapier" type use case (e.g. "For every new customer, I want to use a Python script to hit the Slack API and generate a custom notification.") It'll be interesting to see how Windmill evolves over time.
- debarshri 5y agoThanks for the clarification! Also, recently I discovered retool workflows [1], something I think people would use ops, customer success flows too. In general, automation like these solves relevant problem in an org. [1] https://docs.retool.com/docs/building-a-workflow-with-the-retool-wizard https://docs.retool.com/docs/building-a-workflow-with-the-re...
- metadat 5y agoIs airplane.dev fully OSS and self-hostable?
- Aeolun 5y agoHmm, I can totally see where this is a great idea. When that docker file/image is up I’ll give it a spin.
- philsnow 5y agoI twitched a little when I saw lists and dicts included as default arguments in one of the examples: def main( ..., obj: dict = {"even": "dicts"}, my_list: list = ["or", "lists!"], ... has modern python made these less footgun-y? the issue being that the code looks like 'obj' and 'my_list' are as provided in the default arguments every time the function is called, but the defaults are instantiated once and they can be modified inside the function: def foo(msgs=["hello"]): print(msgs) msgs.append("world") foo() # prints ['hello'] foo() # prints ['hello', 'world'] foo() # prints ['hello', 'world', 'world'] Nonetheless, introspecting on default arguments like that looks like a fine way to turn functions into UI elements if you're going to contrive that those functions aren't going to be re-entered multiple times in the same python runtime.
- animal_spirits 5y agoNope, this is still a "feature" of python and will continue to be
- Aaronstotle 5y agoThis is amazing, at one of my previous employers, a colleague had started learning python and was able to automate a lot of the tedious work with python. Something like this would have been perfect.
- tegiddrone 5y agoStill trying to get my mind around it... I can make python scripts with infinite potential but to invoke them I have to put them behind a CLI, webhook, cron, and then orchestrate the data into these python scripts. So this would be like the orchestration glue? Write a script, have it conform to a schema and then Windmill can orchestrate it's execution and input/output... with a nice UI for manual invokation? Kick ass
- cyanydeez 5y agoHow's this compare to red-node? I love python and JavaScript and think there's a whole world of automation focused jupyternotebook like tasks
- ellyagg 5y agoThis is brilliant. The world has been needing something like Zapier, but not opaque and locked down. Everything Zapier does is a good idea, but also so easy and the whole model is way more powerful if it's open and extensible.
- edmcnulty101 5y agoThis looks really cool. Is there error handling? For example, from the github example, if the email sending fails for some reason can I either select the flow to continue/log or to stop entirely? Can flows branch? For example after sending an email, can two flows be triggered like a tree and two flows off of each one of those ad infinitum (ad no resourcium)?
- techplex 5y agoSeems like my email in the format word.other@gmail.com doesn't pass some check. I'm getting this error: Sql error: error returned from database: new row for relation "workspace_invite" violates check constraint "proper_email"
- mvolfik 5y agoWhat I couldn't find in the docs is the following: is it only restricted to an entry-form > some script runs > notifies via service (slack?), or is it also possible to do something like airplane workbooks: a form, then a Python script runs an SQL query, then another form pre-populated with some data from the query, and then another script?