tetron

8 ms·

31.
by tetron 8y ago
The codebase at my job is usually complex - Python, Ruby and Go, along with JavaScript/Typescript and a little bit of Java and R. I used to be in the "all programming languages are basically the same" camp but jumping back a
32.
by tetron 8y ago
Curoverse (a Veritas Genetics company) | Boston, MA or REMOTE | https://curoverse.com/about Veritas Genetics is at the cutting edge of direct to consumer whole genome sequencing. Come work on the open source Arvados ( htt
33.
by tetron 8y ago
Curoverse (a Veritas Genetics company) | Boston, MA and remote Work on the open source Arvados ( http://arvados.org ) platform for storage & analysis of biomedical big data, used by major biotech & pharma companies. Small
34.
by tetron 8y ago
Schemas and validation for YAML (and JSON): http://github.com/common-workflow-language/schema_salad
35.
Common Workflow Language Joins Software Freedom Conservancy(sfconservancy.org)
4 points by tetron 8y ago|0 comments
36.
by tetron 9y ago
Or maybe certain professions have a limited number of openings, which tend to go to people with connections, and cooincidentally these professions deal with large amounts of money, so taking 1% commission on a hundred million dollar deal fo
37.
by tetron 9y ago
Agree that Microsoft would be a plausible buyer. They have deep pockets, have hired ex-Docker engineers such as Jess Frazelle on the Azure team, and are investing heavily in the capability to run Linux-based Docker containers seamlessly on
38.
by tetron 9y ago
Some of these responses speak to the difference between "programming" and "software engineering". If you want people to hack code without regard for what it is for, you get programmers. If you want a team that is fully
39.
A portable workflow language for data analysis pipelines(commonwl.org)
3 points by tetron 10y ago|0 comments
40.
by tetron 10y ago
Incidentally, Arvados ( http://arvados.org ) is the software used to host the Harvard PGP data, and is a free software platform for managing large scale storage and analysis aimed at scientific workloads.
41.
by tetron 10y ago
Blender is very much the Emacs equivalent for 3D editing software. It had a kernel of functionality written in C but the majority of the functionality is actually written in Python, which means there is really robust support for extensions
42.
by tetron 10y ago
I haven't used the new Azure cli, but a major issue with the node.js based cli was that it was heavily oriented towards interactive use and not well suited for use by scripts. Hard to parse output, prompt for parameters instead of acc
43.
by tetron 10y ago
Common Workflow Language http://commonwl.org is a spec (with multiple implementations) for wrapping command line tools (which may run inside a Docker container) as functional units.
44.
by tetron 10y ago
Curoverse | Boston, MA | on site or remote | full time Open source startup developing the GPL Arvados ( http://arvados.org ) ( https://github.com/curoverse/arvados ) platform for massive scale storage and proce
45.
by tetron 10y ago
http://github.com/common-workflow-language/schema_salad is another schema language for JSON & YAML based on Apache Avro.
46.
by tetron 10y ago
Try https://github.com/rabix/cottontail
47.
by tetron 10y ago
After Arvados ( http://arvados.org ) Toil has the most complete support for the common workflow language http://commonwl.org ) which is an emerging standard for writing portable workflows that can run on different platf
48.
by tetron 10y ago
(Toil contributor here) Workflow systems are generally used to orchestrate parallel batch jobs on a cluster, which may have hundreds or thousands of individual tasks. So the emphasis is on defining what units of work are independent that c
49.
by tetron 11y ago
Yes, that's right. I would also add data provenance, eg where did a data set come from and how it was computed. This flows directly from the features I mentioned of content addressed storage and recording compute job history.
50.
by tetron 11y ago
There's several features that make Arvados unique. * The content addressed storage system references hashes all data so you can unambiguously reference an immutable data set, similar to git but capable of handling huge invidual files (
51.
by tetron 11y ago
Hello all, Arvados developer here to answer your questions.
52.
Show HN: Arvados, a free software storage and compute platform for big data(github.com)
6 points by tetron 11y ago|5 comments
53.
by tetron 11y ago
Going one step further, using http://CommonWL.org you can wrap Dockerized command line tools into callable Python functions, and abstracting all the details of stdin/stdout redirection and getting files into and out of the
54.
by tetron 11y ago
Basically, designers have to put up with the same management bullshit as developers, except with significantly less respect for the difficulty of their craft.
55.
Common Workflow Language(commonwl.org)
9 points by tetron 11y ago|0 comments
56.
by tetron 11y ago
This is basically the architecture of Arvados Crunch: http://arvados.org
57.
by tetron 11y ago
Buying those 600,000 drives is probably only 10% of your costs, you need to run a data center if you want to actually use that data. Also analysis pipelines generate intermediate results that are also large, and data is stored with 2x or 3x
58.
by tetron 11y ago
Someone has, it is called dtrx ("do the right thing extractor"): http://brettcsmith.org/2007/dtrx/
59.
by tetron 11y ago
The Arvados project https://arvados.org/ is an open source scale-out storage and compute platform designed to address the needs of huge data, like genomics.
60.
by tetron 11y ago
The Common Workflow Language https://common-workflow-language.github.io/ is an open source effort to create a common way of defining portable, reproducible computational workflows to address the needs of the science communi

More ›