tomnicholas1

7 ms·

31.
by tomnicholas1 2y ago
Reporter imagines how we'd cover overseas what's happening to the U.S. right now. This is by far the most clear-eyed description I've seen of the severity of what is happening. Relevant to HackerNews partly because it include
32.
by tomnicholas1 2y ago
I was also going to say this looks similar to one layer of dask - dask takes arbitrary python code and uses cloudpickle to serialise it in order to propagate dependencies to workers, this seems to be an equivalent layer for rust.
33.
by tomnicholas1 2y ago
Is there a world in which GitHub used an open protocol for the social network part of their product like BlueSky's AT protocol[0]? [0] https://docs.bsky.app/docs/advanced-guides/atproto
34.
by tomnicholas1 2y ago
In python there is lithops, which provides nice Executor primitives that can run on a wide range of cloud services (AWS lambda, GCF etc.) https://github.com/lithops-cloud/lithops
35.
by tomnicholas1 2y ago
The python package Hypothesis[0] already does a great job bringing property-based testing to the people! I've used it and it's extremely powerful. [0]: https://github.com/HypothesisWorks/hypothesis
36.
by tomnicholas1 2y ago
Interesting - I'm curious whether you feel that Xarray covers these use cases already? https://xarray.dev/ Especially as I've said before that Hyperspy shares so many features in common with Xarray that Hyperspy s
37.
by tomnicholas1 2y ago
No not necessarily - it will keep growing hotter until the black body radiation emitted by the probe matches the power of the radiation hitting the probe. Then it will stay at constant temperature. It's a standard undergraduate problem
38.
by tomnicholas1 2y ago
That's pretty cool! Quite specific to this file format/workload, but this is an important enough problem that people might well be interested in a tailored solution like this :)
39.
by tomnicholas1 2y ago
> It is sort of analogous to what GribStream is doing already. The difference is presumably that you are doing some large rechunking operation on your server to hide from the user the fact that the data is actually in multiple files? Coo
40.
by tomnicholas1 2y ago
> Why is weather data stored in netcdf instead of tensors or sparse tensors? NetCDF is a "tensor", at least in the sense of being a self-describing multi-dimensional array format. The bigger problem is that it's not a Clou
41.
by tomnicholas1 2y ago
That's partly because the warming experienced over land can be ~50-100% larger than the globally-averaged warming, with the temperatures over the oceans increasing more slowly to make up the difference. https://www.carbonbri
42.
by tomnicholas1 2y ago
You should look at Icechunk. Your imaging data is structured (it's a multidimensional array), so it should be possible be to represent it as "Virtual Zarr". Then you could commit it to an Icechunk store. https://ea
43.
by tomnicholas1 2y ago
If you're wondering this you should look at Icechunk too, which was open-sourced just this week. It's Apache Iceberg but for multidimensional data (e.g. Zarr). https://earthmover.io/blog/icechunk https:&#x2F
44.
by tomnicholas1 2y ago
So the equivalent of these balloons in oceanography are called ARGO floats, which similarly cannot be driven laterally but can control their own depth like a submarine. So far millions of timeseries have been collected across the world ocea