whinvik

10 ms·

1. 121.▲
   by whinvik 2y ago

   As someone who deals with geospatial processes like this daily, I have 2 notes. 1. STAC implementations are already complicated. Not everyone has good catalogues, or that work uniformly well for all types of queries. 2. Using STAC geoparque
2. 122.▲
   by whinvik 2y ago

   Feels like the wrong move. Turning a chip company around has timescales of a decade. Getting rid of a CEO 3 years in simply means no turnaround is going to happen.
3. 123.▲
   by whinvik 2y ago

   As someone having inherited a Hashicorp Nomad setup I realize that while it is much simpler, no one really builds anything on top of Nomad. If I wanted a DAG, I can use Argo Worklows with k8s, nothing with Nomad. If I wanted to do lots of d
4. 124.▲
   by whinvik 2y ago

   I am not your target audience but I have been thinking of building a very minified version of this using [0] Pooch and [1] S3FS. Right now we spend a lot of time downloading various stuff from HTTP or S3 links and then figuring out folder s
5. 125.▲
   by whinvik 2y ago

   Thanks. That makes sense
6. 126.▲
   by whinvik 2y ago

   Curious. Why would someone want 1M buckets? What would be the usecase. And how would 1 manage that large amount of buckets. Create new dashboards?
7. 127.▲
   by whinvik 2y ago

   Struggling to compare it to k8s but maybe that's just my lack of knowledge on k8s. However, to the general HN crowd my question is what would be the correct abstraction for something that wants to replace k8s to make one think that thi
8. 128.▲
   by whinvik 2y ago

   Yes, I have generally tried to do that. But sometimes there are third party libraries that depend on GDAL and then there's no way around it.
9. 129.▲
   by whinvik 2y ago

   But can it do GDAL. (This is the Doom equivalent joke for Python environments!)
10. 130.▲
    by whinvik 2y ago

    > We're building multimodal search infrastructure, think of it like an index for your S3 bucket. Sounds interesting. Do you have any writeups on what that means?
11. 131.▲
    by whinvik 2y ago

    Highly recommend Cogram. I interviewed with them last year. While I missed out in the end, I really liked the overall experience and talking to people there.
12. 132.▲
    by whinvik 2y ago

    Can someone who understands it explain what dbt is and how it is used. I hear a lot about it but I just haven't figured out what it is useful for.
13. 133.▲
    by whinvik 2y ago

    I feel like both of them should crash the application. The only benefit is that you don't see the full stack trace in production, which depending on circumstances you don't want to be shown.
14. 134.▲
    by whinvik 2y ago

    Is this github only?
15. 135.▲
    by whinvik 2y ago

    Streamlit. Well maintained and easy for beginners.
16. 136.▲
    by whinvik 2y ago

    Thanks. I did not know dbx was deprecated and replaced. Wouldn't targeting smaller teams lead to a lot of pricing pressure? Or do you think there's enough volume to justify that?
17. 137.▲
    by whinvik 2y ago

    Feels a bit like what Databricks already does with Dbx etc.
18. 138.▲
    by whinvik 2y ago

    Is there any advantage to having both a Data Lake setup as well as Snowflake. Why would one also want Snowflake after doing such an extensive data lake setup?
19. 139.▲
    by whinvik 2y ago

    Orbio | Remote Sensing Data Scientist | Remote (European Time) | Full-time | https://www.orbio.earth Orbio uses satellite imagery to track methane emissions from the oil & gas industry. We sell the data to operators that wan
20. 140.▲
    by whinvik 2y ago

    Don't know how HN hivemind works but there's nothing in this question that should lead to downvotes.
21. 141.▲
    by whinvik 2y ago

    At least for Python, it's usually easy. Just build a Docker image and push from Github action after merge and it's deployed.
22. 142.▲
    by whinvik 2y ago

    Would love to see some of the talks. What is Pycon's policy on releasing talks to Youtube?
23. 143.▲
    by whinvik 2y ago

    What distribution would you recommend?
24. 144.▲
    by whinvik 2y ago

    Have been using Continue and am happy with it. https://docs.continue.dev/intro
25. 145.▲
    by whinvik 2y ago

    Sounds to me like there's a long blog post waiting to be written.
26. 146.▲
    by whinvik 2y ago

    I feel like this would just have been shrugworthy news if SpaceX didn't exist. Most NASA projects used to be like this. The problem for all the other companies is that SpaceX showed up.
27. 147.▲
    by whinvik 2y ago

    Are there Arrow libraries. Feel like a lot of applications that read Parquet actually outsource raw reads to Arrow.
28. 148.▲
    by whinvik 2y ago

    Parquet is a file format. Should a file format be in Debian?
29. 149.▲
    The Birth of Parquet(sympathetic.ink)

    145 points by whinvik 2y ago|79 comments
30. 150.▲
    by whinvik 2y ago

    Are you trying to replace SLURM?