PolarizedPoutin

8 ms·

1. 31.▲
   by PolarizedPoutin 3y ago

   Thank you! Yeah haha some of the benchmarks took several hours (and a few re-runs) and there was a lot of learning done along the way.
2. 32.▲
   by PolarizedPoutin 3y ago

   1. Yeah to me it seems very hard to saturate I/O with Postgres unless maybe you insert into an unlogged table. I guess there's quite a bit of overhead to get all the nice stability/consistency and crash-resistance. 2. That is
3. 33.▲
   by PolarizedPoutin 3y ago

   Haha very cool use! Yeah reading up on TimescaleDB vs. Clickhouse it seems like columnar storage and Clickhouse will be faster and better compress the time series data. For now I'm sticking to TimescaleDB to learn Postgres and PostGIS,
4. 34.▲
   by PolarizedPoutin 3y ago

   I have not analyzed any data yet and the purpose of plotting the time series was to show an example of the data as a function of time. As others have already mentioned, the swing in Durban temperatures over the seasonal cycle is ~25°C while
5. 35.▲
   by PolarizedPoutin 3y ago

   Thanks for the insight! It is true that I started with a micro-benchmark of 20k rows for slower inserts, but I also did some longer benchmarks with ~772 million rows.
6. 36.▲
   by PolarizedPoutin 3y ago

   Yeah I'm curious about this too. Been meaning to ask on the Timescale forums. My only guess is that there's some small extra overhead due to hypertable chunking. I know Timescale has a blog post from 2017 claiming a 20x higher ins
7. 37.▲
   Loading a trillion rows of weather data into TimescaleDB(aliramadhan.me)

   405 points by PolarizedPoutin 3y ago|177 comments
8. 38.▲
   by PolarizedPoutin 5y ago

   Oh that's a neat idea. I should check out Advent of Code. Summing would probably work for a lot of problems but some problems mix strings with numbers in the output
9. 39.▲
   by PolarizedPoutin 5y ago

   I'm seeing a "404 Not Found" when I visit (after logging into NextJournal).
10. 40.▲
    by PolarizedPoutin 5y ago

    Very cool! Didn't know there was a classical mechanics counterpart to SICM. Could inspire some new problems for the website.
11. 41.▲
    by PolarizedPoutin 5y ago

    Thanks for the suggestion! I'm all for quality of life improvements. Definitely wouldn't be hard to add I think.
12. 42.▲
    by PolarizedPoutin 5y ago

    The code gets uploaded to the server which sends it to an "engine" sitting in a Docker container. The engine generates test cases and send them with the code to another Docker container where the code is run. Then the output from
13. 43.▲
    by PolarizedPoutin 5y ago

    Nice to see another Julia user! Yup right now we only have 27 problems but there will hopefully be many more in the future. Some people might even contribute some new problems.
14. 44.▲
    by PolarizedPoutin 5y ago

    Thanks for the suggestion. Thread is too deep to reply to your actual message, but I will look around to see how other "online judge" software run arbitrary code securely. There's probably some low hanging fruit in configurin
15. 45.▲
    by PolarizedPoutin 5y ago

    I'm open to suggestions and advice if you have any. This website is a free open-source side project and not business critical.
16. 46.▲
    by PolarizedPoutin 5y ago

    That sounds awesome! Definitely let us know if we can help.
17. 47.▲
    by PolarizedPoutin 5y ago

    Thank you and sorry about the slow load time. We never expected this much traffic and everything is hosted on a tiny DigitalOcean server including the code runner haha. If you visit a bit later it should be much faster. I can think of some
18. 48.▲
    by PolarizedPoutin 6y ago

    I agree 100%. It seems that almost any real-world science problem these days requires a computational solution/approach at some level. Taking a computational approach in parallel would be super useful.
19. 49.▲
    by PolarizedPoutin 6y ago

    Yeah the other two replies explain why we used code stubs that would at least run but not pass. Don't think it's an idiom (at least not one that I've seen). It's definitely not the nicest code, especially for simpler pro
20. 50.▲
    by PolarizedPoutin 6y ago

    Haha thanks! Didn't know about codewars.com, will look into it. Might be a cool way to learn Rust syntax.
21. 51.▲
    by PolarizedPoutin 6y ago

    Java is definitely on the list of languages to support soon! I'm also thinking of how we can add a "manual submission mode" that lets you use any language that you run locally.
22. 52.▲
    by PolarizedPoutin 6y ago

    Yeah haha they're definitely simple if you have almost any education in math or science. I guess we started simple so that the barrier to entry is low, but we should have some tougher and more interesting problems soon!
23. 53.▲
    by PolarizedPoutin 6y ago

    I'm excited to learn Rust actually so we might support it soon! It's not trivial to add support new languages since the engine app that runs user-submitted code needs to be able to run arbitrary code in your chosen language and ne
24. 54.▲
    by PolarizedPoutin 6y ago

    There should be a limit on how long the Docker container can run code for, but it might be unnecessarily long right now. I'm not a web developer by trade or anything so I'll have to learn how to secure the Docker container from ma
25. 55.▲
    by PolarizedPoutin 6y ago

    Thank you so much for taking a look and letting us know! Definitely not surprised that there are many security vulnerabilities... I'm a grad student in computational science working on this as a hobby so building and securing websites
26. 56.▲
    by PolarizedPoutin 6y ago

    Right yeah it's a good question. I think the reason we did it is so that we could visualize user output/solutions. You can learn new things by looking at your solutions to various test cases! One example would be if you submit a s
27. 57.▲
    by PolarizedPoutin 6y ago

    That is true. It's not super easy to add new languages and I'm not familiar with Fortran (thankfully our group switched from Fortran to Julia recently!) but we already support C since it's easy to call C from Python. Perhaps
28. 58.▲
    by PolarizedPoutin 6y ago

    Thanks for checking it out! I don't use Go so might take a while unless someone decides to swoop in and add support for it! Supporting new languages takes some effort since we need to be able to run arbitrary Go code and communicate wi
29. 59.▲
    by PolarizedPoutin 6y ago

    Thanks for checking it out! Will look into the LaTeX rendering issue. Yeah maybe we went a little too easy on the earlier problems but there will be plenty of harder problems coming up hopefully! I actually think Project Euler gets hard pre
30. 60.▲
    by PolarizedPoutin 6y ago

    Yeah haha, this is somewhat intentional! We thought it would be tedious to write tons of code to make sure users don't take the easy way out for each problem and language, so we figured we might as well allow it. We can't force us