bluestreak

13 ms·

91.
by bluestreak 6y ago
This is running on a single AWS c5.metal instance. It has 196GB RAM, but we are using only 40GB at peak. The instance provides two 24-core CPUs, but we are only using one of them on 23-threads. So far, it's handling the flow of connect
92.
by bluestreak 6y ago
We will find a way to fix this and have grid scroll without problems at the same time. This is not a dedicated website. This is exactly the same web console that is packages with our free product. It wasn't meant to run on the internet
93.
by bluestreak 6y ago
while HN load is high, this is just because poor implementation on our side. Filters are single threaded, not yet vectorized in any way. This will be fixed eventually to bring in line with queries that we've already optimised
94.
by bluestreak 6y ago
Ordering would be memory intensive. There are configurable artificial limits in place on this demo instance. This is just because database takes internet traffic directly. In any other settings these limits can be relaxed, which is default
95.
by bluestreak 6y ago
someone already brought this up, there is rationale for it at bottom of this thread
96.
by bluestreak 6y ago
Thank you! there is no 'GROUP BY' keyword support. Optimiser works out group by queries without it. We are planning to put a noop there, just to be nice to finger memoory. EXPLAIN would be quite boring in our case, we use a myriad
97.
by bluestreak 6y ago
All good so far! Thank you for the feedback. Filter's are single threaded and not very fast. We will take them to the next level shortly
98.
by bluestreak 6y ago
the reason was grid scroll using touchpad on OSX, when you swipe the grid from left to right there is 50% chance of losing web console due to back button being triggered by the same motion.
99.
by bluestreak 6y ago
The data layout is indeed the same for both cases. We don't assemble row as such. In QuestDB row is a collection of data addresses, each of which is calculated lazily using row_id and bitshift
100.
by bluestreak 6y ago
this is accurate, thank you. You can try asof join: SELECT pickup_datetime, cab_type, trip_type, tempF, skyCover, windSpeed FROM trips ASOF JOIN weather; This joins 1.6B to 130K roughly. Generally we provide both row and column based access
101.
by bluestreak 6y ago
Thanks, not all queries are optimal yet. We are going to get there.
102.
by bluestreak 6y ago
Our startup sequence is sub-optimal. We query JDK to index timezone database and JDK generates a tonne of garbage in the process. The whole feat requires roughly 500MB RAM. Once out of startup sequence all this is collected and we can opera
103.
by bluestreak 6y ago
Athena has been benchmaked here with similar dataset (1.1B vs our 1.6Bn): https://tech.marksblogg.com/benchmarks.html
104.
by bluestreak 6y ago
Author here. A few weeks ago, we wrote about how we implemented SIMD instructions to aggregate a billion rows in milliseconds [1] thanks in great part to Agner Fog’s VCL library [2]. Although the initial scope was limited to table-wide aggr
105.
Show HN: Query 1.6B rows in milliseconds, live(try.questdb.io)
271 points by bluestreak 6y ago|93 comments
106.
by bluestreak 6y ago
It is possible, we've done it.
107.
by bluestreak 6y ago
I have not worked on this project, so I don't have details. Mongo is in a nutshell to replace a database trader setup under their desk and that grew quite important. There is nothing pretty about that. I'm quite sure they will hav
108.
by bluestreak 6y ago
This was in London HSBC HQ, Equities
109.
by bluestreak 6y ago
I was there in Aug 2019 and I did hear them talking about mongo, There was a programme at the time to "refactor" their estate, a part of that they wanted to replace bunch of ad-hoc databases with clunky- or no data governance with
110.
by bluestreak 6y ago
We take data integrity very seriously. This is one of the reasons QuestDB actually exists. We haven't had reports of data corruption or loss yet. We can commit durably if needed too. To manipulate data we support "add" and &q
111.
by bluestreak 6y ago
Doesn’t redirect me :/
112.
IoT on QuestDB(davidgs.com)
28 points by bluestreak 6y ago|22 comments
113.
by bluestreak 6y ago
Thank you Radford! This is a long piece and it would take me a while to absorb properly. Does your algorithm lend itself to vectorised implementation?
114.
by bluestreak 6y ago
Thank you for taking time to elaborate on the subject. This is both informative and enjoyable to read! We knew that all real numbers cannot be represented using IEEE-754. It wasn’t all clear though because toString() appears to be perfectly
115.
by bluestreak 6y ago
Thank you! QuestDB is in production. Scaling is a part of commercial product, built using QuestDB.
116.
by bluestreak 6y ago
Author here. About a month ago, I posted about using SIMD instructions to make aggregation calculations faster. I am very thankful for the feedback so far, this post is the result of the comments we received last time. Many comments suggest
117.
Things we learned about sums(questdb.io)
43 points by bluestreak 6y ago|9 comments
118.
QuestDB 4.2.1 Release. SIMD performance gain 20%. Clickhouse benchmark(github.com)
6 points by bluestreak 6y ago|0 comments
119.
by bluestreak 7y ago
This is because OracleJDK has more intrinsics than OpenJDK.
120.
by bluestreak 7y ago
sorry, i think we tested with this one: KDB+ 4.0 2020.03.17 Copyright (C) 1993-2020 Kx Systems l64/ 12(16)core 63960MB EXPIRE 2021.03.26

More ›