felipe_aramburu

8 ms·

31.
by felipe_aramburu 8y ago
And as we state in the post. This does not include ML times since we wanted to show how much faster we were making ETL. The ML timing differences are even starker than ETL.
32.
by felipe_aramburu 8y ago
Did you see the steps being performed in the workflow? How would you perform these steps on hadoop and what kind of times would you expect for the different steps in the workflow?
33.
by felipe_aramburu 8y ago
An etl job could take an infinite amount of time. I am not trying to be cute by saying this but the point is that some ETL jobs might be as simple as loading data from a file others could involve millions of transformations. There is no lim
34.
by felipe_aramburu 8y ago
1. You don't have to fit your whole workload on the GPU you can process it in batches like you would for a workload that doesn't fit into memory on a non gpu solution. You don't need PB of GPU memory to run PB workloads. 2. T
35.
by felipe_aramburu 8y ago
Do you have an example of workloads that are easy to reproduce using clickhouse so we can see how much effort it would be to make a comparison? We are always happy to test against other tools!
36.
by felipe_aramburu 8y ago
The GPU workload is used in feeding XGBOOST to perform classification. We are omitting this timing on purpose because the training time on purpose becuase SPARK took 1000's of times longer for this part of the workflow. You can see the
37.
by felipe_aramburu 8y ago
As for size we will be releasing much larger workloads for GTC and are working hard on finishing up our work to make the workloads public and reproducible as well as releasing a new version of our engine where you can run much much larger w
38.
by felipe_aramburu 8y ago
1. It really isn't the kind of problem we are trying to solve. 2. Last time we did this we were over 100x faster and so we said. Ok its slow as &#@%. We are more interested in trying to compare to tools that can provide workflows w
39.
by felipe_aramburu 8y ago
Most of the execution kernels that we use are in https://github.com/rapidsai/cudf . The engine itself is not open source though it is free to use and we have a dockerhub link which is pretty convenient. https:/&#
40.
by felipe_aramburu 8y ago
Uncoordinated distributed is coming soon. We have been building this tool with distribution in mind and right now are finishing up our first distributed version of BlazingSQL which we will be releasing just before GTC ~March 15th. We have
41.
by felipe_aramburu 8y ago
You will notice that the second size does NOT fit into GPU ram on this benchmark since we ran it on a GPU that has less memory available than the DATA it is running on. Spark is feeble, perhaps but it is one of the tools being used to feed
42.
BlazingSQL – GPU SQL Engine Now Runs Over 20X Faster Than Apache Spark(blog.blazingdb.com)
192 points by felipe_aramburu 8y ago|76 comments
43.
Data Lake to GPUs – BlazingSQL and RAPIDS Initial Benchmark(blog.blazingdb.com)
6 points by felipe_aramburu 8y ago|0 comments
44.
by felipe_aramburu 8y ago
If you mean something like network card ==> device via RDMA we are not doing that yet but are currently working on this implementation. These past few months we have focused more on adding functionality to libgdf. We are now optimizing
45.
by felipe_aramburu 8y ago
I am going to look into this. Its a fantastic question but we are too ignorant on the matter to be able to comment properly at the time. I will respond here, eventually :).
46.
by felipe_aramburu 8y ago
Hopefull the library that the user is employing! In our case we have a few different components that actually make up BlazingSQL. We have Relational Algebra nodes that are stateless and can do nothing but receive inputs and interpret relati
47.
by felipe_aramburu 8y ago
Yah we are all solving different parts of this ENORMOUS puzzle. Our focus has always been on making reducing the pains of ETLing data into the GPUs from all of the crazy sources they can come from.
48.
by felipe_aramburu 8y ago
Also to be clear as of right now the only thing that has to fit in memory are the inputs and the outputs to this system since these are being passed via CUDA IPC.
49.
by felipe_aramburu 8y ago
You can reach out to me (my first name at blazingdb) if you want to talk about this. In a nutshell users interact with blazing through the python API for the most part. If you have small result sets like those that normally go into a dashbo
50.
by felipe_aramburu 8y ago
The data is in the data lake. You need to get it. Either onto the CPU or the GPU and guess what the bottlekneck there will rarely be your interconnect between the CPU and the GPU but rather your interconnect with the data lake. You say triv
51.
BlazingSQL – A GPU SQL Engine for RAPIDS Open-Source Software from Nvidia(blog.blazingdb.com)
125 points by felipe_aramburu 8y ago|35 comments
52.
BlazingDB 2.0 – GPU Fast SQL on Apache Parquet(blog.blazingdb.com)
2 points by felipe_aramburu 9y ago|0 comments
53.
by felipe_aramburu 9y ago
Right now we support RLE, RLE Delta RLE, Delta RLE, Dictionary, Bitpacking. Many of these are combined together. It does look interesting I am checking it out.
54.
by felipe_aramburu 9y ago
Thanks bud. Looks like the GOAI people are making some moves! We are also happy to be sharing in that experience with you all.
55.
by felipe_aramburu 9y ago
Sure the very first time you run a query. But with multi tiered caching the data you frequently access sits closer and closer to the gpus so that alleviates that bottlekneck over time to an extent. Also what is a fantastic way of improving
56.
by felipe_aramburu 9y ago
Ok I see your point there. It does feel like that is quite a bit to manage. Our approach would be to perhaps try and show you how it could be used in these different kinds of workloads but would never try to undertake them ourselves. I&#x27
57.
by felipe_aramburu 9y ago
I just glanced at about 15 of them and did not notice anything we did not support. There seems to be alot of nesting and whatnot but nothing here really jumps out at me. I have not looked at the full set yet.
58.
by felipe_aramburu 9y ago
So depending on your sources, e.g. if they are compressed or not, the GPU can greatly speed upi/o by compressing and decompressing directly on the GPU. This is particularly meaningful when you can store it in compressed state when you
59.
by felipe_aramburu 9y ago
Good Point. I tried to convince someone very recently to go this route so I could piggy back and test on it as well :).
60.
by felipe_aramburu 9y ago
People in the GPU space and slow storage systems like hadoop and S3) To caveat, I saw slow compared to something like storing your queryable data in system memory. The answer is that your clients are not normally GPU people. They are aware

More ›