8 ms·
Baidu Takes FPGA Approach to Accelerating SQL at Scale
- srigan 10y agoisn't this already implemented in datawarehouse appliances like Netezza (now part of IBM)?
- sixdimensional 10y agoMost likely I believe the answer is yes. But since these appliances have been proprietary, not much is publicly known about what they do internally. I suspect that some design like the one we are seeing here at Baidu has been their secret sauce for some time.
- miguelbar 10y agoIt's no secret. Netezza has been using FPGAs since the beginning. Here is a link to a paper that describes the Netezza approach: http://www.redbooks.ibm.com/redpapers/pdfs/redp4725.pdf http://www.redbooks.ibm.com/redpapers/pdfs/redp4725.pdf The Baidu approach seems very similar.
- srigan 10y agoisn't this already implemented in Datawarehouse appliances like Netezza (now part of IBM), which targeted solving the problem of analytical queries on huge sets of data??
- liprais 10y agoI guess Baidu hired some former Netezza guys and they decided to copy/steal from their former company to meet with their KPIs,which is quite common here in China
- weq 10y agoAhh yes, my old boss was next door neighbours with the Netezza CEO back in the early days circa '06. I worked a little with the platform back then, and i saw this article and was like, thats they they did! Back then, u had to shard ur database to take advantage of the parrallel processing, not sure how far its come now.
- CodeSheikh 10y agoOnline stock brokers and finance companies have been using FPGAs for quite some time now because of latency as a microsecond can literally cost them millions. It is good to see these beasts making their way into crunching "Big Data" in non-finance domain.
- tmostak 10y agoVery cool work but since its easy to get memory-bandwidth bound with SQL I would wonder if it would make more sense to use GPUs with 10X the bandwidth. I know FPGAs will be getting HBM as well so this might help. Disclaimer: I work at MapD, a GPU database company. (http://www.mapd.com http://www.mapd.com)
- tkyjonathan 10y agoFinally! In 2011, when I was working on optimizing 100Gb databases on dedicated hardware for small businesses, everyone in the database world was talking SQL chips and PCI-E fast storage. Fast forward to today when cloud and virtualisation and now I'm working on "big data" 400Mb RDS instances. Its time we go "retro" and #gophysical for some things.
- discodave 10y agoIf anybody has the resources to start building dedicated FPGA (or whatever) instances for SQL workloads it would be one of the public cloud vendors (AWS, Azure, GCP).
- tkyjonathan 10y agoGood! They will have to set aside servers that do just databases with SQL chips, but either way, I would be happy if they do that. I would also be interested to see if there is a new trend to go back to owning commodity hardware, putting CoreOS+docker on it + a database with a hopefully inexpensive SQL chip. Moving away from VMware, massively expensive servers, massively expensive SANs, proprietary backups.. Would be very interesting to see.
- koolba 10y agoThis looks pretty cool. Any idea of which database they're doing this on? I don't see it listed out in the article anywhere. I would guess it's a FOSS database (i.e. either Postgres or MySQL) but also wouldn't be surprised if at their scale they've created something entirely in house.
- turingbook 10y agoMySQL is dominant in China Internet companies.
- Diederich 10y agoThat's pretty interesting. Do you have a source for that?
- stevesun21 10y agoNOTE: most of these website are in Chinese Taobao http://mysql.taobao.org/index.php?title=首页 http://mysql.taobao.org/index.php?title=首页 360buy http://www.infoq.com/cn/articles/exploration-of-distributed-mysql-cluster-scheme http://www.infoq.com/cn/articles/exploration-of-distributed-... Tencent http://tencentdba.com http://tencentdba.com DiDi http://os.51cto.com/art/201604/508514.htm http://os.51cto.com/art/201604/508514.htm
- snaky 10y ago"Tencent is using PostgreSQL for data mart in their big data system" - https://pgconf.ru/en/2016/89691 https://pgconf.ru/en/2016/89691
- Scarbutt 10y agoI would say mysql is dominant in general.
- snaky 10y ago> Postgres is getting its momentum in China, especially in 2015, one of the biggest insurance company is adopting Postgres, and Alibaba is providing Postgres service in their public cloud, also there are a lot of significant progress about the adoption. This talk will give an overview about the Postgres adoption in 2015 in China https://pgconf.ru/en/2016/89691 https://pgconf.ru/en/2016/89691 > Alibaba has provided a relational database service (RDS) for postgres in our public cloud platform (aliyun.com, the currently biggest public cloud in China). We are also enabling internal applications to use postgres in our other internet business and we can share our experience https://pgconf.ru/en/2016/89715 https://pgconf.ru/en/2016/89715
- kiril-me 10y agoThey are crazy! Very interesting case. How they support such DB? As I know FPGA has limits on updates. What tools they use to create relational algebra?
- blackguardx 10y agoMost FPGAs (all except for a dozen or so) use SRAM to hold the configuration while in operation. SRAM has no limit to how many times it can change states. Off-chip flash typically stores the configuration for long term needs and the FPGA loads the configuration from that at power-up. Applications that require frequent changing of the configuration use processors to load from somewhere else instead (hdd, network, some other storage meachanism.)
- duskwuff 10y ago> As I know FPGA has limits on updates. I'm not sure what you're trying to refer to here. Are you confusing FPGAs with flash memory?
- teemwerk 10y agoThere are FPGAs (mostly Microsemi) that use internal flash memory to hold their configuration, but you are correct, most use eeproms/
- MaDeuce 10y agoProbably not. A hardware description language (HDL) is used to program the behavior of the FPGA (that's the FP in FPGA). The HDL is usually stored in flash or EEPROM. As you mention, flash supports a finite number of writes; I'm not sure about EEPROM behavior. The bottom line is that the typical FPGA can only be programmed a fixed number of times (e.g., I think Altera is ~100). Some FPGAs can only be programmed once.
- theyoungestgun 10y agoMany FPGAs these days have flash memories that are good for ~10K writes.
- malkia 10y agoNaive question, but wouldn't this work only with ECC-enabled graphics cards like NVidia Quadros? Possibly that's the only reasonable choice for servers, but the price is much higher.
- astrodust 10y agoThey're using a Xilinx FPGA card, not a GPU. These are two totally different things, though the FGPA card is usually a lot more expensive than any GPU on the market.
- smilliken 10y agoFor GPUs, not FPGAs, but PostgreSQL has PGStrom[1] for offloading large table scans. Still a work in progress. [1] https://wiki.postgresql.org/wiki/PGStrom https://wiki.postgresql.org/wiki/PGStrom
- ksec 10y agoCUDA Only. Is PGStrom in Active development? Or is it more of an proof of concept stage?
- snaky 10y agoLooks active at github, https://github.com/pg-strom/devel https://github.com/pg-strom/devel
- snaky 10y agoAnd machine learning in on the way - https://postgrespro.com/roadmap/56513 https://postgrespro.com/roadmap/56513
- astrodust 10y agoIt's no wonder Intel made a big FPGA acquisition lately. This sort of acceleration has enormous upside potential, especially if this sort of thing comes part and parcel with your CPU.
- snaky 10y agoIntel is not alone - http://openpowerfoundation.org/blogs/fpga-acceleration-in-a-power8-cloud/ http://openpowerfoundation.org/blogs/fpga-acceleration-in-a-... > In addition to the CAPI 2.0 coherent links running atop PCI-Express 4.0, there is a further enhanced CAPI protocol that runs atop the 25 Gb/sec Bluelink ports that is much more streamlined and we think is akin to something like NVM-Express for flash running over PCI-Express in that it eliminates a lot of protocol overhead from the PCI-Express bus http://www.nextplatform.com/2016/08/24/big-blue-aims-sky-power9/ http://www.nextplatform.com/2016/08/24/big-blue-aims-sky-pow...
- sixdimensional 10y agoThis is kind of amazing. I have often really wondered what would happen if you basically created a directed acyclic graph/dataflow for data processing (for example, how Apache Spark distributes processing/operations), and then accelerated the operations using physical implementations in FPGAs. After all, a SQL query, when optimized, is essentially a graph of operators that data flows through. You do have to pass your data through the accelerator to get the processing... which potentially means huge volumes of data moving into this physical processing layer (probably can be done in parallel over a network at high speed) - I would assume this is why shared memory bandwidth was a problem. This would provide some really interesting options though - imagine feeding data from two disparate databases (say, Oracle and SQL server) in a data flow into this thing - now you have accelerated cross database joins (as long as you can handle the bandwidth and processing on the way in). There was this post before on HN previously lamenting the state of tools for working with FPGAs, and my related comment wondering if what Baidu has done here was possible: - https://news.ycombinator.com/item?id=9408881 https://news.ycombinator.com/item?id=9408881 - https://news.ycombinator.com/item?id=9410160 https://news.ycombinator.com/item?id=9410160
- gopalv 10y ago> which potentially means huge volumes of data moving into this physical processing layer You get to cheat at that, because the naive implementation loads too much data, while most of it is going to drop off at the filter level. The ideal TPC-DS schema stores data partitioned by date and maintains a lookup index by item, store and clustered by store's state. In a database organized like that, neither part of the Query-3 item.i_manufact_id = 436 or dt.d_moy=12 should actually look at every row in the data-set in brute-force fashion - you get to skip massive parts of the data by just inspecting the index or going to the specific date partition and do it much faster than a naive C++ full-scan. However, if you were limited to a full-load + scan, this would really be killer to have a columnar format which can be handled by an FPGA. The really interesting quote in the blog (for me) is > the data for the queries is pushed to the accelerator card in columnar format (which is blazingly fast for queries) I sure could use FPGA filters for columnar data ... even after I've done all my indexing, since I can't index down AND clauses or OR clauses - unlike a traditional CPU, the FPGA will be able to evaluate my entire condition in one sequential flow instead of check + branching.
- deleted 10y ago[deleted]
- trhway 10y agosuch FPGAs seems to be on the edge of what is export controlled, recently even regular Intel CPU got blocked for delivery to China : http://www.pcworld.com/article/2908692/us-blocks-intel-from-selling-xeon-chips-to-chinese-supercomputer-projects.html http://www.pcworld.com/article/2908692/us-blocks-intel-from-... and around an year ago some Russian guys got busted for selling the FPGAs to Russia (and those FPGAs were less powerful): http://kron4.com/2015/03/21/sf-business-owner-arrested-for-allegedly-smuggling-programmable-electronics-to-russia-laundering-millions/ http://kron4.com/2015/03/21/sf-business-owner-arrested-for-a...
- eva1984 10y agoBaidu has an office here in US, they could build it here...I guess
- DigitalJack 10y agoI'm an FPGA/ASIC engineer that does some software as a hobby (mostly clojure, some python. C and Pascal if you go back far enough). Although I focus more on verification these days, I've done ASIC design for about 16 years. I'd be very interested in working with anyone on figuring out if we can make something better by leveraging hardware.
- turnip1979 10y agoWhere are you based? I'm a software guy learning about hw :)
- DigitalJack 10y agoPortland, OR, but I've been working remotely for 10 years. Virtual coffee still isn't a thing unfortunately.
- asb 10y agoHi, you might be interested in our lowRISC project which is working towards producing a fully open-source SoC implementing the RISC-V instruction set architecture. Some of the key features we are focusing on, minion cores and tagged memory http://www.lowrisc.org/docs/memo-2014-001-tagged-memory-and-minion-cores/ http://www.lowrisc.org/docs/memo-2014-001-tagged-memory-and-... I think are good examples of how we may be able to improve software through leveraging hardware. You don't have an email address in your profile, so if you want to chat more email me at asb@lowrisc.org
- pldrnt 10y agoI've been working with FPGAs for 4 years and shortly on ASICs before that, did software in a previous life (and still do), do you have something specific in mind?
- DigitalJack 10y agoNo, I personally don't. I was into reconfigurable computing for a while, sort of where I think intel is going with their combined CPU/FPGA project. Just In Time logic configuration. But I figured I would throw my offer out in case there were people thinking they could solve or enhance something with an FPGA, but didn't have the experience to get started.
- nl 10y agoMapD does some of this, but with GPUs: https://www.mapd.com/product/ https://www.mapd.com/product/