3 ms·

Show HN: BigObject – Run your data analysis faster

25 points by chenyuanjen 11y ago

MasterScrat 11y ago
How exactly is that supposed to be faster? What will happen when I "Launch" it, and why should I give it my email address beforehand?
- IanCal 11y ago
  Doesn't look like you need to give it your email address, there's a docker image available https://docs.bigobject.io/docs/running-bigobject-as-a-container https://docs.bigobject.io/docs/running-bigobject-as-a-contai... And some examples of using it: https://docs.bigobject.io/docs/getting-started-in-3-minutes https://docs.bigobject.io/docs/getting-started-in-3-minutes Had to dig around a little though, seeing more of this up on the front page would be nice. What is this, and why should I care. Fast ... doing what? How can I start to tell if this might be relevant to me? Those are some of the questions that come to mind.
- chenyuanjen 11y ago
  Hey yes~ you can also run BigObject with docker container. In that case, you won't need to send us your email address. Email address is for use to send the link for those who chose web service. And also stay up-to-date with our latest release.
  - ahazred8ta 11y ago
    I hate to say it, but your people really need to redesign the entire website. https://docs.bigobject.io https://docs.bigobject.io is a better starting point than your main website. Your "Learn" page only has a video, but most developers would skip your "Quick Start Tour" video and assume it was a waste of time if they can't first spend time reading a detailed text explanation of how BigObject works. No link leads from "Learn" to your docs site. Technically, a link to docs does appear in the sidebar of the Product page, but that page contains so much uninformative text that many readers would lose interest before examining the sidebar. Please consider moving the information at https://docs.bigobject.io/docs/why-bigobject-analytics https://docs.bigobject.io/docs/why-bigobject-analytics to your main page.
    - chenyuanjen 11y ago
      Thanks for your feedback! We are restructuring the website and will elaborate more details at the front.
    - chenyuanjen 11y ago
      Hey your comments are really helpful. We just updated the content and it should be clearer now. We will make the design prettier later too.
buremba 11y ago
It seems like a stream processing engine that uses a custom query language similar to SQL (well, even though the syntax doesn't look like SQL, the documentation says so). The website is not helpful enough considering that your target audience is actually developers. "Run queries 100x to 1000x faster" doesn't mean anything.
- chenyuanjen 11y ago
  Hey thanks buremba. We're working on a more elaborate introduction with clear definition and position in system diagram, use cases, etc.
lootsauce 11y ago
I quickly scanned the white-paper on linked on their site [1] So they are doing some interesting things with a virtual memory space using a tree structure of the data that reflects various relations in the data in a star schema. Those objects are persisted in memory mapped files, mapped into an effectively infinite single shared memory address space in the virtual memory model. They argue that you can then do new more efficient algorithms on these memory mapped files. Sounds like they are seeing performance we are getting in Google BigQuery with queries against 100s of millions of rows taking less than 10 seconds however they did not show how it scales beyond that. Another interesting thing is this is designed to sit on top of existing systems to serve not as the source of truth but the layer to do analytics with realtime data, a very compelling approach IMHO. They liken their data-structure to that of RDD (Resilient Distributed Dataset) used in Spark but unlike RDD it is not temporary and is writeable. The other interesting thing about this is as far as I understand it, this can be seen as a virtualized implementation of the computing model that is coming with new memory hardware [2] where ram == storage [1] http://www.slideshare.net/BigObject/big-object-store-in-place-computing-for-interactive-analysis?ref=http://bigobject.io/ http://www.slideshare.net/BigObject/big-object-store-in-plac... [2] http://www.zdnet.com/article/nanteros-carbon-nanotube-memory-breakthrough/ http://www.zdnet.com/article/nanteros-carbon-nanotube-memory... "The race for a workable NVRAM is in its final stages. HP's memristor, Crossbar's RRAM and now, Nantero's NRAM, are all technically sound, backed by tens of millions of dollars in R&D, and close to broad market release.”
- mdaniel 11y ago
  I thought the slideshare.net link was odd, but it seems that is the only non-paywall-ed version of the file. As a friendly FYI to other interested parties, http://bugmenot.com/view/slideshare.net http://bugmenot.com/view/slideshare.net is a mechanism through which one can use the download button, resulting in a much nicer on the eyes PDF.
  - chenyuanjen 11y ago
    Hey mdaniel that is an IEEE paper we published.

25 points by chenyuanjen 11y ago

How exactly is that supposed to be faster? What will happen when I "Launch" it, and why should I give it my email address beforehand?

IanCal 11y ago
Doesn't look like you need to give it your email address, there's a docker image available https://docs.bigobject.io/docs/running-bigobject-as-a-container https://docs.bigobject.io/docs/running-bigobject-as-a-contai... And some examples of using it: https://docs.bigobject.io/docs/getting-started-in-3-minutes https://docs.bigobject.io/docs/getting-started-in-3-minutes Had to dig around a little though, seeing more of this up on the front page would be nice. What is this, and why should I care. Fast ... doing what? How can I start to tell if this might be relevant to me? Those are some of the questions that come to mind.
chenyuanjen 11y ago
Hey yes~ you can also run BigObject with docker container. In that case, you won't need to send us your email address. Email address is for use to send the link for those who chose web service. And also stay up-to-date with our latest release.
- ahazred8ta 11y ago
  I hate to say it, but your people really need to redesign the entire website. https://docs.bigobject.io https://docs.bigobject.io is a better starting point than your main website. Your "Learn" page only has a video, but most developers would skip your "Quick Start Tour" video and assume it was a waste of time if they can't first spend time reading a detailed text explanation of how BigObject works. No link leads from "Learn" to your docs site. Technically, a link to docs does appear in the sidebar of the Product page, but that page contains so much uninformative text that many readers would lose interest before examining the sidebar. Please consider moving the information at https://docs.bigobject.io/docs/why-bigobject-analytics https://docs.bigobject.io/docs/why-bigobject-analytics to your main page.
  - chenyuanjen 11y ago
    Thanks for your feedback! We are restructuring the website and will elaborate more details at the front.
  - chenyuanjen 11y ago
    Hey your comments are really helpful. We just updated the content and it should be clearer now. We will make the design prettier later too.

buremba 11y ago

It seems like a stream processing engine that uses a custom query language similar to SQL (well, even though the syntax doesn't look like SQL, the documentation says so). The website is not helpful enough considering that your target audience is actually developers. "Run queries 100x to 1000x faster" doesn't mean anything.

chenyuanjen 11y ago
Hey thanks buremba. We're working on a more elaborate introduction with clear definition and position in system diagram, use cases, etc.

lootsauce 11y ago

I quickly scanned the white-paper on linked on their site [1] So they are doing some interesting things with a virtual memory space using a tree structure of the data that reflects various relations in the data in a star schema. Those objects are persisted in memory mapped files, mapped into an effectively infinite single shared memory address space in the virtual memory model. They argue that you can then do new more efficient algorithms on these memory mapped files. Sounds like they are seeing performance we are getting in Google BigQuery with queries against 100s of millions of rows taking less than 10 seconds however they did not show how it scales beyond that. Another interesting thing is this is designed to sit on top of existing systems to serve not as the source of truth but the layer to do analytics with realtime data, a very compelling approach IMHO. They liken their data-structure to that of RDD (Resilient Distributed Dataset) used in Spark but unlike RDD it is not temporary and is writeable. The other interesting thing about this is as far as I understand it, this can be seen as a virtualized implementation of the computing model that is coming with new memory hardware [2] where ram == storage [1] http://www.slideshare.net/BigObject/big-object-store-in-place-computing-for-interactive-analysis?ref=http://bigobject.io/ http://www.slideshare.net/BigObject/big-object-store-in-plac... [2] http://www.zdnet.com/article/nanteros-carbon-nanotube-memory-breakthrough/ http://www.zdnet.com/article/nanteros-carbon-nanotube-memory... "The race for a workable NVRAM is in its final stages. HP's memristor, Crossbar's RRAM and now, Nantero's NRAM, are all technically sound, backed by tens of millions of dollars in R&D, and close to broad market release.”

mdaniel 11y ago
I thought the slideshare.net link was odd, but it seems that is the only non-paywall-ed version of the file. As a friendly FYI to other interested parties, http://bugmenot.com/view/slideshare.net http://bugmenot.com/view/slideshare.net is a mechanism through which one can use the download button, resulting in a much nicer on the eyes PDF.
- chenyuanjen 11y ago
  Hey mdaniel that is an IEEE paper we published.