9 ms·

Sorry to leave the technical detail part real quick. But is anyone else concerned about using a DB solely from a company built specifically around that DB? Afte

Sorry to leave the technical detail part real quick. But is anyone else concerned about using a DB solely from a company built specifically around that DB? After Rethink DB (sustainability issue) and Foundation DB (bought and shuttered/hidden) and Riak (admittedly haven't kept up but I saw [0]), I am wary of using any DB that is not built by a large community or is not built as a non-core project from a large tech company. Sorry TimescaleDB, I see you have raised a decent amount of funding, but I have to choose my DBs w/ trepidation these days. 0 - https://www.theregister.co.uk/2017/07/13/will_the_last_person_at_basho_get_the_lights_oh_too_late/ https://www.theregister.co.uk/2017/07/13/will_the_last_perso...

walterbell 9y ago
Is the concern mitigated by this being an open-source Postgres extension? One question would be whether they have attracted external OSS contributors.
- kodablah 9y ago
  Depends on your reliance I suppose. Appears most of TimescaleDB's vendor-specific pieces are in the DDL area, but there are plenty of DML-specific extensions. I'm not saying TimescaleDB might not get to a level of maturity where I'd use it, I personally just can't justify it today. I've found DBs and ops/orchestration engines to be the most painful to move off from if you need to.
  - pas 9y ago
    They also provide the most bang for the feature-complexity buck. (A DB is something you don't want to roll on your own, and you want to use the purpose-built one for your use case. A columnar store for time series, a scatter-gather SSD + random read optimized engine for hot OLTP, a fancy multi level cache/tier BLOB store for photos/files, CockroachDB for geo consistency, etcd for local consistent config, and so on.)
gaius 9y ago
If Timescale goes bust you still have all your data in Postgres, which isn’t going anywhere. The risk is much lower.
greenleafjacob 9y ago
Basho is in receivership, close to bankruptcy. https://www.theregister.co.uk/2017/07/31/end_of_the_road_for_basho_as_court_puts_biz_into_receivership/ https://www.theregister.co.uk/2017/07/31/end_of_the_road_for...
- btown 9y ago
  For anyone just seeing this, its assets were purchased and are planned to be open-sourced: https://news.ycombinator.com/item?id=15182566 https://news.ycombinator.com/item?id=15182566
antirez 9y ago
What you say is another way to see the argument about OSS Project vs OSS licensed software which is actually a company and not a project.
siculars 9y ago
Riak the database survives Basho the company. All assets (IP) were bought by bet365 (A large Basho customer) and made open source. Development continues. /former Basho employee.
- hyder_m29 9y ago
  And RethinkDB has moved under the Linux Foundation.
- j_s 9y ago
  > made open source I hope things work out at least as well as they have for RethinkDB's resurrection/transition! The Riak Users mailing list seems like the best place to follow along: http://riak-users.197444.n3.nabble.com http://riak-users.197444.n3.nabble.com https://github.com/basho/riak_kv/tree/develop-2.2.5 https://github.com/basho/riak_kv/tree/develop-2.2.5 (November 2017) http://bet365techblog.com/riak-workshop-summary http://bet365techblog.com/riak-workshop-summary (October 2017) >the immediate goal of releasing a known good build as soon as possible [...] all code would be released under the Apache 2 license [...] contact the HLL developers to aid in the dependency and GPL removal https://news.ycombinator.com/item?id=15182566#15183008 https://news.ycombinator.com/item?id=15182566#15183008 (September 2017) >andrew_deane_: bet365 have signed the agreement to purchase all Basho IP [...] our intention is to open source all code, help rebuild the community and collaboratively take the development of RIAK forward
  - andrew_deane__ 9y ago
    All IP has been made open by bet365 (including RIAK TS) and development is continuing, both in the community and within the company. bet365 are actively recruiting to support this effort, and are committed to the long term future of RIAK. Latest release is to be cut Jan / Feb with the help of the community.
linsomniac 9y ago
There are tons of justifications you can come up with for not using something you don't want to use. The down side is that you aren't going to be getting the upsides you'd get if you used a new technology, possibly to the extent of being no longer competitive with people who do use that technology. The important thing is balancing chasing the new hotness vs. always using the tried and true. Last year I ran into a similar sort of situation at work, the funds to buy Ansible Tower required a battle I wasn't willing to fight. The argument was "We've spent so much money on tools that then get abandoned." We spent nearly a year with me being the only one in the company that could run Ansible playbooks because of this, until I finally found RunDeck. Now, we aren't paying for Rundeck, does that make it more likely to be abandoned? Was the risk in our operations if I suddenly became unavailable worth the saved licensing fees? Well, in this case it worked out ok, but it could have gone very badly. As far as timeseries goes, there are huge advantages here. We used to use collectd/graphite/carbon, but a year or more ago I switched us over to InfluxDB/telegraf/grafana. It was a little rocky a start, the first version of Influx I used had plenty of problems. At the time Prometheus was probably a better choice, but I couldn't come up with a way to make it's pull-centric design fit into our environment. Then InfluxDB settled down and things have been great since. We have way better metrics, with a way more usable dashboard, and significantly lower load. Literally, disabling collectd ended the complaints about performance on our dev/stg environments.
- kodablah 9y ago
  > The down side is that you aren't going to be getting the upsides you'd get if you used a new technology True of course. I've found that at least "premature abstraction" is almost a requirement when using persistence. I'm not talking an ORM or anything, but put a service in front of it if you have multiple things needing it, or put an in-code abstraction if you don't. And make each abstraction's/service's operation very specific to its caller. So a simple "fetchMarketData(timeRange, tickerSymbols) -> StreamOfData" will save you so much in the future.
krylon 9y ago
> But is anyone else concerned about using a DB solely from a company built specifically around that DB? It worked out pretty well for Oracle. (SCNR)
- kodablah 9y ago
  I'd almost rather be locked in to a cheap, may-deprecate DB than a really expensive one. The former at least forces you to move, the latter just costs your business a lot especially when non-doers run the tech show and will sink money over re-impl every time (as opposed to making measured decisions).
  - krylon 9y ago
    FWIW, I found a table with prices for Oracle database licenses once, and it made Microsoft SQL Server look cheap. I am not sure if that means MSSQL is cheap, though. At least I am fairly confident that Microsoft is not going out of business anytime soon. In my current job, I needed to get intimately familiar with the database our ERP software uses, because my boss needed lots of reports that the ERP software just did not do (unless paid an expensive consultant to customize the software for you). And the idea of having to move all that data to a different database makes me shiver in horror. I can understand why a business would be very reluctant to move their data from one database (both schema and engine) to another. For business-critical tasks, I prefer a boring, reliable solution. But it is good to know that there are still interesting developments going on in the database area, building tomorrow's boring, reliable solutions.
qaq 9y ago
Timescale is open source PostgreSQL extension that is much safer vs RethinkDB.
j_s 9y ago
There was a similar sub-discussion on the Datomic Cloud announcement last week: https://news.ycombinator.com/item?id=16168041#16168743 https://news.ycombinator.com/item?id=16168041#16168743 As an open source product it seems like TimescaleDB is less risky than Datomic. (Not that the comparison is even necessary -- I don't think they compete in the same category apart from both being non-relational DBs.) [edit:link fixed, thanks! (and thanks for taking the time to correct my misunderstanding re:TimescaleDB as a relational DB)]
- stalller 9y ago
  Hi, the link you posted seems broken. Also just wanted to clarify that TimescaleDB is in fact a relational database, being built as a PostgreSQL extension. Looking at the nature of time-series workloads (namely appending data about recent events) we can architect the database differently for time-series workloads and applications powered by time-series data; scaling insert rates and storage capacity, increasing query performance and simplifying operations, while retaining full-SQL and the reliability of Postgres.
Steeeve 9y ago
You mean like Oracle? Or Sybase? Or Informix? Or Terradata? Well, I guess you can stick to DB2.
- kodablah 9y ago
  Having worked at places where they paid tons for options like those, and suffering from the inability to move and the difficulty of finding non-company tooling and community around them, I'd say they have the same problem. Point being: if your ecosystem is a single company, it's at the whims of that company's success and give-a-shitness. Non-DB companies and open source communities tend to gouge less and share more.
lbruder 9y ago
We use a combination of SQLite and a sharding frontend service. One SQLite database file per device, one table per sensor, table contents are timestamp and measured value. As simple as it gets, easy to scale, and damn fast. But try telling people you're using SQLite to store critical data...
- otterley 9y ago
  What's your replication/HA story?
  - j_s 9y ago
    Yes, collecting/aggregating for querying seems like the sticking point.
    - tzakrajs 9y ago
      What does collecting or aggregating have to do with replication/HA?
      - j_s 9y ago
        It was my improvisational approach to pointing in the right direction. https://en.wikipedia.org/wiki/Yes,_and... https://en.wikipedia.org/wiki/Yes,_and.... I don't really see how replication/HA is an issue in this setup beyond normal file system replication/HA (which seems to be a solved "spend money on NAS" problem in the enterprise world). If your filesystem isn't reliable / delivering what's needed then you're gonna have a bad time no matter the storage approach; I doubt some unusual option is involved though it would be cool to be proven wrong. The only time the filesystem could even potentially have read/write conflicts (assuming 1-to-1 correlation between devices writing to files) is when querying. I did wind up muddling the terminology in the interest of jumping to the actual problem.
  - tomc1985 9y ago
    Wouldn't be hard to rsync that data to rotating s3 buckets/nfs shares/ftp drops/whatever
oiuoiu87 9y ago
I'll see your July 2017 article, and raise with this August 2017: https://www.theregister.co.uk/2017/07/31/end_of_the_road_for_basho_as_court_puts_biz_into_receivership/ https://www.theregister.co.uk/2017/07/31/end_of_the_road_for...
camel_gopher 9y ago
Data portability is certainly an important issue here to consider here given the points you mentioned. When adopting a vendor, the relevant question is "How much work is it to get data in?". And when planning for the contingency of vendor loss, the question is "How much work is it to get the data out?". The first question is tied closely to the interfaces provided by the database, and those interfaces are usually commensurate with the associated tooling. In the case of time series databases, they include statsd, collectd, and good old JSON. Relational databases utilize SQL as expected. Take the example of migrating from Oracle to Postgres; you shouldn't have to spend an inordinate amount of retooling your applications because the data interfaces are similar. Data egress is similar; SQL databases generally provide SQL/CSV exports, and most TSDBs (the good ones at least) allow you to get the data out via JSON through a REST interface. Whether or not the DB you choose is open source, or from a small vendor, you should pick the right tool to solve your problem with data portability in mind. (disclosure, I work for http://irondb.io http://irondb.io, and have used Postgres for 18 years)
Johnny555 9y ago
There's also snowflakedb which was in the news today after getting a pile of funding. Though it seems to be cloud-only, not something you can run yourself.
j4ship 9y ago
those are concerns of every project , we will see if timescale is different But is it good enough , as a product it needs to prove to me its good enough to switch. Many people dont need a specific time database. Many people have created solutions on top of their current database. We have hadoop and other post processors that can allow us to off load old data but raise features to the user to get data in the past. Is this better? And how much more does it cost? Oracle didnt get successful becuase people couldnt store data before them. They were successful becuase it became a need: You have a DB solution or you dont and if you dont you cant be secure in your "data assets" (Martin Fowler talks about this). So if I dont have a Time database ... what cant I do? Who is going to not do business with me unless I have a time db? I hope you have some good enterprise friends.
kunthar 9y ago
riak is not dead and still have strong community of developers, users and enthusiasts. bet365 bought all the assets including enterprise parts and made them open source. you can join postriak slack channel and see with your own eyes there is active and vibrant community working about a lot of exciting features. so, no, party is not over yet!
tomc1985 9y ago
Honestly I have found it is simply easier to master PostgreSQL and sqlite, and between the two (and with the right know-how) pretty much all use-cases are covered. Played with a lot of databases, sql or no, and I always end up going back to PG

walterbell 9y ago

Is the concern mitigated by this being an open-source Postgres extension? One question would be whether they have attracted external OSS contributors.

kodablah 9y ago
Depends on your reliance I suppose. Appears most of TimescaleDB's vendor-specific pieces are in the DDL area, but there are plenty of DML-specific extensions. I'm not saying TimescaleDB might not get to a level of maturity where I'd use it, I personally just can't justify it today. I've found DBs and ops/orchestration engines to be the most painful to move off from if you need to.
- pas 9y ago
  They also provide the most bang for the feature-complexity buck. (A DB is something you don't want to roll on your own, and you want to use the purpose-built one for your use case. A columnar store for time series, a scatter-gather SSD + random read optimized engine for hot OLTP, a fancy multi level cache/tier BLOB store for photos/files, CockroachDB for geo consistency, etcd for local consistent config, and so on.)

gaius 9y ago

If Timescale goes bust you still have all your data in Postgres, which isn’t going anywhere. The risk is much lower.

greenleafjacob 9y ago

Basho is in receivership, close to bankruptcy. https://www.theregister.co.uk/2017/07/31/end_of_the_road_for_basho_as_court_puts_biz_into_receivership/ https://www.theregister.co.uk/2017/07/31/end_of_the_road_for...

btown 9y ago
For anyone just seeing this, its assets were purchased and are planned to be open-sourced: https://news.ycombinator.com/item?id=15182566 https://news.ycombinator.com/item?id=15182566

antirez 9y ago

What you say is another way to see the argument about OSS Project vs OSS licensed software which is actually a company and not a project.

siculars 9y ago

Riak the database survives Basho the company. All assets (IP) were bought by bet365 (A large Basho customer) and made open source. Development continues. /former Basho employee.

hyder_m29 9y ago
And RethinkDB has moved under the Linux Foundation.
j_s 9y ago
> made open source I hope things work out at least as well as they have for RethinkDB's resurrection/transition! The Riak Users mailing list seems like the best place to follow along: http://riak-users.197444.n3.nabble.com http://riak-users.197444.n3.nabble.com https://github.com/basho/riak_kv/tree/develop-2.2.5 https://github.com/basho/riak_kv/tree/develop-2.2.5 (November 2017) http://bet365techblog.com/riak-workshop-summary http://bet365techblog.com/riak-workshop-summary (October 2017) >the immediate goal of releasing a known good build as soon as possible [...] all code would be released under the Apache 2 license [...] contact the HLL developers to aid in the dependency and GPL removal https://news.ycombinator.com/item?id=15182566#15183008 https://news.ycombinator.com/item?id=15182566#15183008 (September 2017) >andrew_deane_: bet365 have signed the agreement to purchase all Basho IP [...] our intention is to open source all code, help rebuild the community and collaboratively take the development of RIAK forward
- andrew_deane__ 9y ago
  All IP has been made open by bet365 (including RIAK TS) and development is continuing, both in the community and within the company. bet365 are actively recruiting to support this effort, and are committed to the long term future of RIAK. Latest release is to be cut Jan / Feb with the help of the community.

linsomniac 9y ago

There are tons of justifications you can come up with for not using something you don't want to use. The down side is that you aren't going to be getting the upsides you'd get if you used a new technology, possibly to the extent of being no longer competitive with people who do use that technology. The important thing is balancing chasing the new hotness vs. always using the tried and true. Last year I ran into a similar sort of situation at work, the funds to buy Ansible Tower required a battle I wasn't willing to fight. The argument was "We've spent so much money on tools that then get abandoned." We spent nearly a year with me being the only one in the company that could run Ansible playbooks because of this, until I finally found RunDeck. Now, we aren't paying for Rundeck, does that make it more likely to be abandoned? Was the risk in our operations if I suddenly became unavailable worth the saved licensing fees? Well, in this case it worked out ok, but it could have gone very badly. As far as timeseries goes, there are huge advantages here. We used to use collectd/graphite/carbon, but a year or more ago I switched us over to InfluxDB/telegraf/grafana. It was a little rocky a start, the first version of Influx I used had plenty of problems. At the time Prometheus was probably a better choice, but I couldn't come up with a way to make it's pull-centric design fit into our environment. Then InfluxDB settled down and things have been great since. We have way better metrics, with a way more usable dashboard, and significantly lower load. Literally, disabling collectd ended the complaints about performance on our dev/stg environments.

kodablah 9y ago
> The down side is that you aren't going to be getting the upsides you'd get if you used a new technology True of course. I've found that at least "premature abstraction" is almost a requirement when using persistence. I'm not talking an ORM or anything, but put a service in front of it if you have multiple things needing it, or put an in-code abstraction if you don't. And make each abstraction's/service's operation very specific to its caller. So a simple "fetchMarketData(timeRange, tickerSymbols) -> StreamOfData" will save you so much in the future.

krylon 9y ago

> But is anyone else concerned about using a DB solely from a company built specifically around that DB? It worked out pretty well for Oracle. (SCNR)

kodablah 9y ago
I'd almost rather be locked in to a cheap, may-deprecate DB than a really expensive one. The former at least forces you to move, the latter just costs your business a lot especially when non-doers run the tech show and will sink money over re-impl every time (as opposed to making measured decisions).
- krylon 9y ago
  FWIW, I found a table with prices for Oracle database licenses once, and it made Microsoft SQL Server look cheap. I am not sure if that means MSSQL is cheap, though. At least I am fairly confident that Microsoft is not going out of business anytime soon. In my current job, I needed to get intimately familiar with the database our ERP software uses, because my boss needed lots of reports that the ERP software just did not do (unless paid an expensive consultant to customize the software for you). And the idea of having to move all that data to a different database makes me shiver in horror. I can understand why a business would be very reluctant to move their data from one database (both schema and engine) to another. For business-critical tasks, I prefer a boring, reliable solution. But it is good to know that there are still interesting developments going on in the database area, building tomorrow's boring, reliable solutions.

qaq 9y ago

Timescale is open source PostgreSQL extension that is much safer vs RethinkDB.

j_s 9y ago

There was a similar sub-discussion on the Datomic Cloud announcement last week: https://news.ycombinator.com/item?id=16168041#16168743 https://news.ycombinator.com/item?id=16168041#16168743 As an open source product it seems like TimescaleDB is less risky than Datomic. (Not that the comparison is even necessary -- I don't think they compete in the same category apart from both being non-relational DBs.) [edit:link fixed, thanks! (and thanks for taking the time to correct my misunderstanding re:TimescaleDB as a relational DB)]

stalller 9y ago
Hi, the link you posted seems broken. Also just wanted to clarify that TimescaleDB is in fact a relational database, being built as a PostgreSQL extension. Looking at the nature of time-series workloads (namely appending data about recent events) we can architect the database differently for time-series workloads and applications powered by time-series data; scaling insert rates and storage capacity, increasing query performance and simplifying operations, while retaining full-SQL and the reliability of Postgres.

Steeeve 9y ago

You mean like Oracle? Or Sybase? Or Informix? Or Terradata? Well, I guess you can stick to DB2.

kodablah 9y ago
Having worked at places where they paid tons for options like those, and suffering from the inability to move and the difficulty of finding non-company tooling and community around them, I'd say they have the same problem. Point being: if your ecosystem is a single company, it's at the whims of that company's success and give-a-shitness. Non-DB companies and open source communities tend to gouge less and share more.

lbruder 9y ago

We use a combination of SQLite and a sharding frontend service. One SQLite database file per device, one table per sensor, table contents are timestamp and measured value. As simple as it gets, easy to scale, and damn fast. But try telling people you're using SQLite to store critical data...

otterley 9y ago
What's your replication/HA story?
- j_s 9y ago
  Yes, collecting/aggregating for querying seems like the sticking point.
  - tzakrajs 9y ago
    What does collecting or aggregating have to do with replication/HA?
    - j_s 9y ago
      It was my improvisational approach to pointing in the right direction. https://en.wikipedia.org/wiki/Yes,_and... https://en.wikipedia.org/wiki/Yes,_and.... I don't really see how replication/HA is an issue in this setup beyond normal file system replication/HA (which seems to be a solved "spend money on NAS" problem in the enterprise world). If your filesystem isn't reliable / delivering what's needed then you're gonna have a bad time no matter the storage approach; I doubt some unusual option is involved though it would be cool to be proven wrong. The only time the filesystem could even potentially have read/write conflicts (assuming 1-to-1 correlation between devices writing to files) is when querying. I did wind up muddling the terminology in the interest of jumping to the actual problem.
- tomc1985 9y ago
  Wouldn't be hard to rsync that data to rotating s3 buckets/nfs shares/ftp drops/whatever

oiuoiu87 9y ago

I'll see your July 2017 article, and raise with this August 2017: https://www.theregister.co.uk/2017/07/31/end_of_the_road_for_basho_as_court_puts_biz_into_receivership/ https://www.theregister.co.uk/2017/07/31/end_of_the_road_for...

camel_gopher 9y ago

Data portability is certainly an important issue here to consider here given the points you mentioned. When adopting a vendor, the relevant question is "How much work is it to get data in?". And when planning for the contingency of vendor loss, the question is "How much work is it to get the data out?". The first question is tied closely to the interfaces provided by the database, and those interfaces are usually commensurate with the associated tooling. In the case of time series databases, they include statsd, collectd, and good old JSON. Relational databases utilize SQL as expected. Take the example of migrating from Oracle to Postgres; you shouldn't have to spend an inordinate amount of retooling your applications because the data interfaces are similar. Data egress is similar; SQL databases generally provide SQL/CSV exports, and most TSDBs (the good ones at least) allow you to get the data out via JSON through a REST interface. Whether or not the DB you choose is open source, or from a small vendor, you should pick the right tool to solve your problem with data portability in mind. (disclosure, I work for http://irondb.io http://irondb.io, and have used Postgres for 18 years)

Johnny555 9y ago

There's also snowflakedb which was in the news today after getting a pile of funding. Though it seems to be cloud-only, not something you can run yourself.

j4ship 9y ago

those are concerns of every project , we will see if timescale is different But is it good enough , as a product it needs to prove to me its good enough to switch. Many people dont need a specific time database. Many people have created solutions on top of their current database. We have hadoop and other post processors that can allow us to off load old data but raise features to the user to get data in the past. Is this better? And how much more does it cost? Oracle didnt get successful becuase people couldnt store data before them. They were successful becuase it became a need: You have a DB solution or you dont and if you dont you cant be secure in your "data assets" (Martin Fowler talks about this). So if I dont have a Time database ... what cant I do? Who is going to not do business with me unless I have a time db? I hope you have some good enterprise friends.

kunthar 9y ago

riak is not dead and still have strong community of developers, users and enthusiasts. bet365 bought all the assets including enterprise parts and made them open source. you can join postriak slack channel and see with your own eyes there is active and vibrant community working about a lot of exciting features. so, no, party is not over yet!

tomc1985 9y ago

Honestly I have found it is simply easier to master PostgreSQL and sqlite, and between the two (and with the right know-how) pretty much all use-cases are covered. Played with a lot of databases, sql or no, and I always end up going back to PG