10 ms·

Developer Preview: RethinkDB Now Available for Windows

161 points by coffeemug 11y ago

fucsia 11y ago
many of the underlying design decisions in libuv clash with decisions that we’ve made in the RethinkDB core Can you expand on this please?
- atnnn 11y ago
  RethinkDB uses custom abstractions to hide the lower-level OS-specific code. We were able to port the low-level code to Windows with very little changes in these abstractions, and thus minimal changes to the rest of the code base. The proper way to use libuv would be to replace those abstractions with libuv's own, which would affect a lot more code and possibly break a lot of implicit invariants. In retrospect, perhaps we should have given alternatives like libuv, libevent or boost::asio more consideration.
dylz 11y ago
your mascot is adorable.
- SamReidHughes 11y ago
  Wait 'til it puts on its robe and wizard hat!
  - dylz 11y ago
    haha, it's more that it's rare that I see such detail put into ui/ux/branding in many/most products, especially ones that aren't geared toward common end users. big kudos :)
giancarlostoro 11y ago
This is one of those databases I wanted to eventually arrive on Windows, I really enjoyed the look of it and wanted to try it out, and although I have used from time to time Linux as my main OS I just don't feel comfortable using a database I can't run on multiple platforms, it seems like a barrier not being able to use it locally from Windows, although Vagrant and vagrant boxes like Scotch Box are making this "issue" a little less relevant these days, it's still nice to be able to run a database without installing an entire OS. Thanks for the work and time invested in bringing Rethink into Windows, it is truly appreciated.
iheartmemcache 11y ago
I love how they're reaching out to C# users (and conversely I love how MS is reaching out to Linux and OS X users by opening sourcing their CLR, giving away their IDE, integrating Node.js and JSX as first-class citizens, etc!) but Rethink-ers if you're reading this - I briefly perused your source code and to a C#'er (who has written his fair share of JS, both in the "Visual Studio ASP.NET and SharePoint" style, as well as the "node.js" style) -- this doesn't read like idiomatic C# at all. I'm sure it works well, but the API feels like someone who writes JS for a living was given a deadline to add this feature and tacked it on half-heartedly. There's no reason in this day and age that you should be have any deps on stuff like Cygwin, which you claim you don't: >> You won’t find any POSIX compatibility layers or other similar hacks–RethinkDB uses native Windows APIs on the Windows platform. It looks like you do[1] from the whole Cygwin section and the env var's you have to add to PATH (in a somewhat inelegant way). That dependency "tree" is... extensive to say the least and a lot of it can be eliminated or eased with a combination of NuGet and chocolatey. Having in your instruction notes to build Goog's RE2: >> Build it somehow (I forgot to record how) [2] doesn't really make me too confident that you're going to support Windows as a first party platform. The API exposed to the end-user should be conventionally written in a more LINQ-y way - I.e. Fluent style code should be written with Object.Blah().Filter(x => x % 2).ToList() or whatever. Just little idiomatic things like that are pretty pervasive. Regardless, thanks for investing the time on working for this. As a curmudgeon and someone who spends his spare time reading journal papers in the ACM about concurrency models from hardware transactional locks to ICFP papers which implement similar concepts on the exact opposite side of the spectrum, it's my duty to be wary of new technologies and slow to adopt them into production-- but RDB and VoltDB are two products I really think are technically sound. Contact me if you want a more thorough critique to make this more idiomatic, because your product really is great! [1] https://github.com/rethinkdb/rethinkdb/blob/a90bb0805160362125a03024eb9f309e578408c0/WINDOWS.md https://github.com/rethinkdb/rethinkdb/blob/a90bb08051603621... (Errata: see edit) [2] See [1]: RE2 Edit: I was referring to this revision of Windows.md, which at the time was what was available on their master branch. My critique largely revolves around that version. Proceeding my post, another branch's version was swapped in, to a more clean version in which the build phases of the dependencies are removed, and replaced with pre-compiled binaries as you now see: https://github.com/rethinkdb/rethinkdb/blob/next/WINDOWS.md https://github.com/rethinkdb/rethinkdb/blob/next/WINDOWS.md This solves the aesthetics issue but just pushes the problem into another domain. You're using binary deps (some of which are cygwin'd afaik, but I haven't taken IDA to it so I can't say definitively) but if you want to build from "source" a la the FreeBSD ports way -- where every single .c or .cc is available for one to see and modify as their environment demands -- the problem's still there just pushed under the rug.
mizchief2 11y ago
"We launched an ambitious engineering project to port RethinkDB to Windows–an undertaking that required a year of intensive development, touching nearly every part of the database." What a waste of time.
pjmlp 11y ago
As mostly a Windows developer nowadays, many thanks and congratulations.
talles 11y ago
Not in love with MongoDB, DocumentDB is Azure only. Finally a (great) document database option for me (I work in a .NET shop). Kudos to you guys. But... but... what's with that C# API? It doesn't look like typical C# at all.
- giancarlostoro 11y ago
  Well, they are looking for people to help out, if you have time and can invest in it you could be one of the contributors I'm sure. Otherwise I'm sure with Windows being a platform for it someone will come along and work on either a new driver (API) or improve the existing one.
- hardwaresofton 11y ago
  From the little I know about C#, the LINQ (IQueryable?) interface is very well designed/used -- maybe some work could be done to look into exposing that sort of API? I can at least say that the C# API looks extremely consistent with rethink in other languages.
  - Analemma_ 11y ago
    >the LINQ (IQueryable?) interface is very well designed/used It's very well designed/used if someone else does the heavy lifting of creating the data provider for you. Speaking as someone who has done it, writing your own IQueryable LINQ provider is a serious hassle, though fortunately it only has to be done once. If it's something you really wanted, I would try to lean on RethinkDB to do it rather than implementing it yourself.
    - hardwaresofton 11y ago
      Oh that's what I meant -- RethinkDB is thoroughy open source though, so I'm sure if someone in the community found the time to do it, the rdb devs would give it a look and improve the code, and possibly merge.
- UK-AL 11y ago
  RavenDb?
polskibus 11y ago
Does it make any sense from licensing point of view to host database on windows if there is a Linux version available? As far as I understand each user hitting that database directly or indirectly via an app needs a CAL for that Windows Server. Outside of an enterprise (ie. Cloud) it gets veery expensive quickly. Enterprises embraced Linux long ago so not sure if rethinkdb will have much ROI on this effort.
- talles 11y ago
  I work in a big company (20k+ employees) and we are MS from top to bottom relying as least as possible on the cloud.
- newman314 11y ago
  Depends if your only options are SQL Server or Oracle (it happens a lot with enterprise software). In that case, SQL Server is often the less costly option. I would do everything possible to avoid an Oracle product based on their licensing schemes. Architecture by licensing is not a fun exercise at all.
- UK-AL 11y ago
  You don't have a cal for each user. Say you have rest api server, which talks to sql server. This doens't require a CAL for each user.
  - polskibus 11y ago
    from http://blogs.technet.com/b/volume-licensing/archive/2014/03/10/licensing-how-to-when-do-i-need-a-client-access-license-cal.aspx http://blogs.technet.com/b/volume-licensing/archive/2014/03/... : The front end servers used to host your website would generally be considered as running “web workloads” and CALs or External Connectors will not be required to access these servers. Once the customer adds a widget to their shopping cart, creates an account and enters their credit card and shipping information to complete the sale – they are now authenticated via your back end commerce servers/application (non-web workload). Since users are accessing the backend commerce servers which web workloads are not running – CALs or External Connectors will be required for users to access these back end servers.
zenlikethat 11y ago
Congratualations to the RethinkDB team on this! I'm sure it was no small feat to get RethinkDB running on Windows and between this and the Jepsen results, they are clearly doing very well right now.
- hardwaresofton 11y ago
  The only problem seems to still be name recognition, though I assume that will come with time. Not enough people know that Rethink exists, and is a fantastic choice for those looking to use a document store in their architectures.
  - ssmoot 11y ago
    Just my take: I really don't like RQL. Why couldn't it just be a document flavored SQL? Having a DSL for writing in-database procedures is fine. But tying all external queries to a Builder is, from a purely aesthetic POV, pretty ugly IMO. But that can be addressed down the line I'm sure. If nothing else by other people just building new DSLs on top of the Builder interfaces. (Note: Yes, I realize the Builder supports map/reduce as well. In practice I'm skeptical how well that works in an on-demand fashion.) Having only one fully-managed host available is a minor negative. Though we use Cloudant right now so that's a bit hypocritical perhaps. Lack of integrated full-text search makes things (application-side) a lot more complex than Cloudant or a traditional RDBMS with integrated full-text search. Maybe not in a "look what I can do" prototype-ish phase, but definitely in a "what would Jepsen do to my system" sort of scenario (see the "Warning" here: https://www.rethinkdb.com/docs/elasticsearch/ https://www.rethinkdb.com/docs/elasticsearch/). That means you're going to want to routinely compare and reindex. Which ugh. That's just my 2c on why I'm not super excited about RethinkDB. Doesn't seem a whole lot to differentiate it from other NoSQL databases outside of being a better MongoDB. It doesn't integrate full-text-search like Cloudant. It doesn't have transactions. It's got this streaming thing. Which seems like a pretty niche feature. But maybe that's just me. It is free though, so that's definitely something. If I were a cash-strapped startup that might be enough to move it near the top of my list. OTOH if I'm paying for a fully managed, hosted service that matters a whole lot less. Feel free to educate me. :-)
gkya 11y ago
A little bit off topic but what is the meaning of "developer preview" in the context of an open source developer tool project? edit: speling
- coffeemug 11y ago
  Slava @ RethinkDB here. "Developer preview" means we're still going through internal testing processes. The code review is still in progress (the windows port hasn't been merged into next yet), and the build hasn't been widely tested in the wild. Everything seems stable and there aren't major known bugs, but there is still work to do before we can label the windows port production ready.
  - gkya 11y ago
    A developer preview is a prerelease of a user software package that's potentially a platform for other software, targeted for developers who would potentially build applications for the package's user base. An OS like windows may have development previews, as tgerr would be developers who would want to start early to develop apps on tge platform. Sublime text may release a development preview in order to allow developers of plugins to start early to update thir plugins to the new version. But a database is not a user application, it is a developer's tool or a component of an application. Thus a developer preview of a thing that will never directly face a user is not logical. You want to say release candidate or simply beta instead. Developer preview implies extensible user application.
    - vvpan 11y ago
      It's a preview and it's for developers.
      - gkya 11y ago
        Bravo, you're smart enough to read. Now try to think. I thought that the phrase was misleading, and I gave feedback. No need to join in with a cool slap to me.

161 points by coffeemug 11y ago

many of the underlying design decisions in libuv clash with decisions that we’ve made in the RethinkDB core Can you expand on this please?

atnnn 11y ago
RethinkDB uses custom abstractions to hide the lower-level OS-specific code. We were able to port the low-level code to Windows with very little changes in these abstractions, and thus minimal changes to the rest of the code base. The proper way to use libuv would be to replace those abstractions with libuv's own, which would affect a lot more code and possibly break a lot of implicit invariants. In retrospect, perhaps we should have given alternatives like libuv, libevent or boost::asio more consideration.

dylz 11y ago

your mascot is adorable.

SamReidHughes 11y ago
Wait 'til it puts on its robe and wizard hat!
- dylz 11y ago
  haha, it's more that it's rare that I see such detail put into ui/ux/branding in many/most products, especially ones that aren't geared toward common end users. big kudos :)

giancarlostoro 11y ago

This is one of those databases I wanted to eventually arrive on Windows, I really enjoyed the look of it and wanted to try it out, and although I have used from time to time Linux as my main OS I just don't feel comfortable using a database I can't run on multiple platforms, it seems like a barrier not being able to use it locally from Windows, although Vagrant and vagrant boxes like Scotch Box are making this "issue" a little less relevant these days, it's still nice to be able to run a database without installing an entire OS. Thanks for the work and time invested in bringing Rethink into Windows, it is truly appreciated.

iheartmemcache 11y ago

I love how they're reaching out to C# users (and conversely I love how MS is reaching out to Linux and OS X users by opening sourcing their CLR, giving away their IDE, integrating Node.js and JSX as first-class citizens, etc!) but Rethink-ers if you're reading this - I briefly perused your source code and to a C#'er (who has written his fair share of JS, both in the "Visual Studio ASP.NET and SharePoint" style, as well as the "node.js" style) -- this doesn't read like idiomatic C# at all. I'm sure it works well, but the API feels like someone who writes JS for a living was given a deadline to add this feature and tacked it on half-heartedly. There's no reason in this day and age that you should be have any deps on stuff like Cygwin, which you claim you don't: >> You won’t find any POSIX compatibility layers or other similar hacks–RethinkDB uses native Windows APIs on the Windows platform. It looks like you do[1] from the whole Cygwin section and the env var's you have to add to PATH (in a somewhat inelegant way). That dependency "tree" is... extensive to say the least and a lot of it can be eliminated or eased with a combination of NuGet and chocolatey. Having in your instruction notes to build Goog's RE2: >> Build it somehow (I forgot to record how) [2] doesn't really make me too confident that you're going to support Windows as a first party platform. The API exposed to the end-user should be conventionally written in a more LINQ-y way - I.e. Fluent style code should be written with Object.Blah().Filter(x => x % 2).ToList() or whatever. Just little idiomatic things like that are pretty pervasive. Regardless, thanks for investing the time on working for this. As a curmudgeon and someone who spends his spare time reading journal papers in the ACM about concurrency models from hardware transactional locks to ICFP papers which implement similar concepts on the exact opposite side of the spectrum, it's my duty to be wary of new technologies and slow to adopt them into production-- but RDB and VoltDB are two products I really think are technically sound. Contact me if you want a more thorough critique to make this more idiomatic, because your product really is great! [1] https://github.com/rethinkdb/rethinkdb/blob/a90bb0805160362125a03024eb9f309e578408c0/WINDOWS.md https://github.com/rethinkdb/rethinkdb/blob/a90bb08051603621... (Errata: see edit) [2] See [1]: RE2 Edit: I was referring to this revision of Windows.md, which at the time was what was available on their master branch. My critique largely revolves around that version. Proceeding my post, another branch's version was swapped in, to a more clean version in which the build phases of the dependencies are removed, and replaced with pre-compiled binaries as you now see: https://github.com/rethinkdb/rethinkdb/blob/next/WINDOWS.md https://github.com/rethinkdb/rethinkdb/blob/next/WINDOWS.md This solves the aesthetics issue but just pushes the problem into another domain. You're using binary deps (some of which are cygwin'd afaik, but I haven't taken IDA to it so I can't say definitively) but if you want to build from "source" a la the FreeBSD ports way -- where every single .c or .cc is available for one to see and modify as their environment demands -- the problem's still there just pushed under the rug.

mizchief2 11y ago

"We launched an ambitious engineering project to port RethinkDB to Windows–an undertaking that required a year of intensive development, touching nearly every part of the database." What a waste of time.

pjmlp 11y ago

As mostly a Windows developer nowadays, many thanks and congratulations.

talles 11y ago

Not in love with MongoDB, DocumentDB is Azure only. Finally a (great) document database option for me (I work in a .NET shop). Kudos to you guys. But... but... what's with that C# API? It doesn't look like typical C# at all.

giancarlostoro 11y ago
Well, they are looking for people to help out, if you have time and can invest in it you could be one of the contributors I'm sure. Otherwise I'm sure with Windows being a platform for it someone will come along and work on either a new driver (API) or improve the existing one.
hardwaresofton 11y ago
From the little I know about C#, the LINQ (IQueryable?) interface is very well designed/used -- maybe some work could be done to look into exposing that sort of API? I can at least say that the C# API looks extremely consistent with rethink in other languages.
- Analemma_ 11y ago
  >the LINQ (IQueryable?) interface is very well designed/used It's very well designed/used if someone else does the heavy lifting of creating the data provider for you. Speaking as someone who has done it, writing your own IQueryable LINQ provider is a serious hassle, though fortunately it only has to be done once. If it's something you really wanted, I would try to lean on RethinkDB to do it rather than implementing it yourself.
  - hardwaresofton 11y ago
    Oh that's what I meant -- RethinkDB is thoroughy open source though, so I'm sure if someone in the community found the time to do it, the rdb devs would give it a look and improve the code, and possibly merge.
UK-AL 11y ago
RavenDb?

polskibus 11y ago

Does it make any sense from licensing point of view to host database on windows if there is a Linux version available? As far as I understand each user hitting that database directly or indirectly via an app needs a CAL for that Windows Server. Outside of an enterprise (ie. Cloud) it gets veery expensive quickly. Enterprises embraced Linux long ago so not sure if rethinkdb will have much ROI on this effort.

talles 11y ago
I work in a big company (20k+ employees) and we are MS from top to bottom relying as least as possible on the cloud.
newman314 11y ago
Depends if your only options are SQL Server or Oracle (it happens a lot with enterprise software). In that case, SQL Server is often the less costly option. I would do everything possible to avoid an Oracle product based on their licensing schemes. Architecture by licensing is not a fun exercise at all.
UK-AL 11y ago
You don't have a cal for each user. Say you have rest api server, which talks to sql server. This doens't require a CAL for each user.
- polskibus 11y ago
  from http://blogs.technet.com/b/volume-licensing/archive/2014/03/10/licensing-how-to-when-do-i-need-a-client-access-license-cal.aspx http://blogs.technet.com/b/volume-licensing/archive/2014/03/... : The front end servers used to host your website would generally be considered as running “web workloads” and CALs or External Connectors will not be required to access these servers. Once the customer adds a widget to their shopping cart, creates an account and enters their credit card and shipping information to complete the sale – they are now authenticated via your back end commerce servers/application (non-web workload). Since users are accessing the backend commerce servers which web workloads are not running – CALs or External Connectors will be required for users to access these back end servers.

zenlikethat 11y ago

Congratualations to the RethinkDB team on this! I'm sure it was no small feat to get RethinkDB running on Windows and between this and the Jepsen results, they are clearly doing very well right now.

hardwaresofton 11y ago
The only problem seems to still be name recognition, though I assume that will come with time. Not enough people know that Rethink exists, and is a fantastic choice for those looking to use a document store in their architectures.
- ssmoot 11y ago
  Just my take: I really don't like RQL. Why couldn't it just be a document flavored SQL? Having a DSL for writing in-database procedures is fine. But tying all external queries to a Builder is, from a purely aesthetic POV, pretty ugly IMO. But that can be addressed down the line I'm sure. If nothing else by other people just building new DSLs on top of the Builder interfaces. (Note: Yes, I realize the Builder supports map/reduce as well. In practice I'm skeptical how well that works in an on-demand fashion.) Having only one fully-managed host available is a minor negative. Though we use Cloudant right now so that's a bit hypocritical perhaps. Lack of integrated full-text search makes things (application-side) a lot more complex than Cloudant or a traditional RDBMS with integrated full-text search. Maybe not in a "look what I can do" prototype-ish phase, but definitely in a "what would Jepsen do to my system" sort of scenario (see the "Warning" here: https://www.rethinkdb.com/docs/elasticsearch/ https://www.rethinkdb.com/docs/elasticsearch/). That means you're going to want to routinely compare and reindex. Which ugh. That's just my 2c on why I'm not super excited about RethinkDB. Doesn't seem a whole lot to differentiate it from other NoSQL databases outside of being a better MongoDB. It doesn't integrate full-text-search like Cloudant. It doesn't have transactions. It's got this streaming thing. Which seems like a pretty niche feature. But maybe that's just me. It is free though, so that's definitely something. If I were a cash-strapped startup that might be enough to move it near the top of my list. OTOH if I'm paying for a fully managed, hosted service that matters a whole lot less. Feel free to educate me. :-)

gkya 11y ago

A little bit off topic but what is the meaning of "developer preview" in the context of an open source developer tool project? edit: speling

coffeemug 11y ago
Slava @ RethinkDB here. "Developer preview" means we're still going through internal testing processes. The code review is still in progress (the windows port hasn't been merged into next yet), and the build hasn't been widely tested in the wild. Everything seems stable and there aren't major known bugs, but there is still work to do before we can label the windows port production ready.
- gkya 11y ago
  A developer preview is a prerelease of a user software package that's potentially a platform for other software, targeted for developers who would potentially build applications for the package's user base. An OS like windows may have development previews, as tgerr would be developers who would want to start early to develop apps on tge platform. Sublime text may release a development preview in order to allow developers of plugins to start early to update thir plugins to the new version. But a database is not a user application, it is a developer's tool or a component of an application. Thus a developer preview of a thing that will never directly face a user is not logical. You want to say release candidate or simply beta instead. Developer preview implies extensible user application.
  - vvpan 11y ago
    It's a preview and it's for developers.
    - gkya 11y ago
      Bravo, you're smart enough to read. Now try to think. I thought that the phrase was misleading, and I gave feedback. No need to join in with a cool slap to me.