5 ms·
I wonder what the search story is. One technology that really does deliver, and has totally impressed me, is Lucene/ElasticSearch. I'm used to all sorts of hype
by MichaelGG 12y ago
I wonder what the search story is. One technology that really does deliver, and has totally impressed me, is Lucene/ElasticSearch. I'm used to all sorts of hyperbolic claims, but holy shit, ElasticSearch just delivers. We tossed in about 40M documents from a SQL Server DB, and not only did it require less resources (a 30%? reduction in size), the queries are beyond anything that'd be approachable using SQL Server. And I've only touched the surface, using it as a pure plug-n-play setup.
With DocumentDB, not having a local version severely limits what I'd consider this for. Losing that flexibility is a big deal. Maybe this is just a limited preview and they haven't build the management side for local installs.
- curiousDog 12y agoThey released a full-text search service as well today: http://azure.microsoft.com/en-us/services/search/ http://azure.microsoft.com/en-us/services/search/
- nemothekid 12y agoI doubt MS will ever release a non-hosted version. I see this product as a competitive offering to Amazon's Dynamo and Google's BigQuery on AppEngine.
- bigtones 12y agoI think you mean a local version, or a non-hosted version.
- nemothekid 12y agoRight, edited the parent post.
- spicyj 12y agoBigQuery isn't really in the same space; it's meant for analytics queries where speed isn't as important (queries take seconds or minutes). Google Cloud Datastore is a better parallel.
- richardw 12y agoSearch API. Datastore isn't really aimed at search. https://developers.google.com/appengine/docs/python/search/ https://developers.google.com/appengine/docs/python/search/
- Aardshaark 12y agoThe Search API is pretty much for searching text, whereas this is a database. Also the Search API is terrible.
- richardw 12y agoWhile I agree the Search API isn't great, it searches text, numbers, dates and geopoints. That's good enough for most uses.
- phatboyslim 12y agoSimilarly the Azure Search appears to be a competitor to Amazon CloudSearch. I'm not aware of a Lucene based search offering from Google, however.
- johnward 12y agoI haven't used elastic search but I've used lucene/solr. It's similar to a technology our startup created which was acquired by IBM and is now part of the Watson portfolio. I think our indexing engine was faster. I know for sure we were better at getting data into the index than lucene appears to be. What are the common connector frameworks for lucene? Or do you generally just push the documents to the index?
- chippy 12y agoElasticSearch benefits from Lucenes spatial support too, which is a serious benefit.
- liamca 12y agoHi MichaelGG, I am a Program Manager for Azure Search and as curiousDog mentioned, yes along with DocumentDB, we also announced Azure Search which is a PaaS based full text search service. We actually leverage ElasticSearch at the core of this service and as chippy says about spatial search, we do have the ability to provide a pretty solid geo-spatial search capability thanks to Elastic Search and Lucene. To nemothekids's point, is it very unlikely that we will offer this as a local (non-hosted) version because we found that although ElasticSearch is awesome, one of the common complaints many admin's have is the complexity around managing systems such as ElasticSearch/SOLR/Lucene at high scale and how difficult it is to implement more advanced search capabilities such as tuning and relevancy. Those are areas we think we can add a lot of value being a fully managed service. Longer term, we think this will also allow us to bring even more value on top of search by adding in other Microsoft technologies. For example, we could tie in Bing Maps to allow you to easily tie in reverse geo-coding right into your search. Or perhaps allow you to leverage Bing's synonym list so that you could allow people to search yet find results that are synonyms to commonly searched words (i.e., user types in shoes, but in your content it is referred to as footwear). Multi-language support is actually one of the big things we want to tackle in the short term and we believe that the NLP from Office will really help jump start us with this. Liam
- tracker1 12y agoWe don't need the actual infrastructure or even the same software locally.. what's wanted/desired is an API compatible system that can be used to develop against. It also doesn't need to run on a local cluster for development purposes, or smaller shops. I know everyone is all about running huge clusters behind the scenes, but most people simply don't need that, and being able to start small, and buy bigger would be a nice option. For example, a simple node application using leveldb(levelup) for the storage interface could be very effective as a backend for development/testing. From there, you provide an API compatible interface. Open-source that version, with the disclosure/understanding that the Azure hosted version is much more robust. I think you'd see a lot more buy in from the open-source community... even more so if you accepted PRs to make the open version more robust. If you are at a point where you are considering the likes of ES/SOLR/Lucene etc, you are likely ready to make the jump to self-host in the cloud, or use a SaaS provider. Where people get a bit concerned is in the lock in. I know why Azure would want to present that, but I think it's a bad idea without an open implementation that allows for self-hosting for development and on the small scale. Right now, the company I work for is hosting in Azure. I recently switched from a couple RabbitMQ queues to Azure Queues. It works fine, as was a really simple replacement for a flow of near real-time but temporary data keys. I would be much more open to using a hosted MongoDB from MS than DocumentDB, in much the same way I'm happy to see you guys embracing Redis for a cache system.