10 ms·
Amazon CloudSearch - Start Searching in One Hour for Less Than $100 / Month
- jtchang 15y agoYou know this doesn't seem like a bad deal though $100/mo might be high for someone just starting out. Right now my options for search are: Full text SQL search Apache Solr or something similar Google Search Appliance Custom search Google free search on your site Yay for search as a service.
- zackattack 15y agoYeah, there's a side project I've been wanting to build for a long time, and it needs search. But the way these prices have been presented, it seems that CloudSearch is just not economically feasible for a SaaS / free multiuser offering.
- Ecio78 15y agoI'm in the same situation, and I'm thinkin about using Solr in a vps or trying one of those hosted cloud solr solutions
- nl 15y agoCloudSearch is just not economically feasible for a SaaS / free multiuser offering Are you saying it is too much? WebSolr etc have options that are cheaper.
- Ecio78 15y agoLooking for hosted Solr solutions I found also: http://www.opensolr.com/ http://www.opensolr.com/ http://www.solrhq.com/ http://www.solrhq.com/ http://www.midwesternmac.com/services/hosted-solr-search http://www.midwesternmac.com/services/hosted-solr-search http://www.netaphorsearch.com/products/solr-hosting http://www.netaphorsearch.com/products/solr-hosting http://www.acquia.com/products-services/acquia-network/cloud-services/acquia-search http://www.acquia.com/products-services/acquia-network/cloud... (they're mainly targeted to drupal) anything else?
- edouard1234567 15y agohttp://websolr.com/plans http://websolr.com/plans
- Ecio78 15y ago"also" = additional to the already cited websolr ;)
- friendlytuna 15y agoAnd there's also LucidWorks Cloud. http://www.lucidimagination.com/products/lucidworks-search-platform/cloud http://www.lucidimagination.com/products/lucidworks-search-p... Lucid Imagination is run by some of the most experienced Solr and Lucene devs. Specifically Yonik Seeley, who created Solr.
- nl 15y agoCloudSearch is just not economically feasible for a SaaS / free multiuser offering Are you saying it is too much? WebSolr etc have options that are cheaper.
- hkarthik 15y agoSolr or Sphinx based searching is what pushed me out of most of the PaaS offerings and into my own VPS. It's unfortunate that most of the hosted services out there are too expensive for side projects. When I looked at WebSolr, the cost exceeded my entire VPS structure, even for their cheapest plans.
- nzadrozny 15y agoWe think our prices actually compare pretty well to self-hosting. Hosting a search engine is not cheap when it comes to memory and disk IO, and I don't envy anyone trying to shoehorn production Solr traffic into a small VPS. Not to mention, we've got transparent replicated redundancy on all our indexes—one of our better-kept secrets, I really need to update our marketing materials—so double your VPSs there.
- hkarthik 15y agoAfter setting Solr up, I definitely see the value that you guys provide for high traffic sites. But there's a difference between production level needs and hobbyist level needs for a side projects. For side projects I'd be okay with indexing being less aggressive, sizes being more restrictive, and response times being higher if that made the pricing more accessible. I'd be happy to give you guys more money when I've got the traffic to justify it, but it'd be nice to be able to flip that on when I need it.
- Ecio78 15y agoYes, but the billing is hourly based as usual: You'll be billed based on the number of running search instances. There are three search instance sizes (Small, Large, and Extra Large) at prices ranging from $0.12 to $0.68 per hour (these are US East Region prices, since that's where we are launching CloudSearch). PS It looks like it's initially available only in US East Region
- johns 15y agoThere's also: Searchify, Running the full open sourced IndexTank Search as a Service API HoundSleuth, IndexTank Compatible API IndexTanktoGO, IndexTank Compatible API Bimaple, IndexTank Compatible API IndexDen, IndexTank Compatible API
- nzadrozny 15y agoYou seem to like IndexTank ;) There's also my own http://websolr.com/ http://websolr.com/ running Apache Solr. Some other Solr services are mentioned elsewhere on the page. I've also recently launched http://bonsai.io/ http://bonsai.io/ for a hosted ElasticSearch service. Because ElasticSearch is actually quite awesome (and I'm happy to answer questions about why). For Sphinx, there's Flying Sphinx (by Pat Allen of Thinking Sphinx Ruby client fame, great guy), and IndexDen (which is Sphinx, not IndexTank).
- StavrosK 15y agoSo, why is ElasticSearch awesome, apart from being a searchable document/JSON store? That's pretty obviously awesome :P
- nzadrozny 15y agoThere's a lot to be said for ElasticSearch's data distribution. It does sharding and replication really well. That makes my life easier, as a service provider, as well as the life of anyone that has to manage and scale an ES cluster. Or who doesn't want to have to deal with client-side sharding or worry about how many servers can crash before their search goes down. Here's a good video on the subject from ElasticSearch's creator: http://vimeo.com/26710663 http://vimeo.com/26710663 ElasticSearch has very little ceremony around creating a new index and getting started with using it. You will eventually need to do some configuration to tune its behavior for your specific application, but the learning curve is nice and gradual. This makes ES great for exploration. The JSON document store aspect of ElasticSearch is indeed very nice. The RESTful API is simple enough that you don't really need a client, just grab your favorite HTTP client library and start integrating. Plus, coupled with solid distribution, you're looking at a pretty viable standalone data store, IMO. Also, very good documentation. And its user/developer community is all full of the really smart, enthusiastic early-adopter types right now :) Not least, Lucene itself is hands down the last word when it comes to search.
- rb2k_ 15y agoAnd if you want a hassle free scalable search service with automatic sharding/scaling, a lucene underpinning and a nice REST API: Elasticsearch
- deleted 15y ago[deleted]
- espeed 15y agoIndexTank is now open source and the code is here (http://engineering.linkedin.com/open-source/indextank-now-open-source http://engineering.linkedin.com/open-source/indextank-now-op...).
- Mikushi 15y agoAnd if you want an actually good search product there is always SphinxSearch.
- damncabbage 15y agoAnd for the joke entry: Yahoo! BOSS (http://developer.yahoo.com/search/boss/ http://developer.yahoo.com/search/boss/)
- intended 15y agolooks like some people would want help in converting document X into a - "JSON or XML that conforms to our Search Document Format (SDF)" This is still going to be a painfull task for legacy data - it has to be massaged into shape. Should be interesting to see how this gets applied though.
- spullara 15y agoWhy in the world would transforming your structured data into XML or JSON be a painful task? Essentially every application built is massaging data from one format to another, this being one of more straight-forward transformations. Is building an RSS feed hard? Seems about the same level of difficulty.
- Maxious 15y agohttp://tika.apache.org/ http://tika.apache.org/ "detects and extracts metadata and structured text content from various documents using existing parser libraries". I use it all the time for input to solr.
- Terretta 15y agoThat doesn't seem to be included in the "in an hour" part of the process. I can see a lot of small businesses with S3 tools on the desktop getting excited about the ability to search their office document store, then discovering there's a whole lot of programming to do first.
- benmccann 15y agoI was pretty interested by this announcement as I spent this week setting up elasticsearch, which also uses a JSON interface. However, the main difference to me is that elasticsearch will allow you to use nested documents, which is far less restricting.
- staunch 15y agoFeeling for the guys who started IndexTank replacements and other Search-aaS companies. Infrastructure is a poor place to be with AWS around. Just a matter of time until they offer every low level service.
- negamax 15y agoMoreover they keep reducing their costs as they grow capacity. Before AWS all server hosts used to revise their price upwards.
- diego 15y agoThey can have one advantage over AWS: customer service. That's what made IndexTank, we would have never made it as user-friendly without our close relationship with our users.
- nzadrozny 15y agoAbsolutely agreed. My cofounder and I have real conversations with real websolr customers every day, who are thrilled to have access to search experts. Not just customer service, too, but end to end developer experience can be a huge advantage. I'm personally very active in maintaining the popular Sunspot Ruby library for Solr, and have passing familiarity with the internals of half a dozen other clients as well. This makes for improvements at all levels of the stack. While the CloudSearch API looks a bit more reasonable than the recently released DynamoDB, Amazon has traditionally been somewhat poor in terms of end-to-end developer experience.
- cannuk 15y agoI can definitely attest to IndexTank's fantastic customer service making a difference. We used the service for our startup. I remember dealing with Ignacio. Not only did he take the time to help us when we were using the free service, he took the time to look at our site (and give us a much needed ego boost by calling it a good idea). He took the time to understand the idea and figure out just what we needed. That is not going to happen with AWS...
- mthreat 15y ago
- joshu 15y agoanyone know what it is underneath the covers? elasticsearch?
- mryan 15y agoIt says it is based on the same search that powers Amazon.com, so presumably it is A9: http://a9.com/ http://a9.com/
- hboon 15y agoJust like they said AWS is the infrastructure that powers Amazon when AWS started out. I'd read this with a pinch of salt.
- superfork 15y agoThis is false. Amazon only recently started using+moving some of its services/fleets to AWS. To this day, a vast majority run on dedicated fleets.
- deleted 15y ago[deleted]
- aphexairlines 15y agoAmazon never claimed this.
- peyton 15y agoHere's the initial announcement of EC2: http://aws.amazon.com/about-aws/whats-new/2006/08/24/announcing-amazon-elastic-compute-cloud-amazon-ec2---beta/ http://aws.amazon.com/about-aws/whats-new/2006/08/24/announc... > run on Amazon’s proven computing environment Yes, Amazon never explicitly stated, "We run our site on a fleet of EC2 instances, and you should too," but they certainly weaseled a connection with their main site that didn't exist.
- 15y ago
- cbg0 15y agoIt would be interesting to see how this performs, compared to running Lucene, Solr, Sphinx, or what-have-you on an EC2 instance with equal resources.
- mryan 15y agoThis could replace SOLR for one of my sites - I'll need to do some benchmarks to compare them when deciding if it makes sense to move. If so, I can probably post CloudSearch:SOLR comparisons at least.
- huoju 15y agoLucene and Solr eat too many memory, they are not fit to vps users. Cloudsearch is more cheaper than setup a delicate server for search.
- Ecio78 15y agoit seems that the latest Solr 3.5.0. release use less memory: Bug fixes and improvements from Apache Lucene 3.5.0, including a very substantial (3-5X) RAM reduction required to hold the terms index on opening an IndexReader http://lucene.apache.org/solr/solrnews.html http://lucene.apache.org/solr/solrnews.html
- karmi 15y agoRunning ElasticSearch on AWS is pretty easy, see http://www.elasticsearch.org/tutorials/2012/03/21/deploying-elasticsearch-with-chef-solo.html http://www.elasticsearch.org/tutorials/2012/03/21/deploying-...
- zargath 15y agointeresting, I will check out for sure. Any plans for EU release? I am very interested in the facet-search functionality, anybody know if there is sort options on the returned facet's ? Most search engines just sort facets by number of hits.
- edouard1234567 15y agoYou can sort facets in your application, not sure why you'd want the search engine to sort them, doesn't seem like a feature that belongs to the "back-end" search engine.
- jahewson 15y agoSay a query returns a million results, it makes much more sense to sort them on the search server and return the top 10 than to transfer them all to the application server and sort them there. Another use case is a custom ranking function which boosts the score of newer documents.
- nickm12 15y agoIf you want to sort by anything other than the number of hits in every facet, then there is information about the facets that only the search engine really knows. For instance, which facet holds the most relevant results?
- edouard1234567 15y agoThey claim to support realtime indexing. I wonder what they really mean by that and how it impacts performance. SOLR is a great piece of software, I start using it 5 years ago and more recently implemented a near realtime indexing(publishing) integration but I always had to make some kind trade off between high performance and near "realtime indexing"
- stdbrouw 15y agoIt says "near-real time", which is really just a synonym for "fast".
- fellowshipofone 15y agoI am super excited about this announcement, but for now CloudSearch seems to be supporting only 3 types of indexes (http://docs.amazonwebservices.com/cloudsearch/latest/developerguide/configureindexfields.html http://docs.amazonwebservices.com/cloudsearch/latest/develop...), so no geo-search probably and other filters that might be welcome in upcoming releases
- alexro 15y agoIndexTank had one appealing option: it allowed to change associated statistics, like number of votes, without re-indexing the document. It also allowed to dynamically use ranking functions. Also, other solutions allow indexing in different languages, I can't find this in the CloudSearch.
- codexon 15y agoHow can you change statistics like number of votes without reindexing? I don't see how that is possible unless that field wasn't indexed.
- mthreat 15y agoThis would be stored in a numeric value. IndexTank (and Searchify which runs IndexTank) stores numeric values in a normal array in RAM. So an update is essentially equivalent to: array[index] = updatedValue; And then you can sort or filter by these numeric values, as well as the usual text searching. More info in the docs here: http://www.searchify.com/documentation/python-client#document-variables http://www.searchify.com/documentation/python-client#documen... Note that if you update a text field, it does a normal reindex (although it's true real-time in IndexTank).
- marathe 15y agoQuick write-up here: http://webdev360.com/amazon-finally-moves-into-search-with-new-aws-tool-41966.html http://webdev360.com/amazon-finally-moves-into-search-with-n... Interesting to note that Pando Daily reported this as a rumour almost three months ago (though they got the announcement date drastically wrong: http://pandodaily.com/2012/01/17/good-news-for-ec2-customers-amazon-may-launch-new-cloud-search-tomorrow/ http://pandodaily.com/2012/01/17/good-news-for-ec2-customers...
- brainless 15y agoDoes this literally kill the Search as service provides like IndexTank, Unbxd etc.? It becomes a really tough market with AWS doing the same (~very similar) service.
- tybris 15y agoLiterally kill? I certainly hope not.
- paraschopra 15y agoI'm sure it won't "literally" kill anyone.
- edbloom 15y agoVery interesting. I have a project that I'm about to start which is going to have an index of 4 million plus data records where I need high performance faceted search. I was exploring using Solr but may now give this a look as I'm planning on putting the app on EC2. Are there any technical details/tutorials on how their facets are configured?
- mthreat 15y agoHope you'll excuse the shameless plug, but check out Searchify's hosted search. It covers the requirements you mention - easy-to-use, fast faceting. http://www.searchify.com/documentation/tutorial-faceting http://www.searchify.com/documentation/tutorial-faceting
- deleted 15y ago[deleted]
- cpg 15y agoTheoretical, fun, exercise for the reader: How much would it cost in AWS fees to index the same amount as Google and make it available to search? The answer is in two parts: 1) "fixed" cost to upload the data (say in one shot) and 2) the hourly/daily/monthly) cost to make these search instances running.
- mfringel 15y agoThis sounds like the way you get to creating a Web Commerce Server-as-a-Service. Pricing and Inventory goes into the cloud, and you get all the standard retail searchability "for free" i.e. dis-contiguous price searches, etc.
- sdfjkl 15y agoHaving added my own "site search" for an existing project using django-haystack + Xapian, I was surprised how easy the whole implementation was (spent about a day on it, plus a bit of tweaking). Of course that was a comparatively tiny index.
- matan_a 15y agoIt's nice to see Amazon entering this space as they do have the expertise to keep this up and running as well as provide the scale needed as apps grow. What i always tho is the ability to run search queries that also involve dynamic grouping (like grouping by random combinations of facets) and providing those aggregated results. Only thing i've seen that can do this "on the fly" is SenseiDB. CloudSearch/Solr/etc seem to need preprocessing to get this right.
- uptown 15y agoCan anyone recommend the best book or books to learn about the Amazon cloud stuff? I do better with books than online docs, and am coming from a VPS background.