11 ms·
Vespa.ai is spinning out of Yahoo as a separate company
- cfors 3y agoCan anybody speak to how Vespa compares to some other Vector Database solutions? Seems like there's so many options today
- jkb79 3y agoDisclaimer, I work on Vespa. If you look for just pure vector similarity search, there are many alternatives. But Vespa's tensor support, multi-vector indexing and the ability to express models like colBERT (1) or cross-encoders makes it stand out if you need to move beyond pure vector search support. Plus, for RAG use cases, it's a full blown text search engine as well, allowing hybrid ranking combinations. Also with many pure vector databases like Pinecone, you cannot describe an object with more than one vector, if you have different vector models for the object, you need different indexes, and then duplicate metadata across those indexes (if you need filtering + vector search). 1 https://blog.vespa.ai/pretrained-transformer-language-models-for-search-part-3/ https://blog.vespa.ai/pretrained-transformer-language-models...
- ComputerGuru 3y agoYou might want to add a \ before the asterisk.
- deleted 3y ago[deleted]
- spullara 3y agoThey have absolutely the best take on embedding search + normal faceted search. Here is a blog post about how it works: https://blog.vespa.ai/vespa-hybrid-billion-scale-vector-search/ https://blog.vespa.ai/vespa-hybrid-billion-scale-vector-sear...
- SpaceNoodled 3y agoSo what exactly does Yahoo do these days?
- supportengineer 3y agoMail and Finance
- fermentation 3y agoSeems really popular in Japan for some reason.
- toddmorey 3y agoNTT investment/ partnership
- ilickpoolalgae 3y agoI don't think the Japanese entity and US entity have any relation to each other anymore. I think US entity divested itself of Yahoo Japan completely in 2018 or so. Yahoo Japan is now collectively owned by Softbank and Naver under some holding company (it was big news in Japan when the merger happened).
- mathrawka 3y agoThey actually rebranded from "Yahoo! Japan" to "Line Yahoo" https://www.lycorp.co.jp/ja/ https://www.lycorp.co.jp/ja/
- lmm 3y agoIt's not just a rebranding, they've merged with Line.
- mathrawka 3y agoIt's a bit more nuanced than that. The merger was awhile ago, ZHoldings was the parent company of Yahoo! JP and Line. ZHoldings is now Line Yahoo. The footer of yahoo.co.jp has changed from Yahoo! Japan / ZHoldings to Line Yahoo.
- agloe_dreams 3y agoWhat a weird name for a product. Even they need to tell you which Vespa they are. Like, when naming things...maybe not use a name that people use to define a category. (AKA: A Vespa is to Scooters as a PostIt is to sticky notes)
- ginko 3y agoI'm surprized they haven't gotten a nasty letter from Piaggio yet.
- tannhaeuser 3y agoVespa is the Italian (and Latin) word for wasp, not sure Piaggio can claim ownership outside of the domain of commercial vehicles under idk EU or WTO rules.
- bartread 3y agoAnd also pick something that's distinctive and easily Googleable. I'm tired of companies and products picking super generic names that mean I get swamped with spammy SEOd results every time I search for information/help/reviews/etc on them.
- deleted 3y ago[deleted]
- boredumb 3y agoI've always wanted a self driving scooter.
- vinaypai 3y agoTIL: Yahoo still exists in 2023
- orangepurple 3y agoYahoo search delivers superior search results for tech related queries despite supposedly using Bing
- lofaszvanitt 3y agoPlus the best finance page.
- matwood 3y agoAnd fantasy sports.
- sundarurfriend 3y agosuperior search results as in better than Google? superior in any specific way or just generally more relevant results?
- danielheath 3y agoKagi has shown me that “better than google” is not even particularly difficult, if such a small team can do it.
- orangepurple 3y agoSuperior technical results relative to Google
- cscurmudgeon 3y agoStill one of the top website
- m4jor 3y agoMayer didn't kill it off 100%, only like ~95%.
- sph 3y agoPiaggio is not gonna be happy that someone has called their company Vespa.
- rmccue 3y ago“vespa” means wasp in Italian, and it seems unlikely that they’re in the same trademark categories.
- simonebrunozzi 3y agoTry telling that to Apple.
- jollofricepeas 3y agoLittle known fact… It is believed that Apple paid the Beatles a little over $500 million for Apple Corp trademark rights so they could market “Apple Music” https://en.m.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer https://en.m.wikipedia.org/wiki/Apple_Corps_v_Apple_Computer
- quickthrower2 3y agoSuperb rent seeking and squatting by Apple Corps
- rmccue 3y agoAside from the famous case of Apple Corps as noted by jollofricepeas, there are multiple other trademarks for “Apple” as a single word in the US, including for tobacco, horseshoes, books, and travel.
- autoexec 3y agoWasps are a nuisance known for causing pain and property destruction. "vespa" as a name for a product always seemed strange to me. At least the scooters were vaguely waspish looking and buzzed around town. What's yahoo's excuse?
- simonw 3y agoVespa has been around for a LONG time. When I worked at Yahoo back in 2005-2007 the Flickr team were using it to scale Flickr - they were integrating it in order to serve things like public tag pages, to take the load off their MySQL cluster. I remember understanding it as the internal equivalent to something like Solr, although actually it predated it - looks like the first public Solr open source release (coming out of CNET) was January 2006.
- jkb79 3y agoYeah, I worked with the Flickr team on that project. Scaling to billions of photos, with partial update support of popularity for ranking. Back then, the properties had to stand up their own Vespa cluster(s), later on we created a managed service out of it. And, yes, the original plan for Vespa was to be a Vertical Search Platform, that is where the name Vespa comes from. More on the history in this blog post https://blog.vespa.ai/vespa-is-becoming-its-own-company/ https://blog.vespa.ai/vespa-is-becoming-its-own-company/
- hans_castorp 3y ago> Vespa has been around for a LONG time True. I bought my first Vespa in 1984
- zitterbewegung 3y agoWhat hasn’t spun out of Yahoo?
- weare138 3y agoYahoo could have been the next AWS if it hadn't been horribly mismanaged.
- simonebrunozzi 3y agoAnd tens of others: Sun Microsystems, even Google, etc.
- rrdharan 3y agoCould have been Google, Could have owned Google.. https://yourstory.com/2023/05/yahoos-billion-dollar-blunders-google-facebook-microsoft-netflix#:~:text=In%201998%2C%20Yahoo%20had%20the,potential%20of%20the%20search%20engine https://yourstory.com/2023/05/yahoos-billion-dollar-blunders....
- quickthrower2 3y agoCould have plundered Google.
- ramses0 3y agoMaps. Groups. Upcoming/Local. Flickr. Tech was great, groundbreaking at the time (however, that had its own drag on velocity). Product "management" of anything post-acquisition was demonstrably atrocious.
- neilk 3y agoIt definitely could not have done that. In the same year that Google had Borg, and AWS released EC2, at Yahoo, David Filo was still personally reviewing hardware requests.
- 3y ago
- neilk 3y agoVespa was actually very cool at the time as a document-oriented search engine, occupying the same niche as Solr and elaborations of that like ElasticSearch. But I don't know if it's competitive today. This blog post says it's "developed by Yahoo" which is I guess true. But it was originally an acquisition, largely developed by a team in Norway, and apparently most Vespa development still happens there.
- dathinab 3y ago> don't know if it's competitive today very, it's outcompeting most vector databases on features and maturity when it comes to vector search while having very powerful and flexible and proven text search too The tricky part is, it's more a platform to build complex search systems with then "just" a vector database. So if a found a company today which focus is to create clever multi phase search pipelines and train (e.g. domain adopt) LLMs for calculating embeddings etc. then it's probably _the_ best solution by far, you probably can get away with having only AI engines devops and a single programmer (who might also most times just do devops). But if you need to deeply integrate it into a different existing search system things are less grate. And I would love to see some modernization, like having a format for structured queries which is more widely supported then YQL... (eying graphql here)
- bratao 3y agoYeah +1 for VERY competitive. The vector capabilities of Vespa are incredible and the Text/ranking features are amazing. I don´t think any other product have those two sides so developed as them.
- deleted 3y ago[deleted]
- bayesian_limit 3y agoWe conducted benchmark tests on Elastic's queries per second (QPS) performance using datasets of 500,000 and 1 million vectors. Result was Zilliz is 13x and 22x faster, per number of vectors respectively. https://zilliz.com/blog/elasticsearch-cloud-vs-zilliz https://zilliz.com/blog/elasticsearch-cloud-vs-zilliz Feel free to explore our open-source benchmarking tool, which allows you to examine our methodology and even compare it with your vector database. https://github.com/zilliztech/VectorDBBench https://github.com/zilliztech/VectorDBBench
- fzliu 3y agoI was at Yahoo almost a decade ago, when vector search within Vespa was first being rolled out in production use cases. It was already serving similarity search requests for Flickr back then. Even though I'm with Zilliz/Milvus now, I wholeheartedly support and recommend folks check out and try Vespa. Congrats to the Vespa team! EDIT: For folks on Twitter, you should follow Jo (https://twitter.com/jobergum https://twitter.com/jobergum) from Vespa if you aren't already. Great combo of technical content, hot takes, and vector database memes!
- jkb79 3y agoThank you for the shout-out Frank!
- 90-00-09 3y agoOff topic... looking forward to more engineers moving to Mastodon. I have Twitter/X blocked at DNS level and still fairly frequently encounter interesting accounts that I can't check out.
- srameshc 3y agoEvery time I see Twitter/X here, I want to say this.
- SkyMarshal 3y agoIs there a particular mastodon server or set of servers that the engineering community is favoring? I know it technically doesn’t matter in a federated network, but curious anyway.
- deleted 3y ago[deleted]
- tristan957 3y agoI'm on fosstodon. floss.social is also big.
- 3y ago
- daveevad 3y agoAnyone able to share the definitive history of the polar bears that visited the office? https://upload.wikimedia.org/wikipedia/commons/e/e6/Polar_Bears_in_Trondheim.jpg https://upload.wikimedia.org/wikipedia/commons/e/e6/Polar_Be...
- jkb79 3y agoHehe, it was a joke, we don't have polar bears on the mainland of Norway. But, it was fun to show the photo to visitors from different countries. "Be careful when you walk back to the hotel".
- eirikref 3y agoIt had all started a bit earlier with a colleague (I assume that's you up there, jkb79 :) pulling the leg of a guy visiting us from the Bangalore office, about the polar bears roaming the streets of Trondheim. We just thought we'd prank him a bit about having more dangerous animals than they did in India. Nothing too serious. So a few months later I woke up early one morning and saw it had been snowing all night. I made sure to get over to the office before the traffic started ruining the image of barren, desolate place where you could entertain the idea of polar bears roaming the streets, looking for young, fresh engineering meat. I took a few photos, walked inside, and got busy looking for photos of polar bears that could fit into this idea of upping the Dangerous Wildlife War between India and the home of the proper vikings. So I did a little bit of photoshopping, uploaded it to my Flickr account, and sent the link to a few other yahoos around the world. We had a bit of fun with it, but I don't think anyone with access to this worldwide web of information will be too fooled. (It's quite easy to figure out that there are no polar bears on the Norwegian mainland, so I've never been too bothered about leaving it up there.) You can see the ehm... original at https://www.flickr.com/photos/eirikref/3262204184 https://www.flickr.com/photos/eirikref/3262204184 for a few of the comments and fun from back then. Edit: So that's about as definite as you'll ever get it. And I'll never again admit to it being photoshopped in any shape or form ;)
- deleted 3y ago[deleted]
- 3y ago
- peterstjohn 3y ago+1 to everybody that mentioned that Vespa has great vector support _and_ lexical filtering. And you likely will end up needing both. Don't sleep on some of its newer features like multi-vector document fields, either…
- ram_rar 3y agoWow, this brings back soo many old memories of mine @ Yahoo! . Almost a decade ago, vespa inside Yahoo was way ahead of its time. Glad, it finally spun out. I wonder how does this compare with other vector Dbs like pinecone, chroma...
- happytiger 3y agoThis is fabulous and I’m very excited to see where this project goes. It definitely has legs and if the team understands just how enormous the potential could be, annd especially why they could disrupt, I’m even more eager to see what they do.
- joshu 3y agovespa tech was one of the big reasons yahoo was attractive as an acquirer for del.icio.us. unfortunately they promptly withheld it afterwards.
- pests 3y agoThat's a name I haven't heard in awhile. I thought the concept of it was amazing and tags were the solution to everything. Remember machine tags? Oh how things have evolved. Not to mention the naming trend companies took on for a decade afterwards.
- joshu 3y agomachine tags were stupid. tagging was about humans labeling the important aspects for retrieval. not a replacement for all metadata, or for faceted search.
- gyehuda 3y agoVery excited to see this. I used to work at Yahoo. This is a serious team with serious tech.
- fsckboy 3y ago>Yahoo will own a stake in the new company and will be one of Vespa’s biggest customers for a long time to come. what's usually going on in scenarios like this is either: the subsidiary is in growth mode and needs access to public capital markets to raise large amounts for building infrastructure (and sharing the expenses with others) to compete, or what has been built is going into profitability mode and the profits are better protected/hoarded in a private company where disclosure requirements are not onerous. If this move represents the former, expect the latter in a few years, is why yahoo would maintain a large stake. (I'm just pointing this out because there are frequently underlying reasons that have nothing to do with the type of stuff that's in the press release, "successful venture, income/exit event" etc.)
- prepend 3y agoOr I suspect a third where the Vespa management team would leave and start their own company that would appreciate at a higher rate than yahoo due to greater growth potential. So yahoo spins them out and has their equity grow faster than yahoo stock, or they lose the product to a new competitor.
- fsckboy 3y agoThere's truth in what you say. I was interpreting (without studying closely) the scale of what's happening to be beyond the scale where ICs have significant leverage, so not the startup venture scale but more like B-C round. It's already clear this stuff's going to be big, and the biggest players have the most advantages, it's time to scale, quit, or pray for a proprietary technological breakthrough. But you're right about giving small equity stake holders a more pure play though equity is an expensive way to pay people.
- achow 3y agoCisco has its famous 'Spin-in' strategy. A "spin-in" is a form of R&D in which a company is the sole investor in a startup. It sends a team of employees off to build an experimental product and then buys that startup for a predetermined and very healthy price. Cisco has spent on average $763 million to acquire each spin-in that these three men have founded, even though Cisco was also the sole investor in each. https://www.businessinsider.in/why-cisco-has-showered-these-three-men-with-billions-of-dollars/articleshow/43088562.cms https://www.businessinsider.in/why-cisco-has-showered-these-...
- andrewstuart 3y agoI forgot Yahoo exists. The biggest thing I recall about Yahoo is Jerry Yang vetoing sales to Microsoft for $35B on principle because he didn't like Microsoft. Is that what actually happened or is that a false memory?
- xmprt 3y agoYahoo was massive at one point and $35B didn't seem like a lot compared to its peak valuation as well as its competitors. I can see why Jerry Yang didn't want to settle when he had seen his company at a much higher valuation in the past. I really don't know how Yahoo managed to fumble things so badly but they were always one of the better companies as far as technology goes - this announcement today is a testament to that. Just that their business and product side was lacking
- deleted 3y ago[deleted]
- ec109685 3y agoGiven Yahoo's stake in Alibaba that Jerry brokered for getting out of the China market, Yahoo's valuation actually eclipsed that $35B MSFT was going to offer ($80B): https://www.forbes.com/sites/briansolomon/2016/07/25/yahoo-verizon-sold-alibaba-jerry-yang/?sh=7f7a43b5671a https://www.forbes.com/sites/briansolomon/2016/07/25/yahoo-v... "The deal for Alibaba was expensive and risky at the time, but ended up as the most lucrative bet in Silicon Valley: at today's prices, that stake would be worth more than $80 billion. Normally, such a success would preserve a company for years, but on Monday morning Yahoo announced that it would sell its operating business—the web properties without Alibaba and other investments—to Verizon for less than $5 billion, pennies on the dollar it would have commanded at its peak 15 years ago."
- deleted 3y ago[deleted]
- cmcollier 3y agoThis is fun to see, and for a seriously deserving team! Vespa really is an impressive platform. I've been working with search platforms for a while, starting with FAST Instream, MarkLogic, Elastic, Weaviate, and now Vespa. As others have noted, Vespa has a deep technical lineage. You can see a lot of the hard earned lessons in the design, as well as improvements in the flaws of the spiritual predecessors (FAST et. al). The history alone makes the project interesting. But then, in the last couple of years (or maybe earlier), they started introducing all the fundamentals for supporting deep integration of tensors and a custom HNSW implementation. Whether accident or planning, this was a nice strategic move. They were leagues ahead, just in time for the popularization of learned embeddings. Now they have the best combination of traditional lexical search, semantic search with embeddings, and the combination of the two in hybrid search. That's without even getting to all the functionality in the multi-phase ranking, hosted ML models, and document processing engine. Obviously I'm biased (we use Vespa at work for enterprise search products), but I can't recommend this engine/platform enough. Congrats again to the team!
- loondri 3y agoI think Vespa's growth and innovation were driven by Yahoo's resources and infrastructure. Now as a separate entity, it might struggle to maintain the same pace of development and innovation without the backing of a tech giant like Yahoo.
- bborud 3y agoThis is really exciting news! What is perhaps a bit unusual is that the Vespa team has existed for a very long time. Even before the project got its name (2004/2005?). If memory serves it will be 20 years this year or next year since the first bits of what became the Vespa platform were developed. Some of the people working there have worked together since 1998-1999. I think much of the reason for the longevity of the team is that it was kept somewhat separate from the rest of Yahoo! so they could maintain focus.
- david_shi 3y agoHas anyone tried using the GPU acceleration features? Curious if it works well with speeding up vector search.