18 ms·
A search engine for searching books in the Z-Library index on the IPFS network
- jesusofnazarath 4y agolooks great. Is there an api for something like this so i can make it an extension of my terminal?
- baptiste313 4y agoOf course there are examples on their [GitHub](https://github.com/zlib-searcher/zlib-searcher#original-search-api https://github.com/zlib-searcher/zlib-searcher#original-sear...)
- hosh 4y agoYou can self-host it: https://github.com/zlib-searcher/zlib-searcher https://github.com/zlib-searcher/zlib-searcher
- mdaniel 4y agoWow, I had no idea GitHub releases would tolerate such a huge release asset: https://github.com/zlib-searcher/zlib-searcher/releases https://github.com/zlib-searcher/zlib-searcher/releases (index_0.6.zip is 1.48GB and index_0.5.zip is 1.75GB) It seems their API docs do talk about 502s during upload, but don't otherwise specify a limit: https://docs.github.com/en/rest/releases/assets?apiVersion=2022-11-28#upload-a-release-asset https://docs.github.com/en/rest/releases/assets?apiVersion=2... The irony of not hosting them on IPFS does not escape me :-/
- Nican 4y agoI like reading about IPFS, but I do not really have the time to learn about it and get involved. Last I remember, Z-Library was having an issue scaling the DHS to handle the number of files [1]. Did those issues get resolved? How is it going now? Also, is there anything being done to ensure every file has seeders? [1] https://news.ycombinator.com/item?id=33716560 https://news.ycombinator.com/item?id=33716560
- steponlego 4y agoWorks great, that's old news which is no longer current.
- ComputerGuru 4y agoZFS (Zettabyte File System) is very much separate from Z-Library (and its alternative frontends like libgen).
- Nican 4y agoThanks. My brain crossed wires between Z-Library and IPFS. I updated my post.
- ComputerGuru 4y agoCompletely understandable!
- survirtual 4y agoI’ve dived deep into IPFS and built several prototypes on top of it. It ended up not being performant enough for me, and that was after heavily modifying the codebase so that it was true p2p browser & server (their webrtc transport had a lot of issues and they didn’t seem too interested in it, but my needs required it as a backbone). The security was also a concern, and the scaling had issues. Pinning millions of small items got so slow it would not function. Then I ended up having concerns over hashed based addressing being easy to censor with the architecture IPFS was using (more hub & spoke than anything, given signaling and relay servers were centralized). I could go on but I ran into so many issues I ended up implementing my own solution that did everything I wanted. Wanted to squeeze even more performance, I’ve been converting that solution to Rust. This was a couple years ago so maybe things have changed since I used it. Last I checked, they seemed busy on Filecoin. The idea of IPFS is great and I want to see it succeed, but I think that they got so caught up in their jargon and modularity, the project lost track of some fundamentals.
- 4y ago
- nestorD 4y agoI just did a quick test, it could do with slightly fuzzier searching ('Epub' as an extension got me no result while 'epub' did, a drop down menu with options might be better/simpler for some fields) but, otherwise, it seems functional and useful.
- Cort3z 4y agoIt seems case sensitive on some (or all?) of the input fields. At least for the author field in some of my searches.
- ComputerGuru 4y agoFWIW https://libgen.rs https://libgen.rs already has good search for basically the same books. I like the denser layout that makes better use of whitespace than this HN submission does.
- layer8 4y agoHas LibGen already caught up to Z-Library? I thought that was still very much a work in progress.
- makeworld 4y agoYou can use https://annas-archive.org/ https://annas-archive.org/
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- mohamez 4y agoI think the best thing about libgen is their detailed filter to search books with covers which makes it really easy to scan books you are looking for assuming you know the cover pages.
- deleted 4y ago[deleted]
- altilunium 4y agoBut there's no IPFS link there.
- ComputerGuru 4y agoThere is, after you select mirror1, mirror2, etc.
- moffkalast 4y agoFantastic, I knew something would come up eventually after the old site was taken down.
- jarboot 4y agoLoving the simplicity of a typescript frontend + rust backend
- SpelingBeeChamp 4y agoAnyone know of a similar search engine that allows the user to filter search results by year?
- staunch 4y agoI'm looking forward to some new AI system making it possible to generate high quality audiobooks from epub files. This "AI book reader" doesn't exist yet, right?
- rockemsockem 4y agoWorking on this now, cherry-picked audio from such AI systems abound, but submitting arbitrary text for audio synthesis requires a much higher quality bar to be listenable. Planning to have something out early 2023.
- o_____________o 4y agoWhat's the best off the shelf solution you've found? I've been trying a few different apps to read my queued articles to me, but all of the voices are awful.
- therealdrag0 4y agoI wrote my own script to send article text through GCP text to speech, which I then put on a private podcast feed. If you go through the voices there’s some decent ones. I also found that speeding up audio in podcast app greatly improves listen ability. I also find picking an AI voice with an accent (for me British or aussi or Indian) helps trick my brain into tolerance. However I also got tired of manually extracting text and running my script so now I mostly use: https://www.listenlater.fm https://www.listenlater.fm though on this I just use the “Matthew”. For 5-10 min articles it doesn’t really matter, you get used to the voices so long as you can do something else in the meantime lol.
- leobg 4y agoiOS: Voice Dream Reader. Use Ivona “Amy” or “Joey”. Also allows annotations such as comments and highlights and exporting your annotations.
- o_____________o 4y ago
- mordae 4y agoIt would be even cooler if it found books of "Stanisław Lem" with the correct name as well as its ASCII equivalent "Stanislaw Lem".
- mdaniel 4y agoYou may want to try building it locally without their AsciiFoldingFilter and see if it behaves as you wish: https://github.com/zlib-searcher/zlib-searcher/blob/0.6.0/crates/zlib-searcher-core/src/tokenizer.rs#L16 https://github.com/zlib-searcher/zlib-searcher/blob/0.6.0/cr... I don't know tantivy well enough to know if it'd be possible to put the folded in one field and the literal in another and then use the search syntax to choose between them
- deanc 4y agoIs the catalog on IPFS different to the catalog available via one of the libgen front-ends? I just performed a search on both for one title and it found it right away on a libgen front-end, but not here.
- deleted 4y ago[deleted]
- derrida 4y agoI had the opposite... a book I was looking for on a lib gen front end a few days ago but could not find came up here (the Narada translation of Dhammapada if curious)
- satvikpendem 4y ago> libgen front-ends Libgen is not the same as Z-library, there are many books on Z that are not available on libgen.
- Thorentis 4y agoWhere does "Z" exist as an entity, and how does one add books to it, or figure out what books belong to Z or not?
- DoItToMe81 4y agoZ-library began as a Libgen mirror but split off and stopped sharing uploads. It's kind of scummy, but I guess that just means people need to put more effort into uploading its contents back to Libgen.
- tmpburning 4y agoCan't wait for someone to come up for a solution to copyright that will benefit everyone.
- ROTMetro 4y agoSo, you mean, the current copyright solution? That has seen creation of works skyrocket? Or do you mean a system where people can steal what they want because 'reasons' and 'they want to pay less because xyz'? Do you propose a system that takes ownership from authors and redistributes their works maybe? Why shouldn't owners get to set the pricing? Just because people want to steal doesn't mean it's ok to. Just because people don't like the price doesn't mean they should get to set it. The current system not working would look something like no works being released (because the author felt it wasn't worth it). The fact that people put in so much effort to steal these works shows the current system is in fact working. People wanting to steal does not equal not working. Authors can set their prices to free, but funny, they choose not to (as is their right).
- yucky 4y ago> Or do you mean a system where people can steal what they want because 'reasons' and 'they want to pay less because xyz'? If you have a copy of something, and I make a copy of it, and you still have your copy of it, what was stolen? If I find a book in the library and take pictures of each page, is that also theft? What if I write all the words down? What if I use a hand scanner and OCR to copy and transcribe it, is that theft?
- tadfisher 4y agoThe obvious response is that you stole the author's income, by obtaining a copy of their work without compensating them for it. So the obvious retort is that you may not have bothered to purchase the author's work, forgoing your opportunity to read it, thus the author did not lose their sale. The conversation will then turn to defending the existence of copyright as an incentivization to produce valuable works worth reading, and then to how the publishing industry eats those incentives anyway. In the end, no one will exit the conversation having gained any insight.
- benevol 4y agoWell, nice work. But poor authors...
- deleted 4y ago[deleted]
- pixelatedindex 4y agoIf you like a book, buy it. I do. I love physical books. But, my interests are varied enough that I need multiple sources - sometimes I’m only mildly curious. Libby/Overdrive has been a lifesaver in this regard. I would love to have a digital library of my physical books though, and that’s where this is coming in handy because I don’t want to buy it twice, especially when the ebook almost costs the same as the physical one. It would be nice if there was a “for ${1..5} more, get the digital one too!” - but then there’s DRM to fight with also.
- thinkmcfly 4y agoI get the feeling that poor authors stand to lose the least from this kind of thing
- Nemo_bis 4y agoThere is no evidence that piracy hurts authors. It might hurt publishers (although finding evidence of that has been hard too).
- lxgr 4y agoAnd you are absolutely sure that there is no causal relationship between publishers getting paid and authors getting paid? Yes, the status quo is ridiculous, with libraries having to limit "copies" of ebooks and even simulating wear and tear of these on one side, and having to deal with DRM buying books on the other. But according to at least one author that I‘d trust in the matter, publishers also provide valuable services to them, such as taking the financial risk of the first print run (including editing, marketing etc.) and paying authors upfront for it. That said, I really hope that we‘ll see an "iTunes store for ebooks" (open format, no DRM) sooner rather than later.
- brunoqc 4y agoIs that web site accessible with ipfs?
- FounderBurr 4y ago[flagged]
- ROTMetro 4y agoI hope every project you work on has it's source code stolen resulting in failure. And this is coming from a pretty crappy person, but this website sure does love stealing from authors (so that it can enjoy the fruits of their labor).
- xtian 4y agoNo one is entitled to a particular form of revenue. We live in an attention-based economy. Talented and savvy authors can find any number of ways to derive revenue from the attention their work attracts apart from charging IP tolls. If not, they can always get a job where they have no ownership stake in the fruits of their labor, like almost every other job and 99.9% of people writing code for money.
- misnome 4y agoThis exact argument can be used to justify both POV. No one is entitled to have any particular piece of media available to them for free, either? Talented and savvy consumers can find any number of ways to make money to pay for media, like 99.9% of other consumers.
- xtian 4y agoExcept that making a living as an author or via another creative pursuit is a luxury and privilege premised upon the hard labor performed by countless others which actually reproduces our society. It’s also not really a matter of entitlement. It’s just an objective fact that trying to gatekeep knowledge behind IP toll bridges is a losing battle which only serves to hinder the development of many realms of human endeavor.
- ROTMetro 4y agoYou want to kill the goose so you can consume the eggs. You want to steal their products but condemn the creators. You must not know creative types and how much most of them sacrifice to try and create from the passion inside them.
- cratermoon 4y agoA question about z-lib, libgen, and regular libraries Suppose I check an e-book out of my local library. Because Reasons, the library doesn't "own" unlimited "copies" of the book, so each "copy" can only be checked out to one patron at a time and if there are enough holds then a patron can't renew a checked out book. In short, ebooks in libraries are just like regular paper books, except to you don't have leave your house. I had this ebook on hold for three months because it's very popular, but finally have it on my device, but I only have 3 weeks before I can no longer read it because of holds. I get about 2/3rds the way through before lock closes on the bits on my device. It's a very popular book, it will be 3 months before I can read it again. I go online, use some service, find the exact same book, down to all but the locking bits, and download it, finish reading it a few days later, and then forget about it. I might delete it later if I need room. Is that a crime? Have either the author or publisher lost money? On the one hand, I can say yes because if the libraries purchased more "copies", the waiting list wouldn't be so long, I might have been able to renew it and finish it. On the other hand, I wasn't going to buy the book myself, and the library has to balance budget and demand, so they probably wouldn't purchase additional "copies". What ethical questions do authors and readers see here?
- juuular 4y agoYou might find this interesting: Reverse engineering yet another ebook format (Nemanja Mijailovic) https://mijailovic.net/2022/12/25/hkpropel/ https://mijailovic.net/2022/12/25/hkpropel/
- mdaniel 4y agoWith discussion: https://news.ycombinator.com/item?id=34125958 https://news.ycombinator.com/item?id=34125958
- cratermoon 4y agoI did read that. The format he had wasn't really locked down, it was just a regular epub behind a crappy web-based viewer. The library epub downloads really do have DRM locks.
- 4y ago
- Spacemolte 4y agoI miss the recommendations on the old zlib. Anyone know of any alternatives?
- jerojero 4y agoSomething that has always bothered me about z-lib and libgen and so on is the impossibility of being able to tell quality from the different uploads. Sometimes the epubs are very good, with a good cover and metadata as well as chapters and no weird aberrations; other times, they're unreadable. But the only thing you have to judge quality is size and general description of the upload which in this website it's been reduced so everything looks the same. I think the website looks good, but yeah, would be nice if we could rate quality in some way.
- ok123456 4y agoLarge pdf and djvu files generally means low quality scans.
- akiselev 4y agoI think that really depends on the type of book. I've found the large PDFs to be by far the most reliable for books printed before the 20th century because - save for Google Books - no one's OCR is tuned for the kind of printing errors, age related wear and tear, and fonts used over the centuries. Manuscripts (which are admittedly very niche) often have to be high resolution scans or photographs just to be readable, but zlib and libgen don't really have many of those.
- zozbot234 4y agoSurely books published before the 20th century are in the public domain by now, though?
- akiselev 4y agoThey are but someone has to be in possession of the book and go through the trouble of digitizing it. Usually it’s museums, libraries, and private collections and a variety of enthusiasts eventually upload from those sources to Zlib/libgen. Since no one cares about copyright there anyway it can be more complete in some genres than other central databases.
- Piezoid 4y agoWe downloaded libgen sql and zlib sql and exported the necessary data from them. Where does this "zlib sql" come from? Anna's pilimi-zlib2-index?
- rand846633 4y agohttps://libgen.is/dbdumps/ https://libgen.is/dbdumps/
- pilimi_anna 4y agoI believe it is indeed our index. :) Hurray for open source and open data!
- altilunium 4y ago> I hope everyone have a copy of the index locally, so that no need to rely on any centralized service. Is this a static index? Or maybe there's further additions and update in the future?
- xbmcuser 4y agoI think with this you need torrents of the index if too many people start downloading the indexes would put extra load on your servers
- nyolfen 4y agothis is the result of the annas-archive people mirroring the contents of their crawling project onto ipfs -- huge props
- renaissance_tea 4y agoIs it dangerous to host sites like this? I love making tools like this but worry about the repercussions
- Zuiii 4y agoSeeing that the US managed to extradite a British citizen from his own country for hosting links, I would definitely say so. The right way is to do this is to host this on tor or similar and only advertise it on the internet behind a good vpn. It's ridiculous that this is what people need to do to help others, but that's what ridiculous laws and circus courts get you.
- dredmorbius 4y agoWhich case are you referring to?
- lioeters 4y ago> Britain has authorised the extradition to the United States of a student who created a website allowing people to watch films and television shows for free, the interior ministry said on Tuesday. > ..[The 23-year-old student] allegedly earned tens of thousands of pounds (dollars) through advertising on the TVShack website before it was closed down by US authorities. > The student would be the first British citizen extradited for such an offence. He faces jail if found guilty of the charges, which were brought after a crackdown by the US Immigration and Customs Enforcement agency. > His lawyer had argued in court that the website did not store copyright material itself and merely directed users to other sites, making it similar to Google. > The lawyer also argued that his client was being used as a "guinea pig" for copyright law in the United States. https://phys.org/news/2012-03-britain-student-extradition-website.html https://phys.org/news/2012-03-britain-student-extradition-we...
- dredmorbius 4y agoThanks.
- altilunium 4y agoI wonder why the very concept of public library is not illegal at the first place.. I mean, imagine this concept is being popularized at first in the modern era. I'm pretty sure there's a huge opposition, using the very same argument as digital pirates & AI art debate : "It could damage author's income! People will just go read it for free in public library instead of buying it! Public library is morally wrong and should be burned!!"
- tredre3 4y ago> imagine this concept is being popularized at first in the modern era. I'm pretty sure there's a huge opposition Video rentals have been popularized in the modern era. Netflix mail rentals have been popularized in the even more modern era. I'm sure some copyright holders argued that they should be burned down, but clearly there wasn't a "huge opposition" to it?
- gnarbarian 4y agoI don't know how old you are but there was initially a very large amount of opposition to digitizing media delivery over traditional methods for fear of piracy. It was a huge problem for Netflix.
- vouaobrasil 4y agoThat is a crude rendition of the AI art debate, and also of piracy. A public library buys the book. Many people can read it, but it guarantees one purchase, and often many libraries will buy the book. It also limits the amount of readers to those who live relatively nearby. Books were also traditionally written with the ideas of libraries in the author's mind, and also many authors benefited themselves from libraries. AI Digital art has many facets which make it detestable for some: it floods the market with stuff that can be generated at near zero cost, it weakens the connection between people because people are no longer looking at something another human created, it cheapens the spirit of creation. While libraries, Library Genesis, and piracy might have some facets in common, they are separate entities and should not be conflated in any sense.
- fud101 4y agoSuper fast. What's the tech stack?
- deleted 4y ago[deleted]
- Thorentis 4y agoWhat exactly is Z-Library right now? The domains were seized, and there doesn't seem to be a way to access the library or add new books to it. Is it now simply a collection that exists alongside libgen? Is it being run elsewhere? Does the Ipfs mirror get updated from somewhere?
- netfortius 4y agoThere is a discord server now, where you will find all your answers, plus some.
- Thorentis 4y agoWhere does one find an invite to it?
- netfortius 4y agoUnfortunately posting invites in a large community like HN could only endanger the well being of the discord site - wish there was a DM option in HN (that I know of ... maybe there is?!?)
- bscphil 4y agoIf that's the case, then providing the answers should be pretty easy. Can you tell us what they are?
- netfortius 4y agoSorry - missed this. "The" answers to almost all the questions in this thread could not be consolidated in a single post. Looking in the dedicated channels on the discord server dedicated to this topic will allow searching for all related info. Unfortunately posting invites in a large community like HN could only endanger the well being of the discord site - wish there was a DM option in HN (that I know of ... maybe there is?!?)
- Zuiii 4y agoI've always been able to access it via tor which is operating normally. Just be sure to confirm a disposable email address with them or they'll never let you in. Registering an account by itself is no longer enough when using tor and their website will block you and not tell you why.
- ROTMetro 4y agoMy buddy has nudes of his wife on his phone. I want them so I bypass his 'drm' that prevents me from taking what I want and make copies. He still has his copy so no one's hurt. Information wants to be free. I take what I want.
- mr-pink 4y agoffs is there any way people can just not talk about z-library for a year..what is wrong with you all? this is a public forum. you should not share any copyrighted material without the rights to do so
- danbuildrthe333 4y agoDid a test, this is cool!
- Shadowgamer195 4y agoIt is difficult to determine the quality of content on websites like z-lib and libgen because the only available indicators are size and a general description of the upload, which can make all content appear similar and leave users unable to differentiate between high and low quality content.
- uwagar 4y agojesus cloudflare getting in the way.
- unraveller 4y ago>You do not have access to zlib.zu1k.com. The site owner may have set restrictions that prevent you from accessing the site. I think I hit the cf limit after 10 searches.
- bertman 4y agohttps://github.com/zlib-searcher/zlib-searcher https://github.com/zlib-searcher/zlib-searcher
- Jamie9912 4y agoYou mean, the owner has configured Cloudflare to aggressively restrict access
- Jamie9912 4y agoAccess denied Error code 1020 You do not have access to zlib.zu1k.com. The site owner may have set restrictions that prevent you from accessing the site.
- iyedgivehead 4y agoThe site is down
- jraby3 4y agoI got an error when visiting zlib.zu1k.com/. Error code: 1020 Ray ID: 7819e5389c898e4e Country: IL Data center: tlv03 IP: 85.65.187.66 Timestamp: 2022-12-30 09:54:40 UTC
- SanjayMehta 4y agoSame here. Was working a few hours ago.
- 0ct4via 4y agoIt's possible they were "hugged" enough to instigate a firewall rule dramatically cutting down allowed visitors, which would explain the Error 1020. Might be worth reaching out to the site's creator -- info taken from their site [https://zu1k-com/about/ https://zu1k-com/about/]: "If you have any questions to communicate with me, the preferred way is to comment directly in the comment area of the corresponding blog post, and I will receive an email notification In addition, you can also reach me in the following ways: Email: i@zu1k.com Twitter: @zu1k_lv GitHub: @zu1k Keybase: @zu1k Matrix: @zu1k:mozilla.org My PGP public key can be obtained at https://pgp.zu1k.com https://pgp.zu1k.com or hkps: //keyserver.ubuntu.com ( 2A65 F6F3 1EDA D922 D7E7 E97B AE38 1A8F B1EF 2CC8)"
- neop1x 4y agoPeople should stop over-using CloudFlare for everything. IMO it is not good for IFS "hosting", either. They act like gatekeepers. Along with blocking, as in this case, they make browser assumptions that force visitors to use one of the most recent mainstream "approved" and unmodified browsers. For other browsers, they serve "Checking if the site connection is secure" with javascript fingerprinting and a captcha, and sometimes even solving the captcha won't let me in. i have also seen frontends misbehaving due to them not expecting cloudflare error response codes. Some dvelopers apparenly just blindly enable cloudflare and don't care. :(
- nukemandan 4y agoreported https://github.com/zlib-searcher/zlib-searcher/issues/57 https://github.com/zlib-searcher/zlib-searcher/issues/57
- causality0 4y agoA Z-library who's who would be useful. Is there any relation between the original operators and the IPFS effort? What about the new .to domain that's as far as I can tell identical in functionality to the original z-library?
- sfusato 4y agohttps://zlibrary.to/ https://zlibrary.to/ (to save a few guesses for interested minds)
- marcol1n0 4y agoI got an error when visiting zlib.zu1k.com/. Error code: 1020 Ray ID: 781a36f1b9b0babd Country: IT I got an error when visiting zlib.zu1k.com: Error code: 1020 Ray ID: 781a37c3aeacbae1 Country: IT Data center: mxp03 IP: 31.157.83.62 Timestamp: 2022-12-30 10:51:01 UTC
- baptiste313 4y agoIndeed the site seems to be down but you can use this site as an alternative: - https://zlib.freedit.eu https://zlib.freedit.eu Or else the project is opensource you can install it by yourself: - https://github.com/zu1k/zlib-searcher https://github.com/zu1k/zlib-searcher
- deleted 4y ago[deleted]
- bootyclub 4y agohttps://zlib.zu1k.com/ https://zlib.zu1k.com/
- Alifatisk 4y ago”You do not have access to zlib.zu1k.com. The site owner may have set restrictions that prevent you from accessing the site.” What did I do?
- baptiste313 4y agoNothing wrong only the site owner block it temporarily surely with the large number of requests since sharing otherwise I shared in a previous comment another site to access the tool: - https://news.ycombinator.com/item?id=34184167 https://news.ycombinator.com/item?id=34184167
- lannisterstark 4y agoI wish IPFS actually worked as well as they describe it working.
- trogvj 4y ago[dead]
- terrycody 4y agoI think it lacks a lot of data, before when I search "head first" on Z-library, pretty sure it got a ton of results, and now only 5 pages. What's the reason behind this?
- deleted 4y ago[deleted]
- web3d3v 4y agoSite appears to be down. Is there a mirror or open source repo ?
- throwaway923857 4y agoThe link is no longer working: zlib.zu1k.com’s DNS address could not be found. Diagnosing the problem.