wolfgarbe

9 ms·

31.
by wolfgarbe 5y ago
This is most likely a timing problem. If you type c# fast, then the back button brings you back to the previous query. If you pause between typing 'c' and '#'. then it brings you back to the previous character only. That
32.
by wolfgarbe 5y ago
>> you were crawling news.ycombinator.com, right? No, for retrieving the Hacker News Posts we were using the public Hacker News API, which returns the posts in JSON format: https://github.com/HackerNews/API The c
33.
by wolfgarbe 5y ago
No, the preview is NOT the representation of what we have indexed. The preview ist limited to about 200 words to be compliant with fair use legislation. Indexing is limited to 1 MB per document.
34.
by wolfgarbe 5y ago
DeepHN does not copy the whole page, only the first 200 words or less. DeepHN does not copy the whole website but only a fraction of a single page of that web site. We always provide the original link. That should make it fair use. Of cour
35.
by wolfgarbe 5y ago
Thank you all for your great feedback. We will fix all the bugs and add some of the suggested features and release a next iteration of DeepHN within the next few days!
36.
by wolfgarbe 5y ago
It should, but it currently doesn't in DeepHN. We will fix this asap.
37.
by wolfgarbe 5y ago
Thank you. We will look into this and fix it asap.
38.
by wolfgarbe 5y ago
SeekStorm uses dedicated root servers with NVMe SSDs, hosted by IONOS https://www.ionos.com/servers/intel-servers
39.
by wolfgarbe 5y ago
We will look into this.
40.
by wolfgarbe 5y ago
You mean performance in terms of latency or relevancy?
41.
by wolfgarbe 5y ago
Currently the maximum size of a single document is limited to 1 MByte. That is an artificial limitation resulting from the fact, that SeekStorm does not only indexes the content of the document, but also stores the original document. Somewh
42.
by wolfgarbe 5y ago
https://seekstorm.com/about-deephn https://seekstorm.com/ https://seekstorm.com/search-api-features https://seekstorm.com/search-as-a-service-architecture
43.
by wolfgarbe 5y ago
"NIOFSDirectory and MMapDirectory implementations face file-channel issues in Windows and memory release problems respectively. To overcome such environment peculiarities Lucene provides the FSDirectory.open() method. When invoked, it
44.
by wolfgarbe 5y ago
With the crawler which is part of the SeekStorm search-as-a-service.
45.
by wolfgarbe 5y ago
If you were entering deephn.org in your browser address bar, you couldn't use the browser back button to return to the previous page. That was a bug caused by the navigation code within our dynamic HTML page. It is fixed now.
46.
by wolfgarbe 5y ago
That would definitely work. We are working on local client that could index your local PDF and Word documents. So importing emails would be a nice addition.
47.
by wolfgarbe 5y ago
Good idea!
48.
by wolfgarbe 5y ago
The crawler is a part of SeekStorm. "Well-defined": Just a guess: We are doing key text extraction, i.e. we try not to index boilerplate stuff an and menu items. As "well-defined" is within a short list item, it might be
49.
by wolfgarbe 5y ago
>> When trying to go back after clicking the original HN link, there's no response. The original HN link is opened in a new browser page/tab. The search results should remain unchanged in the previous tab. So you don't
50.
by wolfgarbe 5y ago
Not necessarily a spam link. Linked pages from older post have sometimes updated their content. Sometimes this is legitimate, sometimes spam.
51.
by wolfgarbe 5y ago
Crawling speed is between 100...1000 pages per second. We crawled about 4 million linked unique web pages The pricing for an Index like DeepHN would be $99/month: While we are indexing 30 million Hacker news posts, for DeepHN we are co
52.
by wolfgarbe 5y ago
We can add that button, if you promise not to kill ;)
53.
by wolfgarbe 5y ago
There is no secret about the benchmark, its all open source: https://github.com/wolfgarbe/LuceneBench
54.
by wolfgarbe 5y ago
Yes, it does exactly that. If you check "Web" on the left sidebar, and uncheck "Stories" it will find the posts where "friendship" occurs in the linked web pages. But we still show the title of the original hac
55.
by wolfgarbe 5y ago
We are combining auto correction (SymSpell) and auto completion (PruningRadixTrie). That is sometimes tricky. We are using a static spelling correction dictionary which does not contains the term algolia, but angola. SymSpell is very fast,
56.
by wolfgarbe 5y ago
You are right. The preview is far from perfect. We just wanted to ship early. We will continue to fix and improve.
57.
by wolfgarbe 5y ago
DeeepHN is powered by our SeekStorm search-as-a-service, which has a full-featured API: https://seekstorm.com/docs SeekStorm is intended that the user can index and search their own private data. But if there is demand we c
58.
by wolfgarbe 5y ago
We are deriving the tags from the terms and bigrams in title, text and parsed html of linked web pages. Top frequent terms per post, but only if terms are within top 65k tags per index. Stopwords are excluded.
59.
by wolfgarbe 5y ago
We have just fixed the back button bug (make sure to clear the browser cache).
60.
by wolfgarbe 5y ago
Your are right. It IS a bug, and we will fix it asap.

More ›