saltysalt

6 ms·

91.
by saltysalt 11mo ago
You are correct, and the hard reality is that content producers don't get to pick and choose who gets to index their public content because the bad bots don't play by the rules of robots.txt or user-agent strings. In my experience
92.
by saltysalt 1y ago
That website is delightfully old-school. Love it.
93.
The Relentless Pursuit of the Vorschlag(vomtag.com)
2 points by saltysalt 1y ago|0 comments
94.
by saltysalt 1y ago
I agree man, it's depressing.
95.
by saltysalt 1y ago
You make a fair point, I'm just hoping this will happen, but not confident either to be frank.
96.
by saltysalt 1y ago
Exactly! It's a rent-seeking model.
97.
by saltysalt 1y ago
We do, but the vast majority of users interact with centralised models from Open AI, Google Gemini, Grok...
98.
by saltysalt 1y ago
Not sure the dial-up analogy fits, instead I tend to think we are in the mainframe period of AI, with large centralised computing models that are so big and expensive to host, only a few corporations can afford to do so. We rent a computing
99.
by saltysalt 1y ago
I'm pretty certain that the only thing reading my blog these days is AI.
100.
by saltysalt 1y ago
AWS is this generation's mainframe. /joking
101.
by saltysalt 1y ago
Nailed it. The other side of the marketing hype cycle will be saner, when the market forces sort the wheat from the chaff.
102.
by saltysalt 1y ago
This is great advice: "It also means that staying the course when things don’t go your way isn’t just a virtue but a practice. To play the long game, you have to keep showing up even after crushing disappointment without getting cynica
103.
by saltysalt 1y ago
The Empire Strikes Back
104.
The greatest threat to AI adoption is hallucinations [video](youtube.com)
1 points by saltysalt 1y ago|2 comments
105.
by saltysalt 1y ago
If they suspend everyones accounts, the Internet will be a better place. “Arguing with anonymous strangers on the Internet is a sucker's game because they almost always turn out to be—or to be indistinguishable from—self-righteous sixt
106.
by saltysalt 1y ago
Just one IP for now. You are right to worry about being blocked from crawling however, it has happened to me already on a few sites. The key things to help mitigate against this are: 1. Always identify your crawler via a consistent user-a
107.
by saltysalt 1y ago
I'll try to make time soon to write a blog, and post the link here.
108.
by saltysalt 1y ago
You can achieve both in my opinion.
109.
by saltysalt 1y ago
No proxy yet, but I am considering one as many sites are re-directing my crawler based on its IP, which is causing indexing issues. The hardest part BY FAR is the crawler: initially I was using Apache Nutch but it got slower and slower as t
110.
by saltysalt 1y ago
Lucene on local disc.
111.
by saltysalt 1y ago
A single bare-metal server!
112.
by saltysalt 1y ago
Very nice!
113.
by saltysalt 1y ago
Thanks! If I have time, I should write a how-to blog.
114.
by saltysalt 1y ago
Not everyone trusts AI to filter their internet queries.
115.
by saltysalt 1y ago
I'm building my own search engine that is human-curated, as I am also uncomfortable relying on AI to filter my search results. It's free, feedback welcomed: https://greppr.org/
116.
by saltysalt 1y ago
Great comment and you are absolutely right: AI slop content will (ironically) kill AI search. In my opinion, we have gone full circle and arrived back to human curation to provide quality search results.
117.
by saltysalt 1y ago
I share the same concerns, hence built my own search engine that is human-curated. There is already a push back against "AI slop content" online, AI search will online make this worse as it ingests this slop.
118.
by saltysalt 1y ago
I built my own, it's easier than you might think: https://greppr.org/
119.
by saltysalt 1y ago
This is why I use a local NAS for offline backups.
120.
by saltysalt 1y ago
Physically books for the win.

More ›