6 ms·

Why not try writing a search engine specifically for some category dominated by SEO spam? I like to compare search engine results and wrote this tool to ma

• Debug_Overload 5y ago
  The fact that you included Reddit, SO, Google Scholar etc is awesome (I thought it was only for main search engines). Thanks for sharing. Bookmarked.
• noduerme 5y ago
  Okay that's sort of what the !bang in DDG is for, and why it's a meta-engine. What's the blue sky ideal for a real, no-bullshit, everything search engine that doesn't fall prey to the constant flood of garbage? I have an exterminator who comes to my house every couple months, and sets up traps here, poison there. I don't have any rats in my house. I do see rats running across the yard sometimes. The exterminator explains it like this: Rat pressure. The rats overpopulate and there's "pressure" (like, uh, "memory pressure", which is also a fluid concept) so they try to get into your house more, through smaller holes, as a function of how much outside drama is going on, how many they are and how overpopulated, how scarce their food supply is, how cold it is outside, and whatever else drives rats into your house. (I love the dude who's my exterminator). Anyway, this is the same problem every search engine faces. The more surface area they expose, the more pressure they have building, the more ways people have to fake out their systems. We have to go back to the 1990s Yahoo! model. Curated content. A list of websites that are reputable. 1990s Yahoo is the future.
  • loceng 5y ago
    Creating a "trustless" search crawler, where anybody can participate, and then applying an algorithm to determine trust or value feels like it'd be a never-ending arms race - that'd require AI and extensive/expensive resources that is likely better invested in developing real trust networks and curation; curators are corruptible and regulatory capture of policy is possible if the organization is infiltrated or poorly overseen. Carte blanche opening your system up to anyone to inject data seems like the wrong foot to start off on, whereas my curating a moderator, someone I personally know and feel good about, trust to whatever level, and hiring them - ideally making sure they're someone you respect and you're someone they respect, pay them well, and at scale will be able to pay for itself; this did just bring to mind however big pharma and pharmaceutical trials structure and how that system can be/is/has been captured - and so perhaps the pressures when dealing with multi-billion dollar market categories will always lead to shenanigans if ever trying to centralize too much, not allowing for de-risking and broader resource distribution via sales/profits to more parties than the "5-star" rated products. In a thread on HN, I think it was yesterday, a few people posted about review sites where some product reviews are free - but others you had to pay for. A system to facilitate such organizations could allow a highly competitive environment, where organizations develop/build a brand - build trust for their brand as being competent and thorough - and so then over the lifetime of a reader/customer, perhaps they'll spend $1,000 buying reviews (say 333 big purchases over 40 years that you're willing to pay $3 a hit for) to make sure they're ; mind you there will be organized that could be captured to say promote one conglomerate of products over another, perhaps even regionally, but I'm beginning to think it's a necessary layer to combat the shit show that is Amazon (et al) reviews. Ideally these systems and how the reviews present the information, and how thorough - the technical depth and breadth and testing done - will help educate those who dive into using this system, which will sharpen themselves while keeping reviewers on their toes and arguably strengthening their organizations and competency as well.
    • fsflover 5y ago
      > Creating a "trustless" search crawler, where anybody can participate, and then applying an algorithm to determine trust or value feels like it'd be a never-ending arms race - that'd require AI and extensive/expensive resources Not necessarily: https://yacy.net https://yacy.net
    • noduerme 5y ago
      Legitimate question; if this is a real business model (and I believe it could be) then why the fuck does Yahoo.com look like a dead clickbait aggregator instead of, yknow, what it used to look like? i.e. FINANCE ___ [Stocks] [ etc] ENTERTAINMENT ___ [Movies] [TV] Where's that site?
      • loceng 5y ago
        Visionary leadership left the company? And arguably it lost its soul and excitement. I genuinely thought when Marissa Mayer was brought in as CEO and announced they bought Tumblr for $1.1 billion in cash, I thought she could actually turn Yahoo! around - that she understood platforms and holistic systems; perhaps she did but her hands were tied, and then they made terrible decisions like banning porn on Tumblr - so bureaucracy, politics, and arguably the ad industrial complex and "mainstream" pressures (perhaps like billing/financial system being used as a tool to suppress freedom/sexuality/porn because they've not been successful politically) were pressures her or the Board of Directors couldn't counter. Then Google came along and was a better search engine, for a time, that was a traffic leak for Yahoo! - and then Google has now devolved; I also thought Google had a good shot at competing with Facebook, but whomever's pulling the strings there, the launch of various platforms, they don't seem to understand it can take 5-10 years after the MVP of a product is launched for it to mature - but for whatever reason their executives or managers haven't been comfortable pulling the trigger, arguably because anyone with that entrepreneurial spirit just takes their idea and gets funding and owns a large portion of whatever they've done; but then you can never develop a full breadth, holistic ecosystem, that can grow into every crevice, nor as broadly, or nuanced as possible - so they're stuck being Search, Gmail, Calendar, etc. I'm quite certain I've figured out the foundational MVP facilitating an "infinitely" growing system and that would allow 3rd parties to integrate, however I have severe chronic pain that messes up my executive function, so it's difficult for me to actually self-direct and execute - I'm stuck mostly in a low activity, stream of consciousness and go-with-the-flow life of routine - otherwise I would try to launch my plans, which I've done plenty of UX/UI for, as that is simple enough that somehow bypasses higher executive function (moving a pixel and then responding via visual feeling of it isn't complex) - but organizing to turn that into adequate specs to get solid estimates or fixed price quotes for work is extremely difficult to me. On January 11th I do have a surgery that may or may not reduce my pain by 50%+, may or may not improve my executive function, ... I've even attempted to write draft "Show HN:" posts to explain what I am doing, the starting feature sets, the reasoning behind the design decisions I've made - but it just gets too complicated too quickly for me mentally then to be able to organize further or polish it. I think I have the perfect domain name for it too: ENGN (engine), what makes me smile every time I notice it in my layout/mockup of it is in the search input box it says "Search ENGN". My username on HN actually is an older incarnation of a plan I had, loceng being a short form of "local engine" - and ENGN being from engine, a name I brainstormed after Tumblr sold for $1.1 billion - and I realized that eventually I'd want to try to do my "local engine" idea but that that was too long for a brand name. Fortunately engn.com was for sale at the time, I can't remember if it was $2,000 USD or $4,000 USD - either way, not a bad price for a 4-letter .com that's pronounceable to something with meaning. I've wanted to write a book too - on health, health systems, and on these systems we're talking about here. I'm 38 now and I taught myself to program when I was 11, learned SEO at 15, evolved to design as I'm more creative and programming became mindnumbing to me, and eventually thought I'd need (or want) VC money - so I started engaging on Fred Wilson's of USV.com's blog - AVC.com - so I have plenty of self-taught experience. The problem is even going back to my shorter or longer writings, or comments of mine on HN or other, it's nearly impossible for me to try to do the organization of it all - to compile parts, etc. Maybe this surgery goes well and I can begin to do more, or maybe it doesn't; I've tried to hire people or get help over the years but 1) no one has been willing to engage enough as I'd need due to my executive dysfunction, and maybe that's a moot point as 2) it's extremely difficult for me to even manage someone or an ongoing project - whereas I could explain things and direct if people are initiating, if others are directing the conversation, then I could respond - but otherwise I can't do normal oversight and management any longer. The most accurate odds I can give that this surgery will help (piriformis syndrome, my sciatic nerve goes through the piriformis muscles, rather than around it - so there's constant compression + that's worsened with use/engagement of the muscle) is 50/50. There's a high probability that this surgery t's not related to the primary source of my pain, which is from LASIK eye surgery I did 7 years ago - I got arguably the worst of the worst symptoms: central sensitization and hyperalgesia - a hypersensitivity to pain, where all sensations, pain especially, is amplified to as what seems as strongly as possible; and why I must highly limit my activity level as any little stresses on body, likely even normal natural muscle use which causes micro-tears, then compounds the problem and will take many days of very low activity to return to a still difficult-dysfunctional baseline. But perhaps there's a high probability that the sciatic nerve having been compressed for most of my life, my mind, nervous system, could handle that level of pain/sensitization - but then the damage to the cornea that happens in 100% of LASIK surgeries was what finally broke the camels back. If this surgery doesn't go well, doesn't help - which it took me 1.5 years to even find a surgeon who does this type of specialized surgery - then I'm afraid I may end my life because this pain, the lack of productivity, of being stuck, of quite little social interaction overall - HN is likely the most stimulated my mind gets, only possible to write this fluidly when it's been at least 3 days of eating a very low inflammatory diet and very low activity level - and only if I've been mostly inactive, primarily sitting, since waking and getting out of bed - to not trigger any pain in my body. Anyway, it gets boring, repetitive. I've thought of trying to find an Elixir/Phoenix/React/etc developer or agency on Upwork before surgery and try to struggle to get them on at least developing the initial foundation of ENGN, but aside from the struggles I listed above that I'll encounter, it also will cost additional money - and I've not worked in 5 years, I've spent $250,000+ on stem cell treatments to heal old high school football injuries, that I didn't even know I mostly had and only weren't tolerable after LASIK made my nervous system super hypersensitive - and to pay for this surgery my mother is taking $27,000+ USD out of her retirement; I'm in Ontario, Canada, but the healthcare system has been practically useless to me. Even if the foundation for ENGN would cost just $5,000 to $10,000 to get the ball rolling in terms of starting to get users to signup and bringing in revenues - it's more money, but even thinking about that additional stress would put on my mother, then adds to my already overwhelmed nervous system - so there's plenty of resistance there to overcome on its own. There's also always the potential I'd somehow hire a bad contractor or agency, bad in one or many ways, and then the MVP wouldn't get finished - primarily because of my own incompetency-dysfunction, and then I will just be reminded, again, of how stuck my life is and how it barely moves forward - personally or professionally. I'm living a version of the Groundhog Day movie that keeps repeating itself, except where I'm in pain, and so far where I can tell people my story and ask as many people as possible to help and nothing happens. It's why last thing I try is this surgery, though I am supposed to do another PICL stem cell treatment - where they treat tissues inside of my neck - the first one did reduce my neck pain and migraine some - for whiplash related issues, in part from football - because they only treat one side of the tissues, not all of the tissues, the first treatment - and so the second treatment they target the remaining high yield tissues. But I'm certain I'll know after the surgery if there's any improvement or hope that my life can start to become different, and even though there is a stromal stem cell treatment that was developed at University of Pittsburgh - that had very successful human clinical trials, that were fast tracked under compassionate grounds in India, to heal/regenerate deeper corneal tissue for severe scarring and chemical burns - the ETA for it being FDA approved was 5 years to be clinically available in the US, perhaps less time before available in India, but I'll have nothing else significant treatment wise for further pain reduction to look forward to in the near term after getting this surgery - and so why I'm afraid I won't be around much longer if it doesn't help much.