3 ms·

I didn't paraphrase, I put the exact quote[1] I had copied earlier from goodreads in google and got garbage. I use only local LLM's, So if anyone has Open AI's

I didn't paraphrase, I put the exact quote[1] I had copied earlier from goodreads in google and got garbage. I use only local LLM's, So if anyone has Open AI's search tools give it a shot. [1] "Few hundred years of Western society that we have lost the ability to memorise vast amounts of information." ― Lynne Kelly, Memory Craft: Improve your memory using the most powerful methods from around the world

BeefWellington 2y ago
Interestingly: - Bing: Goodreads is third result - DDG: Paywalled NZ Herald article appears to be the only result. Of course what you've really uncovered is that Google search is the only thing that respects robots.txt: https://www.goodreads.com/robots.txt https://www.goodreads.com/robots.txt
- deleted 2y ago
  [deleted]
- Abishek_Muthian 2y ago
  No, you've uncovered that. I gave a thought on whether this could be some copyright issue with Amazon but didn't bother to check the robots.txt file, nicely done!
- atombender 2y ago
  The robots.txt does not prevent Google from scraping the quote page, which is the one that matches for me: https://www.goodreads.com/quotes/11541816-few-hundred-years-of-western-society-that-we-have-lost https://www.goodreads.com/quotes/11541816-few-hundred-years-... The robots.txt seems to exclude things which are genuinely useful to exclude, like RSS feeds.
  - BeefWellington 2y ago
    While true, it seems like every individual quote page also includes: <meta content='noindex' name='robots'> Worth noting as well, their sitemaps all 404.
fivestones 2y ago
I tried with OpenAI’s o1 model and it came up with the right answer on the second try: https://chatgpt.com/share/6774a930-1e44-8013-8591-424c3b58d81f https://chatgpt.com/share/6774a930-1e44-8013-8591-424c3b58d8...
- Abishek_Muthian 2y ago
  Very interesting, Does o1 have a cutoff date for knowlege or does it index HN in realtime?
igor47 2y ago
Open AI fails here: https://chatgpt.com/share/6770c547-2f90-8004-ba41-21bfa4d3a7f3 https://chatgpt.com/share/6770c547-2f90-8004-ba41-21bfa4d3a7... Curious about your local LLM usage -- do you have that documented, or can you recommend sources on how to get started in that domain? I self host most of my infrastructure, but not LLMs so far. Do you need special hardware? How do you interact with the LLMs? How to you keep them updated? Do you fine tune/do any training, or just of the shelf llama? Do you need to know a bunch about quantization? How fast are the responses? Can you use them in your IDE as a coding assistant? How is resource utilization?
- Abishek_Muthian 2y ago
  Update: I have documented how I run local LLMs here - https://abishekmuthian.com/how-i-run-llms-locally/ https://abishekmuthian.com/how-i-run-llms-locally/ .
- Abishek_Muthian 2y ago
  I will document it and share with you.

BeefWellington 2y ago

Interestingly: - Bing: Goodreads is third result - DDG: Paywalled NZ Herald article appears to be the only result. Of course what you've really uncovered is that Google search is the only thing that respects robots.txt: https://www.goodreads.com/robots.txt https://www.goodreads.com/robots.txt

deleted 2y ago
[deleted]
Abishek_Muthian 2y ago
No, you've uncovered that. I gave a thought on whether this could be some copyright issue with Amazon but didn't bother to check the robots.txt file, nicely done!
atombender 2y ago
The robots.txt does not prevent Google from scraping the quote page, which is the one that matches for me: https://www.goodreads.com/quotes/11541816-few-hundred-years-of-western-society-that-we-have-lost https://www.goodreads.com/quotes/11541816-few-hundred-years-... The robots.txt seems to exclude things which are genuinely useful to exclude, like RSS feeds.
- BeefWellington 2y ago
  While true, it seems like every individual quote page also includes: <meta content='noindex' name='robots'> Worth noting as well, their sitemaps all 404.

fivestones 2y ago

I tried with OpenAI’s o1 model and it came up with the right answer on the second try: https://chatgpt.com/share/6774a930-1e44-8013-8591-424c3b58d81f https://chatgpt.com/share/6774a930-1e44-8013-8591-424c3b58d8...

Abishek_Muthian 2y ago
Very interesting, Does o1 have a cutoff date for knowlege or does it index HN in realtime?

igor47 2y ago

Open AI fails here: https://chatgpt.com/share/6770c547-2f90-8004-ba41-21bfa4d3a7f3 https://chatgpt.com/share/6770c547-2f90-8004-ba41-21bfa4d3a7... Curious about your local LLM usage -- do you have that documented, or can you recommend sources on how to get started in that domain? I self host most of my infrastructure, but not LLMs so far. Do you need special hardware? How do you interact with the LLMs? How to you keep them updated? Do you fine tune/do any training, or just of the shelf llama? Do you need to know a bunch about quantization? How fast are the responses? Can you use them in your IDE as a coding assistant? How is resource utilization?

Abishek_Muthian 2y ago
Update: I have documented how I run local LLMs here - https://abishekmuthian.com/how-i-run-llms-locally/ https://abishekmuthian.com/how-i-run-llms-locally/ .
Abishek_Muthian 2y ago
I will document it and share with you.