4 ms·

Ask HN: Does Google Index “Secret” URLs Only Shared Through Gmail?

3 points by macinjosh 9y ago

I am seeing files I upload to S3 in Google search results. I am not sure how Google found the URLs to these files as I've only shared them in email (Gmail).

claudiulodro 9y ago
You should put a robots.txt file in the S3 bucket if you don't want things in it indexed. I (kind-of) doubt that they found the files through your e-mail, but the files in your S3 bucket are technically available for indexing since you don't have a robots.txt.
- macinjosh 9y ago
  I do have a robots.txt, but for whatever reason its not found by Google
  - iurisilvio 9y ago
    Are you using a custom domain? Robots.txt must be in your root folder, doesn't work with S3 urls.
    - macinjosh 9y ago
      yeah its the S3 URLs that are causing the problem
mattbgates 9y ago
Go into Incognito and see if they are showing up there. Google might show you files from Google Drive and other places because you are logged into your Google Account, but it may not show up for anyone else.
- macinjosh 9y ago
  They do show up in incognito.
  - mattbgates 9y ago
    Well then, Google is just being Google and indexing everything. You might have to delete it or check your account settings to see if you can make it private.
ParameterOne 9y ago
Google definitely reads your email (gmail). As an example, if I have the word attachment written in my email and then attempt to send the email without any attachments, google will tell me that I used the word attachment in my email but didn't attach anything.
- MichaelBurge 9y ago
  "Reads your email" is ambiguous. All unencrypted email must be read just to transmit it over the network, or to store it on disk. Sounds silly, but the example you gave is actually even more benign: It could be done with Javascript that never sends anything across the network.
ravigupta 9y ago
Am also facing the same problem.Anyone, know the answer?http://emailsupportnumber.org/yahoo-mail-customer-support-number/ http://emailsupportnumber.org/yahoo-mail-customer-support-nu...
deleted 9y ago
[deleted]
josephcole 9y ago
Do you mind if I quote a couple of your articles as long as I provide credit and sources back to your website? http://www.juegosfriv2019.org/ http://www.juegosfriv2019.org/ My blog is in the very same area of interest as yours and my visitors would certainly benefit from a lot of the information you present here. http://www.juegosfriv2017.link/ http://www.juegosfriv2017.link/ Please let me know if this okay with you. Thank you!

3 points by macinjosh 9y ago

I am seeing files I upload to S3 in Google search results. I am not sure how Google found the URLs to these files as I've only shared them in email (Gmail).

claudiulodro 9y ago

You should put a robots.txt file in the S3 bucket if you don't want things in it indexed. I (kind-of) doubt that they found the files through your e-mail, but the files in your S3 bucket are technically available for indexing since you don't have a robots.txt.

macinjosh 9y ago
I do have a robots.txt, but for whatever reason its not found by Google
- iurisilvio 9y ago
  Are you using a custom domain? Robots.txt must be in your root folder, doesn't work with S3 urls.
  - macinjosh 9y ago
    yeah its the S3 URLs that are causing the problem

mattbgates 9y ago

Go into Incognito and see if they are showing up there. Google might show you files from Google Drive and other places because you are logged into your Google Account, but it may not show up for anyone else.

macinjosh 9y ago
They do show up in incognito.
- mattbgates 9y ago
  Well then, Google is just being Google and indexing everything. You might have to delete it or check your account settings to see if you can make it private.

ParameterOne 9y ago

Google definitely reads your email (gmail). As an example, if I have the word attachment written in my email and then attempt to send the email without any attachments, google will tell me that I used the word attachment in my email but didn't attach anything.

MichaelBurge 9y ago
"Reads your email" is ambiguous. All unencrypted email must be read just to transmit it over the network, or to store it on disk. Sounds silly, but the example you gave is actually even more benign: It could be done with Javascript that never sends anything across the network.

ravigupta 9y ago

Am also facing the same problem.Anyone, know the answer?http://emailsupportnumber.org/yahoo-mail-customer-support-number/ http://emailsupportnumber.org/yahoo-mail-customer-support-nu...

deleted 9y ago

[deleted]

josephcole 9y ago

Do you mind if I quote a couple of your articles as long as I provide credit and sources back to your website? http://www.juegosfriv2019.org/ http://www.juegosfriv2019.org/ My blog is in the very same area of interest as yours and my visitors would certainly benefit from a lot of the information you present here. http://www.juegosfriv2017.link/ http://www.juegosfriv2017.link/ Please let me know if this okay with you. Thank you!