4 ms·

It's a hard one to do.I agree you can do that now. I'm thinking on a solution, but it's not really easy. Right now I have no solution.

untog 10y ago
It shouldn't be too difficult - surely right now you have some sort of task that scrapes the Hacker News API to present the stories? Generate the thumbnail then, store it on S3.
cvrajeesh 10y ago
Another solution I can think of is - encrypt the URL with a symmetric key. Decrypt it to generate the thumbnail
- valentinvieriu 10y ago
  Thank you for the suggestion. I'm not so familiar with this. Will this work on frontend too? If I expose the secrets on the fronted, then it's pointless. Do you have some suggestions on how to do this? Appreciate the help! It's an important aspect and a very resource intensive process that I need to protect.
  - j1436go 10y ago
    Your decryption/encryption secret (key) is stored on the server side only and if a client requests a thumbnail, you can validate the URL value by successfully decrypting it with this secret and probably do a simple HTTP URL validation check of the decrypted value. And you have to encrypt all thumbnail URLs with this secret before passing them to the client.
  - filearts 10y ago
    To the extent that you are pulling new stories directly from Firebase on the client side, encrypting (or signing) urls with a symmetric key will not work. This is because you would need to embed the secret in the front-end code which means the secret is no longer secret ;-) You could also tail the Firebase feed from a server process, generate the relevant images and only serve images for pre-existing urls.
tedmiston 10y ago
You could your scraper service a unique token that random people won't have so they get 401ed.
striking 10y ago
Do what https://hackernewsfilter.com/ https://hackernewsfilter.com/ does, use a Bloom filter. It's simple enough.
- valentinvieriu 10y ago
  Thank you for this idea. This seems like a quite friendly one. So you are using the Firebase API, and each time a website is added you add i to the Bloom filter? This seems it will work for my website, but then the screenshot api can be use only if you previously pushed that url in the bloom filter
mwambua 10y ago
Setting up a same-origin policy (https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy https://developer.mozilla.org/en-US/docs/Web/Security/Same-o...) would prevent people from embedding the image in other websites. You could try generating a unique token for each page-view, and use this as an access-token for all other resources pulled by the page. The reality is that it's not too many people who will misuse your API... so don't pick a mechanism that will place undue load on your server until you really have to.

untog 10y ago

It shouldn't be too difficult - surely right now you have some sort of task that scrapes the Hacker News API to present the stories? Generate the thumbnail then, store it on S3.

cvrajeesh 10y ago

Another solution I can think of is - encrypt the URL with a symmetric key. Decrypt it to generate the thumbnail

valentinvieriu 10y ago
Thank you for the suggestion. I'm not so familiar with this. Will this work on frontend too? If I expose the secrets on the fronted, then it's pointless. Do you have some suggestions on how to do this? Appreciate the help! It's an important aspect and a very resource intensive process that I need to protect.
- j1436go 10y ago
  Your decryption/encryption secret (key) is stored on the server side only and if a client requests a thumbnail, you can validate the URL value by successfully decrypting it with this secret and probably do a simple HTTP URL validation check of the decrypted value. And you have to encrypt all thumbnail URLs with this secret before passing them to the client.
- filearts 10y ago
  To the extent that you are pulling new stories directly from Firebase on the client side, encrypting (or signing) urls with a symmetric key will not work. This is because you would need to embed the secret in the front-end code which means the secret is no longer secret ;-) You could also tail the Firebase feed from a server process, generate the relevant images and only serve images for pre-existing urls.

tedmiston 10y ago

You could your scraper service a unique token that random people won't have so they get 401ed.

striking 10y ago

Do what https://hackernewsfilter.com/ https://hackernewsfilter.com/ does, use a Bloom filter. It's simple enough.

valentinvieriu 10y ago
Thank you for this idea. This seems like a quite friendly one. So you are using the Firebase API, and each time a website is added you add i to the Bloom filter? This seems it will work for my website, but then the screenshot api can be use only if you previously pushed that url in the bloom filter

mwambua 10y ago

Setting up a same-origin policy (https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy https://developer.mozilla.org/en-US/docs/Web/Security/Same-o...) would prevent people from embedding the image in other websites. You could try generating a unique token for each page-view, and use this as an access-token for all other resources pulled by the page. The reality is that it's not too many people who will misuse your API... so don't pick a mechanism that will place undue load on your server until you really have to.