3 ms·

You're right, Anubis sets a week-long cookie: > exp: The token's expiry week after the token was issued This is surprising to me because it effectively nullif

You're right, Anubis sets a week-long cookie: > exp: The token's expiry week after the token was issued This is surprising to me because it effectively nullifies Anubis. > Bots don't maintain session cookies to avoid rate limits Maybe they don't today, but there's absolutely nothing stopping them adding this, so if Anubis gets any traction, in two days' time they will.

kmeisthax 2y ago
You can turn Anubis into Proof of Storage by appending large amounts of random data to the session cookie and then hashing it on each request. Combined with per-session-cookie limits, you can effectively force the LLM bots to store all the data you want if they want to scrape your site. Maybe suckerpinch can work it into a sequel to Harder Drive: Hard drives we didn't want or need[0]. [0] https://www.youtube.com/watch?v=JcJSW7Rprio https://www.youtube.com/watch?v=JcJSW7Rprio
- akoboldfrying 2y ago
  Great clip! I love the idea of making internet miscreants reluctantly store your data! Slightly more seriously though, I think for the Proof of Storage idea to pack enough punch to be a deterrent, you'd need the cookies to be quite large. Is there a way to avoid needing them to send you all the bytes each time? Because that will cost you (the site owner) too. I had the idea of sending requestors a challenge like "Flip the bit at position i of the large chunk of stored data, hash the result and include the hash in your headers". Instead of the site owner keeping the full stored data, they would just keep the RNG seed used to generate it -- this saves on storage, though it still requires them to do the same time-consuming hash computation done by the requestor.
fc417fc802 2y ago
Anubis remains effective because the token is subject to a rate limit and you could additionally limit token lifetime to some maximum number of requests if you wanted to. All of these factors (total requests, rate of requests, associated IPs, associated browser fingerprints) tie in to detecting bad players, who should receive more frequent and larger challenges.

kmeisthax 2y ago

You can turn Anubis into Proof of Storage by appending large amounts of random data to the session cookie and then hashing it on each request. Combined with per-session-cookie limits, you can effectively force the LLM bots to store all the data you want if they want to scrape your site. Maybe suckerpinch can work it into a sequel to Harder Drive: Hard drives we didn't want or need[0]. [0] https://www.youtube.com/watch?v=JcJSW7Rprio https://www.youtube.com/watch?v=JcJSW7Rprio

akoboldfrying 2y ago
Great clip! I love the idea of making internet miscreants reluctantly store your data! Slightly more seriously though, I think for the Proof of Storage idea to pack enough punch to be a deterrent, you'd need the cookies to be quite large. Is there a way to avoid needing them to send you all the bytes each time? Because that will cost you (the site owner) too. I had the idea of sending requestors a challenge like "Flip the bit at position i of the large chunk of stored data, hash the result and include the hash in your headers". Instead of the site owner keeping the full stored data, they would just keep the RNG seed used to generate it -- this saves on storage, though it still requires them to do the same time-consuming hash computation done by the requestor.

fc417fc802 2y ago

Anubis remains effective because the token is subject to a rate limit and you could additionally limit token lifetime to some maximum number of requests if you wanted to. All of these factors (total requests, rate of requests, associated IPs, associated browser fingerprints) tie in to detecting bad players, who should receive more frequent and larger challenges.