6 ms·

I've been thinking about this for a while now. Please check out my web app to solve this problem: https://www.purplerails.com/ https://www.purplerails.com/ The

I've been thinking about this for a while now. Please check out my web app to solve this problem: https://www.purplerails.com/ https://www.purplerails.com/ The main idea is to use a browser extension to automatically save pages that you read to the cloud (including the images, stylesheets etc) in the background. Saved pages are searchable and sharable.

lingben 12y ago
sounds like evernote, if I'm mistaken, please enlighten me :)
- purplerails 12y ago
  1) saving is automatic 2) privacy-first architecture (e.g., plaintext is never uploaded, plaintext URLs are never uploaded etc)
  - lingben 12y ago
    ah, so it saves... everything? wow! that's nuts. I mean, where the heck are you going to store all of that? and it is terribly inefficient to store say, 30,000 examples of the exact same article. or do you have a way to check and not store duplicates? if so, what if a blog post is saved today and has 10 comments and it is saved tomorrow by another person and it has 11 comments. technically the page is different so it would be saved again I think you need to explain exactly how it works a bit better or maybe I'm just not getting it :)
    - purplerails 12y ago
      > ah, so it saves... everything? wow! that's nuts. I mean, where the heck are you going to store all of that? Please see my reply to idlewords. It isn't literally everything; there are some heuristics to detect what was interesting to you. I understand you might find this excessive. I routinely find that useful. :) See also "As We May Think" by Vannevar Bush. > and it is terribly inefficient to store say, 30,000 examples of the exact same article ... You're right: no deduping is and can be done across users (HMACs chained up the user's password are used create dedup hashes). Storage is sufficiently cheap that I feel it's a acceptable tradeoff vs. privacy (i.e., server being unable to confirm that two users have saved the same page).
idlewords 12y ago
An early design idea I had for Pinboard was as a browser plugin that just saved everything it saw in passing to an upstream server. But the problem that stumped me was that there's much more downstream bandwidth than upstream on a typical residential connection, so it was hard to push things to a server in anything like real time. How did you end up dealing with this issue?
- purplerails 12y ago
  Pages are saved in the background. Nothing too fancy. I dedup when uploading: that helps a lot.
  - idlewords 12y ago
    Okay, but how do you handle things like big PDFs or image gallery sites? Or pages that just pull in a lot of javascript includes? That stuff downloads in parallel, but then I would find myself trying to push it upstream through a little straw of bandwidth, sequentially.
    - purplerails 12y ago
      You're right, it takes longer to upload a page than to download it. And image gallery-like pages take long (I know because I save Imgur pages now than then :)). But in practice, this isn't a problem. There are a couple of heuristics to avoid wastefully uploading pages: the full page is uploaded only if the reader expresses "sufficient interest" in the page. Currently the heuristic is 90 seconds of continuous reading of a page, or scrolling to the bottom. If a page is read for a minimum of 10 continuous seconds then only the text of the page is uploaded. Static assets like JS files benefit from deduping: they take time to upload the first time, but subsequently processing them is much faster. Typically, people read multiple pages in a browsing session: I rapidly open many tabs and then read each one for multiple seconds. There's a debug mode in Purple Rails in which a timer counts up when I switch to a tab. I find that typically spend 100+ (usually much more) seconds on a page that I read through to the end. This is usually enough time on a residential broadband connection (I have Sonic DSL) to finish uploading a page. I also use Purple Rails on a tethered 4G connection almost everyday: uploading is slower than DSL but it works. Basically, by the time you finish reading a tab, the previous tab you read would have finished saving. Like I said, nothing too fancy.
hollerith 12y ago
I'd prefer for the pages to be saved to the hard drive of the machine running the web browser. But maybe browser extensions cannot obtain permission to do that?
- purplerails 12y ago
  I see your point. I may release installable native apps that can serve as the local storage backend for the truly paranoid. Time limitations etc: the usual. :) I anticipate that PurpleRails will be used on multiple computers and over several years. Which is why pages are sync'd to the cloud. I've adopted the current architecture because I feel that the energy barrier that needs to be overcome to persuade somebody to install an app is much higher than the one to install an extension.
zargon 12y ago
This sounded really great until I went to the website and saw that I can't use my own cloud storage, only purplerails'. As soon as purplerails disappears all my saved pages are gone. I already have this functionality with diigo and it makes me very uncomfortable not to have a copy of the data.
- purplerails 12y ago
  Excellent point. The ability to download your data in a well-documented format is coming soon. See also my reply to hollerith on a native client. Time limitations are what is preventing me from doing this. Thanks for your feedback! Hope you will use Purplerails. :)
asaddhamani 12y ago
I tried giving purplerails a shot. I use LastPass for password management, and I am not going to type in a 25/30 character password every single time I want to log into an application. I think you're going way too hard on that part. This is the first time I've had this happen to me when using a web app, and it immediately made me close the page.
- purplerails 12y ago
  Thanks for taking the time to write your feedback. I appreciate it. With PurpleRails, you will rarely be typing your password once you successfully login. It works kinda like Gmail/Facebook etc: you remain logged in for months at a time. This page explains why password saving doesn't work yet: https://www.purplerails.com/blog/saving-passwords-why-we-havent-enabled-it.html https://www.purplerails.com/blog/saving-passwords-why-we-hav... I'll see if I can do something fancier than what I do now to allow saved passwords to work. I suspect that using a Javascript-based AJAX authentication system instead of a plain-old HTML form, I can achieve the privacy goals as well as ease of use. I hope you will hang in there and use the current version until I figure out a way to fix this. Thanks!
  - asaddhamani 12y ago
    Thanks for responding. Another 'issue' I've noticed is that it logs me out if I don't have the extension installed and click on the installation button, which, as you might figure, is a pain to deal with when you have a 25 character password. I guess you're saving the web page data from the client side and not through your servers. Another thing I've noticed is that when I add a page, it takes quite some time to show up in the web app. Also, whats with the timer? Apart from these issues, I really believe in your idea, and I have been working on making such a system for myself for months. I wish you the best, and I hope you succeed. I'll update you if I find any new issues, since if all the issues are fixed, I can see myself using this as my primary bookmarking service.
    - asaddhamani 12y ago
      I see it also saves content from emails and all that. That certainly justifies the need for extensive security. What are your plans for pricing? I suggest you don't take the route that Pocket did, $5 for anything and everything(which seems like a norm these days) is ridiculous.

lingben 12y ago

sounds like evernote, if I'm mistaken, please enlighten me :)

purplerails 12y ago
1) saving is automatic 2) privacy-first architecture (e.g., plaintext is never uploaded, plaintext URLs are never uploaded etc)
- lingben 12y ago
  ah, so it saves... everything? wow! that's nuts. I mean, where the heck are you going to store all of that? and it is terribly inefficient to store say, 30,000 examples of the exact same article. or do you have a way to check and not store duplicates? if so, what if a blog post is saved today and has 10 comments and it is saved tomorrow by another person and it has 11 comments. technically the page is different so it would be saved again I think you need to explain exactly how it works a bit better or maybe I'm just not getting it :)
  - purplerails 12y ago
    > ah, so it saves... everything? wow! that's nuts. I mean, where the heck are you going to store all of that? Please see my reply to idlewords. It isn't literally everything; there are some heuristics to detect what was interesting to you. I understand you might find this excessive. I routinely find that useful. :) See also "As We May Think" by Vannevar Bush. > and it is terribly inefficient to store say, 30,000 examples of the exact same article ... You're right: no deduping is and can be done across users (HMACs chained up the user's password are used create dedup hashes). Storage is sufficiently cheap that I feel it's a acceptable tradeoff vs. privacy (i.e., server being unable to confirm that two users have saved the same page).

idlewords 12y ago

An early design idea I had for Pinboard was as a browser plugin that just saved everything it saw in passing to an upstream server. But the problem that stumped me was that there's much more downstream bandwidth than upstream on a typical residential connection, so it was hard to push things to a server in anything like real time. How did you end up dealing with this issue?

purplerails 12y ago
Pages are saved in the background. Nothing too fancy. I dedup when uploading: that helps a lot.
- idlewords 12y ago
  Okay, but how do you handle things like big PDFs or image gallery sites? Or pages that just pull in a lot of javascript includes? That stuff downloads in parallel, but then I would find myself trying to push it upstream through a little straw of bandwidth, sequentially.
  - purplerails 12y ago
    You're right, it takes longer to upload a page than to download it. And image gallery-like pages take long (I know because I save Imgur pages now than then :)). But in practice, this isn't a problem. There are a couple of heuristics to avoid wastefully uploading pages: the full page is uploaded only if the reader expresses "sufficient interest" in the page. Currently the heuristic is 90 seconds of continuous reading of a page, or scrolling to the bottom. If a page is read for a minimum of 10 continuous seconds then only the text of the page is uploaded. Static assets like JS files benefit from deduping: they take time to upload the first time, but subsequently processing them is much faster. Typically, people read multiple pages in a browsing session: I rapidly open many tabs and then read each one for multiple seconds. There's a debug mode in Purple Rails in which a timer counts up when I switch to a tab. I find that typically spend 100+ (usually much more) seconds on a page that I read through to the end. This is usually enough time on a residential broadband connection (I have Sonic DSL) to finish uploading a page. I also use Purple Rails on a tethered 4G connection almost everyday: uploading is slower than DSL but it works. Basically, by the time you finish reading a tab, the previous tab you read would have finished saving. Like I said, nothing too fancy.

hollerith 12y ago

I'd prefer for the pages to be saved to the hard drive of the machine running the web browser. But maybe browser extensions cannot obtain permission to do that?

purplerails 12y ago
I see your point. I may release installable native apps that can serve as the local storage backend for the truly paranoid. Time limitations etc: the usual. :) I anticipate that PurpleRails will be used on multiple computers and over several years. Which is why pages are sync'd to the cloud. I've adopted the current architecture because I feel that the energy barrier that needs to be overcome to persuade somebody to install an app is much higher than the one to install an extension.

zargon 12y ago

This sounded really great until I went to the website and saw that I can't use my own cloud storage, only purplerails'. As soon as purplerails disappears all my saved pages are gone. I already have this functionality with diigo and it makes me very uncomfortable not to have a copy of the data.

purplerails 12y ago
Excellent point. The ability to download your data in a well-documented format is coming soon. See also my reply to hollerith on a native client. Time limitations are what is preventing me from doing this. Thanks for your feedback! Hope you will use Purplerails. :)

asaddhamani 12y ago

I tried giving purplerails a shot. I use LastPass for password management, and I am not going to type in a 25/30 character password every single time I want to log into an application. I think you're going way too hard on that part. This is the first time I've had this happen to me when using a web app, and it immediately made me close the page.

purplerails 12y ago
Thanks for taking the time to write your feedback. I appreciate it. With PurpleRails, you will rarely be typing your password once you successfully login. It works kinda like Gmail/Facebook etc: you remain logged in for months at a time. This page explains why password saving doesn't work yet: https://www.purplerails.com/blog/saving-passwords-why-we-havent-enabled-it.html https://www.purplerails.com/blog/saving-passwords-why-we-hav... I'll see if I can do something fancier than what I do now to allow saved passwords to work. I suspect that using a Javascript-based AJAX authentication system instead of a plain-old HTML form, I can achieve the privacy goals as well as ease of use. I hope you will hang in there and use the current version until I figure out a way to fix this. Thanks!
- asaddhamani 12y ago
  Thanks for responding. Another 'issue' I've noticed is that it logs me out if I don't have the extension installed and click on the installation button, which, as you might figure, is a pain to deal with when you have a 25 character password. I guess you're saving the web page data from the client side and not through your servers. Another thing I've noticed is that when I add a page, it takes quite some time to show up in the web app. Also, whats with the timer? Apart from these issues, I really believe in your idea, and I have been working on making such a system for myself for months. I wish you the best, and I hope you succeed. I'll update you if I find any new issues, since if all the issues are fixed, I can see myself using this as my primary bookmarking service.
  - asaddhamani 12y ago
    I see it also saves content from emails and all that. That certainly justifies the need for extensive security. What are your plans for pricing? I suggest you don't take the route that Pocket did, $5 for anything and everything(which seems like a norm these days) is ridiculous.