13 ms·

Solid Project: All of your data, under your control

350 points by gibsonf1 6y ago

mawise 6y ago
I've never seen a compelling explanation of how this is supposed to work. Do people self-host their pods? Is the pod one more thing to sign up with an external provider? If I'm self-hosting my pod anyway, then wouldn't I want my self-hosting to include the (for example) photo-library viewing layer instead of giving google photos or flickr access to the photos in my pod? If I did give them access, then how is it more private than if they hosted it themselves? Maybe the pitch is more about mobility of services and avoiding lock-in than it is about privacy and security, but that doesn't seem to be the message I get. Does anyone have any pro-solid blog posts or articles you think might be helpful in convincing me? It smells like it is well aligned with my values but I just can't see how it will actually work.
- ocdtrekkie 6y ago
  Solid seems like an idea that only makes sense if your key goal is allowing cloud providers to offer proprietary services that use your data, while still storing it elsewhere. But I'd say most people fall either into "I don't mind proprietary services holding my data" or "I want to self-host the preferably open source application and my data". I don't see the benefit of separating the two at all, and I think the only reason this project keeps coming up and around is the name of the guy pushing it. "Solid is a mid-course correction for the Web by its inventor, Sir Tim Berners-Lee" is very literally the only selling point I think Solid has. Sandstorm or Cloudron have a better model, self-host (or pay someone to host for you) all of your apps and data together, in an easy interface for adding new apps like an app store.
  - pbronez 6y ago
    I really liked Sandstorm’s capability-based security model. Very cool concept for a team app server.
    - rapnie 6y ago
      Spritely project is developing Goblins, implementing the capability-based CapTP transport protocol for distributed programming and the fediverse, which looks very interesting. https://spritelyproject.org#goblins https://spritelyproject.org#goblins
    - ocdtrekkie 6y ago
      It really is. The challenge is it is tough to have that app support with it, everyone expects their apps to work the old way of doing things. ;)
- Vinnl 6y ago
  You can choose whether to self-host or whether to have someone else handle that for you. As for also hosting your apps: I think the answer to that is the same as for why you might still choose to use an email client even if you're hosting your email server. However, I would expect the majority of people not to self-host, but to have that handled by a party they trust. They would then not be limited by that host also having to support the particular photo-viewing layer they are interested in, but instead are able to choose one independently. (Disclosure: I work for Inrupt, but opinions are my own.)
- seph-reed 6y ago
  I had this idea a while ago when trying to figure out how we're going to do web-tech in space. Obviously you don't want to send an entire web page, just the data. If data is clearly templated, it's up to the user to choose a UI for displaying it. Many UIs can compete. They don't have to proprietary. In time, UI would become an art form; like music.
- passive 6y ago
  The hosting story is still a little messy, but in my version of this, I think it needs a trusted third-party to offer hosting, along with an open committee to manage the data specifications. The advantages of separating data storage from usage are similar to those in application development. If you have a robust model for defining and retrieving your data, the tools for working with that data can be iterated on independently from the data itself. So let's say your photos are in Google Photos, because your phone backs them up there automatically, but all your friends who you want to see the photos are on Facebook. Theoretically, "Solid" could be pulling those photos into a standard data specification, and provide easy tools for automating how they get shared into Facebook. Then let's say you wanted to edit some of those photos, so you open up PhotoShop Solid, and get editing. But then you join Instagram, and you want to quickly show off your Photoshop skills. All you need to do is connect Solid. The key aspect here is that you can use as many different tools as you like to work with your data, but it's the same data, and if Facebook goes belly-up tomorrow, or you just don't want your photos there anymore, Solid has your back. One thing that I don't think is well defined yet, but really should be, is how services will request access to your data. This needs some kind of standard interface, very similar to how phones let you customize app access. It should make it very clear what data you are "selling" to the Facebook, and what they are providing in return for that data.
  - mawise 6y ago
    That makes sense, but it also means the selling point is data portability more than data privacy/security. Since I would already need to give an application access to my data, I might as well have them host it for me. It still feels like Solid is a more complex and harder-to-reason-about solution to the same problem that GDPR data takeout tries to solve. Consistent open standards are nice, but it doesn't take that much work for Facebook (or others) to accept a Google Photos takeout dump as an input format. If the only real problem that Solid solves is "I loose my data if provider X locks me out or goes bankrupt", then it isn't even good enough since the third-party pod-hosting company can have the same failure mode. Maybe we should expand on GDPR data takeout legislation to require something like API-driven access that would allow people/companies to build automated backup/export solutions. That becomes a much simpler thing to build and get buy-in for instead of a whole new paradigm which companies aren't incentivized to follow and is hard for users to understand.
- treis 6y ago
  >photo-library viewing layer instead of giving google photos or flickr access to the photos in my pod? IMHO, privacy is the wrong tack for these arguments. The issue is that Flickr or Google Photos can shut down and/or raise prices while all your data is stuck there.
- tasn 6y ago
  You may also like Etebase (I created it), which is an open-source and end-to-end encrypted backend for apps. So it removes the extra layer (an additional provider) but still maintains the privacy and security benefits of self-hosting.
rasengan 6y ago
I read and it says you have to use a third party host or host on your own [1]. I’m not sure how this differs from the WWW - could someone explain? Thank you! [1] https://solidproject.org/users/get-a-pod https://solidproject.org/users/get-a-pod
- FlorinSays 6y ago
  It's an inversion of control over your data. Instead of all the data being stored in multiple places outside of your control, your data is stored on a pod and you have better control over what happens to it.
- XCSme 6y ago
  I was also confused by this, on the home-page it says > Solid lets people store their data securely in decentralized data stores called Pods. Then in the "Get-a-pod" it suggest you should get Amazon for hosting?
deleted 6y ago
[deleted]
woudsma 6y ago
I attended the presentation of Tim Berners Lee at MozFest 2018 where he outlined the Solid Project, and I've been waiting to see this project evolve. I'd like to see some more documentation and a 'get started' kind of tutorial to make this pretty abstract concept understandable. Looking around the project pages doesn't give me much incentive to start using it yet. I'd love to see a 'Solid for dummies' kind of approach in the documentation, things are quite technical still. Some easy to understand examples would be nice!
mfer 6y ago
Solid seems like a great technological idea. It empowers people to own their data and give others and apps access to it. This is empowering. But!!!! people want useful things. Things that help them solve problems so they want to pick up those things. Solid is lacking these solutions right now. Where is my todo list, contact app, calendar app, and so forth... all built on solid? This is an opportunity. But, until these kinds of the solutions exist for solid it's going to stay as a neat technology that's not bridged the gap to be useful.
- Vinnl 6y ago
  There's definitely not many Solid-supporting apps at the moment, but as for the todo list, there is https://noeldemartin.github.io/solid-focus/ https://noeldemartin.github.io/solid-focus/
- ocdtrekkie 6y ago
  Indeed, I feel projects like this can invest for years on platform, but fail to foster or develop the core useful apps people want to use. Ultimately, if your platform lacks that, few people will invest long enough to bring those apps from third parties.
natural219 6y ago
Has anyone built anything with this yet?
twiddlebits 6y ago
too complicated.
passive 6y ago
Lovely to see this come up again. It's really a great time to promote Solid, with the evidence of shady practices at the big corporations mounting, I know lots of folk would like a way to reclaim some of their data. That said, I feel like the big starting point for Solid is exfiltration/syncing of data between services, and that appears to be missing, for the most part. Give folk an app that will pull their music preferences from Spotify, and let them try out Amazon or Apple music with the same collection. It's a very simple data set, and a fairly common and comprehensible use case. Doing photos from Facebook/Instagram/Twitter/etc would be a bigger lift, but not too much different in concept, and I think very compelling in terms of selling the value of the service.
- miguelrochefort 6y ago
  Music, movie, and book history would be trivial to implement. This is timely, given the removal of Goodreads' API. User profiles as well, à la gravatar. How about work history, LinkedIn exports, standard resume formats? A lot could be done with medical data, using FHIR's ontologies. Fitness data also deserves a place there. It's difficult to understand how Solid isn't a major success after all these years.
jopsen 6y ago
How would you build something with this? Given that none my prospective users have a pod, or would want to go through additional sign-up steps. Also all pod providers seems to be experimental, and have no pricing.
kthejoker2 6y ago
I think something like this makes sense at some # of users closer to 10-20, or maybe even higher numbers, kind of monkeysphere / Dunbar's number level. That is, a smaller group of people (an extended family, a neighborhood street, a collective ... fine, a cult) band their resources together to pay for pod hardware to realize some savings / distributed back-up, and have "edge" apps for trusted sharing within the pod, and then P2P (pod-to-pod?) apps for sharing outside the pod. Right now, the economics just don't quite make sense.
- ocdtrekkie 6y ago
  I absolutely think the self-hosting story definitely only works for the widespread world if those of us capable of self-hosting provide services to less-technical users we know.
- passive 6y ago
  If this were to take off, I would expect to see some governments offering it to all citizens, both to support whatever digital services they provide, and as a tool for helping users manage their privacy.
  - oakfr 6y ago
    How would governments finance this? Developing these offerings with the quality of service of the GAFAM (which users take now for granted) costs a fortune (financed by advertising in practice as of today).
  - Vinnl 6y ago
    Such as that of Flanders, for example: https://inrupt.com/flanders-solid https://inrupt.com/flanders-solid
  - azinman2 6y ago
    Exactly what I want in my tech… gov control of my data, combined with gov-paced innovation…
    - 6y ago
smm11 6y ago
I self-host via a 1T SD card in my phone (encrypted). Why pull from/risk "the cloud" when I can have my stuff on me in the first place?
- oakfr 6y ago
  So if you lose your phone, you lose all your data? (asking in a genuine manner)
  - smm11 6y ago
    I have a glacier store of my phone and SD card data that I upload every weekend. I'm good with 5 or so days of phone data loss in that event. (I use a desktop VM that does nothing other than touch Glacier, and my phone, for this.)
- anderspitman 6y ago
  Something like Solid is more useful if you want to share data with other people, or use apps that only support cloud storage. There are many such apps (Google Docs for example), but most of them are currently embedded in their respective cloud storage provider. Solid decouples the apps from the storage.
mikedc 6y ago
Philosophically, and in the long-term, Solid is compelling to me for all of the reasons that the project purports to exist. But we have a classic chicken-and-egg problem with the absence of both reputable pod providers and the development of an application ecosystem. Practically, and immediately, it is compelling to me as a developer of small-ish applications (plugins, etc.) in which I want to give users the ability to store some data (preferences, etc.) in the cloud without my having to manage that data, or the associated services/infrastructure - including having to deal with absorbing or recouping the cost. Dropbox once offered the Datastore API[0], which was a handy little bring-your-own-database service allowing apps read/write access to a key/value store in a user's account (Dropbox accounts being quite common then), but it was deprecated[1] due to lack of traction at the time. [0] https://dropbox.tech/developers/the-datastore-api-a-new-way-to-store-and-sync-app-data https://dropbox.tech/developers/the-datastore-api-a-new-way-... [1] https://dropbox.tech/developers/deprecating-the-sync-and-datastore-apis https://dropbox.tech/developers/deprecating-the-sync-and-dat...
- deleted 6y ago
  [deleted]
stefanha 6y ago
Do pods support queries or just HTTP GET/POST/PUT/DELETE? Queries are important for performance. They avoid transferring all data over the network from the pod to the application. Although it's possible to define an open index format (metadata built from the actual data), it's probably still less efficient to access and also tricky to update without race conditions, data corruption, etc.
egypturnash 6y ago
Every now and then this gets shared again and every time I am struck by what a terrible job the front page does of explaining what Solid is and why anyone would want to use it. Like, okay, yeah, "control my own data" is a nice pitch in this day and age I guess, but... what can I do with it? "Any kind of data can be stored in a Solid pod, including normal files like you might store in Google Drive or Dropbox" - that sounds kinda useful, where's the link to SolidFile? Is there even a SolidFile app or is that just a thing someone could make with this protocol? Hell, where's the front page link to check out the list of any apps using Solid? It's like two or three random clicks to find this page (https://solidproject.org/apps https://solidproject.org/apps) if you are curious, and it's super uninspiring - why does one of the apps in the "Showcase" section at the top not even have a description? Solid always just feels like a solution looking for a problem.
- TimTheTinker 6y ago
  I'm in the early stages of bootstrapping a SaaS app that prioritizes security and privacy. For my business, this would be great: users are far more likely to trust an app if the app won't be storing any of the application data itself -- if they can choose their own data hosting provider. Google Cloud Drive, Box, and others already allow this kind of model to an extent. Third-party apps can access and modify user data hosted there, although not through a standardized API.
  - wernst 6y ago
    This project might be worth a look: https://www.etebase.com/ https://www.etebase.com/
    - TimTheTinker 6y ago
      This is fantastic. Thank you!
      - tasn 6y ago
        Thanks (I'm the founder of Etebase)! Let me know if you have any questions, my email is tom @ the domain.
whoisstan 6y ago
How would querying the data work? Will a pod provide querying capabilities as well? Example use case would be finding all pictures around a certain location. It "seems" unavoidable that certain data still has to be at a service provider since they enrich the original data. How would that work?
kodah 6y ago
Do I understand this (https://solidproject.org/developers/tutorials/getting-started https://solidproject.org/developers/tutorials/getting-starte...) correctly: you can only write Solid applications with frontend Javascript?
- brabel 6y ago
  No, you're looking at a Getting Started Page targeted at web developers, hence they use the tools they are familiar with. These tools take care of auth/data format and other glue code so you can get started easily... But because Solid is based on open standards, you could write a desktop app that makes HTTP requests using your language's clients without issues, but you'll have to write some boilerplate code yourself until there's frameworks in your language that handle that for you. I believe working with Solid is mostly about working with HTTP and RDF... so, if you're into the JVM, try Apache Jena[0]. PHP? Try EasyRDF[1]. Python? rdflib[2] [0] https://jena.apache.org/ https://jena.apache.org/ [1] https://www.easyrdf.org/ https://www.easyrdf.org/ [2] https://rdflib.readthedocs.io/en/stable/ https://rdflib.readthedocs.io/en/stable/
  - kiwidrew 6y ago
    > Solid is mostly about working with HTTP and RDF Surely this incarnation of the RDF-based Semantic Web nirvana will succeed where all other attempts have failed! :)
  - tgragnato 6y ago
    Asking devs to work with RDF is the main issue I have with the approach of Solid. I don't expect SPARQL and ontologies to go mainstream. So why "force your hand" and make solid depend on the standards of the semantic web? There are much simpler and productive approaches to solve the problem of data interoperability.
    - miguelrochefort 6y ago
      > There are much simpler and productive approaches to solve the problem of data interoperability. Really? Which ones?
azinman2 6y ago
This project is doomed in isolation. It needs to be coupled with at least one major popular/killer service, and preferably more than that. Plus make that as easy as uploading data to “the cloud,” and as transparent, plus “free” to meaningfully compete. Right now it just looks like technology in search of a problem.
- cxr 6y ago
  > Right now it just looks like technology in search of a problem. This is a common turn of phrase (which has turned up more than once here, even), but it's not appropriate to describe Solid. Solid is a half-implemented solution to a real problem.
  - _eqet 6y ago
    The whole internet was built on this concept. The Web technologies were built long before e-commerce or dating sites start using it
    - cxr 6y ago
      I'm not sure what you mean.
gklitt 6y ago
1) I am a big fan of the general philosophy of Solid. But as others have mentioned, I've found it extremely difficult to understand specifics of the proposal or to see concrete progress, based on publicly available resources. The best place I've found is Ruben Verborgh's blog (he's a researcher who's affiliated with Solid). For example here's a nice post which goes into more detail on the ideas behind Solid: https://ruben.verborgh.org/blog/2017/12/20/paradigm-shifts-for-the-decentralized-web/ https://ruben.verborgh.org/blog/2017/12/20/paradigm-shifts-f... 2) In terms of applications, I'd personally like to see more of a focus on productivity applications, rather than so much focus on social media. Better interop between cloud SaaS apps would be valuable to businesses and professionals, and would sidestep many thorny challenges like decentralized moderation. As a concrete starting point: what if I could store my Google Docs text files on my own storage layer, and edit them in realtime using a variety of editor apps? This would resemble my ability to edit a .txt file on my computer in vim or TextEdit, but would port that metaphor to the world of modern online collaboration. Here's a short Twitter thread I wrote on this topic: https://twitter.com/geoffreylitt/status/1355255162626068482 https://twitter.com/geoffreylitt/status/1355255162626068482 3) I'm very curious to see more incremental paths from the current web to a decentralized approach. For example, what if we could start annotating existing websites with private data and sharing those annotations P2P, rather than starting over from scratch? I've explored this a bit with my Wildcard project, where users can store annotations in a "spreadsheet" that is linked to a web page: https://www.geoffreylitt.com/wildcard/ https://www.geoffreylitt.com/wildcard/
- er4hn 6y ago
  I think that with a problem statement of "I want to control my data and how third parties use my data" Solid makes a good deal of sense. Personal information should be owned by you and you should be able to take it from service to service as you see fit. The two problems that I find with Solid, based on my own limited understanding are as follows: (1) Services don't need to give back to Solid - Services, Facebook, your medical provider, whatever else you are using, does not have a clear incentive to provide their own data on you back to your Solid pod. It is far easier for them to keep it: it lets them do offline processing, and it keeps you more locked into their service. I'm not sure how one would solve this issue. (2) Much like mobile apps with excessive permissions and the abuse of tracking elements - I don't see how Solid prevents the abuse of its service. If Solid catches on and Facebook has a permissions check saying "Let Facebook do 'SELECT * FROM .;' on your Solid data, how many people will click yes? Even if you request it each time, once the data is copied out, it is out there and can be packaged and resold, used to build advertising profiles, etc. You're back to the original problem of not being able to limit access to your data, but with extra steps. Where I think this could be solved is by Solid not providing the data directly, but by being a service which can answer queries. Queries could be items such as "Does user like cats? y/n/m". Or it could be something like "Here is an anonymized dataset being built out. Please add your input to it." Replies to queries could also have an amount of deliberately wrong or misleading answers given, depending on the service and endpoint to obfuscate your personal data on places that don't need it. While this can still be abused, it raises the bar for abuse.
alpaca128 6y ago
To me the whole pitch is a bit too nebulous and seemingly inconsistent. What does it mean with "portable" and "interoperable" data standards? Does that mean specific file formats? Protocols? Apps? What do they mean with "Linked Data"? What's novel about this, what makes it different from hyperlinks or shared Dropbox folders? Why does it say pods are decentralized data stores, but then on the "about" page it tells me I have to host the pod on a server - how is that decentralized?
- syats 6y ago
  Linked Data does have a concrete definition: RDF data that is accessible through SPARQL, with well defined semantics. That is, following a published ontology and re-using existing vocabularies as much as possible. Linked Data itself is not novel: https://www.w3.org/standards/semanticweb/data https://www.w3.org/standards/semanticweb/data
kevincox 6y ago
Reminds me of https://remotestorage.io/ https://remotestorage.io/ which I have used in a (mostly personal) website. I like that I can provide a service as a static website, and users pay for, manage and have control over their own storage. One of the things that could help it bootstrap is that they also support client-side Google Drive and Dropbox shims, so users who want to use something else can, but most users will just use one or the other. There are downsides though. - The API may be less than ideal for some use cases. This can make a well-performing app hard. For example there is no relational-sql API. - You can't migrate user's data. So if you make changes to the format you need to retain backwards compatibility "forever". - If you want to provide "discovery" or cross-user features you will need to have your own storage anyways.
- anderspitman 6y ago
  I actually started working on just such a shim yesterday. The idea is to represent all major cloud storage through a single simple frontend API. You have to implement each provider's OAuth flow, but once you have the files it's pretty much the same. Dropbox, Google, and my own protocol[0] will come first, but Solid is planned eventually. What I've learned so far: * Google is extremely draconian unless you use their JavaScript picker. To really integrate nicely you would have to spend at least $15000-$75000 getting your app audited by a 3rd party. * Dropbox picker integration is very easy and slick. [0]: https://github.com/gemdrive https://github.com/gemdrive
fuzzybear3965 6y ago
Is this related to Jaron Lanier's vision of union (in the labor sense) groups which sell to companies access to the group's anonymized data? In his model, one individual can belong to many groups and their information is like a share/stock they have in a pool of data with other people. They sell in a cooperative fashion to select companies, gaining power through anonymity and group bargaining power.
- rapnie 6y ago
  Interesting. Do you have any good links?
- rencire 6y ago
  Would love to learn more about this. Any resources/links on this subject?
- Nasrudith 6y ago
  Data in those contexts nearly always have "reciprocial actions" fundamentally and those who gather the data know what they actually want out of it. Company-user interactions would likely produce better data. It feels like half of an understanding of both data usage and union style collective barganing. Unless I am missing something it seems a bit like trying to combine a semitruck and a bicycle to get green bulk cargo transportation. Sure both things individually have these advantages and it would be great to have all in one but implementing it is fundamentally nonviable.
- asdfk-12 6y ago
  Parent seems to be referring to MIDs [1] "Mediators of Individual Data (MIDs) are a union-like organization championed by technologist Jaron Lanier as a framework for addressing the key issues around user-generated data like ownership, monetization, and reparations. An effective MID or data union could perform a number of critical functions on behalf of its members that are currently lacking in the digital economy." There exists also a great Harvard Business Review article, but it is paywalled. [1] https://blog.datadividendproject.com/this-week-in-data-101-whats-a-mid/ https://blog.datadividendproject.com/this-week-in-data-101-w...
sharathr 6y ago
Love the idea but how does Solid prevent any malicious actor from over time replicating all the data that user authorize into a centralized repository - in this case, it only makes it even easier for data consolidation.
- Vinnl 6y ago
  In my (personal) view, Solid is technology that enables service providers to put you in control of the data, not that forces them to do so. In other words, it's just part of the puzzle, where the other part is e.g. customer demand or regulation.
  - cxr 6y ago
    To use an old metaphor: locks on the doors are not what keep people from getting into a house.
hezag 6y ago
This post[0] by Ruben Verborgh helped me understand the problem Solid is trying to solve: > In December 2019, Google and Facebook proudly announced a major milestone, which was echoed in news media all around the world: it is now possible to copy a picture from Facebook to Google Photos. This news came in mere months after we celebrated the 50th anniversary of another technological feat: the moon landing of 20 July 1969, when millions of households witnessed Neil Armstrong take a giant leap for mankind. > So let me get this straight: two of the largest tech companies in history make headlines because in 2019, they move a single photo over the whopping distance of 11 km it takes from the Facebook headquarters in Menlo Park to the Googleplex in Mountain View, whereas in 1969, we sent live video signals from 380,000 km away on the actual moon? > If those two companies, both widely hailed as pinnacles of technology, genuinely consider this to be innovation they are proud of, the only logical conclusion is that data-driven innovation today is fundamentally broken. > The problem is widespread and not limited to technology or social media. Any sector that requires personal data to deliver services, from retail over insurance to health, suffers from the damaging effects of siloization. Companies increasingly need more access to data, but they won’t get there if they keep on collecting that data themselves.[...] 0. https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-fosters-sustainable-innovation/ https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-...
peterburkimsher 6y ago
I like the concept of Solid, and proposed it to the BeWelcome community (who are running a hospitality exchange network like CouchSurfing, but free). Although Solid is a better technical idea, I'm not sure how the migration from PHP/MySQL would work. Nobody on the Solid forum replied to my post asking about it. Another guy, Chagai, is pushing for Matrix and the fediverse. Ultimately, there's many great technical ideas, but the real solution doesn't lie in writing code - it's in building the community. Therefore I'm trying harder to create more activity in BeWelcome's existing community (which is hard when nobody's travelling), like an online meetup every Thursday. If it gets critical mass, I believe it could be as widespread as Wikipedia or OpenStreetMap (which it models in org structure). I wish that Solid could be a part of it, but that requires active developers, and currently it's hard enough to find someone willing to make an API or native iOS app, never mind a breaking back-end change that people aren't asking for.
gameman144 6y ago
How does this work in practice, I didn't see a worked example on the site. For instance, say I give Solid Social access to my contacts and my photos. At that point, Solid Social can access all my contacts and photos, copy them, and then do whatever they please with them (even if I later revoke permissions). In the current web, I upload my photos to a website, which I then trust not to use them inappropriately. In the Solid world, it seems like I give photo-access to a website (which could then copy them) and trust that website to not use them inappropriately. This seems like the current system, but with extra steps. In terms of data compatibility, I think that there's a potentially compelling case (user-driven standards vs. bespoke handling per website), but again I fail to see the practical implications. For example, if I upload a video to a website and it becomes incompatible with other websites, my assumption is that there's some reason for that step. It could be out of walled-garden malice, but it also could be that the website is encoding it and co-locating it in such a way that it'll be really fast for other users to watch. In Solid-world, it's unclear what the flow here should be: does the website only get to consume the videos as I have them on my pod (potentially terrible performance)? Does the website get to still do optimizations (and potentially keep artifacts of my video after I revoke permissions?) I like the drive towards consistent standards for interfacing with data, but other than that I could just be missing the important bits here (and please let me know if so), and I fail to see the positive change in most web-based user-experiences for the most part.
- anderspitman 6y ago
  Solid is just the technical part. You're also going to need the regulatory part that says it's illegal for a company stores data after you revoke access. But the technical part is still necessary.
  - jwblackwell 6y ago
    How is this really any fundamentally different from say GDPR then?
    - anderspitman 6y ago
angst_ridden 6y ago
What Solid really needs to do is create a page that I could show to my nontechnical friends where it explains how it works and what services/sites support it. Without those two things being front and center, I can't see Solid being anything more than a niche interest for technical folks.
- deleted 6y ago
  [deleted]
- zepatrik 6y ago
  I think the problem there is that there is not much to show. But it will require that page at some point. Maybe it would help now as well as managers and other "decision makers" might understand it more. And they have to implement it in the end right?
- volodymyrs 6y ago
  At least for the services/sites/apps which support it, there is a page: https://solidproject.org/apps https://solidproject.org/apps There are lot's of apps, most of them work just fine, which is quite inspiring.
hackpert 6y ago
Last month, I was working with some people on building a decentralized health data store for vaccine waitlists (a la Mailbox) and certificates and we were looking at Solid as one potential backbone. I really admire what they’re trying to do. However, as some other people have observed, their example applications are quite awful and not very useful(?) and they seem to be marketing it to end users, not developers. In my experience, most end users do not care about privacy and security until 1) they either lose a lot of money or other valuables 2) some hacker comes knocking on their door for extortion and they lose a lot of money or other valuables. It would be so much better if they just made it so easy and simple for developers to use that it becomes their default, and not rely on hope that end users will suddenly start using decentralized apps in the name of privacy.
- tasn 6y ago
  I'm the founder of Etebase[1] an open-source and end-to-end encrypted backend for building apps. The goal is to maintain privacy and security while still bringing users the benefits of a central storage. Unlike Solid it's already used in a few places, like GNOME, KDE and etc. [1] https://www.etebase.com https://www.etebase.com
foolinaround 6y ago
a project with similar goals is Sandstorm.io i plan to check it out soon..
dreamling 6y ago
I haven't looked into it for a couple years, I was initially very excited about the concept. TLDR: Unless the user experience/accessibility/hosting experience is improved, I just couldn't suggest anyone use it. Last time I really dug in the user interface and accessibility concerns were not great. I understand the idea was more about the data handling, and that anyone could produce something based off the framework but I had all sorts of issues with the login tokens/usability. (ie, if you logged in with the wrong cert it didn't have an easy way to clear it, and would 'log in' to an error page. ect. ) as far as I can tell, each pod/data host can run different versions of solid, so your experience may vary between them. You can self host as well. I'd have to dig in again to see if things have gotten better, but I don't have time at the moment. on quick look, some of my data sites are no longer working. oh! link says it was shut down in oct. https://lists.w3.org/Archives/Public/public-solid/2020Oct/0005.html https://lists.w3.org/Archives/Public/public-solid/2020Oct/00... That said, the community was pretty nice and willing to answer questions. Maybe things have improved? I was hoping that maybe they'd start using DAT as well, because it seemed like it might be nice to dovetail with some of the beakerbrowser/distributed web stuff going on. re @pfrazee
- dreamling 6y ago
  quick followup, after checking my old links and making a new sign up. it does seem like the inrupt.com pod has a bit better /more up to date design. Accessibility, while not perfect, does seem to be improved. (quick WAVE tool check) Things are still fairly confusing for new users though. And some how tos I wrote down from then no longer work. Which is not the worst thing, as some of the tutorials required you to hover over and wait to be able to edit things. Much easier to look at though. ymmv for historical/hysterical reference, I uploaded my old solid adventuress file, it is not up to date, and some of the css is not always showing the right colors, ect. but you can see some of the directions I was looking into then: https://pod.inrupt.com/metahari/public/solidnotes/SolidAdventures.html https://pod.inrupt.com/metahari/public/solidnotes/SolidAdven...
  - Vinnl 6y ago
    Note: it was only the old solid.community domain that shut down; the same servers are now reachable through solid.community. But yes, the default UI used there is very lacking and slow to develop. I also built a more developer-focused interface that allows you to view the data in your Pod, and that should be more accessible. If you're interested, you can try it at https://penny.vincenttunru.com https://penny.vincenttunru.com
eivarv 6y ago
I think it's important to point out (if only for end-users) that Solid puts your data "under your control" in the "allows you to authorize third party access to your data" – not in the "can actually control your data in practice, post-access". It'll let you granularly authorize first party access what data you have in your pod, but there can’t be any technical guarantees with regards to illegitimate sharing or otherwise copying (many might at least cache, for instance) – nor about what is collected and shared outside this system. It's own documentation even states this, if you dig a little. I keep seeing data-hubs and identity-providers touting themselves as solutions to the web's privacy issues, but I don't see how they actually solve anything. It seems like an attempted technical solution to a social problem to me. The real problem with data how data is used these days is really that a bunch of data is collected opaquely, unethically, and in some cases illegally – and possibly shared. The whole system including data brokers and real time bidding is out of control.
dang 6y ago
If curious see also 2019 (a bit) https://news.ycombinator.com/item?id=20017493 https://news.ycombinator.com/item?id=20017493 2019 (a bit) https://news.ycombinator.com/item?id=20617506 https://news.ycombinator.com/item?id=20617506 2018 https://news.ycombinator.com/item?id=18100895 https://news.ycombinator.com/item?id=18100895 2018 https://news.ycombinator.com/item?id=16355311 https://news.ycombinator.com/item?id=16355311 2018 (a bit) https://news.ycombinator.com/item?id=18119227 https://news.ycombinator.com/item?id=18119227 2016 https://news.ycombinator.com/item?id=12280764 https://news.ycombinator.com/item?id=12280764 Others?
yiyus 6y ago
From a quick look at the site and a quick look at perkeep's site, both projects look quite similar. Could anyone with some experience comment on their differences?
anderspitman 6y ago
Solid is awesome and I'm bullish on this model in the long-run, but it seems to be caught in a classic chicken/egg problem. Nobody is going to use it until there are good apps. and nobody is going to make good apps until there is a market of users. IMO this is the same problem that kept Sandstorm and remoteStorage from getting big. I see 2 solutions currently: 1. Make an awesome storage product, that just happens to implement Solid. It might have to start out also implementing S3, which unfortunately doesn't support OAuth, which basically kills it for apps. You could try making it compatible with another proprietary protocol like Google Drive, or provide a frontend library that easily talks to all the major cloud storage providers, plus Solid... 2. Take the slow track. If developers themselves want to use it, they'll write useful open source apps over time. This can take forever though. Even after 30 years there are still large feature gaps on Linux because the market is too small to attract big app developers.
- cxr 6y ago
  > It might have to start out also implementing S3, which unfortunately doesn't support OAuth The expectation that Solid is backed by a smart server that's intimately aware that it's hosting a Solid pod and what exactly it takes to do so is a critical failure in Solid's design. If Solid's protocols instead better accommodated "dead" formats that can be served by static web hosts, so your $USERNAME.github.io could be the front door to your data store—at least the parts of it that you already make available to the public, anyway—then we'd have passed the chicken-and-egg stage years ago and Solid would already be a runaway success. At the end of the day, it's supposed to be about data, after all—linked data—and not about the endpoints/servers themselves. Compare to the ease of creating an RSS feed for your blog (you can get your static site generator to spit one out pretty easily) and contrast it to needing to provision and actively administer a Mastodon server, for example.
  - walterbell 6y ago
    > "dead" formats that can be served by static web hosts, so your $USERNAME.github.io could be the front door to your data store—at least the parts of it that you already make available to the public Could you share some examples of such data, other than "news" feeds? FOAF, for example? http://www.foaf-project.org/ http://www.foaf-project.org/
EGreg 6y ago
Tim went on to leave MIT, found Inrupt and get funded to the tune of several million dollars. Some great guys there on the solid team, including Melvin Carvalho and others who maintain ties with the MaidSAFE people. Give it time, it’s the future.
hikerclimb 6y ago
This is probably a lie just like our media is a lie... did you know that robinhood was supposed to raise 6 billion but only raised 1 billion according to nytimes. They probably deleted the article so in case they get screwed. I hope robinhood and citadel get screwed over and fail. Karmas a bitch ain’t it?
matt_f 6y ago
I want to love this. But, in trying to imagine any scenario in which tech corporations might conceivably start hosting their internal user data on these external "Solid Pods", I'm left drawing a blank. Until we have a way that tech companies start storing users' data in user-controlled infrastructure, I think this idea is a noble fantasy. I would love to hear an argument that will convince me I'm wrong on this.
- Vinnl 6y ago
  The only scenario that I think is likely for tech corporations are either legislative pressure, or a tech corporation for which user tracking is not their core competency (e.g. Apple) using it as their USP as a moat against Google and Facebook. That said, there's a lot of user data at non-tech companies. Especially given that they're not particularly skilled at securing that data, a changing regulatory climate turns that data more and more into a liability. If, as a user, you can ensure that that data is stored at a trusted and competent party (potentially yourself), that alone would already be a major win, in my (personal) opinion.

350 points by gibsonf1 6y ago

I've never seen a compelling explanation of how this is supposed to work. Do people self-host their pods? Is the pod one more thing to sign up with an external provider? If I'm self-hosting my pod anyway, then wouldn't I want my self-hosting to include the (for example) photo-library viewing layer instead of giving google photos or flickr access to the photos in my pod? If I did give them access, then how is it more private than if they hosted it themselves? Maybe the pitch is more about mobility of services and avoiding lock-in than it is about privacy and security, but that doesn't seem to be the message I get. Does anyone have any pro-solid blog posts or articles you think might be helpful in convincing me? It smells like it is well aligned with my values but I just can't see how it will actually work.

ocdtrekkie 6y ago
Solid seems like an idea that only makes sense if your key goal is allowing cloud providers to offer proprietary services that use your data, while still storing it elsewhere. But I'd say most people fall either into "I don't mind proprietary services holding my data" or "I want to self-host the preferably open source application and my data". I don't see the benefit of separating the two at all, and I think the only reason this project keeps coming up and around is the name of the guy pushing it. "Solid is a mid-course correction for the Web by its inventor, Sir Tim Berners-Lee" is very literally the only selling point I think Solid has. Sandstorm or Cloudron have a better model, self-host (or pay someone to host for you) all of your apps and data together, in an easy interface for adding new apps like an app store.
- pbronez 6y ago
  I really liked Sandstorm’s capability-based security model. Very cool concept for a team app server.
  - rapnie 6y ago
    Spritely project is developing Goblins, implementing the capability-based CapTP transport protocol for distributed programming and the fediverse, which looks very interesting. https://spritelyproject.org#goblins https://spritelyproject.org#goblins
  - ocdtrekkie 6y ago
    It really is. The challenge is it is tough to have that app support with it, everyone expects their apps to work the old way of doing things. ;)
Vinnl 6y ago
You can choose whether to self-host or whether to have someone else handle that for you. As for also hosting your apps: I think the answer to that is the same as for why you might still choose to use an email client even if you're hosting your email server. However, I would expect the majority of people not to self-host, but to have that handled by a party they trust. They would then not be limited by that host also having to support the particular photo-viewing layer they are interested in, but instead are able to choose one independently. (Disclosure: I work for Inrupt, but opinions are my own.)
seph-reed 6y ago
I had this idea a while ago when trying to figure out how we're going to do web-tech in space. Obviously you don't want to send an entire web page, just the data. If data is clearly templated, it's up to the user to choose a UI for displaying it. Many UIs can compete. They don't have to proprietary. In time, UI would become an art form; like music.
passive 6y ago
The hosting story is still a little messy, but in my version of this, I think it needs a trusted third-party to offer hosting, along with an open committee to manage the data specifications. The advantages of separating data storage from usage are similar to those in application development. If you have a robust model for defining and retrieving your data, the tools for working with that data can be iterated on independently from the data itself. So let's say your photos are in Google Photos, because your phone backs them up there automatically, but all your friends who you want to see the photos are on Facebook. Theoretically, "Solid" could be pulling those photos into a standard data specification, and provide easy tools for automating how they get shared into Facebook. Then let's say you wanted to edit some of those photos, so you open up PhotoShop Solid, and get editing. But then you join Instagram, and you want to quickly show off your Photoshop skills. All you need to do is connect Solid. The key aspect here is that you can use as many different tools as you like to work with your data, but it's the same data, and if Facebook goes belly-up tomorrow, or you just don't want your photos there anymore, Solid has your back. One thing that I don't think is well defined yet, but really should be, is how services will request access to your data. This needs some kind of standard interface, very similar to how phones let you customize app access. It should make it very clear what data you are "selling" to the Facebook, and what they are providing in return for that data.
- mawise 6y ago
  That makes sense, but it also means the selling point is data portability more than data privacy/security. Since I would already need to give an application access to my data, I might as well have them host it for me. It still feels like Solid is a more complex and harder-to-reason-about solution to the same problem that GDPR data takeout tries to solve. Consistent open standards are nice, but it doesn't take that much work for Facebook (or others) to accept a Google Photos takeout dump as an input format. If the only real problem that Solid solves is "I loose my data if provider X locks me out or goes bankrupt", then it isn't even good enough since the third-party pod-hosting company can have the same failure mode. Maybe we should expand on GDPR data takeout legislation to require something like API-driven access that would allow people/companies to build automated backup/export solutions. That becomes a much simpler thing to build and get buy-in for instead of a whole new paradigm which companies aren't incentivized to follow and is hard for users to understand.
treis 6y ago
>photo-library viewing layer instead of giving google photos or flickr access to the photos in my pod? IMHO, privacy is the wrong tack for these arguments. The issue is that Flickr or Google Photos can shut down and/or raise prices while all your data is stuck there.
tasn 6y ago
You may also like Etebase (I created it), which is an open-source and end-to-end encrypted backend for apps. So it removes the extra layer (an additional provider) but still maintains the privacy and security benefits of self-hosting.

rasengan 6y ago

I read and it says you have to use a third party host or host on your own [1]. I’m not sure how this differs from the WWW - could someone explain? Thank you! [1] https://solidproject.org/users/get-a-pod https://solidproject.org/users/get-a-pod

FlorinSays 6y ago
It's an inversion of control over your data. Instead of all the data being stored in multiple places outside of your control, your data is stored on a pod and you have better control over what happens to it.
XCSme 6y ago
I was also confused by this, on the home-page it says > Solid lets people store their data securely in decentralized data stores called Pods. Then in the "Get-a-pod" it suggest you should get Amazon for hosting?

deleted 6y ago

[deleted]

woudsma 6y ago

I attended the presentation of Tim Berners Lee at MozFest 2018 where he outlined the Solid Project, and I've been waiting to see this project evolve. I'd like to see some more documentation and a 'get started' kind of tutorial to make this pretty abstract concept understandable. Looking around the project pages doesn't give me much incentive to start using it yet. I'd love to see a 'Solid for dummies' kind of approach in the documentation, things are quite technical still. Some easy to understand examples would be nice!

mfer 6y ago

Solid seems like a great technological idea. It empowers people to own their data and give others and apps access to it. This is empowering. But!!!! people want useful things. Things that help them solve problems so they want to pick up those things. Solid is lacking these solutions right now. Where is my todo list, contact app, calendar app, and so forth... all built on solid? This is an opportunity. But, until these kinds of the solutions exist for solid it's going to stay as a neat technology that's not bridged the gap to be useful.

Vinnl 6y ago
There's definitely not many Solid-supporting apps at the moment, but as for the todo list, there is https://noeldemartin.github.io/solid-focus/ https://noeldemartin.github.io/solid-focus/
ocdtrekkie 6y ago
Indeed, I feel projects like this can invest for years on platform, but fail to foster or develop the core useful apps people want to use. Ultimately, if your platform lacks that, few people will invest long enough to bring those apps from third parties.

natural219 6y ago

Has anyone built anything with this yet?

twiddlebits 6y ago

too complicated.

passive 6y ago

Lovely to see this come up again. It's really a great time to promote Solid, with the evidence of shady practices at the big corporations mounting, I know lots of folk would like a way to reclaim some of their data. That said, I feel like the big starting point for Solid is exfiltration/syncing of data between services, and that appears to be missing, for the most part. Give folk an app that will pull their music preferences from Spotify, and let them try out Amazon or Apple music with the same collection. It's a very simple data set, and a fairly common and comprehensible use case. Doing photos from Facebook/Instagram/Twitter/etc would be a bigger lift, but not too much different in concept, and I think very compelling in terms of selling the value of the service.

miguelrochefort 6y ago
Music, movie, and book history would be trivial to implement. This is timely, given the removal of Goodreads' API. User profiles as well, à la gravatar. How about work history, LinkedIn exports, standard resume formats? A lot could be done with medical data, using FHIR's ontologies. Fitness data also deserves a place there. It's difficult to understand how Solid isn't a major success after all these years.

jopsen 6y ago

How would you build something with this? Given that none my prospective users have a pod, or would want to go through additional sign-up steps. Also all pod providers seems to be experimental, and have no pricing.

kthejoker2 6y ago

I think something like this makes sense at some # of users closer to 10-20, or maybe even higher numbers, kind of monkeysphere / Dunbar's number level. That is, a smaller group of people (an extended family, a neighborhood street, a collective ... fine, a cult) band their resources together to pay for pod hardware to realize some savings / distributed back-up, and have "edge" apps for trusted sharing within the pod, and then P2P (pod-to-pod?) apps for sharing outside the pod. Right now, the economics just don't quite make sense.

ocdtrekkie 6y ago
I absolutely think the self-hosting story definitely only works for the widespread world if those of us capable of self-hosting provide services to less-technical users we know.
passive 6y ago
If this were to take off, I would expect to see some governments offering it to all citizens, both to support whatever digital services they provide, and as a tool for helping users manage their privacy.
- oakfr 6y ago
  How would governments finance this? Developing these offerings with the quality of service of the GAFAM (which users take now for granted) costs a fortune (financed by advertising in practice as of today).
- Vinnl 6y ago
  Such as that of Flanders, for example: https://inrupt.com/flanders-solid https://inrupt.com/flanders-solid
- azinman2 6y ago
  Exactly what I want in my tech… gov control of my data, combined with gov-paced innovation…
  - 6y ago

smm11 6y ago

I self-host via a 1T SD card in my phone (encrypted). Why pull from/risk "the cloud" when I can have my stuff on me in the first place?

oakfr 6y ago
So if you lose your phone, you lose all your data? (asking in a genuine manner)
- smm11 6y ago
  I have a glacier store of my phone and SD card data that I upload every weekend. I'm good with 5 or so days of phone data loss in that event. (I use a desktop VM that does nothing other than touch Glacier, and my phone, for this.)
anderspitman 6y ago
Something like Solid is more useful if you want to share data with other people, or use apps that only support cloud storage. There are many such apps (Google Docs for example), but most of them are currently embedded in their respective cloud storage provider. Solid decouples the apps from the storage.

mikedc 6y ago

Philosophically, and in the long-term, Solid is compelling to me for all of the reasons that the project purports to exist. But we have a classic chicken-and-egg problem with the absence of both reputable pod providers and the development of an application ecosystem. Practically, and immediately, it is compelling to me as a developer of small-ish applications (plugins, etc.) in which I want to give users the ability to store some data (preferences, etc.) in the cloud without my having to manage that data, or the associated services/infrastructure - including having to deal with absorbing or recouping the cost. Dropbox once offered the Datastore API[0], which was a handy little bring-your-own-database service allowing apps read/write access to a key/value store in a user's account (Dropbox accounts being quite common then), but it was deprecated[1] due to lack of traction at the time. [0] https://dropbox.tech/developers/the-datastore-api-a-new-way-to-store-and-sync-app-data https://dropbox.tech/developers/the-datastore-api-a-new-way-... [1] https://dropbox.tech/developers/deprecating-the-sync-and-datastore-apis https://dropbox.tech/developers/deprecating-the-sync-and-dat...

deleted 6y ago
[deleted]

stefanha 6y ago

Do pods support queries or just HTTP GET/POST/PUT/DELETE? Queries are important for performance. They avoid transferring all data over the network from the pod to the application. Although it's possible to define an open index format (metadata built from the actual data), it's probably still less efficient to access and also tricky to update without race conditions, data corruption, etc.

egypturnash 6y ago

Every now and then this gets shared again and every time I am struck by what a terrible job the front page does of explaining what Solid is and why anyone would want to use it. Like, okay, yeah, "control my own data" is a nice pitch in this day and age I guess, but... what can I do with it? "Any kind of data can be stored in a Solid pod, including normal files like you might store in Google Drive or Dropbox" - that sounds kinda useful, where's the link to SolidFile? Is there even a SolidFile app or is that just a thing someone could make with this protocol? Hell, where's the front page link to check out the list of any apps using Solid? It's like two or three random clicks to find this page (https://solidproject.org/apps https://solidproject.org/apps) if you are curious, and it's super uninspiring - why does one of the apps in the "Showcase" section at the top not even have a description? Solid always just feels like a solution looking for a problem.

TimTheTinker 6y ago
I'm in the early stages of bootstrapping a SaaS app that prioritizes security and privacy. For my business, this would be great: users are far more likely to trust an app if the app won't be storing any of the application data itself -- if they can choose their own data hosting provider. Google Cloud Drive, Box, and others already allow this kind of model to an extent. Third-party apps can access and modify user data hosted there, although not through a standardized API.
- wernst 6y ago
  This project might be worth a look: https://www.etebase.com/ https://www.etebase.com/
  - TimTheTinker 6y ago
    This is fantastic. Thank you!
    - tasn 6y ago
      Thanks (I'm the founder of Etebase)! Let me know if you have any questions, my email is tom @ the domain.

whoisstan 6y ago

How would querying the data work? Will a pod provide querying capabilities as well? Example use case would be finding all pictures around a certain location. It "seems" unavoidable that certain data still has to be at a service provider since they enrich the original data. How would that work?

kodah 6y ago

Do I understand this (https://solidproject.org/developers/tutorials/getting-started https://solidproject.org/developers/tutorials/getting-starte...) correctly: you can only write Solid applications with frontend Javascript?

brabel 6y ago
No, you're looking at a Getting Started Page targeted at web developers, hence they use the tools they are familiar with. These tools take care of auth/data format and other glue code so you can get started easily... But because Solid is based on open standards, you could write a desktop app that makes HTTP requests using your language's clients without issues, but you'll have to write some boilerplate code yourself until there's frameworks in your language that handle that for you. I believe working with Solid is mostly about working with HTTP and RDF... so, if you're into the JVM, try Apache Jena[0]. PHP? Try EasyRDF[1]. Python? rdflib[2] [0] https://jena.apache.org/ https://jena.apache.org/ [1] https://www.easyrdf.org/ https://www.easyrdf.org/ [2] https://rdflib.readthedocs.io/en/stable/ https://rdflib.readthedocs.io/en/stable/
- kiwidrew 6y ago
  > Solid is mostly about working with HTTP and RDF Surely this incarnation of the RDF-based Semantic Web nirvana will succeed where all other attempts have failed! :)
- tgragnato 6y ago
  Asking devs to work with RDF is the main issue I have with the approach of Solid. I don't expect SPARQL and ontologies to go mainstream. So why "force your hand" and make solid depend on the standards of the semantic web? There are much simpler and productive approaches to solve the problem of data interoperability.
  - miguelrochefort 6y ago
    > There are much simpler and productive approaches to solve the problem of data interoperability. Really? Which ones?

azinman2 6y ago

This project is doomed in isolation. It needs to be coupled with at least one major popular/killer service, and preferably more than that. Plus make that as easy as uploading data to “the cloud,” and as transparent, plus “free” to meaningfully compete. Right now it just looks like technology in search of a problem.

cxr 6y ago
> Right now it just looks like technology in search of a problem. This is a common turn of phrase (which has turned up more than once here, even), but it's not appropriate to describe Solid. Solid is a half-implemented solution to a real problem.
- _eqet 6y ago
  The whole internet was built on this concept. The Web technologies were built long before e-commerce or dating sites start using it
  - cxr 6y ago
    I'm not sure what you mean.

gklitt 6y ago

1) I am a big fan of the general philosophy of Solid. But as others have mentioned, I've found it extremely difficult to understand specifics of the proposal or to see concrete progress, based on publicly available resources. The best place I've found is Ruben Verborgh's blog (he's a researcher who's affiliated with Solid). For example here's a nice post which goes into more detail on the ideas behind Solid: https://ruben.verborgh.org/blog/2017/12/20/paradigm-shifts-for-the-decentralized-web/ https://ruben.verborgh.org/blog/2017/12/20/paradigm-shifts-f... 2) In terms of applications, I'd personally like to see more of a focus on productivity applications, rather than so much focus on social media. Better interop between cloud SaaS apps would be valuable to businesses and professionals, and would sidestep many thorny challenges like decentralized moderation. As a concrete starting point: what if I could store my Google Docs text files on my own storage layer, and edit them in realtime using a variety of editor apps? This would resemble my ability to edit a .txt file on my computer in vim or TextEdit, but would port that metaphor to the world of modern online collaboration. Here's a short Twitter thread I wrote on this topic: https://twitter.com/geoffreylitt/status/1355255162626068482 https://twitter.com/geoffreylitt/status/1355255162626068482 3) I'm very curious to see more incremental paths from the current web to a decentralized approach. For example, what if we could start annotating existing websites with private data and sharing those annotations P2P, rather than starting over from scratch? I've explored this a bit with my Wildcard project, where users can store annotations in a "spreadsheet" that is linked to a web page: https://www.geoffreylitt.com/wildcard/ https://www.geoffreylitt.com/wildcard/

er4hn 6y ago
I think that with a problem statement of "I want to control my data and how third parties use my data" Solid makes a good deal of sense. Personal information should be owned by you and you should be able to take it from service to service as you see fit. The two problems that I find with Solid, based on my own limited understanding are as follows: (1) Services don't need to give back to Solid - Services, Facebook, your medical provider, whatever else you are using, does not have a clear incentive to provide their own data on you back to your Solid pod. It is far easier for them to keep it: it lets them do offline processing, and it keeps you more locked into their service. I'm not sure how one would solve this issue. (2) Much like mobile apps with excessive permissions and the abuse of tracking elements - I don't see how Solid prevents the abuse of its service. If Solid catches on and Facebook has a permissions check saying "Let Facebook do 'SELECT * FROM .;' on your Solid data, how many people will click yes? Even if you request it each time, once the data is copied out, it is out there and can be packaged and resold, used to build advertising profiles, etc. You're back to the original problem of not being able to limit access to your data, but with extra steps. Where I think this could be solved is by Solid not providing the data directly, but by being a service which can answer queries. Queries could be items such as "Does user like cats? y/n/m". Or it could be something like "Here is an anonymized dataset being built out. Please add your input to it." Replies to queries could also have an amount of deliberately wrong or misleading answers given, depending on the service and endpoint to obfuscate your personal data on places that don't need it. While this can still be abused, it raises the bar for abuse.

alpaca128 6y ago

To me the whole pitch is a bit too nebulous and seemingly inconsistent. What does it mean with "portable" and "interoperable" data standards? Does that mean specific file formats? Protocols? Apps? What do they mean with "Linked Data"? What's novel about this, what makes it different from hyperlinks or shared Dropbox folders? Why does it say pods are decentralized data stores, but then on the "about" page it tells me I have to host the pod on a server - how is that decentralized?

syats 6y ago
Linked Data does have a concrete definition: RDF data that is accessible through SPARQL, with well defined semantics. That is, following a published ontology and re-using existing vocabularies as much as possible. Linked Data itself is not novel: https://www.w3.org/standards/semanticweb/data https://www.w3.org/standards/semanticweb/data

kevincox 6y ago

Reminds me of https://remotestorage.io/ https://remotestorage.io/ which I have used in a (mostly personal) website. I like that I can provide a service as a static website, and users pay for, manage and have control over their own storage. One of the things that could help it bootstrap is that they also support client-side Google Drive and Dropbox shims, so users who want to use something else can, but most users will just use one or the other. There are downsides though. - The API may be less than ideal for some use cases. This can make a well-performing app hard. For example there is no relational-sql API. - You can't migrate user's data. So if you make changes to the format you need to retain backwards compatibility "forever". - If you want to provide "discovery" or cross-user features you will need to have your own storage anyways.

anderspitman 6y ago
I actually started working on just such a shim yesterday. The idea is to represent all major cloud storage through a single simple frontend API. You have to implement each provider's OAuth flow, but once you have the files it's pretty much the same. Dropbox, Google, and my own protocol[0] will come first, but Solid is planned eventually. What I've learned so far: * Google is extremely draconian unless you use their JavaScript picker. To really integrate nicely you would have to spend at least $15000-$75000 getting your app audited by a 3rd party. * Dropbox picker integration is very easy and slick. [0]: https://github.com/gemdrive https://github.com/gemdrive

fuzzybear3965 6y ago

Is this related to Jaron Lanier's vision of union (in the labor sense) groups which sell to companies access to the group's anonymized data? In his model, one individual can belong to many groups and their information is like a share/stock they have in a pool of data with other people. They sell in a cooperative fashion to select companies, gaining power through anonymity and group bargaining power.

rapnie 6y ago
Interesting. Do you have any good links?
rencire 6y ago
Would love to learn more about this. Any resources/links on this subject?
Nasrudith 6y ago
Data in those contexts nearly always have "reciprocial actions" fundamentally and those who gather the data know what they actually want out of it. Company-user interactions would likely produce better data. It feels like half of an understanding of both data usage and union style collective barganing. Unless I am missing something it seems a bit like trying to combine a semitruck and a bicycle to get green bulk cargo transportation. Sure both things individually have these advantages and it would be great to have all in one but implementing it is fundamentally nonviable.
asdfk-12 6y ago
Parent seems to be referring to MIDs [1] "Mediators of Individual Data (MIDs) are a union-like organization championed by technologist Jaron Lanier as a framework for addressing the key issues around user-generated data like ownership, monetization, and reparations. An effective MID or data union could perform a number of critical functions on behalf of its members that are currently lacking in the digital economy." There exists also a great Harvard Business Review article, but it is paywalled. [1] https://blog.datadividendproject.com/this-week-in-data-101-whats-a-mid/ https://blog.datadividendproject.com/this-week-in-data-101-w...

sharathr 6y ago

Love the idea but how does Solid prevent any malicious actor from over time replicating all the data that user authorize into a centralized repository - in this case, it only makes it even easier for data consolidation.

Vinnl 6y ago
In my (personal) view, Solid is technology that enables service providers to put you in control of the data, not that forces them to do so. In other words, it's just part of the puzzle, where the other part is e.g. customer demand or regulation.
- cxr 6y ago
  To use an old metaphor: locks on the doors are not what keep people from getting into a house.

hezag 6y ago

This post[0] by Ruben Verborgh helped me understand the problem Solid is trying to solve: > In December 2019, Google and Facebook proudly announced a major milestone, which was echoed in news media all around the world: it is now possible to copy a picture from Facebook to Google Photos. This news came in mere months after we celebrated the 50th anniversary of another technological feat: the moon landing of 20 July 1969, when millions of households witnessed Neil Armstrong take a giant leap for mankind. > So let me get this straight: two of the largest tech companies in history make headlines because in 2019, they move a single photo over the whopping distance of 11 km it takes from the Facebook headquarters in Menlo Park to the Googleplex in Mountain View, whereas in 1969, we sent live video signals from 380,000 km away on the actual moon? > If those two companies, both widely hailed as pinnacles of technology, genuinely consider this to be innovation they are proud of, the only logical conclusion is that data-driven innovation today is fundamentally broken. > The problem is widespread and not limited to technology or social media. Any sector that requires personal data to deliver services, from retail over insurance to health, suffers from the damaging effects of siloization. Companies increasingly need more access to data, but they won’t get there if they keep on collecting that data themselves.[...] 0. https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-fosters-sustainable-innovation/ https://ruben.verborgh.org/blog/2020/12/07/a-data-ecosystem-...

peterburkimsher 6y ago

I like the concept of Solid, and proposed it to the BeWelcome community (who are running a hospitality exchange network like CouchSurfing, but free). Although Solid is a better technical idea, I'm not sure how the migration from PHP/MySQL would work. Nobody on the Solid forum replied to my post asking about it. Another guy, Chagai, is pushing for Matrix and the fediverse. Ultimately, there's many great technical ideas, but the real solution doesn't lie in writing code - it's in building the community. Therefore I'm trying harder to create more activity in BeWelcome's existing community (which is hard when nobody's travelling), like an online meetup every Thursday. If it gets critical mass, I believe it could be as widespread as Wikipedia or OpenStreetMap (which it models in org structure). I wish that Solid could be a part of it, but that requires active developers, and currently it's hard enough to find someone willing to make an API or native iOS app, never mind a breaking back-end change that people aren't asking for.

gameman144 6y ago

How does this work in practice, I didn't see a worked example on the site. For instance, say I give Solid Social access to my contacts and my photos. At that point, Solid Social can access all my contacts and photos, copy them, and then do whatever they please with them (even if I later revoke permissions). In the current web, I upload my photos to a website, which I then trust not to use them inappropriately. In the Solid world, it seems like I give photo-access to a website (which could then copy them) and trust that website to not use them inappropriately. This seems like the current system, but with extra steps. In terms of data compatibility, I think that there's a potentially compelling case (user-driven standards vs. bespoke handling per website), but again I fail to see the practical implications. For example, if I upload a video to a website and it becomes incompatible with other websites, my assumption is that there's some reason for that step. It could be out of walled-garden malice, but it also could be that the website is encoding it and co-locating it in such a way that it'll be really fast for other users to watch. In Solid-world, it's unclear what the flow here should be: does the website only get to consume the videos as I have them on my pod (potentially terrible performance)? Does the website get to still do optimizations (and potentially keep artifacts of my video after I revoke permissions?) I like the drive towards consistent standards for interfacing with data, but other than that I could just be missing the important bits here (and please let me know if so), and I fail to see the positive change in most web-based user-experiences for the most part.

anderspitman 6y ago
Solid is just the technical part. You're also going to need the regulatory part that says it's illegal for a company stores data after you revoke access. But the technical part is still necessary.
- jwblackwell 6y ago
  How is this really any fundamentally different from say GDPR then?
  - anderspitman 6y ago

angst_ridden 6y ago

What Solid really needs to do is create a page that I could show to my nontechnical friends where it explains how it works and what services/sites support it. Without those two things being front and center, I can't see Solid being anything more than a niche interest for technical folks.

deleted 6y ago
[deleted]
zepatrik 6y ago
I think the problem there is that there is not much to show. But it will require that page at some point. Maybe it would help now as well as managers and other "decision makers" might understand it more. And they have to implement it in the end right?
volodymyrs 6y ago
At least for the services/sites/apps which support it, there is a page: https://solidproject.org/apps https://solidproject.org/apps There are lot's of apps, most of them work just fine, which is quite inspiring.

hackpert 6y ago

Last month, I was working with some people on building a decentralized health data store for vaccine waitlists (a la Mailbox) and certificates and we were looking at Solid as one potential backbone. I really admire what they’re trying to do. However, as some other people have observed, their example applications are quite awful and not very useful(?) and they seem to be marketing it to end users, not developers. In my experience, most end users do not care about privacy and security until 1) they either lose a lot of money or other valuables 2) some hacker comes knocking on their door for extortion and they lose a lot of money or other valuables. It would be so much better if they just made it so easy and simple for developers to use that it becomes their default, and not rely on hope that end users will suddenly start using decentralized apps in the name of privacy.

tasn 6y ago
I'm the founder of Etebase[1] an open-source and end-to-end encrypted backend for building apps. The goal is to maintain privacy and security while still bringing users the benefits of a central storage. Unlike Solid it's already used in a few places, like GNOME, KDE and etc. [1] https://www.etebase.com https://www.etebase.com

foolinaround 6y ago

a project with similar goals is Sandstorm.io i plan to check it out soon..

dreamling 6y ago

I haven't looked into it for a couple years, I was initially very excited about the concept. TLDR: Unless the user experience/accessibility/hosting experience is improved, I just couldn't suggest anyone use it. Last time I really dug in the user interface and accessibility concerns were not great. I understand the idea was more about the data handling, and that anyone could produce something based off the framework but I had all sorts of issues with the login tokens/usability. (ie, if you logged in with the wrong cert it didn't have an easy way to clear it, and would 'log in' to an error page. ect. ) as far as I can tell, each pod/data host can run different versions of solid, so your experience may vary between them. You can self host as well. I'd have to dig in again to see if things have gotten better, but I don't have time at the moment. on quick look, some of my data sites are no longer working. oh! link says it was shut down in oct. https://lists.w3.org/Archives/Public/public-solid/2020Oct/0005.html https://lists.w3.org/Archives/Public/public-solid/2020Oct/00... That said, the community was pretty nice and willing to answer questions. Maybe things have improved? I was hoping that maybe they'd start using DAT as well, because it seemed like it might be nice to dovetail with some of the beakerbrowser/distributed web stuff going on. re @pfrazee

dreamling 6y ago
quick followup, after checking my old links and making a new sign up. it does seem like the inrupt.com pod has a bit better /more up to date design. Accessibility, while not perfect, does seem to be improved. (quick WAVE tool check) Things are still fairly confusing for new users though. And some how tos I wrote down from then no longer work. Which is not the worst thing, as some of the tutorials required you to hover over and wait to be able to edit things. Much easier to look at though. ymmv for historical/hysterical reference, I uploaded my old solid adventuress file, it is not up to date, and some of the css is not always showing the right colors, ect. but you can see some of the directions I was looking into then: https://pod.inrupt.com/metahari/public/solidnotes/SolidAdventures.html https://pod.inrupt.com/metahari/public/solidnotes/SolidAdven...
- Vinnl 6y ago
  Note: it was only the old solid.community domain that shut down; the same servers are now reachable through solid.community. But yes, the default UI used there is very lacking and slow to develop. I also built a more developer-focused interface that allows you to view the data in your Pod, and that should be more accessible. If you're interested, you can try it at https://penny.vincenttunru.com https://penny.vincenttunru.com

eivarv 6y ago

I think it's important to point out (if only for end-users) that Solid puts your data "under your control" in the "allows you to authorize third party access to your data" – not in the "can actually control your data in practice, post-access". It'll let you granularly authorize first party access what data you have in your pod, but there can’t be any technical guarantees with regards to illegitimate sharing or otherwise copying (many might at least cache, for instance) – nor about what is collected and shared outside this system. It's own documentation even states this, if you dig a little. I keep seeing data-hubs and identity-providers touting themselves as solutions to the web's privacy issues, but I don't see how they actually solve anything. It seems like an attempted technical solution to a social problem to me. The real problem with data how data is used these days is really that a bunch of data is collected opaquely, unethically, and in some cases illegally – and possibly shared. The whole system including data brokers and real time bidding is out of control.

dang 6y ago

If curious see also 2019 (a bit) https://news.ycombinator.com/item?id=20017493 https://news.ycombinator.com/item?id=20017493 2019 (a bit) https://news.ycombinator.com/item?id=20617506 https://news.ycombinator.com/item?id=20617506 2018 https://news.ycombinator.com/item?id=18100895 https://news.ycombinator.com/item?id=18100895 2018 https://news.ycombinator.com/item?id=16355311 https://news.ycombinator.com/item?id=16355311 2018 (a bit) https://news.ycombinator.com/item?id=18119227 https://news.ycombinator.com/item?id=18119227 2016 https://news.ycombinator.com/item?id=12280764 https://news.ycombinator.com/item?id=12280764 Others?

yiyus 6y ago

From a quick look at the site and a quick look at perkeep's site, both projects look quite similar. Could anyone with some experience comment on their differences?

anderspitman 6y ago

Solid is awesome and I'm bullish on this model in the long-run, but it seems to be caught in a classic chicken/egg problem. Nobody is going to use it until there are good apps. and nobody is going to make good apps until there is a market of users. IMO this is the same problem that kept Sandstorm and remoteStorage from getting big. I see 2 solutions currently: 1. Make an awesome storage product, that just happens to implement Solid. It might have to start out also implementing S3, which unfortunately doesn't support OAuth, which basically kills it for apps. You could try making it compatible with another proprietary protocol like Google Drive, or provide a frontend library that easily talks to all the major cloud storage providers, plus Solid... 2. Take the slow track. If developers themselves want to use it, they'll write useful open source apps over time. This can take forever though. Even after 30 years there are still large feature gaps on Linux because the market is too small to attract big app developers.

cxr 6y ago
> It might have to start out also implementing S3, which unfortunately doesn't support OAuth The expectation that Solid is backed by a smart server that's intimately aware that it's hosting a Solid pod and what exactly it takes to do so is a critical failure in Solid's design. If Solid's protocols instead better accommodated "dead" formats that can be served by static web hosts, so your $USERNAME.github.io could be the front door to your data store—at least the parts of it that you already make available to the public, anyway—then we'd have passed the chicken-and-egg stage years ago and Solid would already be a runaway success. At the end of the day, it's supposed to be about data, after all—linked data—and not about the endpoints/servers themselves. Compare to the ease of creating an RSS feed for your blog (you can get your static site generator to spit one out pretty easily) and contrast it to needing to provision and actively administer a Mastodon server, for example.
- walterbell 6y ago
  > "dead" formats that can be served by static web hosts, so your $USERNAME.github.io could be the front door to your data store—at least the parts of it that you already make available to the public Could you share some examples of such data, other than "news" feeds? FOAF, for example? http://www.foaf-project.org/ http://www.foaf-project.org/

EGreg 6y ago

Tim went on to leave MIT, found Inrupt and get funded to the tune of several million dollars. Some great guys there on the solid team, including Melvin Carvalho and others who maintain ties with the MaidSAFE people. Give it time, it’s the future.

hikerclimb 6y ago

This is probably a lie just like our media is a lie... did you know that robinhood was supposed to raise 6 billion but only raised 1 billion according to nytimes. They probably deleted the article so in case they get screwed. I hope robinhood and citadel get screwed over and fail. Karmas a bitch ain’t it?

matt_f 6y ago

I want to love this. But, in trying to imagine any scenario in which tech corporations might conceivably start hosting their internal user data on these external "Solid Pods", I'm left drawing a blank. Until we have a way that tech companies start storing users' data in user-controlled infrastructure, I think this idea is a noble fantasy. I would love to hear an argument that will convince me I'm wrong on this.

Vinnl 6y ago
The only scenario that I think is likely for tech corporations are either legislative pressure, or a tech corporation for which user tracking is not their core competency (e.g. Apple) using it as their USP as a moat against Google and Facebook. That said, there's a lot of user data at non-tech companies. Especially given that they're not particularly skilled at securing that data, a changing regulatory climate turns that data more and more into a liability. If, as a user, you can ensure that that data is stored at a trusted and competent party (potentially yourself), that alone would already be a major win, in my (personal) opinion.