3 ms·

How many people are actually effectively organizing their data in files? If a system doesn't work for vast majority of users, then it does deserve rethinking. I

by zokier 5y ago

How many people are actually effectively organizing their data in files? If a system doesn't work for vast majority of users, then it does deserve rethinking. Its great that desktop model works for you, but I strongly suspect you are not representative of the greater user base.

asymptosis 5y ago
Rethinking is fine, if it leads to action. If the desktop isn't working for the vast majority of users, then it's certainly the prerogative of anyone in the world to code up something which is guaranteed to make every one of those users happy.
superkuh 5y ago
>If a system doesn't work for vast majority of users, then it does deserve rethinking. No, if a commercial for profit system doesn't work for the vast majority of users, then it does deserve rethinking. Linux is a great example of an operating system that was not written for the highest profitable denominator until recently.
BuyMyBitcoins 5y ago
I feel like the author’s proposed solution of better searching through tagging and semantic associations over manual organization also requires user care and effort. I don’t think the desktop model is broken so much as the average person isn’t very organized to begin with.
- marrs 5y ago
  This tagging idea resurfaces every few years. I have to confess I don’t really see where the revolutionary improvement is. Filenames are already namespaced tags and symlinks allow for multiple tags. Not quite the same thing, I know, but close enough for tagging to appear to me to be a convenient enhancement rather than something game changing. And as you imply, when you have a lot of files that you need to organise, you tend to start compiling them into directories.
  - imiric 5y ago
    So your solution to organizing files... is more files? How would multiple files use the same "tag" under that scheme, without duplicating it everywhere? Tags are very powerful if used properly. They describe sets by definition and allow quick filtering with intersections and other set operations. If the filtering is a bit smarter it can support boolean operators, or use tag distance and order as a meaningful data point so that e.g. "discussion board" doesn't return the same results as "discussion snow board". They're a more flexible way to organize data than hierarchical directories. Hierarchy can easily be expressed with tags (use any character as separator, e.g. "os.linux"), but files and directories are not nearly as expressive enough for all the use cases tags can be used for.
    - u210601 5y ago
      I've been working on a file organization system that incorporates tags, and I must say I agree with the GP. Tags improve display and search over hierarchies, but they require no less discipline on the part of the user. Without some way to automatically mass-assign meaningful tags to items, which is an insanely difficult problem in the general case, the user is forced to manually tag every single item they add to the system. AI could help do this, but not much. First, consider that mistagging an item can make it practically impossible to find again. Second, consider that semantic tagging can be almost arbitrarily abstract; imagine tagging a text document with "cyberpunk" or an image file with "parody".
      - heisenbit 5y ago
        The genius of Google search was leveraging hyperlinks to add semantics. The only way I can see something somewhat equivalent happening is by exposing all my interactions incl. e.g. the juggling and browsing of other documents beyond the one worked on to search. I would find this scary.
    - marrs 5y ago
      > So your solution to organizing files... is more files? How would multiple files use the same "tag" under that scheme, without duplicating it everywhere? You could put the same file in multiple directories. You could have a file in ~/vat and symlink it to ~/urgent and ~/accountant.
smoldesu 5y ago
The majority of users don't even realize they're interacting with files. The mantle falls entirely on developers to ensure that their programs are simple.
Quekid5 5y ago
Most users have no idea of what is even possible with a computer, organization-wise. I don't disagree that interfaces should be designed for users, but they should be designed to EMPOWER and teach users, not to just dumb everything down to the lowest common denominator. Because, let's face it, most people are awful at organizing stuff that materially matters a lot to their lives and well-being.
Veserv 5y ago
Except it does not work for those users because automatically managing vast unordered, uncategorized data sets is not a problem we know how to solve. If we knew how to solve that problem, we could just implement it in the existing hierarchical manually-organized paradigm by just having one hierarchical level and dumping all your data into it. Given that the new model fits in the old model, the only reason for demolishing the old model is for efficiency reasons and there is very little in the way of efficiency that can be gained by converting the relatively shallow organizational trees that are commonly used for a single-level store. Why do I believe we do not know how to solve the problem? As the article inadvertently points out, the unorganized data they want is not accessible and Mac Spotlight, a tool that operates on unorganized data, does an "OK Job, but not a miraculous one". If Apple knew how to super search the vast trove of unorganized data and file formats for content-specific data and always get it right would there be any reason they would not do so? It would clearly be strictly superior to their existing offering. For that matter, they bring up browsing history in Safari which is an even easier problem and also claim it hardly works. For evidence outside of Apple, think of all the in-app search systems that can barely even search plaintext for things you know exist. I frequently have problems with Gmail not finding emails despite giving exact string matches. These problems are all subsets of the proposed problem, which must be solved to solve the harder problem, and which would offer immediate material benefits for their solution yet do not or barely exist. The problem is not a mismatch between solution and user workflow. The problem is that we do not know how to solve the problem for all meaningful workflows. The best we have achieved is creating and supporting a workflow that allows the problem to be solved in many useful cases. Luckily, if the problem can actually be solved in the future, it can easily be bolted onto the existing solution to test viability before going all-in.
- TeMPOraL 5y ago
  > It would clearly be strictly superior to their existing offering. For that matter, they bring up browsing history in Safari which is an even easier problem and also claim it hardly works. For evidence outside of Apple For evidence outside of Apple, just try to use browser history in Chrome or Firefox. They're completely unreliable, entirely useless. For me, it's 50/50 chance I'll get a result when searching for a site that I visited few days earlier. > think of all the in-app search systems that can barely even search plaintext for things you know exist Exactly. Most of the search systems I've worked with have one, big problem: they don't feel reliable. You type in a query, a system starts a search. You get some indicator of progress, which eventually expires. Typically, search systems don't tell you if the search was exhaustive - did it check every possible thing that could match, or did it bail out after hitting some time limit? They also often meddle with the query, modifying it or doing fuzzy matches, in ways not communicated to you in the user interface. With such systems, if the thing I'm looking for isn't in the result set, I don't feel confident it isn't there. Manual file organization (or any direct data access) has this property of being exhaustive: a file is either there or it isn't. Direct file access means that if you don't trust your file searcher, you can always look for yourself. There are many other challenges ahead if we want "universal search" to work, but a big step forward would be addressing these trust issues. It's entirely an UI issue. Instead of "0 results found", say "0 results found after searching contents of all text files in Documents folder (symbolic links not followed)". Instead of "about 123 results", say "123 results found, there may be more matches, [click here] to read about limitations of the search method". (And on a general point: users aren't as dumb as the common claim is. Our industry is treating them as dumb, taking away every opportunity to learn and build mental models - and then complains that users "are dumb".)
wvenable 5y ago
> How many people are actually effectively organizing their data in files? Most people, in their jobs, every day. Just as they did before the advent of computers, too.
nitwit005 5y ago
You'll get a biased answer because you'll include a group of people who won't succeed no matter what approach you take. People have tried that approach and ended up with things like Microsoft Bob. The real question is what is effective for individuals and business trying to accomplish real work, who are at least somewhat capable of learning or training staff.

by zokier 5y ago

asymptosis 5y ago

Rethinking is fine, if it leads to action. If the desktop isn't working for the vast majority of users, then it's certainly the prerogative of anyone in the world to code up something which is guaranteed to make every one of those users happy.

superkuh 5y ago

>If a system doesn't work for vast majority of users, then it does deserve rethinking. No, if a commercial for profit system doesn't work for the vast majority of users, then it does deserve rethinking. Linux is a great example of an operating system that was not written for the highest profitable denominator until recently.

BuyMyBitcoins 5y ago

I feel like the author’s proposed solution of better searching through tagging and semantic associations over manual organization also requires user care and effort. I don’t think the desktop model is broken so much as the average person isn’t very organized to begin with.

marrs 5y ago
This tagging idea resurfaces every few years. I have to confess I don’t really see where the revolutionary improvement is. Filenames are already namespaced tags and symlinks allow for multiple tags. Not quite the same thing, I know, but close enough for tagging to appear to me to be a convenient enhancement rather than something game changing. And as you imply, when you have a lot of files that you need to organise, you tend to start compiling them into directories.
- imiric 5y ago
  So your solution to organizing files... is more files? How would multiple files use the same "tag" under that scheme, without duplicating it everywhere? Tags are very powerful if used properly. They describe sets by definition and allow quick filtering with intersections and other set operations. If the filtering is a bit smarter it can support boolean operators, or use tag distance and order as a meaningful data point so that e.g. "discussion board" doesn't return the same results as "discussion snow board". They're a more flexible way to organize data than hierarchical directories. Hierarchy can easily be expressed with tags (use any character as separator, e.g. "os.linux"), but files and directories are not nearly as expressive enough for all the use cases tags can be used for.
  - u210601 5y ago
    I've been working on a file organization system that incorporates tags, and I must say I agree with the GP. Tags improve display and search over hierarchies, but they require no less discipline on the part of the user. Without some way to automatically mass-assign meaningful tags to items, which is an insanely difficult problem in the general case, the user is forced to manually tag every single item they add to the system. AI could help do this, but not much. First, consider that mistagging an item can make it practically impossible to find again. Second, consider that semantic tagging can be almost arbitrarily abstract; imagine tagging a text document with "cyberpunk" or an image file with "parody".
    - heisenbit 5y ago
      The genius of Google search was leveraging hyperlinks to add semantics. The only way I can see something somewhat equivalent happening is by exposing all my interactions incl. e.g. the juggling and browsing of other documents beyond the one worked on to search. I would find this scary.
  - marrs 5y ago
    > So your solution to organizing files... is more files? How would multiple files use the same "tag" under that scheme, without duplicating it everywhere? You could put the same file in multiple directories. You could have a file in ~/vat and symlink it to ~/urgent and ~/accountant.

smoldesu 5y ago

The majority of users don't even realize they're interacting with files. The mantle falls entirely on developers to ensure that their programs are simple.

Quekid5 5y ago

Most users have no idea of what is even possible with a computer, organization-wise. I don't disagree that interfaces should be designed for users, but they should be designed to EMPOWER and teach users, not to just dumb everything down to the lowest common denominator. Because, let's face it, most people are awful at organizing stuff that materially matters a lot to their lives and well-being.

Veserv 5y ago

Except it does not work for those users because automatically managing vast unordered, uncategorized data sets is not a problem we know how to solve. If we knew how to solve that problem, we could just implement it in the existing hierarchical manually-organized paradigm by just having one hierarchical level and dumping all your data into it. Given that the new model fits in the old model, the only reason for demolishing the old model is for efficiency reasons and there is very little in the way of efficiency that can be gained by converting the relatively shallow organizational trees that are commonly used for a single-level store. Why do I believe we do not know how to solve the problem? As the article inadvertently points out, the unorganized data they want is not accessible and Mac Spotlight, a tool that operates on unorganized data, does an "OK Job, but not a miraculous one". If Apple knew how to super search the vast trove of unorganized data and file formats for content-specific data and always get it right would there be any reason they would not do so? It would clearly be strictly superior to their existing offering. For that matter, they bring up browsing history in Safari which is an even easier problem and also claim it hardly works. For evidence outside of Apple, think of all the in-app search systems that can barely even search plaintext for things you know exist. I frequently have problems with Gmail not finding emails despite giving exact string matches. These problems are all subsets of the proposed problem, which must be solved to solve the harder problem, and which would offer immediate material benefits for their solution yet do not or barely exist. The problem is not a mismatch between solution and user workflow. The problem is that we do not know how to solve the problem for all meaningful workflows. The best we have achieved is creating and supporting a workflow that allows the problem to be solved in many useful cases. Luckily, if the problem can actually be solved in the future, it can easily be bolted onto the existing solution to test viability before going all-in.

TeMPOraL 5y ago
> It would clearly be strictly superior to their existing offering. For that matter, they bring up browsing history in Safari which is an even easier problem and also claim it hardly works. For evidence outside of Apple For evidence outside of Apple, just try to use browser history in Chrome or Firefox. They're completely unreliable, entirely useless. For me, it's 50/50 chance I'll get a result when searching for a site that I visited few days earlier. > think of all the in-app search systems that can barely even search plaintext for things you know exist Exactly. Most of the search systems I've worked with have one, big problem: they don't feel reliable. You type in a query, a system starts a search. You get some indicator of progress, which eventually expires. Typically, search systems don't tell you if the search was exhaustive - did it check every possible thing that could match, or did it bail out after hitting some time limit? They also often meddle with the query, modifying it or doing fuzzy matches, in ways not communicated to you in the user interface. With such systems, if the thing I'm looking for isn't in the result set, I don't feel confident it isn't there. Manual file organization (or any direct data access) has this property of being exhaustive: a file is either there or it isn't. Direct file access means that if you don't trust your file searcher, you can always look for yourself. There are many other challenges ahead if we want "universal search" to work, but a big step forward would be addressing these trust issues. It's entirely an UI issue. Instead of "0 results found", say "0 results found after searching contents of all text files in Documents folder (symbolic links not followed)". Instead of "about 123 results", say "123 results found, there may be more matches, [click here] to read about limitations of the search method". (And on a general point: users aren't as dumb as the common claim is. Our industry is treating them as dumb, taking away every opportunity to learn and build mental models - and then complains that users "are dumb".)

wvenable 5y ago

> How many people are actually effectively organizing their data in files? Most people, in their jobs, every day. Just as they did before the advent of computers, too.

nitwit005 5y ago

You'll get a biased answer because you'll include a group of people who won't succeed no matter what approach you take. People have tried that approach and ended up with things like Microsoft Bob. The real question is what is effective for individuals and business trying to accomplish real work, who are at least somewhat capable of learning or training staff.