5 ms·

Too many ideas given the time, but now I'm focused on two: 1. Building an open source bicycle computer. It's been over 20 years since they arrived at the scene

Too many ideas given the time, but now I'm focused on two: 1. Building an open source bicycle computer. It's been over 20 years since they arrived at the scene, and there still isn't one that you could hack! An outrage! No published sources yet, I'm getting stuck on not having much experience building physical things. 2. Getting rid of the directory hierarchy. I have 10K photos, 30K emails, 10000km of GPS tracks, and 10 years of chat logs. Why can't I find anything among them? I own a computer, after all. I have 10 folders called some variation of photos/Cologne/2020/flowers! Having to organize them myself is tedious and a fool's errand, so I'm leaning towards using a database as a file system, to let me just query for files. Geo queries using a map? Yes please. Selecting bounds on a timeline? Oh yeah! Turns out I'm not alone, Microsoft tried this with WinFS, and failed. But the idea lives on: https://www.nayuki.io/page/designing-better-file-organization-around-tags-not-hierarchies https://www.nayuki.io/page/designing-better-file-organizatio... 3. Writing. I hope I can find the time to expand on the above on my blog.

Doctor_Fegg 5y ago
Have a look at Bangle.js. It's nominally an open source smartwatch, but the difference between that and a bike GPS is just a handlebar mount. v2 has just gone on Kickstarter - I've ordered one specifically for bike mounting.
- dcz_self 5y ago
  Nice, thanks! Now I need to reconsider. My goals were to have 180+ days battery life and wired connectors, but maybe no one really wants those.
  - Doctor_Fegg 5y ago
    If you do go for Bangle.js, ping me if you get to the stage of putting any code up on github - I'd be very interested.
    - dcz_self 5y ago
      Would you be interested in cooperating on that? You can ping me over email (see my profile or the blog). My second impression about the Bangle.js is: "oh no, JavaScript only?". Still, it's much better than building the same thing from scratch, except it may still take effort on the software side. Having another interested person involved would be amazing for my motivation! Btw, one of the reasons I wanted hackable hardware was to map road surfaces on OSM ;) GPS alone doesn't add much to OSM in my area any more.
bspammer 5y ago
For your second point, you might want to have a look at Perkeep: https://perkeep.org/ https://perkeep.org/ It supports the kind of queries you are after, but it’s a bit of a passion project and development is slow. What’s already there works well though.
- dcz_self 5y ago
  Thanks for confirming. I intended to check on perkeep, but my frustration never reached that point before. I hope it can be adjusted to my needs: I expect I'm going to have to find solutions for tracking files' sources, for snapshots, and for remote/removable storage, in addition to all the UI work.
  - bspammer 5y ago
    The included web UI is quite nice (they really should include a screenshot on the website), and setting up backups/remotes is very simple, everything is content addressed so you can just copy stuff without fear. Snapshots I suspect wouldn’t be too hard.
sideshowb 5y ago
I presume you've checked out the likes of Lightroom for photo organizing? Though don't let that put you off something better!
- dcz_self 5y ago
  Lightroom is closed and comes from Adobe, so it was never on my radar. Being able to modify the software is a must: photos are only part of the problem, so any solution must be open to integration with other sources of data. It would have been interesting to see how it organizes the UI, but there's no demo version for Linux :P
  - sideshowb 5y ago
    Fair enough on the closed source thing! My take is that we have specialist requirements for photos that are probably better served by a photo specific UI. Lightroom's doesn't inspire me to say it's great, but it's adequate. Can search by metadata like camera or capture date, save previous search results as 'smart folders' (e.g. I have one for all pictures since I last had a book printed). It indexes thumbnails for quick scrolling. My ancient version doesn't do geosearch but maybe later ones do. You can clone pictures (like a hardlink) rather than copy into different albums. You can also clone pictures but apply different non destructive edits to the same source file. Almost all the edit operations I want are close at hand. Order is preserved in albums. Rate photos 0-5* and tag with different colours. As to how all that is organized, I'm sure it could be improved on. I've also yet to see a good photo solution for my parents who (like many non tech folk) still struggle to understand folder hierarchies and where they saved stuff. Picasa used to be ok-ish but discontinued. Google photos is a bit of a pain over their ADSL. Good luck with it anyway!
    - dcz_self 5y ago
      Thanks, that's a great summary. It does seem it's most of the way there already. How does it handle removable storage? Ironically, the fact that a topic-specific UI is needed for topic-related tasks is another strike against closed software. Something open would have been a starting point for other sorts of views (email? PDFs?), but when there's no modification, there's no ability to do that.
chalcolithic 5y ago
How much would you pay for a program that would help you do 2 way more effectively than just by querying it?
- dcz_self 5y ago
  That depends on how easily I could compile it, and how much more effective it would be than querying. At the moment I can't come up with anything better than queries. I'm willing to part with some money to have someone implement it for me if I see there's demand.
  - chalcolithic 5y ago
    Queries kind of suck when it comes hierarchy, do they? And you sure have some folder hierarchy, even if you're not happy about it?
    - dcz_self 5y ago
      I identify the hierarchy as the source of my problems. I don't need hierarchy, and when I use it, it's because that's the one tool I have. There's no hierarchy between time and place, and I shouldn't be forced to establish it. There might be some hierarchy between tags: PhD/laser experiment, PhD/dissertation, but there's rarely a need for it to be more than 1-2 levels deep, so I prefer to suffer the pains of inadequate hierarchy rather than inadequate queries. Or at least I would if I could try it out.
      - chalcolithic 5y ago
        I was mostly referring to the fact that filesystem kind of enforces hierarchy, so you have it even if you dislike it. When I'm sorting out the mess that is my hard drive I defenitely value information about hierarchy overlap (due to (incomplete) copies), but I still have not found a tool that would help me an I can't imagine a query language that would be helpful.
        5y ago
drran 5y ago
StagesCycling Dash L50/M50 are open to hack: bootloader is not protected, SD card can be replaced, and ssh access is open (when device is connected by USB).
- dcz_self 5y ago
  Do you have any links confirming that? I'm failing to find any resources.
  - Doctor_Fegg 5y ago
    https://www.reddit.com/r/linuxmasterrace/comments/ehn19r/how_to_hack_stages_cycling_dash_l50_or_m50/ https://www.reddit.com/r/linuxmasterrace/comments/ehn19r/how...
joshxyz 5y ago
Maybe the answer is tags/attributes for filtering, instead of nested directories? Could be more tedious but given the image recognition ArTiFiCiAl iNtElLiGeNcE maybe it could be easier
- Zababa 5y ago
  I think it is. Directory hierarchies are trees, tags are graphs. Trees are great when you need to optimise the physical retrieving time, but don't map well to stuff like pictures. For example, tags of who is in the picture are way better than making folders by person. Same thing for places.
- dcz_self 5y ago
  That's what I'm thinking about too. Most directories are describing what's already available for the computer to extract: the place of something being created, the time of it, the source where it came from (e.g. who sent this as email attachment). Only a minority needs to be explicitly added, like emotional meaning ("my favorites"), or external organization ("experiment 1", "driver's licence"). While we can't use the computer for the latter, it's nonsense that we leverage computers so little for the former.
delegate 5y ago
I've been working on 2. since January. All your photos, chats, e-mails, messages, health data (eventually all your data) into a single database, which can be distributed onto multiple removable 'volumes' (disks). Plug in the volume and the db index is seamlessly merged and queries run on all the connected volumes. No files. No cloud. Data on your disks only. There are processing nodes that run in the background indexing the content, finding people and locations in the pictures, chats. It allows you to build up a fully search-able archive of all your data, which you can physically keep in a locker or safe. But also, if you store and keep all your data on disks that you control, then many of the services that we use could just run as local apps on your server, reading and writing from your archive via an API. Social apps are just two people's archives sharing content between each other, no 3rd party services required. I worked a lot on it until about 1.5 months ago, when I got discouraged by how hard and big this project is. Breaking up with my girlfriend around that time did not help either. But now I see you mention it and it's kind of inspiring, since I've been doing a lot of thinking and gathering strength for another crack at getting a prototype out. It would be interesting to see what other people have been thinking/doing in this area. But since there's at least one person who wants it, I might actually jump back to work ;) Thank you !
- dcz_self 5y ago
  Do you have any writeups/examples in the public? I am on purpose not addressing the exchange of information between people – never considered it, no vision, would just confuse me. I'm also not so keen on indexing. I'm envisioning this more as a file system layer, where data is added to the database as it gets saved. Ideally there would be no way to access the data outside of the database (that would cause desyncs and the need to reindex stuff). We don't need no POSIX file systems ;)
- aydwi 5y ago
  I can empathize with this to some extent. I realized that homebrew is too slow and bloaty for me, so I decided to write an alternative from scratch (also meant as a learning project). Eventually got discouraged by the sheer amount of real-world workarounds and corner cases that need to be accomodated rather than some inexplicable technical hurdle. And broke up with my long-time girlfriend. Now I have little desire to go back, and things just feel meh. I play basketball sometimes to keep my sanity.
benibela 5y ago
I want the opposite for 2. Keep all my photos in the directory hierarchy, but have a tool to sort them and sync them between devices If I sync it with rsync, I might accidentally delete important files. Git would sync properly, but then it keeps the data in the repo rather than the directory hierarchy

Doctor_Fegg 5y ago

Have a look at Bangle.js. It's nominally an open source smartwatch, but the difference between that and a bike GPS is just a handlebar mount. v2 has just gone on Kickstarter - I've ordered one specifically for bike mounting.

dcz_self 5y ago
Nice, thanks! Now I need to reconsider. My goals were to have 180+ days battery life and wired connectors, but maybe no one really wants those.
- Doctor_Fegg 5y ago
  If you do go for Bangle.js, ping me if you get to the stage of putting any code up on github - I'd be very interested.
  - dcz_self 5y ago
    Would you be interested in cooperating on that? You can ping me over email (see my profile or the blog). My second impression about the Bangle.js is: "oh no, JavaScript only?". Still, it's much better than building the same thing from scratch, except it may still take effort on the software side. Having another interested person involved would be amazing for my motivation! Btw, one of the reasons I wanted hackable hardware was to map road surfaces on OSM ;) GPS alone doesn't add much to OSM in my area any more.

bspammer 5y ago

For your second point, you might want to have a look at Perkeep: https://perkeep.org/ https://perkeep.org/ It supports the kind of queries you are after, but it’s a bit of a passion project and development is slow. What’s already there works well though.

dcz_self 5y ago
Thanks for confirming. I intended to check on perkeep, but my frustration never reached that point before. I hope it can be adjusted to my needs: I expect I'm going to have to find solutions for tracking files' sources, for snapshots, and for remote/removable storage, in addition to all the UI work.
- bspammer 5y ago
  The included web UI is quite nice (they really should include a screenshot on the website), and setting up backups/remotes is very simple, everything is content addressed so you can just copy stuff without fear. Snapshots I suspect wouldn’t be too hard.

sideshowb 5y ago

I presume you've checked out the likes of Lightroom for photo organizing? Though don't let that put you off something better!

dcz_self 5y ago
Lightroom is closed and comes from Adobe, so it was never on my radar. Being able to modify the software is a must: photos are only part of the problem, so any solution must be open to integration with other sources of data. It would have been interesting to see how it organizes the UI, but there's no demo version for Linux :P
- sideshowb 5y ago
  Fair enough on the closed source thing! My take is that we have specialist requirements for photos that are probably better served by a photo specific UI. Lightroom's doesn't inspire me to say it's great, but it's adequate. Can search by metadata like camera or capture date, save previous search results as 'smart folders' (e.g. I have one for all pictures since I last had a book printed). It indexes thumbnails for quick scrolling. My ancient version doesn't do geosearch but maybe later ones do. You can clone pictures (like a hardlink) rather than copy into different albums. You can also clone pictures but apply different non destructive edits to the same source file. Almost all the edit operations I want are close at hand. Order is preserved in albums. Rate photos 0-5* and tag with different colours. As to how all that is organized, I'm sure it could be improved on. I've also yet to see a good photo solution for my parents who (like many non tech folk) still struggle to understand folder hierarchies and where they saved stuff. Picasa used to be ok-ish but discontinued. Google photos is a bit of a pain over their ADSL. Good luck with it anyway!
  - dcz_self 5y ago
    Thanks, that's a great summary. It does seem it's most of the way there already. How does it handle removable storage? Ironically, the fact that a topic-specific UI is needed for topic-related tasks is another strike against closed software. Something open would have been a starting point for other sorts of views (email? PDFs?), but when there's no modification, there's no ability to do that.

chalcolithic 5y ago

How much would you pay for a program that would help you do 2 way more effectively than just by querying it?

dcz_self 5y ago
That depends on how easily I could compile it, and how much more effective it would be than querying. At the moment I can't come up with anything better than queries. I'm willing to part with some money to have someone implement it for me if I see there's demand.
- chalcolithic 5y ago
  Queries kind of suck when it comes hierarchy, do they? And you sure have some folder hierarchy, even if you're not happy about it?
  - dcz_self 5y ago
    I identify the hierarchy as the source of my problems. I don't need hierarchy, and when I use it, it's because that's the one tool I have. There's no hierarchy between time and place, and I shouldn't be forced to establish it. There might be some hierarchy between tags: PhD/laser experiment, PhD/dissertation, but there's rarely a need for it to be more than 1-2 levels deep, so I prefer to suffer the pains of inadequate hierarchy rather than inadequate queries. Or at least I would if I could try it out.
    - chalcolithic 5y ago
      I was mostly referring to the fact that filesystem kind of enforces hierarchy, so you have it even if you dislike it. When I'm sorting out the mess that is my hard drive I defenitely value information about hierarchy overlap (due to (incomplete) copies), but I still have not found a tool that would help me an I can't imagine a query language that would be helpful.
      - 5y ago

drran 5y ago

StagesCycling Dash L50/M50 are open to hack: bootloader is not protected, SD card can be replaced, and ssh access is open (when device is connected by USB).

dcz_self 5y ago
Do you have any links confirming that? I'm failing to find any resources.
- Doctor_Fegg 5y ago
  https://www.reddit.com/r/linuxmasterrace/comments/ehn19r/how_to_hack_stages_cycling_dash_l50_or_m50/ https://www.reddit.com/r/linuxmasterrace/comments/ehn19r/how...

joshxyz 5y ago

Maybe the answer is tags/attributes for filtering, instead of nested directories? Could be more tedious but given the image recognition ArTiFiCiAl iNtElLiGeNcE maybe it could be easier

Zababa 5y ago
I think it is. Directory hierarchies are trees, tags are graphs. Trees are great when you need to optimise the physical retrieving time, but don't map well to stuff like pictures. For example, tags of who is in the picture are way better than making folders by person. Same thing for places.
dcz_self 5y ago
That's what I'm thinking about too. Most directories are describing what's already available for the computer to extract: the place of something being created, the time of it, the source where it came from (e.g. who sent this as email attachment). Only a minority needs to be explicitly added, like emotional meaning ("my favorites"), or external organization ("experiment 1", "driver's licence"). While we can't use the computer for the latter, it's nonsense that we leverage computers so little for the former.

delegate 5y ago

I've been working on 2. since January. All your photos, chats, e-mails, messages, health data (eventually all your data) into a single database, which can be distributed onto multiple removable 'volumes' (disks). Plug in the volume and the db index is seamlessly merged and queries run on all the connected volumes. No files. No cloud. Data on your disks only. There are processing nodes that run in the background indexing the content, finding people and locations in the pictures, chats. It allows you to build up a fully search-able archive of all your data, which you can physically keep in a locker or safe. But also, if you store and keep all your data on disks that you control, then many of the services that we use could just run as local apps on your server, reading and writing from your archive via an API. Social apps are just two people's archives sharing content between each other, no 3rd party services required. I worked a lot on it until about 1.5 months ago, when I got discouraged by how hard and big this project is. Breaking up with my girlfriend around that time did not help either. But now I see you mention it and it's kind of inspiring, since I've been doing a lot of thinking and gathering strength for another crack at getting a prototype out. It would be interesting to see what other people have been thinking/doing in this area. But since there's at least one person who wants it, I might actually jump back to work ;) Thank you !

dcz_self 5y ago
Do you have any writeups/examples in the public? I am on purpose not addressing the exchange of information between people – never considered it, no vision, would just confuse me. I'm also not so keen on indexing. I'm envisioning this more as a file system layer, where data is added to the database as it gets saved. Ideally there would be no way to access the data outside of the database (that would cause desyncs and the need to reindex stuff). We don't need no POSIX file systems ;)
aydwi 5y ago
I can empathize with this to some extent. I realized that homebrew is too slow and bloaty for me, so I decided to write an alternative from scratch (also meant as a learning project). Eventually got discouraged by the sheer amount of real-world workarounds and corner cases that need to be accomodated rather than some inexplicable technical hurdle. And broke up with my long-time girlfriend. Now I have little desire to go back, and things just feel meh. I play basketball sometimes to keep my sanity.

benibela 5y ago

I want the opposite for 2. Keep all my photos in the directory hierarchy, but have a tool to sort them and sync them between devices If I sync it with rsync, I might accidentally delete important files. Git would sync properly, but then it keeps the data in the repo rather than the directory hierarchy