11 ms·

now pages of 7k+ personal sites

586 points by louisbarclay 3y ago

Hi HN! It’s hard to find interesting people to work with on your ideas. Our solution: index the /about, /ideas, /now pages of 1000s of personal websites. There are thousands of cool personal sites out there, with amazing ideas on them, but there’s nowhere to easily search through. So we built a simple site that indexes 7k+ personal sites [0]. We were inspired by Derek Sivers’ Now page movement [1] and other IndieWeb directories [2], but we figured that it would be more useful if we: * Let you search directly across personal sites without having to visit them * Take the content from 3 specific pages, /about, /now and /ideas, to structure everything * Define /ideas pages as a space to articulate things you want to work on We hope this’ll be a cool place for people to find others to collaborate with - would love your feedback. If you’d like your site to appear at the top, add it via the form and add a last updated date of today (any format). It’s completely open source (MIT) and open to contributions [3]! Peter & Louis [0] gathered from: 1) https://nownownow.com https://nownownow.com and similar sites 2) checking all HN posts since 2020 with more than 100 upvotes [1] https://nownownow.com https://nownownow.com [2] https://personalsit.es https://personalsit.es [3] https://github.com/lindylearn/aboutideasnow https://github.com/lindylearn/aboutideasnow

sarimkx 3y ago
This is amazing. I was just in the process of building a directory of personal websites.
Beefin 3y ago
this is amazing, so many people struggling with coming up with cool ideas haha
- phgn 3y ago
  For me it's actually that I have too many ideas and none of them are great. What makes ideas great I think is having the right people to try things out with -- hence this tool. So there are a lot of uses for this :)
  - thebestmoshe 3y ago
    I have the same issue. I have too many ideas with very limited amount of time to work on them. When I meet people, I want to share my ideas and get feedback about what they would find interesting, use, pay for, etc. I'm building a platform to do this, so I can share my ideas. My page is at https://ideas.moshegrunwald.com https://ideas.moshegrunwald.com
- sph 3y ago
  Ideas are not worth very much, and many try to focus on world-changing ideas, which is the wrong approach. I sit on two or three profitable ideas at any time, there's just not enough hours in a day to dedicate serious effort to any.
  - phgn 3y ago
    I agree! What would you say is the most important then? The right people, or sticking with something for long enough?
    - sph 3y ago
      Effort trumps everything. The difference between a dumb idea and a multi-million dollar business is how much effort you put in. So effectively the problem most people have (including me) is finding an idea you will keep believing in even after the hype and excitement has faded down, and the only thing standing in front of you is the mountain you still have to climb.
natashasalant 3y ago
Amazing idea and such a cool interface!
bantunes 3y ago
Hugged to death right now, getting a 500
- phgn 3y ago
  Fixed now, sorry! It was actually a bad deploy.
  - ramon156 3y ago
    Still down
    - phgn 3y ago
      Error querying the database: FATAL: Max client connections reached It's weird since the SvelteKit load() server function should be cached. I added a fallback just now, sorry again!
- phgn 3y ago
  Uh oh, the search overloaded the basic typesense.org cloud instance :( I'm upgrading it now!
INTPenis 3y ago
Also /uses[1]. 1. https://uses.tech/ https://uses.tech/
- louisbarclay 3y ago
  Oh wow, that is so cool! We hadn't come across it, nice UX too. Thanks for posting
- aranw 3y ago
  Oh nice I really like that. Ace definitely will add one to my site!
- theharshpat 3y ago
  Super interesting, thanks!
raduan 3y ago
looks amazing, will scroll and find someone interesting today.
afrnz 3y ago
Love it - I knew /about and /now but /ideas is new to me. I am adding this to my site as well. Do you filter results based on last update?
- phgn 3y ago
  Yep at least the /ideas pages that turn up on the page load are sorted by recency. You can also see the most recent /now pages by clicking the filter icon next to the search bar and selecting /now. And we re-scrape all indexed pages every few days. For now the search is purely sorted by relevancy (keyword + embeddings), but it yes would be nice to prioritise recent posts somehow. Since those are much more actionable.
- phgn 3y ago
  And to clarify, we made up the /ideas thing. But IMO it makes so much sense since /about and /now are about the past & present. Because where else do you write about the future?
- jamesdwilson 3y ago
  The only thing missing really is a search engine with an LLM. How do you plan to handle our new overlord indexing?
nonethewiser 3y ago
Cool site. Maybe implement /ideas and /now pages? I kinda figured they'd be there but they are 404.
- Retr0id 3y ago
  Perhaps they could redirect to https://github.com/lindylearn/aboutideasnow/issues https://github.com/lindylearn/aboutideasnow/issues and https://github.com/lindylearn/aboutideasnow/pulls https://github.com/lindylearn/aboutideasnow/pulls, respectively
  - phgn 3y ago
    Very nice idea, I just implemented this! Feel free to create some GitHub issues at https://aboutideasnow.com/ideas https://aboutideasnow.com/ideas to grow the list :)
isuckatcoding 3y ago
Seeing some obviously bad dates: “Dec 31, 1969”
- phgn 3y ago
  Sorry for this! I believe it's fixed now. In which timezone do you live by the way?
city41 3y ago
A lot of cards say "Updated December 31, 1969", what date are you all using to populate that?
- phgn 3y ago
  GPT-3.5: https://github.com/lindylearn/aboutideasnow/blob/main/apps/api/src/common/meta.ts#L40 https://github.com/lindylearn/aboutideasnow/blob/main/apps/a... But in this case the date seems very close to the unix timestamp 0 which we use when there's no date found. Maybe it's a local timezone issue... Edit: this should be fixed now!
  - city41 3y ago
    Oh interesting. I never would have thought AI would be used for this. Does it also find things like the meta "revised" tag or anything like that? Doing some Googling it seems like officially it should be "revision", but seems like it's very common to use "revised"
    - phgn 3y ago
      I tried using https://www.npmjs.com/package/metascraper https://www.npmjs.com/package/metascraper before which I believe does check this meta tag. But a few websites set their updated date to the current date which was annoying, maybe to rank better in Google? And some people (including me) only mention the update time in the page text content. I've used GPT to parse human formatted dates in another project too, it's quite reliable if you validate the output timestamp. And relatively cheap too if you only pass in the first part of the page text.
      - city41 3y ago
        I can see how it's a tricky problem. I wish html had more structure here (and people followed the structure, a whole other problem...). FWIW, my page has a "last updated" date on its now page but comes up as 1969 in aboutideasnow. Oh, now aboutideasnow shows no date at all.
desireco42 3y ago
Like others said, I love it. This is good way to connect people by connecting their independent sites.
- phgn 3y ago
  That's the idea! And especially if more people start writing /ideas pages I think the search will become better and the results more actionable. Also, if you find something to improve, people open a GitHub issue: https://github.com/lindylearn/aboutideasnow/issues https://github.com/lindylearn/aboutideasnow/issues
bilsbie 3y ago
Neat idea! But where are all these websites coming from? People have personal websites AND they share random ideas? Btw I don’t have a website but here are some ideas I’d love to work on: Anything applying LLM to robots Device that rides on top of roomba and taps its bumper (to turn it) if it sees something bad. An employment agent for letting people try out new careers. Cheap simple pickleball out ball detection
- phgn 3y ago
  Yes seems like a good number of people randomly decided to add /ideas to their website. /now is more of a standard, see nownownow.com There are some good results for your third idea, maybe you can talk to those people :) https://aboutideasnow.com/?q=employment+agent+for+letting+people+try+out+new+careers https://aboutideasnow.com/?q=employment+agent+for+letting+pe...
kgeist 3y ago
Are submissioms manually reviewed by a human? Otherwise it can be abused for spam. Is it reindexed from time to time? Say, the submission looks okay, and then they change the content to something spammy. Maybe a Report button could be added in that case.
- phgn 3y ago
  No, there are just some static rules to exclude missing pages and /ideas pages which don't actually talk about ideas: https://github.com/lindylearn/aboutideasnow/blob/main/apps/api/src/common/filter.ts https://github.com/lindylearn/aboutideasnow/blob/main/apps/a... And yes it's re-scraped every few days. How could a process of reviewing the pages look like? Maybe it can be implemented with GPT.
bilsbie 3y ago
I wonder if you could go through twitter posts and find idea posts.
- 01HNNWZ0MV43FF 3y ago
  Hm or Fediverse https://mastodon.social/tags/idea https://mastodon.social/tags/idea
KomoD 3y ago
Last I checked bleepingcomputer, ifixit, vibilagare.se, psyche.co and libreboot.org aren't personal sites. None of those actually have an /about page, yet your site says they do... Another funny thing: just search 404 or not found and you'll get a lot of 404 pages
- phgn 3y ago
  Yes :( Do you have an idea of how to remove company websites in an automated way? I didn't want to manually review all 7k indexed websites. This is the GPT prompt I used for filtering domains to add, but it gives false positives: You are an API. Return a JSON array of booleans indicating whether each provided domain is someone's personal website. Use common sense. Make sure to return false for company websites.
  - layer8 3y ago
    Maybe change the API so that GPT can express uncertainty (make it a ternary value or even a confidence percentage), and then check the “uncertain” cases manually.
    - phgn 3y ago
      Great idea, I will try this. Thank you!!
    - tetha 3y ago
      Yep, most of our systems end up exposing a parameter like that to the customer. Some people only like the system to take action if the system is very sure, hate incorrect action and prefer unprocessed stuff in a queue. Other customers hate unprocessed items and prefer to cleanup incorrect actions. Takes tinkering to find the best.
  - netghost 3y ago
    For the 404s (assuming the status code isn't a 4xx), use a URL that you strongly suspect won't exist, then you can do a comparison (levenshtein distance, bag of words, etc.) to see if it's very similar to one of about, ideas, etc. pages.
netghost 3y ago
Since I see you're using GPT already, it might be nice to include a summary of any about page for a domain with the non-about results. That way I can see a little something about the person to give me more context.
anon115 3y ago
i feel like this could be its own social media idk.
- bbsz 3y ago
  I had the same thought. Doesn't aggregation of this type fall pretty squarely with federated/activitypub/bluesky -kind-of-a-thing? It became almost pointless to try to host such a small search/aggregator as a separate web (discovery through web search is almost impossible and it's hard to make people stay on a new website today because of activity sinkholes like twitter/reddit. add to it a hassle of managing modern application at scale etc). I don't know. But I feel that projects like such will have a higher 1yr survival rate when developed directly as a "feed".And, AFAIU, that's what activitypub got created for.
  - phgn 3y ago
    That's very interesting... What would you like to see "updates" about?
digging 3y ago
Super cool project. Reading this introduced me to /now and /ideas and /uses, all of which I will be adding to my site this week. (I guess I know what goes first on /now now!)
- phgn 3y ago
  Good luck, and thank you! I personally really enjoy writing and updating my /now and /ideas pages [0], it's quite clarifying to list the things you really care about :) [0] https://lindylearn.io/now https://lindylearn.io/now
burningion 3y ago
This is great, would love to see something like this turn into a social network of people building sites / pursuing ideas publicly. Added my site makeartwithpython.com too.
- sT370ma2 3y ago
  One small personal social media site that often discusses issues surrounding building and hosting personal websites and social media sites is https://bluedwarf.top https://bluedwarf.top.
- thebestmoshe 3y ago
  I’m building something like this. It’s not yet available publicly, but you can see my personal page at https://ideas.moshegrunwald.com https://ideas.moshegrunwald.com You can see more details about this specific idea [here](https://ideas.moshegrunwald.com/ideas/i_mos2 https://ideas.moshegrunwald.com/ideas/i_mos2)
  - phgn 3y ago
    This is so cool! I love the upvote function :) Could you put it at /ideas on your website so it appears on aboutideasnow.com?
    - thebestmoshe 3y ago
      It's currently available as a subdomain of my personal site, since I have not yet finalized the domain for this project. However, each user's page will be at a path of the primary domain (e.g. example.com/moshe). `/ideas` is not supported at a sub-path (e.g. example.com/moshe/ideas). I will be supporting custom domains, so I could support `/ideas` only for those sites. Other options include redirecting/rewriting your personal `/ideas` path to this product. Or, offering subdomains (e.g. moshe.example.com/ideas)
      - phgn 3y ago
        Mhh, I would love aboutideasnow.com to be interoperable with your project. Seems like you have more things planned! The redirect option seems good, or supporting subpaths. If there's something I can do to help please write me an email [0] or open a GitHub issue! [1] We just want more people to write about their ideas, and find others with similar ideas. It doesn't matter to me where that happens :) [0] See https://lindylearn.io https://lindylearn.io [1] https://github.com/lindylearn/aboutideasnow/issues https://github.com/lindylearn/aboutideasnow/issues
spencerflem 3y ago
I like the site, but I wish I could see all the /ideas posts chronologically without applying a filter
- phgn 3y ago
  Mhh why do you want to go through all posts? My thought was that searching would be more useful than a huge list.
  - spencerflem 3y ago
    Certainly its not more useful, and ik the pitch for the site is a way to find collaborators which this wouldnt help at all. I just like seeing personal sites in a people watching kinda way. Its neat, and there's usually something interesting there, but I don't really have an objective behind it. Like scrolling twitter but instead of short form comedy and engagement bait its just peoples random ideas.
    - nbbaier 3y ago
      I agree with this!
panarchy 3y ago
I first thought it would be showing all ideas, about, and now simultaneously, but then noticed all the cards only had /ideas, then I thought I had to do url.com/now only to find that redirected to github instead. Then finally I noticed the little filter button. Kind of feel like the categories shouldn't be behind the button, it's not like they really get in the way visually and with them being the only thing there it feels like just an unnecessary extra click for a common function. /userstory Edit for positivity: It's a cool site though and I'll definitely be checking it out again in the future.
- phgn 3y ago
  We actually had an argument about this and I wanted to remove the filters entirely (since I didn't see a use case for filtering), Louis wanted to keep them. The compromise is the filter button :) I agree that showing all recent posts types would be more expected given the title. But with the stated purpose of the site right now being to "find people to collaborate with", /ideas posts are the most useful IMO [0]. And I really want people to write more /ideas pages. Why did you want to see a list of people's /now pages? [0] An earlier version of this site was actually called ideasideasideas.io and had only /ideas posts. But the other content makes the search much better.
  - panarchy 3y ago
    I agree that showing... That sounds pretty reasonable! I do wonder though if rewording "Find people to talk to or collaborate with by searching across the /about, /ideas and /now pages of 7487 personal websites." might help a bit? Maybe something more along the lines of "Find new /ideas to discuss and collaborate on or even search for what's /new with people and what they're /about" to focus on the ideas coming first and are not just one of the tags of equal importance to search on with the others. Why did you want to see a list of people's /now pages? Just exploratory curiosity (I also tried /about and /ideas, but left that out for brevity). Being pretty unfamiliar with the idea of /ideas and /now pages I just wanted to see what they tended to be like and it seemed like a core part of the website given the name. Given the focus on the ideas part I don't imagine this suggestion to be entirely relevant, but I had an idea about turning your logo header thing that links to your about page (which imo is kind of unintuitive) into the filter. Have it so that that only the current one (or ones?) are colored in and the others are desaturated. And potentially moving the "building open source, starting a community, etc" div/search suggestions to show with the hamburger filter button.
tunnuz 3y ago
I love this!
RamblingCTO 3y ago
Awesome, just added this! I'm looking for someone to grow my "micro PE" of two side projects together, because I sure can build stuff, but not make them grow -> https://andinfinity.eu/about/ https://andinfinity.eu/about/ (or /now) I have paying customers for 2markdown, but not nearly enough for the potential it has.
- phgn 3y ago
  2markdown is actually very interesting. You could expand into handling request proxies + JS rendering for people as well then it would be the easiest way to index web pages. We should definitely chat. I built unclutter.it before and the content ingestion for a reading tech startup. Will send you an email. And thank you for the shout out of aboutideasnow.com :) Making connections like this is exactly the purpose of the website.
  - RamblingCTO 3y ago
    Would love to hear from you! I actually pondered going down that road. There's heavy competition though. Getting LLMs in touch with the web, not so much. So I decided to stay in that niche, as almost all customers come from there as well. And I'm integrated into langchain, that drives a big part of the traffic ;)
1123581321 3y ago
This is really cool. I'd love to see duplicates or near-duplicates (as determined by the LLM?) to help match the thinkers.
- phgn 3y ago
  YES! Submit your website on the site and enter an email address in case this ever gets implemented :)
simonw 3y ago
The semantic search actually works really well - https://aboutideasnow.com/?q=data+journalism https://aboutideasnow.com/?q=data+journalism ?q=data+journalism gave me a bunch of interesting people thinking about data journalism and related ideas.
- breadchris 3y ago
  yes, this. it feels pretty magical to immediately find people across the Internet who have most likely "gone deep" on something I also have. it is surprisingly hard to connect with people like this in 2024.
joeriddles 3y ago
I added my site and received the following response: > josephriddle.com/ideas without update time I looked into the source code to determine how it's finding the update time. Come to find out, it's using ChatGPT! [0] It appears to only be looking at the article contents for the date, not at any page metadata. [0] https://github.com/lindylearn/aboutideasnow/blob/main/apps/api/src/common/meta.ts#L41 https://github.com/lindylearn/aboutideasnow/blob/main/apps/a...
- phgn 3y ago
  Yep but there is a fallback to metascraper [0] which does check the HTML tags. However the fallback didn't work in case GPT returns a 1970 date -- I just fixed this! [1] I think you can now remove the date from your post content and it should still work. If you submit your website again it should do a re-scrape if you changed the content text. Thanks for catching this :) [0] https://metascraper.js.org/#/ https://metascraper.js.org/#/ [1] https://github.com/lindylearn/aboutideasnow/commit/8b0ea5b468a320a9ff47da368ed484e1ecaaa137 https://github.com/lindylearn/aboutideasnow/commit/8b0ea5b46...
  - nicbou 3y ago
    Does it also look at JSON+LD?
  - plumeria 3y ago
    It would be nice if it also supported the If-Modified-Since and If-Unmodified-Since precondition headers.
    - phgn 3y ago
      Good idea, I just created https://github.com/lindylearn/aboutideasnow/issues/7 https://github.com/lindylearn/aboutideasnow/issues/7 for this!
- jseliger 3y ago
  I tried to add https://jakeseliger.com https://jakeseliger.com, and I got an error saying that there is no "about" page. But if you look at https://jakeseliger.com/about https://jakeseliger.com/about, there is in fact one!
a_petrov 3y ago
Thanks for sharing this great project! Randomly browsing through the first page results, I ended up reading some of the ideas. Reading that someone else, somewhere else, has a similar idea, has made my day. Cheers.
- phgn 3y ago
  Thank you! Reach out to them and make the idea a reality :)
bhasi 3y ago
Great! I don't have an /about or /ideas page, but I do have a /now and a /uses page. https://gurudas.dev https://gurudas.dev I currently only have a short blurb on my landing homepage instead of a full /about page because I struggle with anxiety when trying to describe myself and my journey so far (also impostor syndrome). I like the /ideas page too, but I may not end up finishing ideas if I get my dopamine fix early by publishing them to the world.
- phgn 3y ago
  That's fine, good that we index all three page types. I'm wishing you all the best in San Jose! :)
  - bhasi 3y ago
    Thanks a lot! I really love this superset of /now pages with your own great idea of an /ideas page.
lqcfcjx 3y ago
This is a great idea. Just a quick feedback: when I search for "llm", I got quite random results.
- phgn 3y ago
  Well the results are indeed "quite random"... A good test case to improve the spam filtering with :) It seems to work much better with capital letters for some reason: https://aboutideasnow.com/?q=LLM https://aboutideasnow.com/?q=LLM
jasondigitized 3y ago
I can only seem to surface the first 15 ideas pages. No other search surfaces any other ideas pages. Surely there is more than 15 idea pages out of 7000. Great concept though.
jameschensmith 3y ago
Awesome! I recently started an "Awesome Website Paths" [1] repository. I didn't know about /ideas, and I haven't tracked /about yet. I'll add them later tonight and point both to https://aboutideasnow.com https://aboutideasnow.com for now. In the past I've had each endpoint mention a specific page that's dedicated to the endpoint. I can make an exception until I find one of those dedicated pages, but I'll make sure to still keep a reference to your site when I do make that change. Thanks for sharing! [1] https://github.com/jameschensmith/awesome-website-paths https://github.com/jameschensmith/awesome-website-paths
- louisbarclay 3y ago
  This is an awesome awesome list! Thank you for adding /about and /ideas! It also reminded me to have a proper look for awesome-personal-sites or similar, and there are a few: https://github.com/logancyang/awesome-personal-websites https://github.com/logancyang/awesome-personal-websites https://github.com/christopherkade/awesome-dev-websites https://github.com/christopherkade/awesome-dev-websites https://github.com/tk-notes/awesome-personal-websites https://github.com/tk-notes/awesome-personal-websites
- phgn 3y ago
  This is so cool, thank you!! Likely didn't find any reference of /ideas because we made it up :)
- surprisetalk 3y ago
  I've been trying to make /wish pages a thing: [1] https://taylor.town/wish-manifesto https://taylor.town/wish-manifesto I'll open a PR :)
quickthrower2 3y ago
Fuzzy search? “gaza” search returned results without that string or anything close.
sivers 3y ago
Very cool. I've got all the /now pages from https://nownownow.com/ https://nownownow.com/ in my PostgreSQL database (I'm the creator/admin of it) so feel free to contact me if you'd like a more direct data connection. https://sive.rs/contact https://sive.rs/contact I'm currently going through every site on nownownow.com by hand, removing the dead and spam ones, since after years of automated scripts it deserved a personal visit to verify. While doing that, I'm updating things like when each site was last-updated and last-visited. Anyone reading this, if you've got a /now page on your site, please email me so I can add you to nownownow.com
- phgn 3y ago
  Yep definitely everyone add your /now site to nownownow.com too! I feel like the directories each have a different purpose: aboutideasnow.com to connect people with similar ideas, nownownow.com as a personal look into people's lives. Derek I sent you an email :)
notso411 3y ago
[dead]
raytopia 3y ago
A little tangential but is there a good place to share one's personal website?
mrkramer 3y ago
This was my vision of discovery engine for the Web! I'm glad that someone turned it into reality but I think you should scale it to the whole Web not just personal websites. Amazing job, you guys did!
- phgn 3y ago
  Thank you!! :) What kind of websites do you want to discover other than personal websites? Why not use general purpose web search?
  - mrkramer 3y ago
    All kind of websites....ecommerce, corporate, government, non-profit etc. I think we need to evolve old-school type web directories and make what I call discovery engine because we already have search engines. For example we should think how we can evolve old directory like this one: https://wiki.archiveteam.org/index.php/Dmoz https://wiki.archiveteam.org/index.php/Dmoz Here is the new revived version of it: https://www.curlie.org/ https://www.curlie.org/ but for sure something more sophisticated can and should exist.
samsquire 3y ago
I don't know if my website is eligible. My homepage is a small introduction and a table of projects with links to those projects, which are also github repositories with a blog underneath. This community standardisation I really like /about /now /ideas, I would like people to create github repositories with this format: "wants" -> things that this person wants "perspectives" -> technical perspectives on different subjects "work" -> work that someone thinks need doing "collaboration" -> what someone would be willing to work with others on "commissions" -> what you'll pay someone for "ideas" -> ideas "blog" -> if your blog is hosted on github "dotfiles" -> if you have them
- phgn 3y ago
  Every page is eligible actually because we treat / as /about if /about is not present. Just try submitting your site :) Another commenter here created a repo to track more standard webpages: https://github.com/jameschensmith/awesome-website-paths https://github.com/jameschensmith/awesome-website-paths For GitHub repos I'm not sure of any standardisation except README.md, LICENCE.md and CONTRIBUTING.md?
  - samsquire 3y ago
    Thank you phgn I submitted my personal homepage. Those GitHub conventions are helpful. I was hoping people would create entire repositories in a standard convention too. Similar to Rails conventions.
peter_vukovic 3y ago
Excellent work. This is a perfect demonstration of how valuable content curation can be, paired with an excellent presentation and user experience. You made it much easier to discover interesting people in the corners of the Internet, and I am grateful for that.
- phgn 3y ago
  Ohh, thank you! :)
nbbaier 3y ago
This is such a cool project! I do wish that there was a way to see all websites in a each of the categories. A mega list of /ideas, for example
quantonganh 3y ago
I attempted to add my website and received the message "Indexed your website successfully! Found posts...". However, when I searched for something related to my /about page or my domain, it did not appear in the results. How can I verify that my site has been successfully added?
matteohorvath 3y ago
Wow. Your site is awesome. I could use some similar stuff for books.

586 points by louisbarclay 3y ago

sarimkx 3y ago

This is amazing. I was just in the process of building a directory of personal websites.

Beefin 3y ago

this is amazing, so many people struggling with coming up with cool ideas haha

phgn 3y ago
For me it's actually that I have too many ideas and none of them are great. What makes ideas great I think is having the right people to try things out with -- hence this tool. So there are a lot of uses for this :)
- thebestmoshe 3y ago
  I have the same issue. I have too many ideas with very limited amount of time to work on them. When I meet people, I want to share my ideas and get feedback about what they would find interesting, use, pay for, etc. I'm building a platform to do this, so I can share my ideas. My page is at https://ideas.moshegrunwald.com https://ideas.moshegrunwald.com
sph 3y ago
Ideas are not worth very much, and many try to focus on world-changing ideas, which is the wrong approach. I sit on two or three profitable ideas at any time, there's just not enough hours in a day to dedicate serious effort to any.
- phgn 3y ago
  I agree! What would you say is the most important then? The right people, or sticking with something for long enough?
  - sph 3y ago
    Effort trumps everything. The difference between a dumb idea and a multi-million dollar business is how much effort you put in. So effectively the problem most people have (including me) is finding an idea you will keep believing in even after the hype and excitement has faded down, and the only thing standing in front of you is the mountain you still have to climb.

natashasalant 3y ago

Amazing idea and such a cool interface!

bantunes 3y ago

Hugged to death right now, getting a 500

phgn 3y ago
Fixed now, sorry! It was actually a bad deploy.
- ramon156 3y ago
  Still down
  - phgn 3y ago
    Error querying the database: FATAL: Max client connections reached It's weird since the SvelteKit load() server function should be cached. I added a fallback just now, sorry again!
phgn 3y ago
Uh oh, the search overloaded the basic typesense.org cloud instance :( I'm upgrading it now!

INTPenis 3y ago

Also /uses[1]. 1. https://uses.tech/ https://uses.tech/

louisbarclay 3y ago
Oh wow, that is so cool! We hadn't come across it, nice UX too. Thanks for posting
aranw 3y ago
Oh nice I really like that. Ace definitely will add one to my site!
theharshpat 3y ago
Super interesting, thanks!

raduan 3y ago

looks amazing, will scroll and find someone interesting today.

afrnz 3y ago

Love it - I knew /about and /now but /ideas is new to me. I am adding this to my site as well. Do you filter results based on last update?

phgn 3y ago
Yep at least the /ideas pages that turn up on the page load are sorted by recency. You can also see the most recent /now pages by clicking the filter icon next to the search bar and selecting /now. And we re-scrape all indexed pages every few days. For now the search is purely sorted by relevancy (keyword + embeddings), but it yes would be nice to prioritise recent posts somehow. Since those are much more actionable.
phgn 3y ago
And to clarify, we made up the /ideas thing. But IMO it makes so much sense since /about and /now are about the past & present. Because where else do you write about the future?
jamesdwilson 3y ago
The only thing missing really is a search engine with an LLM. How do you plan to handle our new overlord indexing?

nonethewiser 3y ago

Cool site. Maybe implement /ideas and /now pages? I kinda figured they'd be there but they are 404.

Retr0id 3y ago
Perhaps they could redirect to https://github.com/lindylearn/aboutideasnow/issues https://github.com/lindylearn/aboutideasnow/issues and https://github.com/lindylearn/aboutideasnow/pulls https://github.com/lindylearn/aboutideasnow/pulls, respectively
- phgn 3y ago
  Very nice idea, I just implemented this! Feel free to create some GitHub issues at https://aboutideasnow.com/ideas https://aboutideasnow.com/ideas to grow the list :)

isuckatcoding 3y ago

Seeing some obviously bad dates: “Dec 31, 1969”

phgn 3y ago
Sorry for this! I believe it's fixed now. In which timezone do you live by the way?

city41 3y ago

A lot of cards say "Updated December 31, 1969", what date are you all using to populate that?

phgn 3y ago
GPT-3.5: https://github.com/lindylearn/aboutideasnow/blob/main/apps/api/src/common/meta.ts#L40 https://github.com/lindylearn/aboutideasnow/blob/main/apps/a... But in this case the date seems very close to the unix timestamp 0 which we use when there's no date found. Maybe it's a local timezone issue... Edit: this should be fixed now!
- city41 3y ago
  Oh interesting. I never would have thought AI would be used for this. Does it also find things like the meta "revised" tag or anything like that? Doing some Googling it seems like officially it should be "revision", but seems like it's very common to use "revised"
  - phgn 3y ago
    I tried using https://www.npmjs.com/package/metascraper https://www.npmjs.com/package/metascraper before which I believe does check this meta tag. But a few websites set their updated date to the current date which was annoying, maybe to rank better in Google? And some people (including me) only mention the update time in the page text content. I've used GPT to parse human formatted dates in another project too, it's quite reliable if you validate the output timestamp. And relatively cheap too if you only pass in the first part of the page text.
    - city41 3y ago
      I can see how it's a tricky problem. I wish html had more structure here (and people followed the structure, a whole other problem...). FWIW, my page has a "last updated" date on its now page but comes up as 1969 in aboutideasnow. Oh, now aboutideasnow shows no date at all.

desireco42 3y ago

Like others said, I love it. This is good way to connect people by connecting their independent sites.

phgn 3y ago
That's the idea! And especially if more people start writing /ideas pages I think the search will become better and the results more actionable. Also, if you find something to improve, people open a GitHub issue: https://github.com/lindylearn/aboutideasnow/issues https://github.com/lindylearn/aboutideasnow/issues

bilsbie 3y ago

Neat idea! But where are all these websites coming from? People have personal websites AND they share random ideas? Btw I don’t have a website but here are some ideas I’d love to work on: Anything applying LLM to robots Device that rides on top of roomba and taps its bumper (to turn it) if it sees something bad. An employment agent for letting people try out new careers. Cheap simple pickleball out ball detection

phgn 3y ago
Yes seems like a good number of people randomly decided to add /ideas to their website. /now is more of a standard, see nownownow.com There are some good results for your third idea, maybe you can talk to those people :) https://aboutideasnow.com/?q=employment+agent+for+letting+people+try+out+new+careers https://aboutideasnow.com/?q=employment+agent+for+letting+pe...

kgeist 3y ago

Are submissioms manually reviewed by a human? Otherwise it can be abused for spam. Is it reindexed from time to time? Say, the submission looks okay, and then they change the content to something spammy. Maybe a Report button could be added in that case.

phgn 3y ago
No, there are just some static rules to exclude missing pages and /ideas pages which don't actually talk about ideas: https://github.com/lindylearn/aboutideasnow/blob/main/apps/api/src/common/filter.ts https://github.com/lindylearn/aboutideasnow/blob/main/apps/a... And yes it's re-scraped every few days. How could a process of reviewing the pages look like? Maybe it can be implemented with GPT.

bilsbie 3y ago

I wonder if you could go through twitter posts and find idea posts.

01HNNWZ0MV43FF 3y ago
Hm or Fediverse https://mastodon.social/tags/idea https://mastodon.social/tags/idea

KomoD 3y ago

Last I checked bleepingcomputer, ifixit, vibilagare.se, psyche.co and libreboot.org aren't personal sites. None of those actually have an /about page, yet your site says they do... Another funny thing: just search 404 or not found and you'll get a lot of 404 pages

phgn 3y ago
Yes :( Do you have an idea of how to remove company websites in an automated way? I didn't want to manually review all 7k indexed websites. This is the GPT prompt I used for filtering domains to add, but it gives false positives: You are an API. Return a JSON array of booleans indicating whether each provided domain is someone's personal website. Use common sense. Make sure to return false for company websites.
- layer8 3y ago
  Maybe change the API so that GPT can express uncertainty (make it a ternary value or even a confidence percentage), and then check the “uncertain” cases manually.
  - phgn 3y ago
    Great idea, I will try this. Thank you!!
  - tetha 3y ago
    Yep, most of our systems end up exposing a parameter like that to the customer. Some people only like the system to take action if the system is very sure, hate incorrect action and prefer unprocessed stuff in a queue. Other customers hate unprocessed items and prefer to cleanup incorrect actions. Takes tinkering to find the best.
- netghost 3y ago
  For the 404s (assuming the status code isn't a 4xx), use a URL that you strongly suspect won't exist, then you can do a comparison (levenshtein distance, bag of words, etc.) to see if it's very similar to one of about, ideas, etc. pages.

netghost 3y ago

Since I see you're using GPT already, it might be nice to include a summary of any about page for a domain with the non-about results. That way I can see a little something about the person to give me more context.

anon115 3y ago

i feel like this could be its own social media idk.

bbsz 3y ago
I had the same thought. Doesn't aggregation of this type fall pretty squarely with federated/activitypub/bluesky -kind-of-a-thing? It became almost pointless to try to host such a small search/aggregator as a separate web (discovery through web search is almost impossible and it's hard to make people stay on a new website today because of activity sinkholes like twitter/reddit. add to it a hassle of managing modern application at scale etc). I don't know. But I feel that projects like such will have a higher 1yr survival rate when developed directly as a "feed".And, AFAIU, that's what activitypub got created for.
- phgn 3y ago
  That's very interesting... What would you like to see "updates" about?

digging 3y ago

Super cool project. Reading this introduced me to /now and /ideas and /uses, all of which I will be adding to my site this week. (I guess I know what goes first on /now now!)

phgn 3y ago
Good luck, and thank you! I personally really enjoy writing and updating my /now and /ideas pages [0], it's quite clarifying to list the things you really care about :) [0] https://lindylearn.io/now https://lindylearn.io/now

burningion 3y ago

This is great, would love to see something like this turn into a social network of people building sites / pursuing ideas publicly. Added my site makeartwithpython.com too.

sT370ma2 3y ago
One small personal social media site that often discusses issues surrounding building and hosting personal websites and social media sites is https://bluedwarf.top https://bluedwarf.top.
thebestmoshe 3y ago
I’m building something like this. It’s not yet available publicly, but you can see my personal page at https://ideas.moshegrunwald.com https://ideas.moshegrunwald.com You can see more details about this specific idea [here](https://ideas.moshegrunwald.com/ideas/i_mos2 https://ideas.moshegrunwald.com/ideas/i_mos2)
- phgn 3y ago
  This is so cool! I love the upvote function :) Could you put it at /ideas on your website so it appears on aboutideasnow.com?
  - thebestmoshe 3y ago
    It's currently available as a subdomain of my personal site, since I have not yet finalized the domain for this project. However, each user's page will be at a path of the primary domain (e.g. example.com/moshe). `/ideas` is not supported at a sub-path (e.g. example.com/moshe/ideas). I will be supporting custom domains, so I could support `/ideas` only for those sites. Other options include redirecting/rewriting your personal `/ideas` path to this product. Or, offering subdomains (e.g. moshe.example.com/ideas)
    - phgn 3y ago
      Mhh, I would love aboutideasnow.com to be interoperable with your project. Seems like you have more things planned! The redirect option seems good, or supporting subpaths. If there's something I can do to help please write me an email [0] or open a GitHub issue! [1] We just want more people to write about their ideas, and find others with similar ideas. It doesn't matter to me where that happens :) [0] See https://lindylearn.io https://lindylearn.io [1] https://github.com/lindylearn/aboutideasnow/issues https://github.com/lindylearn/aboutideasnow/issues

spencerflem 3y ago

I like the site, but I wish I could see all the /ideas posts chronologically without applying a filter

phgn 3y ago
Mhh why do you want to go through all posts? My thought was that searching would be more useful than a huge list.
- spencerflem 3y ago
  Certainly its not more useful, and ik the pitch for the site is a way to find collaborators which this wouldnt help at all. I just like seeing personal sites in a people watching kinda way. Its neat, and there's usually something interesting there, but I don't really have an objective behind it. Like scrolling twitter but instead of short form comedy and engagement bait its just peoples random ideas.
  - nbbaier 3y ago
    I agree with this!

panarchy 3y ago

I first thought it would be showing all ideas, about, and now simultaneously, but then noticed all the cards only had /ideas, then I thought I had to do url.com/now only to find that redirected to github instead. Then finally I noticed the little filter button. Kind of feel like the categories shouldn't be behind the button, it's not like they really get in the way visually and with them being the only thing there it feels like just an unnecessary extra click for a common function. /userstory Edit for positivity: It's a cool site though and I'll definitely be checking it out again in the future.

phgn 3y ago
We actually had an argument about this and I wanted to remove the filters entirely (since I didn't see a use case for filtering), Louis wanted to keep them. The compromise is the filter button :) I agree that showing all recent posts types would be more expected given the title. But with the stated purpose of the site right now being to "find people to collaborate with", /ideas posts are the most useful IMO [0]. And I really want people to write more /ideas pages. Why did you want to see a list of people's /now pages? [0] An earlier version of this site was actually called ideasideasideas.io and had only /ideas posts. But the other content makes the search much better.
- panarchy 3y ago
  I agree that showing... That sounds pretty reasonable! I do wonder though if rewording "Find people to talk to or collaborate with by searching across the /about, /ideas and /now pages of 7487 personal websites." might help a bit? Maybe something more along the lines of "Find new /ideas to discuss and collaborate on or even search for what's /new with people and what they're /about" to focus on the ideas coming first and are not just one of the tags of equal importance to search on with the others. Why did you want to see a list of people's /now pages? Just exploratory curiosity (I also tried /about and /ideas, but left that out for brevity). Being pretty unfamiliar with the idea of /ideas and /now pages I just wanted to see what they tended to be like and it seemed like a core part of the website given the name. Given the focus on the ideas part I don't imagine this suggestion to be entirely relevant, but I had an idea about turning your logo header thing that links to your about page (which imo is kind of unintuitive) into the filter. Have it so that that only the current one (or ones?) are colored in and the others are desaturated. And potentially moving the "building open source, starting a community, etc" div/search suggestions to show with the hamburger filter button.

tunnuz 3y ago

I love this!

RamblingCTO 3y ago

Awesome, just added this! I'm looking for someone to grow my "micro PE" of two side projects together, because I sure can build stuff, but not make them grow -> https://andinfinity.eu/about/ https://andinfinity.eu/about/ (or /now) I have paying customers for 2markdown, but not nearly enough for the potential it has.

phgn 3y ago
2markdown is actually very interesting. You could expand into handling request proxies + JS rendering for people as well then it would be the easiest way to index web pages. We should definitely chat. I built unclutter.it before and the content ingestion for a reading tech startup. Will send you an email. And thank you for the shout out of aboutideasnow.com :) Making connections like this is exactly the purpose of the website.
- RamblingCTO 3y ago
  Would love to hear from you! I actually pondered going down that road. There's heavy competition though. Getting LLMs in touch with the web, not so much. So I decided to stay in that niche, as almost all customers come from there as well. And I'm integrated into langchain, that drives a big part of the traffic ;)

1123581321 3y ago

This is really cool. I'd love to see duplicates or near-duplicates (as determined by the LLM?) to help match the thinkers.

phgn 3y ago
YES! Submit your website on the site and enter an email address in case this ever gets implemented :)

simonw 3y ago

The semantic search actually works really well - https://aboutideasnow.com/?q=data+journalism https://aboutideasnow.com/?q=data+journalism ?q=data+journalism gave me a bunch of interesting people thinking about data journalism and related ideas.

breadchris 3y ago
yes, this. it feels pretty magical to immediately find people across the Internet who have most likely "gone deep" on something I also have. it is surprisingly hard to connect with people like this in 2024.

joeriddles 3y ago

I added my site and received the following response: > josephriddle.com/ideas without update time I looked into the source code to determine how it's finding the update time. Come to find out, it's using ChatGPT! [0] It appears to only be looking at the article contents for the date, not at any page metadata. [0] https://github.com/lindylearn/aboutideasnow/blob/main/apps/api/src/common/meta.ts#L41 https://github.com/lindylearn/aboutideasnow/blob/main/apps/a...

phgn 3y ago
Yep but there is a fallback to metascraper [0] which does check the HTML tags. However the fallback didn't work in case GPT returns a 1970 date -- I just fixed this! [1] I think you can now remove the date from your post content and it should still work. If you submit your website again it should do a re-scrape if you changed the content text. Thanks for catching this :) [0] https://metascraper.js.org/#/ https://metascraper.js.org/#/ [1] https://github.com/lindylearn/aboutideasnow/commit/8b0ea5b468a320a9ff47da368ed484e1ecaaa137 https://github.com/lindylearn/aboutideasnow/commit/8b0ea5b46...
- nicbou 3y ago
  Does it also look at JSON+LD?
- plumeria 3y ago
  It would be nice if it also supported the If-Modified-Since and If-Unmodified-Since precondition headers.
  - phgn 3y ago
    Good idea, I just created https://github.com/lindylearn/aboutideasnow/issues/7 https://github.com/lindylearn/aboutideasnow/issues/7 for this!
jseliger 3y ago
I tried to add https://jakeseliger.com https://jakeseliger.com, and I got an error saying that there is no "about" page. But if you look at https://jakeseliger.com/about https://jakeseliger.com/about, there is in fact one!

a_petrov 3y ago

Thanks for sharing this great project! Randomly browsing through the first page results, I ended up reading some of the ideas. Reading that someone else, somewhere else, has a similar idea, has made my day. Cheers.

phgn 3y ago
Thank you! Reach out to them and make the idea a reality :)

bhasi 3y ago

Great! I don't have an /about or /ideas page, but I do have a /now and a /uses page. https://gurudas.dev https://gurudas.dev I currently only have a short blurb on my landing homepage instead of a full /about page because I struggle with anxiety when trying to describe myself and my journey so far (also impostor syndrome). I like the /ideas page too, but I may not end up finishing ideas if I get my dopamine fix early by publishing them to the world.

phgn 3y ago
That's fine, good that we index all three page types. I'm wishing you all the best in San Jose! :)
- bhasi 3y ago
  Thanks a lot! I really love this superset of /now pages with your own great idea of an /ideas page.

lqcfcjx 3y ago

This is a great idea. Just a quick feedback: when I search for "llm", I got quite random results.

phgn 3y ago
Well the results are indeed "quite random"... A good test case to improve the spam filtering with :) It seems to work much better with capital letters for some reason: https://aboutideasnow.com/?q=LLM https://aboutideasnow.com/?q=LLM

jasondigitized 3y ago

I can only seem to surface the first 15 ideas pages. No other search surfaces any other ideas pages. Surely there is more than 15 idea pages out of 7000. Great concept though.

jameschensmith 3y ago

Awesome! I recently started an "Awesome Website Paths" [1] repository. I didn't know about /ideas, and I haven't tracked /about yet. I'll add them later tonight and point both to https://aboutideasnow.com https://aboutideasnow.com for now. In the past I've had each endpoint mention a specific page that's dedicated to the endpoint. I can make an exception until I find one of those dedicated pages, but I'll make sure to still keep a reference to your site when I do make that change. Thanks for sharing! [1] https://github.com/jameschensmith/awesome-website-paths https://github.com/jameschensmith/awesome-website-paths

louisbarclay 3y ago
This is an awesome awesome list! Thank you for adding /about and /ideas! It also reminded me to have a proper look for awesome-personal-sites or similar, and there are a few: https://github.com/logancyang/awesome-personal-websites https://github.com/logancyang/awesome-personal-websites https://github.com/christopherkade/awesome-dev-websites https://github.com/christopherkade/awesome-dev-websites https://github.com/tk-notes/awesome-personal-websites https://github.com/tk-notes/awesome-personal-websites
phgn 3y ago
This is so cool, thank you!! Likely didn't find any reference of /ideas because we made it up :)
surprisetalk 3y ago
I've been trying to make /wish pages a thing: [1] https://taylor.town/wish-manifesto https://taylor.town/wish-manifesto I'll open a PR :)

quickthrower2 3y ago

Fuzzy search? “gaza” search returned results without that string or anything close.

sivers 3y ago

Very cool. I've got all the /now pages from https://nownownow.com/ https://nownownow.com/ in my PostgreSQL database (I'm the creator/admin of it) so feel free to contact me if you'd like a more direct data connection. https://sive.rs/contact https://sive.rs/contact I'm currently going through every site on nownownow.com by hand, removing the dead and spam ones, since after years of automated scripts it deserved a personal visit to verify. While doing that, I'm updating things like when each site was last-updated and last-visited. Anyone reading this, if you've got a /now page on your site, please email me so I can add you to nownownow.com

phgn 3y ago
Yep definitely everyone add your /now site to nownownow.com too! I feel like the directories each have a different purpose: aboutideasnow.com to connect people with similar ideas, nownownow.com as a personal look into people's lives. Derek I sent you an email :)

notso411 3y ago

[dead]

raytopia 3y ago

A little tangential but is there a good place to share one's personal website?

mrkramer 3y ago

This was my vision of discovery engine for the Web! I'm glad that someone turned it into reality but I think you should scale it to the whole Web not just personal websites. Amazing job, you guys did!

phgn 3y ago
Thank you!! :) What kind of websites do you want to discover other than personal websites? Why not use general purpose web search?
- mrkramer 3y ago
  All kind of websites....ecommerce, corporate, government, non-profit etc. I think we need to evolve old-school type web directories and make what I call discovery engine because we already have search engines. For example we should think how we can evolve old directory like this one: https://wiki.archiveteam.org/index.php/Dmoz https://wiki.archiveteam.org/index.php/Dmoz Here is the new revived version of it: https://www.curlie.org/ https://www.curlie.org/ but for sure something more sophisticated can and should exist.

samsquire 3y ago

I don't know if my website is eligible. My homepage is a small introduction and a table of projects with links to those projects, which are also github repositories with a blog underneath. This community standardisation I really like /about /now /ideas, I would like people to create github repositories with this format: "wants" -> things that this person wants "perspectives" -> technical perspectives on different subjects "work" -> work that someone thinks need doing "collaboration" -> what someone would be willing to work with others on "commissions" -> what you'll pay someone for "ideas" -> ideas "blog" -> if your blog is hosted on github "dotfiles" -> if you have them

phgn 3y ago
Every page is eligible actually because we treat / as /about if /about is not present. Just try submitting your site :) Another commenter here created a repo to track more standard webpages: https://github.com/jameschensmith/awesome-website-paths https://github.com/jameschensmith/awesome-website-paths For GitHub repos I'm not sure of any standardisation except README.md, LICENCE.md and CONTRIBUTING.md?
- samsquire 3y ago
  Thank you phgn I submitted my personal homepage. Those GitHub conventions are helpful. I was hoping people would create entire repositories in a standard convention too. Similar to Rails conventions.

peter_vukovic 3y ago

Excellent work. This is a perfect demonstration of how valuable content curation can be, paired with an excellent presentation and user experience. You made it much easier to discover interesting people in the corners of the Internet, and I am grateful for that.

phgn 3y ago
Ohh, thank you! :)

nbbaier 3y ago

This is such a cool project! I do wish that there was a way to see all websites in a each of the categories. A mega list of /ideas, for example

quantonganh 3y ago

I attempted to add my website and received the message "Indexed your website successfully! Found posts...". However, when I searched for something related to my /about page or my domain, it did not appear in the results. How can I verify that my site has been successfully added?

matteohorvath 3y ago

Wow. Your site is awesome. I could use some similar stuff for books.