7 ms·

Show HN: Ghuser.io – Complete GitHub profiles

157 points by lourot 8y ago

lourot 8y ago
OP here. The idea came because on your GitHub profile: - I can't see all your code contributions since the beginning of time at a glance. - For each of your pinned repos, I can't tell at the first glance whether you just fixed a typo or you wrote 90% of it. ghuser.io tries to close this gap. You can show off with your entire GitHub "portfolio" and it gets refreshed every day. And hopefully this is temporary. The best for the community would be to have all this directly on GitHub. The sooner ghuser.io can be switched off, the better ;)
- franciscop 8y ago
  Hi OP, seems nice but "Get my profile" seems to attempt to load but never gets done. Github username: franciscop. Slashdot effect? My profile is weird? Been coding for ~8 years OSS in Github and I've got a quite public profile [1], so I might have quite a bit of data there. [1] http://git-awards.com/users/franciscop http://git-awards.com/users/franciscop
  - lourot 8y ago
    indeed, you're hitting what I just described in my other comment. Sorry about that, I really didn't think it would happen this fast (it's not the first time I post this on a forum). Taking a few minutes now to display a temporary message that we're a bit overloaded.
    - MrQuincle 8y ago
      This is a different type of forum: you're experiencing kind of a "hug of death". Be proud and move fast. https://en.wikipedia.org/wiki/Slashdot_effect https://en.wikipedia.org/wiki/Slashdot_effect
- lourot 8y ago
  This is already more popular than I thought, the queue of profile requests (and AWS SQS) contains already more than 10 users, at this point any new user user coming will have the sensation that nothing happens when they want to get a profile. I didn't expect to reach this point so fast (thanks!). I'll work on improving this right after this wave of new users. For now, just making sure that you all get your profile.
  - franciscop 8y ago
    Expect 10-20k visits if it rides the front page of HN, with anywhere between 100-2000 users creating their profile.
  - gadgetoid 8y ago
    Just requested mine! I'm not ready to abandon GitHub just yet- I use it for the "Hub", not the "Git" and this sort of change is just the kind of thing I'd like to see more of- let's hope some of these ideas get integrated into GitHub itself. I'm still optimistic that Microsoft will be keen to make some quality-of-life improvements when they take the wheel, as a show of good faith.
    - lourot 8y ago
      We had a chat with the "profile team" at GitHub recently and they are doing a great job as we speak at improving the GitHub profiles :)
- rmetzler 8y ago
  I'm the guy fixing the typos in everybody's README :)
  - lourot 8y ago
    Hehe :) And this is really useful, I hope my message didn't make it sound like the opposite. (Unfortunately our readme is probably full of typos!)
  - TomK32 8y ago
    damn it, you've been quicker at it, again...
  - michaelmior 8y ago
    Don't you tire of it after a while? I still do it from time to time but I've started to feel apathetic after seeing so many READMEs just put together as an afterthought.
- TomK32 8y ago
  Thanks, my gh profile has become utterly useless due to overload.
toyg 8y ago
Eh, you were too honest in your FAQs - chances are that Github will now close the hole. I expect there is a reason your use-case was not enabled in the API despite being already supported by the official frontend...
- brillout 8y ago
  ghuser.io contributor here. We talked with GitHub already. They think that what we are doing is cool and we agreed to tell them our roadmap and they tell us theirs. We would be glad if GitHub copies us.
  - ghoshbishakh 8y ago
    GitHub copying you! Now that's cool! achievement unlocked
lourot 8y ago
OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!
- KenanSulayman 8y ago
  Isn't 200 requests in an hour ... just three requests per minute? What is this service doing?
  - iokanuon 8y ago
    Maybe it's cloning all GitHub repos of a user to compute activity?
    - brillout 8y ago
      We don't clone repos. Instead we use GitHub's API to get activity info. E.g. https://api.github.com/repos/aurelienlourot/ghuser.io/contributors https://api.github.com/repos/aurelienlourot/ghuser.io/contri...
  - brillout 8y ago
    GitHub's API does't provide the exhaustive list of all your contributions. Instead we crawl GitHub's website which is slow. Details at https://github.com/AurelienLourot/github-contribs#how-does-it-work https://github.com/AurelienLourot/github-contribs#how-does-i... We are in talks with GitHub and they know that we are crawling GitHub.
    - ChristianBundy 8y ago
      > Instead we crawl GitHub's website which is computationally expensive I'm surprised to hear that you're bottlenecking on CPU time. Could you verify that my understanding is correct? I would've thought your bottleneck would be networking and connectivity as you have to wait for GitHub to process all of the requests.
delinka 8y ago
Am I understanding correctly that this is a manual process and not currently automated? (And can I cancel my request so you can move on to others for now?)
- lourot 8y ago
  It's cheeply automated to handle 10 profile requests per day, which is more than what we got in the past few months. So now we're giving it some human help and we'll have to rethink the system. What is your username? I'll cancel your request, thanks!
  - social_quotient 8y ago
    Seems like the perfect match for aws lambdas. I’d consider setting the tasks for crawling in to SQS and then trigger the lambdas to go do single crawl functions. For better control over the throttle and concurrency you can leverage dynamodb... I love it for controlling lambdas but not for storage. If you need more power than a lambda then you can do a similar process with EC2. Populate the SQS Trigger the aws lambda to turn on EC2 machine. Consider spot instances to save a ton of money. If you need ideas I’m sure HN readers would be glad to help solution for afar.
- brillout 8y ago
  Code-wise it is fully automated. But it's slow and it doesn't scale. We have to spin up new servers manually. As OP said, we'll need to make changes for ghuser.io to be scalable. Ideally GitHub adds an API that lists all your contribs. Which is (obviously) not in our hands but we are talking with GitHub.
fabricexpert 8y ago
Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well? It would also be really cool if we could pull out basic private repo info as well (with the user's permission). E.g. Worked on X private repos covering 60% Go, 20% Javascript etc. I am also a bit confused as to why you would build this if you don't work at github?
- brillout 8y ago
  > Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well?" Yes totally. Although we currently think that the best for the community is GitHub improving their GitHub profiles making ghuser.io obsolete. Right now we'd rather do this with GitHub and not against GitHub. But we don't know how much GitHub is willing to help us. (We need changes to the GitHub APIs for ghuser.io to be scalable.) > private repo Yea we could do that as well. Give permission to our public key and we're good to go. > why you would build this if you don't work at github? Because Aurelien (OP) wanted/needed that so he just built it. To the both of us it's basically a fun experiment. Also for me it's cool because we use the web framework Reframe (https://github.com/reframejs/reframe https://github.com/reframejs/reframe) which I'm currently building.
  - sytse 8y ago
    I love these profiles. At GitLab we're very open to contributions to enhance user profiles. One of the hard things might be the performance of these pages, for that you probably need to compute metrics in advance in a background job.
  - jdormit 8y ago
    Just so you know, you have a name collision with a popular ClojureScript SPA library, re-frame: https://github.com/Day8/re-frame https://github.com/Day8/re-frame
OnlyRepliesToBS 8y ago
at some point, programming was about making software
ohnoesjmr 8y ago
Given you disabled onboarding, is my onboarding request queued somewhere or will I have to comeback and re-request once you got over the hump?
- lourot 8y ago
  We're working on serving all the requests that came in before we disabled onboarding (about 200 requests). If you give me your github username I can tell you if you're part of them :)
  - ohnoesjmr 8y ago
    I am pretty sure I am not, as I received the message that onboarding is disabled. If you have user requests in the logs, you should queue them up.
ocdtrekkie 8y ago
Pretty bummed I missed a chance to try this! I had manually gone through my merged PRs list before to try and assemble a count of my contributions to each public project to come up with some sort of profile of stuff I've participated in/contributed to, as I really didn't see any of these type of pages doing that. This looks much more like what I'd like to see.
- lourot 8y ago
  hopefully we'll get back soon with a scalable system and you'll be able to try it out :)
sergey4096 8y ago
Check out https://sourcerer.io https://sourcerer.io - GitHub and GitLab profiles with extensive timeline, language, library, repo analysis. Customizable.
- kelsolaar 8y ago
  Just signed up, love it! Your https://github.com/sourcerer-io/hall-of-fame https://github.com/sourcerer-io/hall-of-fame is great too!
  - sergey4096 8y ago
    thank you!
7oorih 8y ago
grafana/.github/CONTRIBUTING.md

157 points by lourot 8y ago

OP here. The idea came because on your GitHub profile: - I can't see all your code contributions since the beginning of time at a glance. - For each of your pinned repos, I can't tell at the first glance whether you just fixed a typo or you wrote 90% of it. ghuser.io tries to close this gap. You can show off with your entire GitHub "portfolio" and it gets refreshed every day. And hopefully this is temporary. The best for the community would be to have all this directly on GitHub. The sooner ghuser.io can be switched off, the better ;)

franciscop 8y ago
Hi OP, seems nice but "Get my profile" seems to attempt to load but never gets done. Github username: franciscop. Slashdot effect? My profile is weird? Been coding for ~8 years OSS in Github and I've got a quite public profile [1], so I might have quite a bit of data there. [1] http://git-awards.com/users/franciscop http://git-awards.com/users/franciscop
- lourot 8y ago
  indeed, you're hitting what I just described in my other comment. Sorry about that, I really didn't think it would happen this fast (it's not the first time I post this on a forum). Taking a few minutes now to display a temporary message that we're a bit overloaded.
  - MrQuincle 8y ago
    This is a different type of forum: you're experiencing kind of a "hug of death". Be proud and move fast. https://en.wikipedia.org/wiki/Slashdot_effect https://en.wikipedia.org/wiki/Slashdot_effect
lourot 8y ago
This is already more popular than I thought, the queue of profile requests (and AWS SQS) contains already more than 10 users, at this point any new user user coming will have the sensation that nothing happens when they want to get a profile. I didn't expect to reach this point so fast (thanks!). I'll work on improving this right after this wave of new users. For now, just making sure that you all get your profile.
- franciscop 8y ago
  Expect 10-20k visits if it rides the front page of HN, with anywhere between 100-2000 users creating their profile.
- gadgetoid 8y ago
  Just requested mine! I'm not ready to abandon GitHub just yet- I use it for the "Hub", not the "Git" and this sort of change is just the kind of thing I'd like to see more of- let's hope some of these ideas get integrated into GitHub itself. I'm still optimistic that Microsoft will be keen to make some quality-of-life improvements when they take the wheel, as a show of good faith.
  - lourot 8y ago
    We had a chat with the "profile team" at GitHub recently and they are doing a great job as we speak at improving the GitHub profiles :)
rmetzler 8y ago
I'm the guy fixing the typos in everybody's README :)
- lourot 8y ago
  Hehe :) And this is really useful, I hope my message didn't make it sound like the opposite. (Unfortunately our readme is probably full of typos!)
- TomK32 8y ago
  damn it, you've been quicker at it, again...
- michaelmior 8y ago
  Don't you tire of it after a while? I still do it from time to time but I've started to feel apathetic after seeing so many READMEs just put together as an afterthought.
TomK32 8y ago
Thanks, my gh profile has become utterly useless due to overload.

toyg 8y ago

Eh, you were too honest in your FAQs - chances are that Github will now close the hole. I expect there is a reason your use-case was not enabled in the API despite being already supported by the official frontend...

brillout 8y ago
ghuser.io contributor here. We talked with GitHub already. They think that what we are doing is cool and we agreed to tell them our roadmap and they tell us theirs. We would be glad if GitHub copies us.
- ghoshbishakh 8y ago
  GitHub copying you! Now that's cool! achievement unlocked

lourot 8y ago

OP again, we're way overloaded. We got 200 profile requests already in an hour and we didn't see it coming. We're extremely thankful and sorry to disappoint you at the same time. We're closing the "Get your profile" feature for now. Spinning up many EC2 instances to process the 200 profile requests faster in parallel as we speak. And we'll come back soon with a system that can handle this load. Many many thanks!

KenanSulayman 8y ago
Isn't 200 requests in an hour ... just three requests per minute? What is this service doing?
- iokanuon 8y ago
  Maybe it's cloning all GitHub repos of a user to compute activity?
  - brillout 8y ago
    We don't clone repos. Instead we use GitHub's API to get activity info. E.g. https://api.github.com/repos/aurelienlourot/ghuser.io/contributors https://api.github.com/repos/aurelienlourot/ghuser.io/contri...
- brillout 8y ago
  GitHub's API does't provide the exhaustive list of all your contributions. Instead we crawl GitHub's website which is slow. Details at https://github.com/AurelienLourot/github-contribs#how-does-it-work https://github.com/AurelienLourot/github-contribs#how-does-i... We are in talks with GitHub and they know that we are crawling GitHub.
  - ChristianBundy 8y ago
    > Instead we crawl GitHub's website which is computationally expensive I'm surprised to hear that you're bottlenecking on CPU time. Could you verify that my understanding is correct? I would've thought your bottleneck would be networking and connectivity as you have to wait for GitHub to process all of the requests.

delinka 8y ago

Am I understanding correctly that this is a manual process and not currently automated? (And can I cancel my request so you can move on to others for now?)

lourot 8y ago
It's cheeply automated to handle 10 profile requests per day, which is more than what we got in the past few months. So now we're giving it some human help and we'll have to rethink the system. What is your username? I'll cancel your request, thanks!
- social_quotient 8y ago
  Seems like the perfect match for aws lambdas. I’d consider setting the tasks for crawling in to SQS and then trigger the lambdas to go do single crawl functions. For better control over the throttle and concurrency you can leverage dynamodb... I love it for controlling lambdas but not for storage. If you need more power than a lambda then you can do a similar process with EC2. Populate the SQS Trigger the aws lambda to turn on EC2 machine. Consider spot instances to save a ton of money. If you need ideas I’m sure HN readers would be glad to help solution for afar.
brillout 8y ago
Code-wise it is fully automated. But it's slow and it doesn't scale. We have to spin up new servers manually. As OP said, we'll need to make changes for ghuser.io to be scalable. Ideally GitHub adds an API that lists all your contribs. Which is (obviously) not in our hands but we are talking with GitHub.

fabricexpert 8y ago

Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well? It would also be really cool if we could pull out basic private repo info as well (with the user's permission). E.g. Worked on X private repos covering 60% Go, 20% Javascript etc. I am also a bit confused as to why you would build this if you don't work at github?

brillout 8y ago
> Could you make this work for gitlab, bitbucket and maybe even just regular old git repos as well?" Yes totally. Although we currently think that the best for the community is GitHub improving their GitHub profiles making ghuser.io obsolete. Right now we'd rather do this with GitHub and not against GitHub. But we don't know how much GitHub is willing to help us. (We need changes to the GitHub APIs for ghuser.io to be scalable.) > private repo Yea we could do that as well. Give permission to our public key and we're good to go. > why you would build this if you don't work at github? Because Aurelien (OP) wanted/needed that so he just built it. To the both of us it's basically a fun experiment. Also for me it's cool because we use the web framework Reframe (https://github.com/reframejs/reframe https://github.com/reframejs/reframe) which I'm currently building.
- sytse 8y ago
  I love these profiles. At GitLab we're very open to contributions to enhance user profiles. One of the hard things might be the performance of these pages, for that you probably need to compute metrics in advance in a background job.
- jdormit 8y ago
  Just so you know, you have a name collision with a popular ClojureScript SPA library, re-frame: https://github.com/Day8/re-frame https://github.com/Day8/re-frame

OnlyRepliesToBS 8y ago

at some point, programming was about making software

ohnoesjmr 8y ago

Given you disabled onboarding, is my onboarding request queued somewhere or will I have to comeback and re-request once you got over the hump?

lourot 8y ago
We're working on serving all the requests that came in before we disabled onboarding (about 200 requests). If you give me your github username I can tell you if you're part of them :)
- ohnoesjmr 8y ago
  I am pretty sure I am not, as I received the message that onboarding is disabled. If you have user requests in the logs, you should queue them up.

ocdtrekkie 8y ago

Pretty bummed I missed a chance to try this! I had manually gone through my merged PRs list before to try and assemble a count of my contributions to each public project to come up with some sort of profile of stuff I've participated in/contributed to, as I really didn't see any of these type of pages doing that. This looks much more like what I'd like to see.

lourot 8y ago
hopefully we'll get back soon with a scalable system and you'll be able to try it out :)

sergey4096 8y ago

Check out https://sourcerer.io https://sourcerer.io - GitHub and GitLab profiles with extensive timeline, language, library, repo analysis. Customizable.

kelsolaar 8y ago
Just signed up, love it! Your https://github.com/sourcerer-io/hall-of-fame https://github.com/sourcerer-io/hall-of-fame is great too!
- sergey4096 8y ago
  thank you!

7oorih 8y ago

grafana/.github/CONTRIBUTING.md