5 ms·

Honest question: how can VCs consider the 'star' system reliable? Users who add stars often stop following the project, so poorly maintained projects can have m

by dafi70 6mo ago

Honest question: how can VCs consider the 'star' system reliable? Users who add stars often stop following the project, so poorly maintained projects can have many stars but are effectively outdated. A better system, but certainly not the best, would be to look at how much "life" issues have, opening, closing (not automatic), and response times. My project has 200 stars, and I struggle like crazy to update regularly without simple version bumps.

askl 6mo ago
Stars are a simple metric even someone like a VC investor can understand. Your "better system" sounds far too complicated and time consuming.
HighlandSpring 6mo ago
I wonder if there's a more graph oriented score that could work well here - something pagerank ish so that a repo scores better if it has issues reported by users who themselves have a good score. So it's at least a little resilient to crude manipulation attempts
- 3form 6mo ago
  It would be more resilient indeed, I think. Definitely needs a way to figure out which users should have a good score, though - otherwise it's just shifting the problem somewhat. Perhaps it could be done with a reputation type of approach, where the initial reputation would be driven by a pool of "trusted" open source contributors from some major projects. That said, I believe the core problem is that GitHub belongs to Microsoft, and so it will still go more towards operating like a social network than not - i.e. engagement matters. It will still take a good will to get rid of Social Network Disease at scale.
  - az226 6mo ago
    Reputation doesn’t equal good taste in judging other projects. There are much better ways of finding those who have good taste.
- az226 6mo ago
  GitHub has all kinds of private internal metrics that could update the system to show a much higher signal/quality score. A score that is impervious to manipulation. And extremely well correlated with actual quality and popularity and value, not noise. Two projects could look exactly the same from visible metrics, and one is complete shell and the other a great project. But they choose not to publish it. And those same private signals more effectively spot the signal-rich stargazers than PageRank.
- mech422 6mo ago
  Sounds like 'advogato' for github... Social trust metrics https://web.archive.org/web/20170715120119/http://advogato.org/ https://web.archive.org/web/20170715120119/http://advogato.o...
3form 6mo ago
The stars have fallen to the classic problem of becoming a goal and stopping being a good metric. This can apply to your measure just as well: issues can also be gamed to be opened, closed and responded to quickly, especially now with LLMs.
- test1235 6mo ago
  "When a measure becomes a target, it ceases to be a good measure" https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- sunrunner 6mo ago
  Was it ever a good metric? A star from another account costs nothing and conveys nothing about the sincerity, knowledge, importance or cultural weight of the star giver. As a signal it's as weak as 'hitting that like button'. If the number of stars are in the thousands, tens of thousands, or hundreds of thousands, that might correlate with a serious project. But that should be visible by real, costly activity such as issues, PRs, discussion and activity.
  - noosphr 6mo ago
    There was a time when total number of hyperlinks to a site was an amazing metric measuring its quality.
    - kang 6mo ago
      at that time having a website took work, while having a github account can be cheaply used to sybil attack/signal marketing
      - noosphr 6mo ago
        Forums, news groups and mailing lists counted towards pagerank in the early days.
    - embedding-shape 6mo ago
faangguyindia 6mo ago
because VC don't care about anything being legitimate, if it can fool VCs it can also fool market participants, then VC can profit off of it. one VC told me, you'll get more funding and upvotes if u don't put "india" in your username.
foresterre 6mo ago
With the advent of AI, these "life" events are probably even simpler to fake than AI though, and unlike the faking of stars not against the ToS.
Se_ba 6mo ago
This is a good idea, but from my experience most VCs (I’m not talking about the big leagues) aren’t technical, they tend to repeat buzzwords, so they don’t really understand how star systems works.
logicallee 6mo ago
>Honest question: how can VCs consider the 'star' system reliable? Founders need the ability to get traction, so if a VC gets a pitch and the project's repo has 0 stars, that's a strong signal that this specific team is just not able to put themselves out there, or that what they're making doesn't resonate with anyone. When I mentioned that a small feature I shared got 3k views when I just mentioned it on Reddit, then investors' ears perked right up and I bet you're thinking "I wonder what that is, I'd like to see that!" People like to see things that are popular. By the way, congrats on 200 stars on your project, I think that is definitely a solid indicator of interest and quality, and I doubt investors would ignore it.
csomar 6mo ago
Because VCs love quantifiable metrics regardless of how reliable they actually are. They raise money from outside investors and are under pressure to deploy it. The metrics give them something concrete to justify their thesis and move on with their life.
az226 6mo ago
Much more important is who starred it. And are they selective about giving out stars or bookmarking everything. Forks is a closer signal to usage than stargazing.
- whilenot-dev 6mo ago
  Indeed, GitHub should set up a monthly quota for available stars to give and correlate the account age with it: either make something up like a "trusted-age-factor" that multiplies any given star by that factor, or scale the available quota accordingly by that factor (and let users star repos repeatedly). GitHub should also introduce a way to bookmark a repo, additional to the existing options of sponsor/watch/fork/star-ing it.
ethegwo 6mo ago
Many VCs are only doing one thing: how to use some magical quantitative metrics to assess whether a project is reliable without knowing the know-how. Numbers are always better than no numbers.
- dukeyukey 6mo ago
  Honestly I don't know if that's true. Picking up on vibes might be better than something like GitHub stats.
  - ethegwo 6mo ago
    When a partner decides to recommend a startup to the investment committee, he needs some explicit reasons to convince the committee, not some kind of implicit vibe
    - dukeyukey 6mo ago
      But given the amount of astroturfing and star-buying out there, relying on star counts may well select for deceptive founders.
      - ethegwo 6mo ago
        Yes, I think VCs have already switched to using other metrics that are less easy to fake, such as download per month or customer interviews (or more direct, ARR, even for really early stage startups). I just want to explain the background reason of it.
scotty79 6mo ago
> how can VCs consider the 'star' system reliable I think VCs just know that there are no reliable systems, so they go with whatever's used.
hobofan 6mo ago
Unless something has changed in the last ~3 years, I think the article vastly overstates the credibility with VC's. Even 10 years ago most VCs we spoke to had wisened up and discarded Github stars as a vanity metric.
- evilsocket 6mo ago
  Agree that sophisticated funds don't, but the ecosystem hasn't caught up. StarHub/GitStar pricing pages still sell to "seed-stage founders pre-fundraise"
JimDabell 6mo ago
You are looking for different things to VCs. You are looking for markers that show software quality over the long-term. They are looking for markers that show rapidly gaining momentum over the short-term. These are often in opposition to one another.
mapmeld 6mo ago
Making the conversations about VCs expecting thousands of stars, is thinking too big. It's probably more often someone pays $20 to make one of their projects look good, for their CV, for vanity, thinking this will get them the push that they need to get clicks on reddit, or noticed over some other open source project. If there is someone offering a 10k star project an investment over 8k without looking at the project or revenue potential, I can only think they are clueless, or picking a student project to fund each summer. The fake accounts often star my old repos to look like real users. They are usually very sketchy if you think for a minute, for example starring 5,000 projects in a month and no other GitHub activity. One time I found a GitHub Sponsor ring, which must be a money laundering / stolen credit cards thing?
Aurornis 6mo ago
> Honest question: how can VCs consider the 'star' system reliable? They don't. I've helped with due diligence on a couple projects. VCs know that metrics can be gamed because they see it all the time. Stars, followers, views, clicks, likes. A portion of entrepreneurs have been gaming every metric since before you and I learned how to program. It has always been this way and always will. Most of the VC-related comments have interpreted this article to mean that VCs are so dumb that they haven't realized that stars can be faked, but in reality VCs spend so much time sorting through fake metrics that they understand this probably better than most here. If you've ever gone through due diligence for an acquisition or big investment round it's amazing how much work you have to do in order to prove that your metrics are real. When things got crazy after COVID there was a short time when VCs were trying to move so fast that they skipped this, but it resulted in some high profile fraud cases. During normal times, you will get grilled on metrics. They might see stars as a signal for rising stars, but they're not throwing money at projects based on star count like many commenters assume. They will do a deeper dive before investing and they will call it off if things aren't adding up. The amount of diligence scales with the investment, so someone getting a $10K check can get away with a lot of fraud but that $2mm funding round isn't going to cross the finish line based on star count.

by dafi70 6mo ago

askl 6mo ago

Stars are a simple metric even someone like a VC investor can understand. Your "better system" sounds far too complicated and time consuming.

HighlandSpring 6mo ago

I wonder if there's a more graph oriented score that could work well here - something pagerank ish so that a repo scores better if it has issues reported by users who themselves have a good score. So it's at least a little resilient to crude manipulation attempts

3form 6mo ago
It would be more resilient indeed, I think. Definitely needs a way to figure out which users should have a good score, though - otherwise it's just shifting the problem somewhat. Perhaps it could be done with a reputation type of approach, where the initial reputation would be driven by a pool of "trusted" open source contributors from some major projects. That said, I believe the core problem is that GitHub belongs to Microsoft, and so it will still go more towards operating like a social network than not - i.e. engagement matters. It will still take a good will to get rid of Social Network Disease at scale.
- az226 6mo ago
  Reputation doesn’t equal good taste in judging other projects. There are much better ways of finding those who have good taste.
az226 6mo ago
GitHub has all kinds of private internal metrics that could update the system to show a much higher signal/quality score. A score that is impervious to manipulation. And extremely well correlated with actual quality and popularity and value, not noise. Two projects could look exactly the same from visible metrics, and one is complete shell and the other a great project. But they choose not to publish it. And those same private signals more effectively spot the signal-rich stargazers than PageRank.
mech422 6mo ago
Sounds like 'advogato' for github... Social trust metrics https://web.archive.org/web/20170715120119/http://advogato.org/ https://web.archive.org/web/20170715120119/http://advogato.o...

3form 6mo ago

The stars have fallen to the classic problem of becoming a goal and stopping being a good metric. This can apply to your measure just as well: issues can also be gamed to be opened, closed and responded to quickly, especially now with LLMs.

test1235 6mo ago
"When a measure becomes a target, it ceases to be a good measure" https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
sunrunner 6mo ago
Was it ever a good metric? A star from another account costs nothing and conveys nothing about the sincerity, knowledge, importance or cultural weight of the star giver. As a signal it's as weak as 'hitting that like button'. If the number of stars are in the thousands, tens of thousands, or hundreds of thousands, that might correlate with a serious project. But that should be visible by real, costly activity such as issues, PRs, discussion and activity.
- noosphr 6mo ago
  There was a time when total number of hyperlinks to a site was an amazing metric measuring its quality.
  - kang 6mo ago
    at that time having a website took work, while having a github account can be cheaply used to sybil attack/signal marketing
    - noosphr 6mo ago
      Forums, news groups and mailing lists counted towards pagerank in the early days.
  - embedding-shape 6mo ago

faangguyindia 6mo ago

because VC don't care about anything being legitimate, if it can fool VCs it can also fool market participants, then VC can profit off of it. one VC told me, you'll get more funding and upvotes if u don't put "india" in your username.

foresterre 6mo ago

With the advent of AI, these "life" events are probably even simpler to fake than AI though, and unlike the faking of stars not against the ToS.

Se_ba 6mo ago

This is a good idea, but from my experience most VCs (I’m not talking about the big leagues) aren’t technical, they tend to repeat buzzwords, so they don’t really understand how star systems works.

logicallee 6mo ago

>Honest question: how can VCs consider the 'star' system reliable? Founders need the ability to get traction, so if a VC gets a pitch and the project's repo has 0 stars, that's a strong signal that this specific team is just not able to put themselves out there, or that what they're making doesn't resonate with anyone. When I mentioned that a small feature I shared got 3k views when I just mentioned it on Reddit, then investors' ears perked right up and I bet you're thinking "I wonder what that is, I'd like to see that!" People like to see things that are popular. By the way, congrats on 200 stars on your project, I think that is definitely a solid indicator of interest and quality, and I doubt investors would ignore it.

csomar 6mo ago

Because VCs love quantifiable metrics regardless of how reliable they actually are. They raise money from outside investors and are under pressure to deploy it. The metrics give them something concrete to justify their thesis and move on with their life.

az226 6mo ago

Much more important is who starred it. And are they selective about giving out stars or bookmarking everything. Forks is a closer signal to usage than stargazing.

whilenot-dev 6mo ago
Indeed, GitHub should set up a monthly quota for available stars to give and correlate the account age with it: either make something up like a "trusted-age-factor" that multiplies any given star by that factor, or scale the available quota accordingly by that factor (and let users star repos repeatedly). GitHub should also introduce a way to bookmark a repo, additional to the existing options of sponsor/watch/fork/star-ing it.

ethegwo 6mo ago

Many VCs are only doing one thing: how to use some magical quantitative metrics to assess whether a project is reliable without knowing the know-how. Numbers are always better than no numbers.

dukeyukey 6mo ago
Honestly I don't know if that's true. Picking up on vibes might be better than something like GitHub stats.
- ethegwo 6mo ago
  When a partner decides to recommend a startup to the investment committee, he needs some explicit reasons to convince the committee, not some kind of implicit vibe
  - dukeyukey 6mo ago
    But given the amount of astroturfing and star-buying out there, relying on star counts may well select for deceptive founders.
    - ethegwo 6mo ago
      Yes, I think VCs have already switched to using other metrics that are less easy to fake, such as download per month or customer interviews (or more direct, ARR, even for really early stage startups). I just want to explain the background reason of it.

scotty79 6mo ago

> how can VCs consider the 'star' system reliable I think VCs just know that there are no reliable systems, so they go with whatever's used.

hobofan 6mo ago

Unless something has changed in the last ~3 years, I think the article vastly overstates the credibility with VC's. Even 10 years ago most VCs we spoke to had wisened up and discarded Github stars as a vanity metric.

evilsocket 6mo ago
Agree that sophisticated funds don't, but the ecosystem hasn't caught up. StarHub/GitStar pricing pages still sell to "seed-stage founders pre-fundraise"

JimDabell 6mo ago

You are looking for different things to VCs. You are looking for markers that show software quality over the long-term. They are looking for markers that show rapidly gaining momentum over the short-term. These are often in opposition to one another.

mapmeld 6mo ago

Making the conversations about VCs expecting thousands of stars, is thinking too big. It's probably more often someone pays $20 to make one of their projects look good, for their CV, for vanity, thinking this will get them the push that they need to get clicks on reddit, or noticed over some other open source project. If there is someone offering a 10k star project an investment over 8k without looking at the project or revenue potential, I can only think they are clueless, or picking a student project to fund each summer. The fake accounts often star my old repos to look like real users. They are usually very sketchy if you think for a minute, for example starring 5,000 projects in a month and no other GitHub activity. One time I found a GitHub Sponsor ring, which must be a money laundering / stolen credit cards thing?

Aurornis 6mo ago

> Honest question: how can VCs consider the 'star' system reliable? They don't. I've helped with due diligence on a couple projects. VCs know that metrics can be gamed because they see it all the time. Stars, followers, views, clicks, likes. A portion of entrepreneurs have been gaming every metric since before you and I learned how to program. It has always been this way and always will. Most of the VC-related comments have interpreted this article to mean that VCs are so dumb that they haven't realized that stars can be faked, but in reality VCs spend so much time sorting through fake metrics that they understand this probably better than most here. If you've ever gone through due diligence for an acquisition or big investment round it's amazing how much work you have to do in order to prove that your metrics are real. When things got crazy after COVID there was a short time when VCs were trying to move so fast that they skipped this, but it resulted in some high profile fraud cases. During normal times, you will get grilled on metrics. They might see stars as a signal for rising stars, but they're not throwing money at projects based on star count like many commenters assume. They will do a deeper dive before investing and they will call it off if things aren't adding up. The amount of diligence scales with the investment, so someone getting a $10K check can get away with a lot of fraud but that $2mm funding round isn't going to cross the finish line based on star count.