8 ms·

Not great... MD5 password hashing: https://github.com/swituo/openbilibili-go-common/blob/8866d1359a2a501009b976b02bb27e4949cc4e77/app/service/main/passport-gam

Not great... MD5 password hashing: https://github.com/swituo/openbilibili-go-common/blob/8866d1359a2a501009b976b02bb27e4949cc4e77/app/service/main/passport-game/service/passport_login.go#L185 https://github.com/swituo/openbilibili-go-common/blob/8866d1... Hardcoded credentials: https://github.com/swituo/openbilibili-go-common/blob/8866d1359a2a501009b976b02bb27e4949cc4e77/app/admin/ep/marthe/dao/tapd.go#L17 https://github.com/swituo/openbilibili-go-common/blob/8866d1... More hard coded secrets: https://github.com/swituo/openbilibili-go-common/blob/8866d1359a2a501009b976b02bb27e4949cc4e77/app/service/ep/footman/cmd/tapd/tapd.go#L54 https://github.com/swituo/openbilibili-go-common/blob/8866d1... This configuration is my favourite: https://github.com/swituo/openbilibili-go-common/blob/8866d1359a2a501009b976b02bb27e4949cc4e77/library/queue/databus/report/conf.go https://github.com/swituo/openbilibili-go-common/blob/8866d1... And of course, RSA keys which they use for all of their RSA encryption: https://github.com/swituo/openbilibili-go-common/blob/8866d1359a2a501009b976b02bb27e4949cc4e77/app/service/main/passport-game/service/passport_key.go https://github.com/swituo/openbilibili-go-common/blob/8866d1... ... their problem is not that the source code is all public over the internet now... their problem is the engineering team. If source code leaks the worst outcome should be some IP leakage, but not a compromised live system. That can and should be easily avoided by not having everything in your source code, especially when you are such a big company with so many employees...

alias_neo 8y ago
I've always wondered, how it could be that someone can be smart enough to write what on the surface is some fairly clean Golang, and yet at the same time, dumb enough to put secrets in the code. I can forgive the use of MD5, because they probably just don't know their hashing/crypto but secrets? It's literally in the name. There is so much material in your 5 links alone, that anyone who desires could utterly own their infrastructure, and then some.
- nemothekid 8y ago
  It’s one of those things that you dangerously start when your project is small then when you balloon in size, you find that everyone is hard coding secrets in code and standing up some secrets infrastructure would take weeks to get right. It’s easier now with tools like Vault but let’s say you joined bilibili today - where do you even begin? You have a massive cultural problem before you even begin to tackle the technical one. Even a smart engineer may just resign to doing things the wrong way than trying to fight a huge political battle.
  - alias_neo 8y ago
    Oh you're absolutely right, but there are some fairly straightforward steps to take to mitigate the risk with little effort. You can take arguments or env vars or config files (not added to Git) for your secrets. If you begin with a system of not putting the secrets in the code, ever, it's fairly straightforward to not make this mistake. A few minutes of setup on a repository and a mindfulness to be sure not to commit any new secret files that may be in use (and add them to the .gitignore) is a great start before getting to secret management a la Vault.
    - tracker1 8y ago
      For reference, here is my getConfig which uses the environment for configuration options. It's really easy enough to start with something like this, and add it in at the baseline. https://gist.github.com/tracker1/fcc39f40a0d14648501d329c7bdb8499 https://gist.github.com/tracker1/fcc39f40a0d14648501d329c7bd...
  - heavenlyblue 8y ago
    >> you find that everyone is hard coding secrets in code and standing up some secrets infrastructure would take weeks to get right. You open up the code, find all of the secrets (e.g. using high-entropy substring search), replace them with access to a global variable, and set it from a file set by a configuration from a command line. Done.
    - colejohnson66 8y ago
      And when your manager says don’t do that because it’s a waste of time?
      - bmm6o 8y ago
        If you aren't allowed to do it right, I'm not sure what kind of advice will be useful.
      - alias_neo 8y ago
- raxxorrax 8y ago
  > dumb enough to put secrets in the code. Man, I have tons of auth data in services like AWS just in environment variables. But pushing your rsa key to github must have happened on a bad monday. I do often have auth info in code, plainly because of time constraints. You just have to remember it before pushing anything on github. But aside from that, is it possible to file a DMCA for anything that has been forked if it was published under a license that permitted that action?
  - alias_neo 8y ago
    I think they examples you've listed are different, and more acceptable than pushing secrets and private keys to Github because you had dozens of them hard-coded. As for the aside, I imagine (with no background knowledge here) that as "owner" if you accidentally published something within that licensed code that does not belong or isn't covered by that license, you should probably have the right to remove it
    - kiallmacinnes 8y ago
      What if I accidentally contributed a little bit too much code to the Linux kernel? What if that code had been in the last 10 releases? What about the users and distributions running those versions? I'm not sure the answer is, or should be, as simple as "derp - delete immediately, this was never meant to be Open Source". That said, if the code was stolen, and then published under an OS licence - it's not OS. The "publisher" never had the rights to make it OS, so the "contract" is null and void...
      - alias_neo 8y ago
        This is what I was thinking. If it's something that wasn't meant to be published, I assume your right as the owner hold. How you'd solve this, logistically in an example like yours (Linux kernel) I have no idea, but I presume by the time it's been made public long enough to be cloned/forked (a la the company in question in OP) all of your efforts should be focused on damage control.
  - vthriller 8y ago
- tinus_hn 8y ago
  Much of it is api keys they would distribute in the deployed app anyway. Not really ‘secret’.
  - chrismeller 8y ago
    That’s how it starts though. If everything is provided at deploy and nothing is ever embedded in source code there’s no way you can end up in this situation. In some industries it’s also an audit or legal requirement that developers not have access to production credentials, so there’s no other way to reasonably handle that. Edit: also, rebuilding your source because an API key changed... no thanks.
    - tinus_hn 8y ago
      The choices are: 1. api key is publicly readable in a configuration files you ship 2. api key is compiled into the binary you ship. There is only obfuscation. Then again api keys are not security keys.
      - chrismeller 8y ago
        But literally in this case it was security keys. Even including an API key into the binary build is avoidable. Add an OAuth-style negotiation for the key as the first startup process. Start digging deeper and there are fewer and fewer reasons.
        dpark 8y ago
        > Add an OAuth-style negotiation for the key as the first startup process. How does this possibly work? You must have some “bootstrap key” that you would use to fetch the API key. You’re going to ship something in the app that says “hey, I’m really your app” or else you’re doing to allow anyone to fetch your API key. All you can do is obfuscate the process of getting the API key. You cannot actually keep it secret when you need clients to have access to it.
        8y ago
- jasonlotito 8y ago
  > dumb enough to put secrets in the code. Even Apple has released code doing "dumb" things. goto goto for example [1]. This is a simple mistake, easily caught using proper code reviewing techniques and tools, and yet it still happened. This means they could have prevented someone making this mistake if they invested the time and energy doing things properly. This is Apple here. We aren't even talking about mistakes from Microsoft or Amazon or other major software companies. And these people aren't dumb. Facebook isn't filled with "dumb" people, and yet, they've done far worse than Bilibili here with just their 'mistakes'. The reality is, smart people do dumb things all the time because they are trying to get things done. I'm not going to pass judgement on what other people did. I'm going to assume they did the best they could, and while they made mistakes, it doesn't make them dumb. Maybe someone got lazy, maybe someone was under pressure, and things just piled up. "It's bad, but we'll get to it later when we have the time." No one plans to have their code shared out to the public. I wonder how many of us could honestly come out clean for code they've written along the way. To not have someone say "Oh, you are using an older library there that's got a security bug" or "You shouldn't have done this" and what not. [1]: https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-goto-fail-apples-ssl-bug-explained-plus-an-unofficial-patch/ https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-got...
bArray 8y ago
Repository also appearing in GitLab: https://gitlab.com/wkingfly/openbilibili https://gitlab.com/wkingfly/openbilibili https://gitlab.com/panxue/openbilibili https://gitlab.com/panxue/openbilibili https://gitlab.com/efsg/openbilibili https://gitlab.com/efsg/openbilibili Either way, it's now spread to far and they need to take actions to protect their users.
dustinmoris 8y ago
I don't know what to make of this, but this all feels like a deliberate attempt to damage this company. Here are some interesting things I noticed: - GitHub has a lot of DMCAs each month and going through them it seems that all repos have been taken down by GitHub, but in this case the entire source code is still online despite it being posted here on HN for hours now and after they have been notified. - None of the other DMCAs (some of them really interesting) have ever trended on HN - The above linked repo has been forked more than 5k times, which is so much more than what any other DMCA reported repo has been ever forked from what I could see - The repo with the source code put a link to https://996.icu https://996.icu in the description - The person who posted the DMCA here on HN seems to be a new user who has only posted or commented on topics related to 996. Potentially the person/group has also gamed HN to get this link to the front page There is no proof, but it feels like there is a very coordinated and deliberate attempt to harm Bilibili which is kind of sad.
- kevin_thibedeau 8y ago
  The only sad thing here is a fraudulent DMCA takedown for a trademark violation.
- GoblinSlayer 8y ago
  What harm? It's nice to have the source.
pavelbr 8y ago
I'm a new developer (an intern, actually). I just started writing a system that requires a couple secret strings. Currently I just have them as constants with my code, with the idea that I'll figure out something to do with them once I make sure everything is working. What should I do with those secrets though? I'm not sure how to store them securely. So far I've been considering putting them in the server configuration so they can be read from environment variables, but that seems inconvenient for me and other developers and also not that much more secure.
- farisjarrah 8y ago
  How to handle organizational secrets is definitely your concern, however, you are probably too junior to be making decisions on implementing security best practices in production. Likely your company has methodologies in place to deal with deployment secrets. Ask a senior dev how they handle secret management. In many companies there are key management tools such as Hashicorp Vault or Ansible vault. Basically without knowing your environment its hard to tell you what to do, but there are lots of options out there, and your company may have already implemented some of them.
- nickflood 8y ago
  You read them from a config file and fill them into the config by hand while deploying. Never push secrets embedded into code or portions of the config file to your source repo. You can hardcode the secrets to test stuff, but the first time you push the code to the repo should be the time you change it to reading from config. And add config to gitignore cause even if you don't stage the particular lines with the secrets in them, there will come one time where you'll rush or will have too long of a day when you'll push those secrets by accident. If you've got a public repo, then it's over. On a private repo then you may not notice this or not remember to remove it with a force push. A point in time when you get tired of juggling config files manually in dev/prod is the point in time you explore the system for secret management and auto build/deployment as clearly your project has become useful/popular enough. Those are my IMO and what I use as thresholds. Of course, if your environment is more relaxed there's no limit on further improving this practice.
- danenania 8y ago

alias_neo 8y ago

I've always wondered, how it could be that someone can be smart enough to write what on the surface is some fairly clean Golang, and yet at the same time, dumb enough to put secrets in the code. I can forgive the use of MD5, because they probably just don't know their hashing/crypto but secrets? It's literally in the name. There is so much material in your 5 links alone, that anyone who desires could utterly own their infrastructure, and then some.

nemothekid 8y ago
It’s one of those things that you dangerously start when your project is small then when you balloon in size, you find that everyone is hard coding secrets in code and standing up some secrets infrastructure would take weeks to get right. It’s easier now with tools like Vault but let’s say you joined bilibili today - where do you even begin? You have a massive cultural problem before you even begin to tackle the technical one. Even a smart engineer may just resign to doing things the wrong way than trying to fight a huge political battle.
- alias_neo 8y ago
  Oh you're absolutely right, but there are some fairly straightforward steps to take to mitigate the risk with little effort. You can take arguments or env vars or config files (not added to Git) for your secrets. If you begin with a system of not putting the secrets in the code, ever, it's fairly straightforward to not make this mistake. A few minutes of setup on a repository and a mindfulness to be sure not to commit any new secret files that may be in use (and add them to the .gitignore) is a great start before getting to secret management a la Vault.
  - tracker1 8y ago
    For reference, here is my getConfig which uses the environment for configuration options. It's really easy enough to start with something like this, and add it in at the baseline. https://gist.github.com/tracker1/fcc39f40a0d14648501d329c7bdb8499 https://gist.github.com/tracker1/fcc39f40a0d14648501d329c7bd...
- heavenlyblue 8y ago
  >> you find that everyone is hard coding secrets in code and standing up some secrets infrastructure would take weeks to get right. You open up the code, find all of the secrets (e.g. using high-entropy substring search), replace them with access to a global variable, and set it from a file set by a configuration from a command line. Done.
  - colejohnson66 8y ago
    And when your manager says don’t do that because it’s a waste of time?
    - bmm6o 8y ago
      If you aren't allowed to do it right, I'm not sure what kind of advice will be useful.
    - alias_neo 8y ago
raxxorrax 8y ago
> dumb enough to put secrets in the code. Man, I have tons of auth data in services like AWS just in environment variables. But pushing your rsa key to github must have happened on a bad monday. I do often have auth info in code, plainly because of time constraints. You just have to remember it before pushing anything on github. But aside from that, is it possible to file a DMCA for anything that has been forked if it was published under a license that permitted that action?
- alias_neo 8y ago
  I think they examples you've listed are different, and more acceptable than pushing secrets and private keys to Github because you had dozens of them hard-coded. As for the aside, I imagine (with no background knowledge here) that as "owner" if you accidentally published something within that licensed code that does not belong or isn't covered by that license, you should probably have the right to remove it
  - kiallmacinnes 8y ago
    What if I accidentally contributed a little bit too much code to the Linux kernel? What if that code had been in the last 10 releases? What about the users and distributions running those versions? I'm not sure the answer is, or should be, as simple as "derp - delete immediately, this was never meant to be Open Source". That said, if the code was stolen, and then published under an OS licence - it's not OS. The "publisher" never had the rights to make it OS, so the "contract" is null and void...
    - alias_neo 8y ago
      This is what I was thinking. If it's something that wasn't meant to be published, I assume your right as the owner hold. How you'd solve this, logistically in an example like yours (Linux kernel) I have no idea, but I presume by the time it's been made public long enough to be cloned/forked (a la the company in question in OP) all of your efforts should be focused on damage control.
- vthriller 8y ago
tinus_hn 8y ago
Much of it is api keys they would distribute in the deployed app anyway. Not really ‘secret’.
- chrismeller 8y ago
  That’s how it starts though. If everything is provided at deploy and nothing is ever embedded in source code there’s no way you can end up in this situation. In some industries it’s also an audit or legal requirement that developers not have access to production credentials, so there’s no other way to reasonably handle that. Edit: also, rebuilding your source because an API key changed... no thanks.
  - tinus_hn 8y ago
    The choices are: 1. api key is publicly readable in a configuration files you ship 2. api key is compiled into the binary you ship. There is only obfuscation. Then again api keys are not security keys.
    - chrismeller 8y ago
      But literally in this case it was security keys. Even including an API key into the binary build is avoidable. Add an OAuth-style negotiation for the key as the first startup process. Start digging deeper and there are fewer and fewer reasons.
      - dpark 8y ago
        > Add an OAuth-style negotiation for the key as the first startup process. How does this possibly work? You must have some “bootstrap key” that you would use to fetch the API key. You’re going to ship something in the app that says “hey, I’m really your app” or else you’re doing to allow anyone to fetch your API key. All you can do is obfuscate the process of getting the API key. You cannot actually keep it secret when you need clients to have access to it.
        8y ago
jasonlotito 8y ago
> dumb enough to put secrets in the code. Even Apple has released code doing "dumb" things. goto goto for example [1]. This is a simple mistake, easily caught using proper code reviewing techniques and tools, and yet it still happened. This means they could have prevented someone making this mistake if they invested the time and energy doing things properly. This is Apple here. We aren't even talking about mistakes from Microsoft or Amazon or other major software companies. And these people aren't dumb. Facebook isn't filled with "dumb" people, and yet, they've done far worse than Bilibili here with just their 'mistakes'. The reality is, smart people do dumb things all the time because they are trying to get things done. I'm not going to pass judgement on what other people did. I'm going to assume they did the best they could, and while they made mistakes, it doesn't make them dumb. Maybe someone got lazy, maybe someone was under pressure, and things just piled up. "It's bad, but we'll get to it later when we have the time." No one plans to have their code shared out to the public. I wonder how many of us could honestly come out clean for code they've written along the way. To not have someone say "Oh, you are using an older library there that's got a security bug" or "You shouldn't have done this" and what not. [1]: https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-goto-fail-apples-ssl-bug-explained-plus-an-unofficial-patch/ https://nakedsecurity.sophos.com/2014/02/24/anatomy-of-a-got...

bArray 8y ago

Repository also appearing in GitLab: https://gitlab.com/wkingfly/openbilibili https://gitlab.com/wkingfly/openbilibili https://gitlab.com/panxue/openbilibili https://gitlab.com/panxue/openbilibili https://gitlab.com/efsg/openbilibili https://gitlab.com/efsg/openbilibili Either way, it's now spread to far and they need to take actions to protect their users.

dustinmoris 8y ago

I don't know what to make of this, but this all feels like a deliberate attempt to damage this company. Here are some interesting things I noticed: - GitHub has a lot of DMCAs each month and going through them it seems that all repos have been taken down by GitHub, but in this case the entire source code is still online despite it being posted here on HN for hours now and after they have been notified. - None of the other DMCAs (some of them really interesting) have ever trended on HN - The above linked repo has been forked more than 5k times, which is so much more than what any other DMCA reported repo has been ever forked from what I could see - The repo with the source code put a link to https://996.icu https://996.icu in the description - The person who posted the DMCA here on HN seems to be a new user who has only posted or commented on topics related to 996. Potentially the person/group has also gamed HN to get this link to the front page There is no proof, but it feels like there is a very coordinated and deliberate attempt to harm Bilibili which is kind of sad.

kevin_thibedeau 8y ago
The only sad thing here is a fraudulent DMCA takedown for a trademark violation.
GoblinSlayer 8y ago
What harm? It's nice to have the source.

pavelbr 8y ago

I'm a new developer (an intern, actually). I just started writing a system that requires a couple secret strings. Currently I just have them as constants with my code, with the idea that I'll figure out something to do with them once I make sure everything is working. What should I do with those secrets though? I'm not sure how to store them securely. So far I've been considering putting them in the server configuration so they can be read from environment variables, but that seems inconvenient for me and other developers and also not that much more secure.

farisjarrah 8y ago
How to handle organizational secrets is definitely your concern, however, you are probably too junior to be making decisions on implementing security best practices in production. Likely your company has methodologies in place to deal with deployment secrets. Ask a senior dev how they handle secret management. In many companies there are key management tools such as Hashicorp Vault or Ansible vault. Basically without knowing your environment its hard to tell you what to do, but there are lots of options out there, and your company may have already implemented some of them.
nickflood 8y ago
You read them from a config file and fill them into the config by hand while deploying. Never push secrets embedded into code or portions of the config file to your source repo. You can hardcode the secrets to test stuff, but the first time you push the code to the repo should be the time you change it to reading from config. And add config to gitignore cause even if you don't stage the particular lines with the secrets in them, there will come one time where you'll rush or will have too long of a day when you'll push those secrets by accident. If you've got a public repo, then it's over. On a private repo then you may not notice this or not remember to remove it with a force push. A point in time when you get tired of juggling config files manually in dev/prod is the point in time you explore the system for secret management and auto build/deployment as clearly your project has become useful/popular enough. Those are my IMO and what I use as thresholds. Of course, if your environment is more relaxed there's no limit on further improving this practice.
danenania 8y ago