5 ms·

1. We have a lot of code paths, there isn't an integration test for everything. And for runtime, just because something is rarely called doesn't mean it never c

1. We have a lot of code paths, there isn't an integration test for everything. And for runtime, just because something is rarely called doesn't mean it never called; a bank can have yearly processes 2. We'll do service mesh too but this was probably an easier first step. We have rolled our own mesh with envoy sidecars and moving it towards istio style behaviour isn't trivial

tjungblut 7y ago
> We have a lot of code paths, there isn't an integration test for everything. but you will have the service names that you're contacting configured somewhere (or in the worst case hardcoded). I would think this is much easier to analyze than the traffic flows.
- YawningAngel 7y ago
  It's done by importing a protobuf, so I imagine rpcmap just checks all the protos imported by a service
  - p10jkle 7y ago
    It's slightly more complicated than that because you can import protos to use a constant or to consume an event from another service. But that is the gist
- p10jkle 7y ago
  That's what we're doing, basically
  - tjungblut 7y ago
    then why do you need the runtime bit?
    - p10jkle 7y ago
      We don't have one. I think you may have misinterpreted my comments above. Its all code analysis
speedplane 7y ago
> We have a lot of code paths, there isn't an integration test for everything. This is just saying that writing "tests is hard". I agree, it is. I always wonder about the value of static analysis. I've used it myself and have found bugs with it, but they are mostly easy bugs to find, and there's a lot of noise. The hard bugs are often only found with serious integration tests. I'll feel much more confident with a codebase that's 90% covered by tests than one that's statically analyzed.
- bostik 7y ago
  You should think of static analysis as a form of linting. It's there to catch stupid errors and brainfarts that should not need to even reach a reviewer. Dynamic analysis, instrumentation and intelligent logging are invaluable in catching a good chunk of the remaining non-trivial errors. (I have a soft spot in my heart for fuzzing, which really should be considered as just another form of exploratory/destructive testing.)
  - speedplane 7y ago
    > You should think of static analysis as a form of linting. It's there to catch stupid errors and brainfarts that should not need to even reach a reviewer. I actually agree. But the implication is that static analysis should really live in the IDE, not the compiler or test suite. It's best when catching the silly whoopsy bugs as you're typing them. But I've yet to see a static analyzer that can catch race conditions, network or I/O errors, or something that can verify what you expect to happen actually does happen. Thinking about static analysis on a higher level, arguably they shouldn't be necessary at all. If a system can statically analyze your code and find faults, then it implies that there is a deficiency in the language itself.

tjungblut 7y ago

> We have a lot of code paths, there isn't an integration test for everything. but you will have the service names that you're contacting configured somewhere (or in the worst case hardcoded). I would think this is much easier to analyze than the traffic flows.

YawningAngel 7y ago
It's done by importing a protobuf, so I imagine rpcmap just checks all the protos imported by a service
- p10jkle 7y ago
  It's slightly more complicated than that because you can import protos to use a constant or to consume an event from another service. But that is the gist
p10jkle 7y ago
That's what we're doing, basically
- tjungblut 7y ago
  then why do you need the runtime bit?
  - p10jkle 7y ago
    We don't have one. I think you may have misinterpreted my comments above. Its all code analysis

speedplane 7y ago

> We have a lot of code paths, there isn't an integration test for everything. This is just saying that writing "tests is hard". I agree, it is. I always wonder about the value of static analysis. I've used it myself and have found bugs with it, but they are mostly easy bugs to find, and there's a lot of noise. The hard bugs are often only found with serious integration tests. I'll feel much more confident with a codebase that's 90% covered by tests than one that's statically analyzed.

bostik 7y ago
You should think of static analysis as a form of linting. It's there to catch stupid errors and brainfarts that should not need to even reach a reviewer. Dynamic analysis, instrumentation and intelligent logging are invaluable in catching a good chunk of the remaining non-trivial errors. (I have a soft spot in my heart for fuzzing, which really should be considered as just another form of exploratory/destructive testing.)
- speedplane 7y ago
  > You should think of static analysis as a form of linting. It's there to catch stupid errors and brainfarts that should not need to even reach a reviewer. I actually agree. But the implication is that static analysis should really live in the IDE, not the compiler or test suite. It's best when catching the silly whoopsy bugs as you're typing them. But I've yet to see a static analyzer that can catch race conditions, network or I/O errors, or something that can verify what you expect to happen actually does happen. Thinking about static analysis on a higher level, arguably they shouldn't be necessary at all. If a system can statically analyze your code and find faults, then it implies that there is a deficiency in the language itself.