6 ms·

This might be the best introduction post I've read. Lays the foundation (get it?) for who the people are and what they've built. Then explains how the current

This might be the best introduction post I've read. Lays the foundation (get it?) for who the people are and what they've built. Then explains how the current thing they are building is a result of the previous thing. It feels that they actually want this problem solved for everyone because they have experienced how good the solution feels. Then tells us about the teams (pretty big names with complex systems) that have already used it. All of these wrapped in good writing that appeals to developers/founders. Landing page is great too!

getoffmycase 3y ago
The entire testing system they describe feels like something I can strive towards too. They make you want their solution because it offers a way of life and thinking and doing like you've never experienced before
foobarqux 3y ago
Except it doesn't actually explain in what it does: Is it fuzzing? Do you supply your own test cases? Is it testing hardware non-determinism?
- Aeolun 3y ago
  Yeah. I could figure out the global idea, but then the mechanics of how it would actually work were very sparse.
- wwilson 3y ago
  Post author here. Sorry it was vague, but there's only so much detail you can go into in a blog post aimed at general audiences. Our documentation (https://antithesis.com/docs/ https://antithesis.com/docs/) has a lot more info. Here's my attempt at a more complete answer: think of the story of the blind men and the elephant. There's a thing, called fuzzing, invented by security researchers. There's a thing, called property-based testing, invented by functional programmers. There's a thing, called network simulation, invented by distributed systems people. There's a thing, called rare-event simulation, invented by physicists (!). But if you squint, all of these things are really the same kind of thing, which we call "autonomous testing". It's where you express high-level properties of your system, and have the computer do the grunt work to see if they're true. Antithesis is our attempt to take the best ideas from each of these fields, and turn them into something really usable for the vast majority of software. We believe the two fundamental problems preventing widespread adoption of autonomous testing are: (1) most software is non-deterministic, but non-determinism breaks the core feedback loop that guides things like coverage-guided fuzzing. (2) the state space you're searching is inconceivably vast, and the search problem in full generality is insolubly hard. Antithesis tries to address both of these problems. So... is it fuzzing? Sort of, except you can apply it to whole interacting networked systems, not just standalone parsers and libraries. Is it property-based testing? Sort of, except you can express properties that require a "global" view of the entire state space traversed by the system, which could never be locally asserted in code. Is it fault injection or chaos testing? Sort of, except that it can use the techniques of coverage guided fuzzing to get deep into the nooks and crannies of your software, and determinism to ensure that every bug is replayable, no matter how weird it is. It's hard to explain, because it's hard to wrap your arms around the whole thing. But our other big goal is to make all of this easy to understand and easy to use. In some ways, that's proved to be even harder than the very hard technological problems we've faced. But we're excited and up for it, and we think the payoff could be big for our whole industry. Your feedback about what's explained well and what's explained poorly is an important signal for us in this third very hard task. Please keep giving it to us!
  - criddell 3y ago
    > turn them into something really usable for the vast majority of software Would it work for debugging, say, Notepad on Windows?
  - jldugger 3y ago
    I remember watching the Strange Loop video on your testing strategy, and now I need to go back and relearn how it differed from model checking (ie Promela or TLA+). Model checking is probably the big QA story that tech companies ignore because it requires dramatically more education, especially from QA departments typically seen as "inferior" to SWE.
    - rhodin 3y ago
      Video of [0] the Strangeloop talk [1]. [0] https://www.youtube.com/watch?v=4fFDFbi3toc https://www.youtube.com/watch?v=4fFDFbi3toc [1] https://thestrangeloop.com/2014/testing-distributed-systems-w-slash-deterministic-simulation.html https://thestrangeloop.com/2014/testing-distributed-systems-...
  - randomdata 3y ago
    > most software is non-deterministic Doesn't Antithesis rely on the fact that software is always deterministic? Reproducibility appears to be its top selling feature – something that wouldn't be possible if software were non-deterministic.
    - wwilson 3y ago
      We can force any* software to be deterministic. * Offer only good for x86-64 software that runs on Linux whose dependencies you can install locally or mock. The first two restrictions we will probably relax someday.
      - randomdata 3y ago
        Aren't you just 'forcing' determinism in the inputs, relying on the software to be always deterministic for the same inputs?
- kretaceous 3y ago
  Sure, it doesn't go into details. And that is exactly why I termed it an excellent introduction and a sales pitch. I haven't heard of deterministic testing before. Nor have I heard of FoundationDB or the related things. And I went from knowing zero things about them to getting impressed and interested. This led me to go into their docs, blogs, landing page, etc. to know more.
deleted 3y ago
[deleted]
k__ 3y ago
Did you read a different article than me? The linked article is 3/4 about some history and rationale before it actually tells you what they build. It's like those pesky recipe blogs that tell you about the authors childhood, when you just want to make vegan pancakes.
chinchilla2020 3y ago
It seems like marketing copy. Not a technical blog post. It would be nice to see some actual use cases and examples. Instead, the writer just name-dropped a few big companies and claimed to have a revolutionary product that works magically. Then include the typical buzzwords like '10x programmer' and 'stealth mode'. The latter doesn't make sense because they also name-drop clients.
- MattRix 3y ago
  It absolutely doesn’t read like typical marketing copy, and yes it’s not a dense technical blog post either. I’m sure the use cases and examples will come, but putting them in this post would have been overkill. Also, stealth mode just means your company isn’t public, you can still have clients.
- amw-zero 3y ago
  I'm assuming you aren't aware of FoundationDB: https://www.foundationdb.org/files/fdb-paper.pdf https://www.foundationdb.org/files/fdb-paper.pdf Having that context puts the post in a much better perspective. It's definitely an introduction post (the company has been developing this in stealth mode for the past few years), but it is most certainly _not_ a marketing post. These people developed extremely novel testing techniques for FoundationDB and are now generalizing them to work with any containerized application. It's a big deal.
  - chinchilla2020 3y ago
    I have heard of it but know very little about it. I'm reading the paper. It's very intriguing but so is the marketing material for the new Tesla. I need to work with things like this before I believe the claims. Too many people here on HN are getting caught up in a barely-tested new technology and hailing it as some revolution.
deleted 3y ago
[deleted]

getoffmycase 3y ago

The entire testing system they describe feels like something I can strive towards too. They make you want their solution because it offers a way of life and thinking and doing like you've never experienced before

foobarqux 3y ago

Except it doesn't actually explain in what it does: Is it fuzzing? Do you supply your own test cases? Is it testing hardware non-determinism?

Aeolun 3y ago
Yeah. I could figure out the global idea, but then the mechanics of how it would actually work were very sparse.
wwilson 3y ago
Post author here. Sorry it was vague, but there's only so much detail you can go into in a blog post aimed at general audiences. Our documentation (https://antithesis.com/docs/ https://antithesis.com/docs/) has a lot more info. Here's my attempt at a more complete answer: think of the story of the blind men and the elephant. There's a thing, called fuzzing, invented by security researchers. There's a thing, called property-based testing, invented by functional programmers. There's a thing, called network simulation, invented by distributed systems people. There's a thing, called rare-event simulation, invented by physicists (!). But if you squint, all of these things are really the same kind of thing, which we call "autonomous testing". It's where you express high-level properties of your system, and have the computer do the grunt work to see if they're true. Antithesis is our attempt to take the best ideas from each of these fields, and turn them into something really usable for the vast majority of software. We believe the two fundamental problems preventing widespread adoption of autonomous testing are: (1) most software is non-deterministic, but non-determinism breaks the core feedback loop that guides things like coverage-guided fuzzing. (2) the state space you're searching is inconceivably vast, and the search problem in full generality is insolubly hard. Antithesis tries to address both of these problems. So... is it fuzzing? Sort of, except you can apply it to whole interacting networked systems, not just standalone parsers and libraries. Is it property-based testing? Sort of, except you can express properties that require a "global" view of the entire state space traversed by the system, which could never be locally asserted in code. Is it fault injection or chaos testing? Sort of, except that it can use the techniques of coverage guided fuzzing to get deep into the nooks and crannies of your software, and determinism to ensure that every bug is replayable, no matter how weird it is. It's hard to explain, because it's hard to wrap your arms around the whole thing. But our other big goal is to make all of this easy to understand and easy to use. In some ways, that's proved to be even harder than the very hard technological problems we've faced. But we're excited and up for it, and we think the payoff could be big for our whole industry. Your feedback about what's explained well and what's explained poorly is an important signal for us in this third very hard task. Please keep giving it to us!
- criddell 3y ago
  > turn them into something really usable for the vast majority of software Would it work for debugging, say, Notepad on Windows?
- jldugger 3y ago
  I remember watching the Strange Loop video on your testing strategy, and now I need to go back and relearn how it differed from model checking (ie Promela or TLA+). Model checking is probably the big QA story that tech companies ignore because it requires dramatically more education, especially from QA departments typically seen as "inferior" to SWE.
  - rhodin 3y ago
    Video of [0] the Strangeloop talk [1]. [0] https://www.youtube.com/watch?v=4fFDFbi3toc https://www.youtube.com/watch?v=4fFDFbi3toc [1] https://thestrangeloop.com/2014/testing-distributed-systems-w-slash-deterministic-simulation.html https://thestrangeloop.com/2014/testing-distributed-systems-...
- randomdata 3y ago
  > most software is non-deterministic Doesn't Antithesis rely on the fact that software is always deterministic? Reproducibility appears to be its top selling feature – something that wouldn't be possible if software were non-deterministic.
  - wwilson 3y ago
    We can force any* software to be deterministic. * Offer only good for x86-64 software that runs on Linux whose dependencies you can install locally or mock. The first two restrictions we will probably relax someday.
    - randomdata 3y ago
      Aren't you just 'forcing' determinism in the inputs, relying on the software to be always deterministic for the same inputs?
kretaceous 3y ago
Sure, it doesn't go into details. And that is exactly why I termed it an excellent introduction and a sales pitch. I haven't heard of deterministic testing before. Nor have I heard of FoundationDB or the related things. And I went from knowing zero things about them to getting impressed and interested. This led me to go into their docs, blogs, landing page, etc. to know more.

deleted 3y ago

[deleted]

k__ 3y ago

Did you read a different article than me? The linked article is 3/4 about some history and rationale before it actually tells you what they build. It's like those pesky recipe blogs that tell you about the authors childhood, when you just want to make vegan pancakes.

chinchilla2020 3y ago

It seems like marketing copy. Not a technical blog post. It would be nice to see some actual use cases and examples. Instead, the writer just name-dropped a few big companies and claimed to have a revolutionary product that works magically. Then include the typical buzzwords like '10x programmer' and 'stealth mode'. The latter doesn't make sense because they also name-drop clients.

MattRix 3y ago
It absolutely doesn’t read like typical marketing copy, and yes it’s not a dense technical blog post either. I’m sure the use cases and examples will come, but putting them in this post would have been overkill. Also, stealth mode just means your company isn’t public, you can still have clients.
amw-zero 3y ago
I'm assuming you aren't aware of FoundationDB: https://www.foundationdb.org/files/fdb-paper.pdf https://www.foundationdb.org/files/fdb-paper.pdf Having that context puts the post in a much better perspective. It's definitely an introduction post (the company has been developing this in stealth mode for the past few years), but it is most certainly _not_ a marketing post. These people developed extremely novel testing techniques for FoundationDB and are now generalizing them to work with any containerized application. It's a big deal.
- chinchilla2020 3y ago
  I have heard of it but know very little about it. I'm reading the paper. It's very intriguing but so is the marketing material for the new Tesla. I need to work with things like this before I believe the claims. Too many people here on HN are getting caught up in a barely-tested new technology and hailing it as some revolution.

deleted 3y ago

[deleted]