3 ms·

I would like to see how to easily get regression tests from REPL driven development. In general I wonder how to quickly set up mocks and such with chatbot inte

I would like to see how to easily get regression tests from REPL driven development. In general I wonder how to quickly set up mocks and such with chatbot integrations.

diggan 1y ago
> how to easily get regression tests from REPL driven development Usually it's a matter of renaming "comment" to "deftest", and move some things around. Not sure how/why it would be different when using a LLM compared to doing it manually.
- rads 1y ago
  There is often a misunderstanding that “REPL driven development” means typing things into the REPL prompt interactively and accumulating state that’s not easy to reproduce. Writing code down in comment blocks as you go is the solution to that, as you mentioned. With that technique I have no fear of restarting my REPL and getting back to where I was.
  - diggan 1y ago
    That's true. To be fair, if you come from a language like Ruby, JavaScript or Python, a REPL is basically a standalone experience, so I understand why people initially think it's like that, the name is more or less the same :) "Interactive Development" is maybe a better overall term that can lead to people thinking "Oh, what is that?" rather than "I've tried that already, what's the point?"
    - worldsayshi 1y ago
      I like that. So you're saying that the switch between REPL and writing a test is much more seamless in a clojure system. That makes a lot of sense to me.
      - dandelion9 1y ago
        At a basic level any Lisp-like with an editor integrated REPL lets you put your cursor over the `greet` invocation below and evaluate it, and have the results shown in the manner of your choice (IDE window, inline temporary comment, etc). This is all in the file as saved to your version control: (defn greet [name] (-> (str "Hello, " name "!") println)) (comment (greet "worldsayshi")) It's 90% of the way to being a test already, just from the typical workflow.
  - worldsayshi 1y ago
    But if you're maintaining a bunch of code blocks by comment toggling, why not just turn it into tests from the get go? I suppose there's a somewhat fine line between TDD and REPL given the right tooling for it.
    - diggan 1y ago
      Different purposes. I usually start out with comment blocks, for experimenting/prototyping, once happy, cement whatever assertions I did there, but inside deftests. Not everything is moved to a test though, some things are just "helpers" sort of, and those tend to remain inside comment blocks instead of being converted to deftest or similar.
    - rads 1y ago
      I use the REPL as lightweight form of testing while I'm in the flow state, which is complementary to TDD. If there's too much code in the comment block, that's a good signal to move it into a formal test like you're saying, or at least move the code into a reusable function so the comment block can be shorter and focused on usage examples instead of implementation details.
  - reddit_clone 1y ago
    Exactly. I start in the repl, but when it becomes a few expressions deep, I transfer it to the editor and turn it into a proper defun. Start 'sending' the defun to the repl and test it out in repl. Rince repeat.
    - diggan 1y ago
      > I start in the repl [...] I transfer it to the editor That's exactly the kind of misunderstanding parent was talking about :) Why start somewhere else than your editor, if it's already hooked up to the REPL? When I launch my REPL, I don't think it even accepts stdin, because there is no reason it has to.
      - reddit_clone 1y ago
        There is no hard and fast rule (for me). It is easy to start creating objects in the repl (say if you are testing an API) and work with those created objects in the repl, one step at a time. You are able to observe the behaviour of the objects every step of the way. This gives you a much better idea when you are writing the functions (usually copy-pasting from repl) in the editor. I am sure it can be done from the editor itself. (say using the 'comment' block in clojure). It is just matter of preference.
- worldsayshi 1y ago
  > LLM compared to doing it manually. When testing against an LLM you probably want to mock its responses for tests. Which is quite doable by copy-pasting its responses. I'm just curious what the "best" workflow is.
  - diggan 1y ago
    If I'd be using Clojure for this, it would look something like this: 1. Write a function that returns a LLM response 2. Play around with this function in a `comment` block, highlighting what I'm testing 3. If I find a response I'd like to "cement"/save and use for testing, Conjure (what I use to connect to the REPL) has a command that is basically "evaluate what's highlighted in the editor with whatever that returns when evaluated", so that basically copy-pastes the return value into the editor 4. Move it into a test somewhere Not sure there is a better or faster workflow, from deciding "Hmm I want to test against this in the future" to "make test checks this" takes something like 2 minutes with a workflow like this.

I would like to see how to easily get regression tests from REPL driven development. In general I wonder how to quickly set up mocks and such with chatbot integrations.

diggan 1y ago

> how to easily get regression tests from REPL driven development Usually it's a matter of renaming "comment" to "deftest", and move some things around. Not sure how/why it would be different when using a LLM compared to doing it manually.

rads 1y ago
There is often a misunderstanding that “REPL driven development” means typing things into the REPL prompt interactively and accumulating state that’s not easy to reproduce. Writing code down in comment blocks as you go is the solution to that, as you mentioned. With that technique I have no fear of restarting my REPL and getting back to where I was.
- diggan 1y ago
  That's true. To be fair, if you come from a language like Ruby, JavaScript or Python, a REPL is basically a standalone experience, so I understand why people initially think it's like that, the name is more or less the same :) "Interactive Development" is maybe a better overall term that can lead to people thinking "Oh, what is that?" rather than "I've tried that already, what's the point?"
  - worldsayshi 1y ago
    I like that. So you're saying that the switch between REPL and writing a test is much more seamless in a clojure system. That makes a lot of sense to me.
    - dandelion9 1y ago
      At a basic level any Lisp-like with an editor integrated REPL lets you put your cursor over the `greet` invocation below and evaluate it, and have the results shown in the manner of your choice (IDE window, inline temporary comment, etc). This is all in the file as saved to your version control: (defn greet [name] (-> (str "Hello, " name "!") println)) (comment (greet "worldsayshi")) It's 90% of the way to being a test already, just from the typical workflow.
- worldsayshi 1y ago
  But if you're maintaining a bunch of code blocks by comment toggling, why not just turn it into tests from the get go? I suppose there's a somewhat fine line between TDD and REPL given the right tooling for it.
  - diggan 1y ago
    Different purposes. I usually start out with comment blocks, for experimenting/prototyping, once happy, cement whatever assertions I did there, but inside deftests. Not everything is moved to a test though, some things are just "helpers" sort of, and those tend to remain inside comment blocks instead of being converted to deftest or similar.
  - rads 1y ago
    I use the REPL as lightweight form of testing while I'm in the flow state, which is complementary to TDD. If there's too much code in the comment block, that's a good signal to move it into a formal test like you're saying, or at least move the code into a reusable function so the comment block can be shorter and focused on usage examples instead of implementation details.
- reddit_clone 1y ago
  Exactly. I start in the repl, but when it becomes a few expressions deep, I transfer it to the editor and turn it into a proper defun. Start 'sending' the defun to the repl and test it out in repl. Rince repeat.
  - diggan 1y ago
    > I start in the repl [...] I transfer it to the editor That's exactly the kind of misunderstanding parent was talking about :) Why start somewhere else than your editor, if it's already hooked up to the REPL? When I launch my REPL, I don't think it even accepts stdin, because there is no reason it has to.
    - reddit_clone 1y ago
      There is no hard and fast rule (for me). It is easy to start creating objects in the repl (say if you are testing an API) and work with those created objects in the repl, one step at a time. You are able to observe the behaviour of the objects every step of the way. This gives you a much better idea when you are writing the functions (usually copy-pasting from repl) in the editor. I am sure it can be done from the editor itself. (say using the 'comment' block in clojure). It is just matter of preference.
worldsayshi 1y ago
> LLM compared to doing it manually. When testing against an LLM you probably want to mock its responses for tests. Which is quite doable by copy-pasting its responses. I'm just curious what the "best" workflow is.
- diggan 1y ago
  If I'd be using Clojure for this, it would look something like this: 1. Write a function that returns a LLM response 2. Play around with this function in a `comment` block, highlighting what I'm testing 3. If I find a response I'd like to "cement"/save and use for testing, Conjure (what I use to connect to the REPL) has a command that is basically "evaluate what's highlighted in the editor with whatever that returns when evaluated", so that basically copy-pastes the return value into the editor 4. Move it into a test somewhere Not sure there is a better or faster workflow, from deciding "Hmm I want to test against this in the future" to "make test checks this" takes something like 2 minutes with a workflow like this.