4 ms·

Show HN: Wasted hours on Unit tests, so I built VS Code ext for AI tests

• graypegg 3y ago
  Anyone made the opposite product? Human written tests, that an AI tries to make green?
  • fmdz 3y ago
    We are writting ton of such tests, on daily basis haha. That is infinite process, we're all stuck within :-)
  • darkteflon 3y ago
    That’s actually a really interesting idea. Particularly if you take care to add docstrings and comments to the tests. Thanks!
    • graypegg 3y ago
      Keep me posted if you get that working! Would be really interesting to see! Might mess around with it myself sometime too.
  • 8note 3y ago
    Given the "ai Mario just pauses the game and calls it a win" I'd expect that the ai written code doesn't do the right thing outside of the given test cases
  • ramon156 3y ago
    Watch out, we'll end up writing unit tests 40 hrs a week
• lnxg33k1 3y ago
  Is there a place where to see the code you’re trying to test? My primary use of unit tests is to understand if its clean and maintainable code, so im likely your last user case, but wanted to see what kind of code took hours to test
  • fmdz 3y ago
    Do you mean within overview page of extension, on examples gif/images or within real flow in VS Code?
    • lnxg33k1 3y ago
      Just some code examples of what was being tested
      • fmdz 3y ago
        Sure. Here is one example from run I performed just now. TypeScript utility method I generated cases and tests for ------- static merge<T extends { id?: string }>(arr1: T[], arr2: T[]): T[] { arr1 = arr1 || []; arr2 = arr2 || []; // Create a map for quick access to items in arr1 by id const map = new Map<string, T>(arr1.map((item) => [item.id, item])); // Iterate over arr2 and merge or add items arr2.forEach((item) => { if (map.has(item.id)) { // If item exists in arr1, merge it map.set(item.id, { ...map.get(item.id), ...item }); } else { // If item doesn't exist in arr1, add it map.set(item.id, item); } }); // Convert the map back to an array return Array.from(map.values()); } Generated test cases ------------- POSITIVE - Merging two arrays with unique and overlapping objects by id Given: Two arrays of objects with unique and overlapping ids When: Merging two arrays of objects where some objects have matching ids Then: Should return an array of merged objects with no duplicates and combined properties for overlapping ids NEGATIVE - Merging arrays when second array contains objects with undefined ids Given: First array of objects with valid ids and second array with some objects having undefined ids When: Second array contains objects with undefined ids Then: Should throw an error or handle undefined ids gracefully without merging those objects EDGE CASE - Merging arrays where all objects in the second array have matching ids in the first array Given: Two arrays of objects with ids where the second array's objects all have ids that match those in the first array When: Every object in the second array has a matching id in the first array Then: Should return an array with objects from the first array having their properties updated by the second array's objects Generated unit tests by test cases ------------- ---> 1st case --------------- it('Merging two arrays with unique and overlapping objects by id should return an array of merged objects with no duplicates and combined properties for overlapping ids', () => { // Arrange const arr1 = [{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }]; const arr2 = [{ id: '2', name: 'Robert' }, { id: '3', name: 'Charlie' }]; const expectedResult = [ { id: '1', name: 'Alice' }, { id: '2', name: 'Robert' }, // Note: 'Bob' is overwritten by 'Robert' { id: '3', name: 'Charlie' } ]; // Act const result = merge(arr1, arr2); // Assert expect(result).toEqual(expectedResult); }); ---> 2nd case ---------------- it('Merging arrays when second array contains objects with undefined ids should throw an error or handle undefined ids gracefully', () => { // Arrange const arr1 = [{ id: '1', name: 'Alice' }, { id: '2', name: 'Bob' }]; const arr2 = [{ id: '2', age: 30 }, { name: 'Charlie' }]; // Charlie has undefined id // Act and Assert expect(() => MyArrayUtils.merge(arr1, arr2)).toThrow(); }); ---> 3rd case ---------------- it('Merging arrays where all objects in the second array have matching ids in the first array', () => { // Arrange const arr1 = [{ id: '1', propA: 'A1' }, { id: '2', propA: 'A2' }]; const arr2 = [{ id: '1', propB: 'B1' }, { id: '2', propB: 'B2' }]; const expectedResult = [ { id: '1', propA: 'A1', propB: 'B1' }, { id: '2', propA: 'A2', propB: 'B2' } ]; // Act const result = merge(arr1, arr2); // Assert expect(result).toEqual(expectedResult); });
• 8note 3y ago
  I don't really understand what the test cases will do? Is the code expected to already correct, and the tests are documentation? If I imagine a workflow of 1. Updating some code (with a regression) 2. Running the ai to generate tests 3. The tests pass How do I know that the ai didn't assume the regression is expected? Do I read the explanation of the test cases?
  • fmdz 3y ago
    There are positive, negative and edge cases. They are generated by AI, so unit tests will be implemented by case definitions.
• bicepjai 3y ago
  IMHO, Testing is one thing that needs to be inspected when it comes to AI code generation
  • fmdz 3y ago
    It is, I agree. You have full control over generated tests. Also, I put focus on putting as much control as possible, so you can set different inputs to reach higher quality of generated tests. At the end, you add or append file with generated tests, so you have full control over them.