5 ms·
It’s nuts to think that in the near future LLM will be able to do that refactoring in seconds. All we need is enough context window.
by motoboi 3y ago
It’s nuts to think that in the near future LLM will be able to do that refactoring in seconds. All we need is enough context window.
- bigbillheck 3y agoWhy on earth would you possibly think that?
- ctoth 3y agoJust as your human intelligence lead to you writing the same darn comment as another human above you, AI can often write the same code as a human would, without having to even bring creativity into it! For those of us who write code, this can be useful!
- motoboi 3y agoI'm quite confused by the answers I got from this thread. Haven't you tried refactoring with gpt-4 yet?
- bigbillheck 3y ago> Haven't you tried refactoring with gpt-4 yet? I most certainly have not. At work, I do greenfield development in a specialized problem domain, and I would not trust a model (or, for that matter, a junior developer) to do any kind of refactor in an acceptable manner. (That aside, there's no way I'm goingto expose company code to any sort of outside party without the approval of upper management). At home, I program for fun and self-improvement, and a big part of both is thinking hard about problems. Why would I want to wreck that with asking a model to do it for me?
- motoboi 3y agoOh, I understand. What if you have an idea and you tell the computer to implement it and then thoroughly check the code? About the self-improvement part, I suppose you don't operate your graphics card yourself, but delegate it to your driver. The LLM is just another tool.
- smabie 3y agoOh yea your work is far too sophisticated for a llm, got it
- bigbillheck 3y agoSome of us do actual creative work, yes.
- airstrike 3y agoSome people get significance from their ability to write code. To them, admitting an LLM can (or will soon be able to) do their work inflicts cognitive dissonance, so they refuse to believe it. Some refuse to even try it—not realizing that refusing to engage does nothing to hinder the advancement of the tool they fear.
- bsdpufferfish 3y agoWhy would you think this is possible?
- motoboi 3y agobased on my current experience with gpt-4. Have you tried some sort of refactoring in it? Because I have been routinely turning serial scripts into parallel ones with success. Couldn't do the same with larger codebases because the context is not enough for the input code and output refactoring.
- thfuran 3y agoYeah, it's nuts to think that.
- motoboi 3y agoI'm genuinely confused about your point of view. Have you tried refactoring with GPT-4? I have been refactoring code using gpt-4 for some months now and the limiting factor have been the context size. GPT-4 turbo now have 128k context and I can provide it with larger portions of the code base for the refactors. When we have millions of tokens of context, based on what I'm experiencing now, I can see that a refactoring like the one made in ffmpeg would be possible. Or not? What am I missing here?
- rocqua 3y agoRefactoring is really rather well defined. It's " just transformations that are invariant w.r.t. the outcome". The reason they are hard to automate is that 'invariant w.r.t. the outcome' is a lot more lenient than most semantic models van handle. But this kind of well-defined task with a slight amount of nuance (and decently checkable) seems pretty well-suited to an LLM.
- nolist_policy 3y agoAt least for the linux kernel, qemu and other large c projects, this is a solved problem with coccinelle[1]. Compared to AI, it has the added benefit of not doing incorrect changes and/or hallucinating stuff or promt injections or ... I guess you could use AI to help create a coccinelle semantic patch. [1] https://en.wikipedia.org/wiki/Coccinelle_(software) https://en.wikipedia.org/wiki/Coccinelle_(software)
- dataangel 3y agoThe part coccinelle does is the part GPT is good at, the problem is neither of them actually reason about the code
- mrintegrity 3y agoParent post is getting down voted to oblivion but it seems a reasonable belief for someone who is not highly engaged with AI. I have only the vaguest understanding of how it works (and it's probably wrong) and to my layman mind it also seems like a totally fair assumption, based on experience as a user and the constant flood of news. Please explain why the suggestion that a future AI / sufficiently advanced LLM could refactor a complex codebase is so preposterous.
- not_the_fda 3y agoBecause refactoring requires understanding, which LLMs completely lack.
- ctoth 3y ago> Because refactoring requires understanding, which LLMs completely lack. Interesting. Here's a sample question I just asked the AI: We are working on a React mapping application. I am going to give you two components from this application. Our goal is to extract functionality to deduplicate code We can make a single base component, extract code to hooks, or any other strategy which you think is appropriate Here is travel.tsx: <snip> Here is heatmap.tsx: <snip> And here is the response: To deduplicate code and improve maintainability between the `TravelMap` and `Heatmap` components in your React mapping application, we can identify common functionalities and extract them into reusable hooks or components. Here's a strategy considering your code: 1. *Extract Common Hooks*: Both components use similar hooks for managing markers, popups, and map references. We can create custom hooks that encapsulate these functionalities. 2. *Create Shared Utility Functions*: Functions like `initialViewState`, which are identical in both components, can be moved to a separate utility file and imported where needed. 3. *Refactor Map Layers and Sources*: The way map layers and sources are handled is quite similar in both components. A custom hook or component to manage map layers and sources would be beneficial. 4. *Generalize Popup Handling*: The popup handling in both components can be abstracted into a custom hook or a higher-order component. 5. *Combine Common Styling*: Shared styles, such as `MAP_CONTAINER_STYLE`, can be moved to a common CSS module or a styled-component that can be reused. 6. *Refactor Event Handlers*: Functions like `onMapClick`, `onMouseMove`, and `onMouseLeave` have similar structures. These can be part of a custom hook or a utility function with configurable parameters. 7. *Legend and Live Region Components*: These components are used in a similar way in both `TravelMap` and `Heatmap`. Ensure they are designed to be reusable with different props. 8. *Refactor Map Initialization*: The way the map is initialized and configured is quite similar. This can be abstracted into a custom component or hook. ### Example Implementations: #### 1. Custom Hook for Map Markers: ```typescript // useMapMarkers.ts import { useState, useCallback } from 'react'; export const useMapMarkers = (initialLat, initialLng, world) => { // Similar marker logic from your components // ... return { marker, onMarkerDrag, onMarkerDragEnd, updateMarkerPosition, // other marker related states and functions }; }; ``` #### 2. Shared Utility Function: ```typescript // utils/mapUtils.ts export const initialViewState = (map, mapType) => { // Implementation }; ``` #### 3. Custom Hook for Map Layers and Sources: ```typescript // useMapLayers.ts import { useState, useEffect } from 'react'; export const useMapLayers = (props) => { // Logic to handle layers and sources // ... return { dataLayer, patternLayer, // other layer related states and functions }; }; ``` #### 4. Generalized Popup Handling: ```typescript // useMapPopup.ts import { useState } from 'react'; export const useMapPopup = () => { // Popup logic from your components // ... return { popupState, openPopup, closePopup, setPopupData, // other popup related states and functions }; }; ``` ### Integration: Once these hooks and utilities are created, integrate them into your `TravelMap` and `Heatmap` components. Replace the duplicated logic with calls to these shared resources. This will significantly reduce code duplication and make future maintenance easier. ### Testing and Refinement: After refactoring, thoroughly test both components to ensure that all functionalities work as expected. Watch out for any edge cases or specific behaviors that might be affected by the refactoring. For those suggestions, I might use five out of eight of them, and probably do one or two things differently. But you cannot, with a straight face, say the model did not understand. It clearly did. It suggested reasonable refactors. If being able to refactor means understanding, I guess we have understanding! I could continue with this conversation, ask it to produce the full code for the hooks (I have in my custom prompt to provide outlines) and once the hooks are complete, ask it to rewrite the components using the shared code. Have you ever used one of these models?
- ctoth 3y agoI see that you got some responses from people who may have not even used gpt-4 as a coding assistant, but I absolutely agree with you. A larger context window, a framework like Aider, and slightly-better tooling so the AI can do renames and other high-level actions without having to provide the entire changeset as patches, and tests. Lots of tests. Then you can just run the migration 15 times, pick from the one which passes all the tests... run another integration pass to merge ideas from the other runs, rinse and repeat. Of course the outer loops will themselves be automated. The trick to this is continuous iteration and feedback. It's remarkable how far I've gotten with GPT using these simple primitives and I know I'm not the only one.
- beeboobaa 3y agoIf you think a large refactor is just renaming some stuff then it makes sense you think this.
- dataangel 3y agoIf you ask GPT to refactor a single threaded program much smaller than the context window that is truly out of sample into a multithreaded program, its often going to fail. GPT has trouble understanding bit masks in single threaded code, let alone multiple threads.
- bigbillheck 3y ago> so the AI can do renames I do renames in a big window with my IDE.
- beeboobaa 3y agoI have some snake oil to sell you