4 ms·
Things I'd like to know about this tool: - What areas/languages/tasks is it good at and what is it bad at? - How often is it generating code with bugs? - How
by iandanforth 4y ago
Things I'd like to know about this tool:
- What areas/languages/tasks is it good at and what is it bad at?
- How often is it generating code with bugs?
- How often is the code that gets generated used as is vs immediately edited?
- What are the new frustrations that this causes that existing IDE code completion doesn't?
I work with trained models daily and I know that their failure cases are unintuitive, unexpected, and exasperating. I'd like to know as much about the failure cases of this model as possible before diving in.
- bardia95 4y agoReplit team member here: - Ghostwriter is especially good at Python and JS, but supports up to 16 languages (to varying degrees of effectiveness) you can read more here: https://docs.replit.com/ghostwriter/faq#which-programming-languages-does-ghostwriter-support https://docs.replit.com/ghostwriter/faq#which-programming-la... - As for tasks, it's great for reducing the amount of boilerplate code you need to write (Complete Code), writing React components (Complete + Generate Code), explaining code in plain English (Explain Code), translating code between languages (Transform Code), writing exhaustive tests (Complete Code) - No stats on how often its generating code with bugs or how often the code gets generated used as is vs immediately edited, we're interested in getting both to help improve Ghostwriter - Like any LLM, it can get stuck in a long-tail of repetitive loops; we're working pretty hard to improve and mitigate these issues, but, especially for new users, the repetition and hallucination type problems can be distracting.
- deleted 4y ago[deleted]