3 ms·
Thank you for this feedback. We actually have an Auto Wiki v2 in the works which is even higher quality, would be interesting to see how it changes when that co
by oshams 3y ago
Thank you for this feedback. We actually have an Auto Wiki v2 in the works which is even higher quality, would be interesting to see how it changes when that comes out.
- faizshah 3y agoCan you talk a little bit about the crawler or what information are you feeding the agent about the repostory? My main concern is that this is just hallucinating the documentation and that with the more well known repos like React it can pull the data from training data like blogs etc. I think the concept is really great would just like to understand especially for enterprise use cases.
- oshams 3y agoIt's purely based on the code and we force the LLM to cite the code it describes to cut back on hallucination. We are adding a self verification steps (like chain-of-verification / https://arxiv.org/abs/2309.11495 https://arxiv.org/abs/2309.11495) in Wiki V2.
- GrinningFool 3y agoI would expect something like this to output only factually correct documentation since it would be used as reference; but it sounds like even under the upcoming v2 that's not the case?
- HL33tibCe7 3y agoAutogenerating documentation that could then be corrected by a human could still have value, in fairness
- krupan 3y agoReally? How much value would you put on that actually? How would you price that if you were selling it? Would I get a little refund for each mistake I find and have to correct?
- GrinningFool 3y agoFor one-and-done, I agree. But if you're looking at dynamic code base and you want to keep your documentation in sync, having to make 'manually correct the generated output' part of your build process can get expensive. (And tedious for the poor grunt who's got to do the work.)
- oshams 3y agoWe think chain-of-verification and fine tuning will help a lot see the other response. We are really excited to launch v2 ASAP so we can address these concerns.