3 ms·
Even with old GPT, if the summary is decent, it works reasonably well with even no RAG. We are a data management platform and allow users to build data pipeline
by neeleshs 2y ago
Even with old GPT, if the summary is decent, it works reasonably well with even no RAG. We are a data management platform and allow users to build data pipelines around a data model. This is basically a DAG. We autogenerate documentation for these pipelines, using gpt4, and feed a summarized version of the data pipeline - expressed as graphviz dot file format in the prompt. gpt4 understands this format well, and seeminlgy understands the graph itself reasonably well!
It performs poorly expressing the higher level intent of the pipeline, but tactical details are accurately documented. We are trying to push prompting itself more, before turning to RAG & finetuning