What I have done for a similar use case is to have ChatGPT via the OpenAI API generate SQL (or KQL) based on the user request and then run that query and displa

What I have done for a similar use case is to have ChatGPT via the OpenAI API generate SQL (or KQL) based on the user request and then run that query and display the results (with some prose if appropriate). Works fairly well even with GPT-3.5-turbo. With GPT-4 can handle more complex requests (slower). It could even create a custom Chart.js chart on the fly if requested. To me this demonstrates that there is a specific job here, even if you don't want to call it "engineering". Which I would argue is the correct category of job at least. The above project was presented as "let's put a table of data in a vector database and then search it using the embedding of the user query". Here you were suggesting fine-tuning an LLM with the structured data. Again, it makes more sense to just generate the SQL and leave it in the relational database. So there are a few basic things about how this stuff works that are not obvious and require some specialization. Even for programmers. Right now I think it's fair enough to put it in its own job category since there are plenty of software engineers that just don't have any experience with generative AI. But within a few years, I think knowing how to integrate generative AI into a product will be considered core knowledge for a software engineer. So using LLMs or Stable Diffusion will become bullet points on a job requirements list.