Worth noting that Bing Copilot had the exact same bugs when it was announced, even though those documents were far simpler than any real corporate use case. Mic

Worth noting that Bing Copilot had the exact same bugs when it was announced, even though those documents were far simpler than any real corporate use case. Microsoft and OpenAI have surely spent the last year trying to improve this problem...and have seemingly made almost no useful progress. The total inability of transformer LLMs to summarize documents is not a surprise. Document summarization is an incredibly difficult task because it's impossible to do it accurately if you don't understand the semantics of the document. Current LLMs are simply too stupid to have anything but shallow, illusory understanding of human language. The illusion works when you are "kicking the tires" with simple documents, because literally thousands of data contractors have diligently trained the LLM on simple documents. But the illusion badly fails when you feed the LLM interesting documents: - novel ideas are smashed against the training set and flattened into a homogenous mush - specific facts and numbers are replaced with """statistically likely""" facts and numbers, which actually works okay for giant tables of statistics (except it destroys meaningful outliers). But this badly fails for, say, engineering specifications around aircraft engines. Or quarterly profit/loss figures, which Bing's demo screwed up. - Even code generation shits the bed if you're working in an uncommon language. Last I checked, GPT-3.5 had a terrible understanding of F# and would copy-paste dozens of lines verbatim from specific Github projects, even if that code was inappropriate for the task at hand. This illusion of competence is not meaningless if it works: GPT's Python code generation mostly works by translating English sentences into Python sentences, supplemented by plagiarism, but it's a useful tool regardless. The problem is that Copilot's illusion of competence does not work for aircraft piloting. There are not millions of lines of written aircraft logic in Github like there is with Python. The only way I can see this tool working is if Microsoft goes back and pretrains Copilot specifically on aircraft specifications, then hires data contractors for RHLF specific to Robichaux's use case. Otherwise it's simply too unreliable.