3 ms·

I'm very curious about this feature: > analyze a complex graph for work-related data Does this mean that I can take a screenshot of e.g. Apple stock chart and

I'm very curious about this feature: > analyze a complex graph for work-related data Does this mean that I can take a screenshot of e.g. Apple stock chart and it will be able to reason about it and provide insights and analysis? GPT-4 currently can display images but cannot reason or understand them at all. I think it's one thing to have some image recognition and be able to detect that the picture "contains a time-series chart that appears to be displaying apple stock" vs "apple stock appears to be 40% up YTD but 10% down from it's all time high from earlier in July. closing at $176 as of the last recorded date". I'm very curious how capable ChatGPT will be at actually reasoning about complex graphical data.

gdubs 3y ago
Check out their linked paper that goes into details around its current limitations and capabilities. In theory, it will be able to look at a financial chart and perform fairly sophisticated analysis on it. But they're careful to highlight that there are hallucinations still, and also cases where it misreads things like labels on medical images, or diagrams of chemical compounds, etc.
famouswaffles 3y ago
Look at this link of GPT-4 Vision analyzing charts(last image). https://imgur.com/a/iOYTmt0 https://imgur.com/a/iOYTmt0
- laurels-marts 3y ago
  This is brilliant. Thank you very much for this link. The analysis on the last image was impressive and quite thorough (given the simple prompt). Every chart has an equivalent tabular representation. One way to get "charts" analysed like this before GPT Vision was to just pass tabular representations of charts to GPT-4. This makes implementing chart analysis a lot simpler. I do wonder though if for absolute best result it still wouldn't be better to pass both - image of the chart and the tabular representation of the chart. Imagine having a dashboard with 5 different visualisations. You could capture the state of the entire dashboard in one screenshot and then pass tabular representations of the each individual chart all in one prompt to GPT-4 for a very comprehensive analysis and summary.

gdubs 3y ago

Check out their linked paper that goes into details around its current limitations and capabilities. In theory, it will be able to look at a financial chart and perform fairly sophisticated analysis on it. But they're careful to highlight that there are hallucinations still, and also cases where it misreads things like labels on medical images, or diagrams of chemical compounds, etc.

famouswaffles 3y ago

Look at this link of GPT-4 Vision analyzing charts(last image). https://imgur.com/a/iOYTmt0 https://imgur.com/a/iOYTmt0

laurels-marts 3y ago
This is brilliant. Thank you very much for this link. The analysis on the last image was impressive and quite thorough (given the simple prompt). Every chart has an equivalent tabular representation. One way to get "charts" analysed like this before GPT Vision was to just pass tabular representations of charts to GPT-4. This makes implementing chart analysis a lot simpler. I do wonder though if for absolute best result it still wouldn't be better to pass both - image of the chart and the tabular representation of the chart. Imagine having a dashboard with 5 different visualisations. You could capture the state of the entire dashboard in one screenshot and then pass tabular representations of the each individual chart all in one prompt to GPT-4 for a very comprehensive analysis and summary.