4 ms·
I tried my standard "trick" question I use for LLMs: "Give me five papers with code demonstrating the state of the art of machine learning which uses geospatia
by alex-moon 3y ago
I tried my standard "trick" question I use for LLMs:
"Give me five papers with code demonstrating the state of the art of machine learning which uses geospatial data (e.g. GeoJSON) as both input and output."
There is no such state of the art. My hand-wavey understanding is that GIS data is non-continuous, which makes it useless for transformers, and also contextual, which makes it useless for anything else. Will defer to actual ML people for better explanations.
Point is, LLMs invariably give five papers with code that don't actually exist - it's a guaranteed hallucination.
Phind was able to give me five links that do in fact exist, as well as contextual information as to why these five links were not papers with code doing ML with GIS data. This is by far the best answer to this question from an LLM I've received yet.
- jstummbillig 3y agoChatGPT 4 with web browsing: https://chat.openai.com/share/19a425b5-ed37-469e-860d-65ee706cd603 https://chat.openai.com/share/19a425b5-ed37-469e-860d-65ee70... ChatGPT 4 without web browsing: https://chat.openai.com/share/7e11b4a6-52f2-441a-8614-7266c369865a https://chat.openai.com/share/7e11b4a6-52f2-441a-8614-7266c3...
- alex-moon 3y agoChatGPT 4 seems to be better than it was when I was using it (mere months ago)!
- lucubratory 3y agoYeah, all of OpenAI's stuff gets better much quicker than I'm used to. So does the general performance ceiling of all open source models, even if individual models don't improve as much.
- Smith42 3y agoCheck out EarthPT! https://arxiv.org/abs/2309.07207 https://arxiv.org/abs/2309.07207
- geedzmo 3y agoI don't see how this would be relevant for a code model? The code model isn't trained to retrieve papers/articles, it's meant to complete code. Whether or not you find hallucination in a unrelated task isn't particularly interesting.
- deleted 3y ago[deleted]
- alex-moon 3y agoDamn, this is how I learn that HN doesn't have a block function. What a shame. My friend, can you do me a favour and actually click the link and have a play with the app? If you do, you will discover that what you're dealing with there is an LLM. That's literally why it's being compared to other LLMs. No idea what you were trying to achieve with this comment. "The code model isn't trained to retrieve articles." a) neither is any other LLM, what's your point? and b) the app on the other end of that URL retrieves articles - it's not even tangential to the app, it's key functionality.
- NoNoisle 3y agoLmao
- jamessb 3y ago> the state of the art of machine learning which uses geospatial data (e.g. GeoJSON) as both input and output > There is no such state of the art Some GIS work uses vector data: points/lines/polygons representing features (e.g., the location of roads or the outlines of buildings), which can be stored in formats like GeoJSON or WKT. But other work uses remote sensing data/satellite imagery that can be stored in raster formats like GeoTIFF - essentially TIFF image files with additional information stored to georeference them. You can totally do machine learning on satellite imagery where both the input and output are geospatial data (e.g. to categorise land use - the inputs are multispectral images and the outputs can be images where the value of each pixel represents the identified land use). You can also use machine learning for tasks like building footprint detection/delineation (e.g., [1]) based on satellite imagery. The output from such a pipeline can be a set of polygons, which could be saved as GeoJSON. I'd consider either of theses to be examples of "machine learning which uses geospatial data (e.g. GeoJSON) as both input and output". [1]: https://azure.microsoft.com/en-us/blog/how-to-extract-building-footprints-from-satellite-images-using-deep-learning/ https://azure.microsoft.com/en-us/blog/how-to-extract-buildi...
- deleted 3y ago[deleted]