2 ms·
Earlier in the press: Geoscience AI in crisis? (17 June 2024) https://geoscientist.online/sections/viewpoint/geoscience-ai-in-crisis/ https://geoscientist.onli
by defrost 2y ago
Earlier in the press:
Geoscience AI in crisis? (17 June 2024) https://geoscientist.online/sections/viewpoint/geoscience-ai-in-crisis/ https://geoscientist.online/sections/viewpoint/geoscience-ai...
Paul Cleverley raises concerns about big-data artificial intelligence projects in the geosciences
Geologists raise concerns over possible censorship and bias in Chinese chatbot (24 June 2024) https://www.theguardian.com/technology/article/2024/jun/24/geologists-censorship-bias-chinese-chatbot-geogpt https://www.theguardian.com/technology/article/2024/jun/24/g...
Tests on Qwen, part of GeoGPT’s underlying AI, reveal geoscience-related questions can produce answers that appear to be influenced by narratives set by the Chinese Communist party.
For example, when asked how many people have died in a mining operation in Ghana run by the Shaanxi Mining Company, Qwen says: “I’m unable to provide current or specific information about events, including mining accidents, as my knowledge is based on data up until 2021 and I don’t have real-time access to news updates.”
The same question posed to ChatGPT, the chatbot developed by the US company OpenAI, produces the answer: “The Shaanxi Mining Company in Ghana has experienced multiple fatal incidents, resulting in a total of 61 deaths since 2013. This includes a significant explosion in January 2019 that alone claimed 16 lives. ”
Eight days ago on HN: https://news.ycombinator.com/item?id=40773876 https://news.ycombinator.com/item?id=40773876 (1 comment | 2 points)
- droopyEyelids 2y agoAre miner deaths relevant to a geology-focused tool? The example of its use was to explain the fossil record.
- defrost 2y agoNot relevant to surface geochemistry results, no. Not relevant to airbone geophysics results, no. Not relevant to to drill exploration sampling results, no. Not relevant to Technical | Economic Feasibility studies, no. Not relevant to mining environmental impact studies, no; although deaths, injury and illness in surrounding populations is. Relevant to ongoing minesite safety records and operations, yes. It's a question of scope; a geoscience aware AI that can answer questions about resource availibity, resource extraction and associated risks, costs and impacts that includes the kind of information found in, say, https://www.spglobal.com/marketintelligence/en/campaigns/metals-mining https://www.spglobal.com/marketintelligence/en/campaigns/met... would be expected to aware of mining deaths. Just as people want to be aware of nuclear risks for power and deaths in lithium processing from accidents and associated radioactive waste. It's being touted as a Geoscience tool, not just a chatty geology database.
- smopiy 2y agoSocial geoscience (impact of geology on society) is very much core to geoscience and geoethics. Ironically, UNESCO and IUGS authored the paper below calling for a change in how we perceive geoscience..that was cited by the DDE project. https://www.escubed.org/journals/earth-science-systems-and-society/articles/10.3389/esss.2023.10079/full https://www.escubed.org/journals/earth-science-systems-and-s...
- decremental 2y ago[dead]
- csomar 2y agoIt’s a bit unfair. I think a proper comparison would be whether the geoGPT gives answers for other companies/countries screw up. This makes it sound like a conspiracy when it could be just poor bot.
- defrost 2y agoIt's an odd example, to be sure, part of the reason I pulled that from the Guardian article. It's really pitting two general knowledge AI DB's against each other on mining knowledge, it's unclear whether this is a smoking gun on censorship or incomplete data entry. It does highlight a knowledge void though.
- smopiy 2y agoThe Deep-time Digital Earth (DDE) GeoGPT uses Alibaba’s Qwen LLM. It is no conspiracy, the fact is that Chinese AI has to abide by Chinese Censorship Law’s. It does give answers on other companies and countries. Check out Leonard Lin’s excellent independent technical evaluation of Qwen’s Chinese government censorship. https://huggingface.co/blog/leonardlin/chinese-llm-censorship-analysis https://huggingface.co/blog/leonardlin/chinese-llm-censorshi... As Lin points out, Qwen is Chinese Government censored so why bother using it in a global context when there are equally as good models (if not better) that are not aligned - so won’t issue Chinese government propaganda or refuse to answer. It seems odd that the International Union of Geological Sciences (IUGS) which is non-political and non-governmental, has allowed DDE to build such a system for the world’s community of geologists with such Chinese government bias and censorship. Maybe they did not realise what was being developed using their name.
- beambot 2y agoIs it any surprise that models reflect their RLHF training, and thus cultural and governmental underpinnings...? Just look at the old Gemini cultural flubs for a western-centric example. It will be very difficult (impossible?) to remove underlying bias from an AI model, since you'd be hard pressed to get any two people "aligned", let alone entire communities, states, countries & the world.
- smopiy 2y agoIt is not a surprise models reflect their source training data and RLHF. The surprise is the DDE project choosing to use a Chinese government censored model (based on Qwen) for a world-wide audience.