4 ms·

There are methods for getting useful information from LLMs about subjects outside your own area of expertise, but it takes a lot of work. For the example, caus

There are methods for getting useful information from LLMs about subjects outside your own area of expertise, but it takes a lot of work. For the example, causes of the Great War, I'd first ask it to provide a list of the top ten reasons reputable historians cite as primary causes of World War One. Then, for each reason that looked interesting, I'd ask the LLM to provide arguments for and against, including one primary historical document in support of each side. Then, go to Google Scholar or similar and look up that document and see what different reputable historians have said about it. A few iterations of this process should get you to some reliable information even in a field you know nothing about. This is a time-consuming process, and requires active engagement rather than what many people have been trained to do by our atrocious education system and watching television, i.e. passive absorption of content without skepticism.

bsenftner 3y ago
I'm doing something similar with figuring out how to convert my home to solar power. Where I live, the sun is perfect for solar, and as a result our area if flooded with solar startups that are universally using hard sell tactics and undermining one another. After discussions with neighbors, everyone feels the available solar installers are all serving slanted bad deals. So I'm using ChatGPT4-API to create solar installation planning bots, financing evaluation bots, duties and responsibilities checklists for service people, and so on. I think my neighbors and I might just form our own startup or just give it all away because this situation here with the available service companies feels Orwellian; they can like there is no choice, they use fear and hard sell language.
latexr 3y ago
> I'd first ask it to provide a list of the top ten reasons reputable historians cite as primary causes of World War One. And right here you could bump into problems, because how would you know if what you got back is really the opinion of reputable historians? For all you know it could be giving you the opinions of a layman on Reddit. > Then, go to Google Scholar or similar and look up that document and see what different reputable historians have said about it. Looks like that could’ve been the first step. Identify the reputable historians and go from there.
- photochemsyn 3y ago
  That's a good suggestion but I'd do both, e.g. "Provide a list of ten well-known and reputable historians who specialize in the history of the Great War and its origins." Now there might be outliers that are missing from both lists but which might be worth looking at, as it's not that unusual for consensus viewpoints to be overturned by new information and so on, and that's not something LLMs are likely to be very helpful with. Fundamentally, my point is that if skeptical thinking, rational analysis, research strategies etc. aren't taught to people from a young age than it really doesn't matter whether the BS they're subjected to is human-generated or AI-generated.
RandomLensman 3y ago
It might or might not work. How do you know you get to the right set of arguments and documents that way? If you are not an expert in the field, how do you select who is a "reputable historian", for example? How do you know that you correctly understand the language and terms used in a paper not in an area of your expertise?

bsenftner 3y ago

I'm doing something similar with figuring out how to convert my home to solar power. Where I live, the sun is perfect for solar, and as a result our area if flooded with solar startups that are universally using hard sell tactics and undermining one another. After discussions with neighbors, everyone feels the available solar installers are all serving slanted bad deals. So I'm using ChatGPT4-API to create solar installation planning bots, financing evaluation bots, duties and responsibilities checklists for service people, and so on. I think my neighbors and I might just form our own startup or just give it all away because this situation here with the available service companies feels Orwellian; they can like there is no choice, they use fear and hard sell language.

latexr 3y ago

> I'd first ask it to provide a list of the top ten reasons reputable historians cite as primary causes of World War One. And right here you could bump into problems, because how would you know if what you got back is really the opinion of reputable historians? For all you know it could be giving you the opinions of a layman on Reddit. > Then, go to Google Scholar or similar and look up that document and see what different reputable historians have said about it. Looks like that could’ve been the first step. Identify the reputable historians and go from there.

photochemsyn 3y ago
That's a good suggestion but I'd do both, e.g. "Provide a list of ten well-known and reputable historians who specialize in the history of the Great War and its origins." Now there might be outliers that are missing from both lists but which might be worth looking at, as it's not that unusual for consensus viewpoints to be overturned by new information and so on, and that's not something LLMs are likely to be very helpful with. Fundamentally, my point is that if skeptical thinking, rational analysis, research strategies etc. aren't taught to people from a young age than it really doesn't matter whether the BS they're subjected to is human-generated or AI-generated.

RandomLensman 3y ago

It might or might not work. How do you know you get to the right set of arguments and documents that way? If you are not an expert in the field, how do you select who is a "reputable historian", for example? How do you know that you correctly understand the language and terms used in a paper not in an area of your expertise?