If your going to try to construct a meta-analysis of many studies, the first step is going to be reading the materials and methods sections of all the included

If your going to try to construct a meta-analysis of many studies, the first step is going to be reading the materials and methods sections of all the included papers, the goal being to ensure that the data is actually comparable within some (small) margin of error. Disassembling the materials and methods section is the typically the most difficult part of reconstructing a paper's experimental approach as authors often are less than complete, or may refer to other (obscure) papers, or there may be concepts that are so well known in the field that they're taken for granted and not explicitly described. This entails a lot of work and some people are careful and some are sloppy. This is also true for construction of large datasets from individual scientific reports that are then used to produce analytical papers (this is not a 'meta-analysis') based on that dataset, such as in genomic studies, climate studies, etc. The solution here is again to carefully curate the databases and understand their stengths and weaknesses (quality of genomic sequencing matters, for example, and satellite data is not the same as weather ballon data in terms of climate). So if you try to do a meta-analysis of analytical studies based on databases constructed from the results of many individual scientific reports there are all kinds of pitfalls, so it's not surprising that the value of such meta-analysis can be very questionable, unless the authors are extremely careful and clear about their criteria for including or excluding papers from their study. Read enough of them, however, and it becomes easier to identify when the authors are trying to sell their predetermined conclusion via cherry-picking studies that support their position. Unclear methods sections are a good warning sign.