Eric the .5b wrote:1) The single metric is the murder rate.
Mo addressed this. Of all the violent crimes, murder is probably the most easily measured. While coroners can certainly face some tough calls, most murders will produce bodies with obvious stab wounds, bullet holes, etc. There are any number of reasons why a beating might not be reported (or might be falsely reported), and we've had more than enough discussion about rape definitions. But murder is (comparatively) easy to measure.
That said, I agree that it would be important to see follow-up work that looks more broadly than murder, and looks at different crime reporting and classification methodologies. Those problems may be hard to address, but to the extent that they can be addressed it's important that the comparisons be made, to see if this hypothesis makes sense.
2) Actual useful data on the murder rate doesn't start until after the hypothesized time lag since installing lead pipes had passed. Therefore, no growth-in-murder trend can be shown.
3) Nor does the study go to chart drops in crime after those pipes were changed out. So, no time trend whatsoever is shown.
The question tested here is admittedly about comparisons at the same time rather than trends over time, but I actually think that's important. If the claim is that lead exposure affects crime rates, then we shouldn't just see differences over time as lead exposure goes up or down. We should also be able to compare two sets of cities that are otherwise more-or-less-similar but differ in lead exposure (or, more precisely, differ in lead exposure 20 years ago) and see, on average, differences in crime. If we didn't see that effect, it would call into question whether lead exposure actually matters for crime rates.
I admit that the short duration of the study is a problem, which is why I hope that there are follow-ups looking at longer time intervals, more cities (provided that they are roughly apples-to-apples comparisons), and other sets of crime statistics (since, from my admittedly non-expert understanding, different law enforcement agencies and research organizations have often used different methodologies for counting and classifying crimes).
4) The one thing actually compared is Northern vs Southern cities, which mostly didn't use lead pipes. A general difference in the murder rate seems to show up.
A general difference between two culturally and economically different regions needs to be interpreted with caution.
5) However, there's no actual difference found between Southern cities with and without lead pipes, something that would help control for differences of city ages, settlement patterns, early growth of modern ethnic gangsters in Northern cities during the time period, etc. between Northern and Southern cities.
My understanding is that the sample size for Southern cities with lead pipes was small. FWIW, if somebody had found a significant effect in that small set of Southern cities, I would have been dubious about attaching much significance to it.
It seems like a crap study to me. I find myself suspecting that we'll see more crappy studies that won't meaningfully bolster each other's cases, but will be treated as "mounting evidence", so that the first study that isn't eagerly looking to demonstrate this hypothesis will be blown off as an outlier.
I am more optimistic than you are concerning the way that peer-reviewed social science research is conducted (not to be confused with how journalists summarize that research). I predict that if somebody came out with a small-scale study that failed to find the lead-crime effect, the response from most social science researchers would be "Small sample size, so don't dismiss the entire hypothesis based on one study, but definitely needs follow-up." I also predict that if somebody came out with a large, carefully-controlled study that failed to find the effect, the response would be "There are very serious problems with this hypothesis, and we need to carefully compare this with other studies to understand what is going on."
Science doesn't proceed solely through large, comprehensive, unassailably rigorous studies. It proceeds through the accumulation of smaller and often conflicting studies, which collectively point to the need for larger, more comprehensive, and more careful studies to get to the bottom of things.
Also, remember that I've been sporadically commenting on studies I see on this for two years now, and one of my consistent points has been that if this hypothesis is really true then we ought to see more statistical evidence for Kids These Days having better impulse control, at least in the lower tail of the distribution. So I do remain skeptical, but I also think that it is reasonable to view this as a hypothesis with promise.
"They were basically like D&D min maxers, but instead of pissing off their DM, they destroyed the global economy. Also, instead of their DM making a level 7 paladin fight a beholder as punishment, he got a +3 sword of turning."