Quote:
Originally Posted by Aaron W.
Clearly, an article published by a journal that "did not practice academic peer review" undermines the entirety of academia. Because "not peer reviewed" is totally the right standard to consider. The entirety of academic thought rests on maintaining that approach.
But at least it didn't happen in a more science-y journal...
https://www.vox.com/2014/11/21/72592...fic-paper-scam
I agree that Pulvis' argument is ham-fisted and fairly ignorant.
But I don't think we should go down some line of holding scientific journals as sacrosanct. There are legitimate issues with journals. Obviously as you point out, some of this is down to lack of peer review. But even in peer-reviewed journals there has been shown lack of proper editorial review.
There are also other issues, one major one being that proper peer review is often lacking: There is not much testing of papers and research done and published. The onus is often on attempting to do original research, leading to a lot of rehashing old models with a slight refurbish and sometimes outright design analyses to achieve results. A final peeve of mine is more and more reliance on pay-to-view journals and paywalls, academia doesn't thrive in such environments.
I think it is very important to "fess up" to those issues. I don't really think it is a new problem, more an age-old problem in a new frock. It's more that time tends to bury mediocre research so we don't see it anymore. Émile Durkheim's brilliant "Suicide" is a monumental achievement in sociology (and still a great work), but I bet a lot of more forgettable stuff was around at his time.
But I do not think it is really a problem unique to social science, but something all fields struggle with. STEM subjects might go out on weird tangential models where demonstrating perspective becomes more important than demonstrating relevance, medical science might spew out case studies where empirical research would have been better, social science might develop strange statistical frameworks where the models hold up, but their real world relevance is questionable.
For a professional sifting throw the material isn't that hard,and I am guessing that is true for all fields. You develop a pretty good "BS detector", but for laymen it can be tough. And especially dangerous is when journalists "fall for the bait", amplifying questionable papers with even more questionable headlines. We had an example on this very forum where a study supposedly demonstrated that religious children were less moral, but they had never bothered to ask themselves if religious children simply had
different morals.
But the glaring flaw in Pulvis' argument (barring the weird fixation with "postmodernism" which is a weird objection against qualitative research following generally accepted method... one of the thing postmodernists generally despise the most) is that he doesn't demonstrate problems with the paper we are discussing in this thread. He is speaking from prejudice more than understanding.
Last edited by tame_deuces; 04-02-2019 at 06:32 AM.