As you might know, I’ve also given the topic of the role of journals some thought:
I am fully supportive of open publications and a renewal of the publication process.
(And, actually, in addition to SciPost, someone who is very active in this is actually the Nature journals themselves, in particular the recent initiatives they have been pushing to be more open about reviews, publishing review reports, and encouraging reviewers to volunteer to identify themselves, even during the review process.)
But although I like the initiative, SciPost for me missed an important factor: what is, what I like to call, the “perceived impact level” they are targetting?
In shorter words: is SciPost targetting a “quality level” of PRB? Or PRL? Or Nature? Or Nature Physics? Or Solid State Communications? Or European Journal of Physics E?
(A SciPost editor once told me they perceive themselves to be something like PRB “level” papers.)
I like to call in “perceived impact”, or maybe “estimated impact”, since, of course, not even the smartest editors and smartest reviewers in the universe can assess the importance that a paper will turn out to have in 5 or 10 years.
However, whatever you might think of it, there is a process by which your work is reviewed by an editor and by multiple experts in the field who read your work and try to make an estimate of the “level” of your work compared to other works they have read in the journal you have submitted to. This is, at least, something.
And if I am in a committee and I have say 2 hours (in the evening after the kids go to bed since I don’t actually have time to do it during the day) to read 30 CVs and make a snap judgement of who I would invite and who I would not invite for an interview, “something” is highly valuable for me. It is just not possible for me to personally read every one of your papers in depth to determine if your work is of high quality or not: I just don’t have time. And there is no magic fairy that is going to somehow “create” more time for me to do this.
Upshot: Just everybody publishing all their work in SciPost is not a solution, it does not address one of the key things we rely on journals for: an assessment by experts in the field of the perceived impact / importance of a work.
Of course, the review process itself is not perfect, and in particular, not nearly transparent enough. There are attempts to make it more open, such as those by SciPost and Nature. But there is not yet a viable solution: for example, for Nature papers, the review discussions of papers that are rejected are NOT published.
A few years ago, myself and some of my colleagues were brainstorming about a solution to this. The idea is as follows:
- All papers in physics are submitted just to one place, and arxiv-like central server
- These are then sent out to review by certified experts in the field (based on a central database, also to distribute load)
- The authors indicate what they think the “perceived impact level” is by clicking on a button. I would even suggest that we re-use the names of existing journals for these buttons. A “Nature-level” button, a “Science-level” button, a “Nature-Physics-level” button, a “PRX/PRL-level” button, a “Nature-Comms-level” button, etc etc.
- A small set of experts perform a full technical assessment of the work, based on an publicly available dialogue
- There is an editor who then makes a decision that the manuscript is technically correct or not (ie. claims supported by evidence) after fixed number of rounds of review
- After the technical review, the manuscript is sent to a larger number of reviewers for an “impact assessment” review (ie. quick read). They click on one of the “perceived impact level” buttons to indicate what they thing the equivalent journal would correspond to, and provide at least a few sentences justifying their choice. These are also public (though likely anonymous)
- Finally, the editor makes a decision of assigning the “impact level” based on the “impact assessment” reviews, and the manuscript becomes recognised by the community as a “Nature-like” paper, or a “PRL-like” paper, or a “PRB-like” paper, etc.
A huge benefit of this is that there is only ONE review. You do not waste your time and the editors time resubmitting to a different journal. You do not waste reviewer time by needing new reviewers each time you submit. The process is fully open and transparent, this would also help in the objectivity I think a lot (it can still be anonymous). The database would also be able to cross check for correlations and identify reviewer bias.
Crucially, it would also keep the certification of recognition from your peers. You would be able to point out on your CV that your peers felt that your work was relevant and important at the level of so-and-so journal. And this you can put on your CV, and then people like me in committees can use this information in a valuable way (along with the other many pieces of information we use).
I think it would be a great idea, but it would need a lot of things before it could start, like money (EU?) to set up infrastructure, hire editors, etc. I am not going to do this in my free -2 hours per day.
More importantly it would need FULL support of the community. People would have to be on board. People who are publishing Nature papers now would have to accept that “Nature-like” papers are just as good. Of course, why wouldn’t they be? The only diffference is that there is no nature editor. But people would have to really accept this. And ideally the people who submit now to Nature (particularly the senior established one) would have to put their money where their mouth was and really submit their new papers to “Nature-like” instead of Nature!
If we don’t have a critical mass of people who do that right from the start, then we are dead in the water, hopeless. We want to be mainstream, not a bunch of hippies on the sidelines complaining about how unfair the world is. We would need to coordinate large scale community support before the launch and make sure we make a running start.
You asked for my thoughts, and there they are I think it would be a cool idea, but for now it mostly lives in my head (and a google doc I share with some collaborators). I’m not sure how to take it to the next step, but if anyone has some ideas, time, and energy, I am happy to help.