Give the supplementary materials the importance they deserve!

Since I started sending papers to peer-reviewed journals, over 13 years ago (time flies!), many changes to the way we disseminate and publish the results of our research have taken place. Together with the rise of Open Access journals, an important change in scientific publishing is the incorporation of supplementary materials that only appear in the webpage of the journal (SOM hereafter). Such materials include, but are not limited to, additional data, references, software code, videos, maps, pictures, statistical results and details on the methods employed in the article. The use of SOM is nowadays very common in virtually all Ecology/Biology/Multidisciplinary journals, and have becoming increasingly important through time, up the degree that some journals (e.g. Science) directly place the methods of the article as part of the SOM, and in the case of articles published in many other journals, such as Nature, PNAS or Ecology Letters, a careful reading to the SOM is needed in most cases to fully understand what the authors did, and to properly interpret the statistical analyses employed and the results presented in the main text. The SOM are also of paramount importance in the case of quantitative reviews and meta-analyses, as it is typically there where the authors put the raw data and/or the original references where these data come from can be found.

The possibility of using SOM is certainly a welcome innovation in the publishing world, as SOM can greatly enhance the content of published papers while alleviating the pressure on the limited number of pages that most journals can publish every year (online-only journals such as PLoS ONE can circumvent page limitations, but even articles published in these journals commonly use SOM, see a couple of examples from our own research here and here). Journals are actively encourage the (judicious) use of the SOM as a way to reduce the number of pages per published article (Ecology is a clear example of this, see its instructions for the authors) or to enhance the content of the article itself (Methods in Ecology and Evolution is doing this very well), and hence it seems that the importance of SOM will continue growing in the future. Despite its clear advantages, the generalization of SOM has also some downsides that have not been fully acknowledged from my point of view, and that could be easily solved if we give the materials included in the SOM the importance they deserve.

The first issue with the SOM I would like to discuss concerns its use in quantitative reviews and meta-analyses, a topic that has received lots of attention recently in the social media (check here for a summary of a recent interesting discussion in twitter about authorship in meta-analyses that is relevant to the topic of this post). I just would like to highlight a problem already raised by different ecologists so far: the lack of recognitions (in terms of citations) of the articles included in the SOM, which are also the “core” of a quantitative review/meta-analysis. Our work is being evaluated by multiple ways, being the number of citations an important metric for measuring its impact among the scientific community. If our work is used in subsequent reviews, something that I particularly welcome, but the citations appear in the SOM rather than in the main text, then is not taken in consideration by citation aggregators such as Google Scholar or the ISI Web of Science. This is certainly unfair to the authors that collected the original data, and can at the mid to long-term make less attractive to gather primary data than synthesizing them. This problem, however, has an easy solution, which is to count all references of published articles, both in the main text and in the SOM. Jarret Byrnes has put forward an open letter to ask for this, if you think it is a good idea to do so you can sign the letter here (I would advice to do so, I have already signed it!).

The second problem I would like to highlight is related to my own publishing experience. Over the last years I had different papers rejected because the referees asked for particular analyses (or criticized the lack of these analyses), methodological details or data that were included in the SOM and were properly cited throughout the main text as appropriate. Many journals no longer give the opportunity to revising a manuscript unless all the reviewers (or most of them) really support it, and it is certainly discouraging when this apparent lack of information/data/analyses are used to justify a rejection. This gives me the impression that editors and reviewers do not read the SOM with the same care they read the main text. This view is reinforced by some recent changes in the policy of journals such as Nature, which now allows to include all the methodological details that used to be part of the SOM in a full Methods section that appear at the end of the pdf version of the paper, while a summary of this section appear in the printed version of the journal. I am afraid that this may relate to the fact that the methodological details present in the SOM of many papers are not carefully revised during the review process, and this has lead to a lack of reproducibility of some published work. Fortunately this problem has an easy solution:  if SOM are an integral part of most papers published nowadays, and we must use them because the space limitations imposed by many journals, then they should be carefully assessed during the review process! This is what I do when I am reviewing/editing a manuscript, and I would certainly like to see my peers doing the same. Of course, we can always publish in journals that do not put limits in the number of pages per article, and include then all the (detailed) information as part of the main text, but given that these journals are still (and will be for some years I guess) a (growing) minority, we must give the SOM the importance they deserve! 

Please comment if you have similar experiences or identify any additional problems/solutions to the issues discussed above.