Getting the Numbers Right in Medicinal Chemistry
Graphical Abstract
It is argued in this Perspective that not enough attention is paid to the correct representation of quantitative data in Medicinal Chemistry. In particular, results from biological assays are too often reported with unreasonable numbers of significant figures. This unfortunate practice should change, which will probably require more rigorous policies on this matter for Medicinal Chemistry publications.
Abstract
Medicinal Chemistry is a quantitative scientific discipline. Hence, it is of utmost importance that scientists working in this field pay rigorous attention to the correct representation of their quantitative data. However, browsing the Medicinal Chemistry literature, it becomes obvious that this is very often not the case. In particular, results from biological assays are too frequently reported with unreasonable numbers of significant figures. In this Perspective, it is argued that this is poor practice that sheds an unfortunate light on the discipline of Medicinal Chemistry and that therefore, more rigorous policies regarding the presentation of quantitative data in Medicinal Chemistry publications should be considered.
Introduction
In many sub-disciplines of Chemistry, the properties of compounds are studied. These properties are often investigated in experiments that furnish quantitative data. Medicinal Chemistry is one of these sub-disciplines, as Medicinal Chemists often make (or sometimes purchase) compounds and then investigate them for their biological properties. In this regard, Medicinal Chemistry is not completely different from, for instance, Physical Chemistry, with the notable exception that Medicinal Chemists are usually more interested in biological rather than physical properties of chemical matter.
In the field of Medicinal Chemistry, quantitative data obtained from biological assays are often represented with parameters such as IC50 values or percentage remaining activities for target inhibition, or half-lives for stability in relevant biological media. The way these numbers are reported reflects on the precision of the respective experiment. Medicinal Chemists usually learn at rather early stages in their careers that any work with biological material, even in the controlled setting of an in vitro assay, involves significant experimental uncertainties, leading to rather high standard deviations (SD) of the obtained mean values. Hence, SD values of, for example, 10 %, 20 % or even more of the mean value are not uncommon at all in this discipline, which is perfectly acceptable though. Medicinal Chemists usually aim for improvements of biological properties that are way beyond these percentages when they optimise a hit or lead structure, and we all can therefore be completely at ease with these experimental uncertainties.
However, problems arise when such data from biological experiments are reported in a way that is in obvious contradiction to the acceptance of a significant experimental error, that is, when data are provided with unreasonable numbers of significant figures. In this Persepctive, I would like to argue that this is a rather widespread phenomenon in the Medicinal Chemistry literature and that we all should aim to do better. It should be noted that I do not intend to call out specific colleagues or editors, but I would rather like to alert us all (including myself) that we should pay more attention to the way we report quantitiative data in the field of Medicinal Chemistry.
Some General Considerations
In order to make the message of this Perspective as clear as possible, I hereby would like to provide a short reminder of the way quantitative scientific data should ideally be reported.
Significant figures are essential in this context, that is, the digits of a number starting with the digit furthest to the left that is not zero, and ending with the digit furthest to the right. For example, the number 1.24 has three significant figures as has the number 0.124 or the number 0.120. The significant figures of an experimentally obtained number should correspond to the precision of the respective experiment: the last digit to the right usually is the one with the experimental uncertainty. Hence, numbers from very precise experiments should be reported with more significant figures than results from less precise experiments with larger errors.
In synthetic chemistry, it can be a useful rule to report amounts of employed reagents with three significant figures, if this is justified by the experimental precision. The latter is sometimes not the case when volumes of liquids are measured as this is often done with less precision in synthetic laboratories. Thus, an amount of e. g. 5.24 g for a starting material would probably be universally accepted in the synthetic literature, while a number such as 5.2404 g would be universally criticised and should be rounded.
The question therefore arises if a similarly useful rule of thumb can be identified for reporting numbers from biological assays in the field of Medicinal Chemistry. In my estimation, this is perfectly possible when one reflects the aforementioned statements regarding experimental uncertainties in such assays. If SD values of more than 10 % of the mean value are acceptable in most biological assays, then the number of significant figures for the mean value should be limited to two. I would like to further clarify this statement: With such significant experimental errors (>10 %), the experimental uncertainty is usually already reflected in the second significant figure. Hence, any further significant figure would be superfluous and without informational value. The SD then has to be adjusted accordingly so that it has the same number of decimal places as the mean value. However, in my estimation, one should always round up the SD in order to avoid reporting values that appear to be more precise than the experiment has actually been. The latter recommendation might be controversial and not universally accepted though. To illustrate these statements, Table 1 provides some made-up examples of incorrect and correct versions of experimental data.
# |
Incorrect |
Correct |
Mistake[a] |
|---|---|---|---|
1 |
IC50=2.345±0.789 μM |
IC50=2.3±0.8 μM |
Mean has too many significant figures (4 instead of 2) |
2 |
Residual activity=98.12±4.24 % |
Residual activity=98 %±5 % |
Mean has too many significant figures (4 instead of 2) |
3 |
Plasma stability t1/2=5.22±0.12 h |
Plasma stability t1/2=5.2±0.2 h |
Mean has too many significant figures (3 instead of 2) |
4 |
IC50=2.3±0.6789 μM |
IC50=2.3±0.7 μM |
Number of decimal places of mean value and SD do not match |
- [a] Explanation why the ‘incorrect’ version of the number is flawed.
As with any rule of thumb, the proposed guideline to report mean values of data from biological assays with just two significant figures should be applied with some caveats. For instance, some assays might have an intrinsically higher experimental precision than described above, that is, they might consistently furnish SD values significantly below 10 % of the respective mean values. Naturally, such data should be reported with significant figures that reflect the higher experimental precision. It should be noted though that such precise biological assays are the fairly rare exception in most Medicinal Chemistry publications.
Scope of the Problem
I became alert to the problem discussed in this Perspective in a slightly unusual way. We teach a research seminar for our undergraduate students in their last semester prior to their final exam. For this research seminar, groups of two students each present a rather recent paper from the Medicinal Chemistry literature that is then further discussed by the whole class. In this scenario, I have had the opportunity to look at papers in great detail that I probably would just quickly go through when browsing the latest literature in our field. After a while, I have noticed that nearly every presented paper, even those from esteemed journals, had some issues with the way biological data are presented: there simply were too many significant figures provided in most of the tables.
However, statements on such a delicate topic should not be misled by subjective impressions, but should rather be based on objective observations. In the preparation of this contribution, I have therefore gone through the most recent issues of five selected and esteemed journals in the field of Medicinal Chemistry: the Journal of Medicinal Chemistry,1 the European Journal of Medicinal Chemistry,2 ChemMedChem,3 ACS Medicinal Chemistry Letters,4 and RSC Medicinal Chemistry.5 For each of these issues, I have only taken original research papers into account, no reviews or other articles. Papers without numerical data from biological assays were neglected, and no Supporting Informations were taken into account. My original idea had been to generate some sort of statistics by grouping the relevant papers into different categories with respect to the representation of numbers from biological assays. However, I have soon realised that such a meticulous exercise would not be necessary to get the main point across. I have simply looked for mistakes in the style of those depicted in Table 1, and I have found them in the vast majority of all studied research papers. An arbitrary selection of such mistakes is provided in Table 2, with two examples from each of the investigated journal issues.
# |
Source |
Number |
Context[a] |
|---|---|---|---|
1 |
ref. [1] |
IC50=28.69±3.63 μM |
Activity against A549R tumor cells |
2 |
ref. [1] |
F(%)=77.61 |
Oral bioavailability in an in vivo rodent model |
3 |
ref. [2] |
Cell death=96.40±11.4 % |
Protective effect in an induced cell death model |
4 |
ref. [2] |
IC50=59.99±13.63 μM |
Activity against PC-3 tumor cells |
5 |
ref. [3] |
Latency=435.8±93.4 s |
Effect on induced seizures in an in vivo rodent model |
6 |
ref. [3] |
IC50=44±5.81 μM |
Activity against PC-3 tumor cells |
7 |
ref. [4] |
Inhibition=65.02 % |
Antiplasmodial activity against ookinetes |
8 |
ref. [4] |
IC50=2163.50±221.32 nM |
In vitro target inhibition |
9 |
ref. [5] |
IC50=86.37±10.12 μM |
Antiplasmodial activity (P. falciparum 3D7) |
10 |
ref. [5] |
Consumption=121.3±15.8 % |
Stimulated glucose consumption in myotube cells |
- [a] Type of assay or measurement from which the cited number has been obtained.
It is astonishing how similar the examples 1–10 (Table 2) – that can actually be found in recent Medicinal Chemistry literature – are to the made-up examples of ′incorrect’ numbers in Table 1. It should be noted that the imaginary examples listed in Table 1 had been compiled before the described literature search, so they have not been adjusted by any means to what was found in the cited publications. This observed phenomenon is certainly not limited to a specific type of assay or to a certain journal, but appears to be of almost ubiquitous nature. Having in mind that such examples could be found in many papers in all of the five studied journals, one could almost identify an epidemic of superfluous digits in Medicinal Chemistry data.
In this context, it is important to make several clarifications: (i) No concerns with the quality of the actual data and their integrity are meant to be implied. The discussed issue is only with the formal representation of experimental data, not with the experiments themselves. (ii) No specific papers are explicitly cited that could serve as particularly bad examples. This article is intended to be a constructive contribution to our scientific community, not some sort of medieval pillory. In particular because the problem appears to be of nearly ubiquitous nature, it would be pointless to call out some selected authors on it. All of the examples listed in Table 2 can easily be found in the respective journal issues if desired. (iii) Sometimes, overly precise numbers are provided with respect to readability. For instance, if a table has nM activity data for most of its entries, it can make sense to list all entries with nM numbers, even if that means that some of the entries will have three (or even more) significant figures. In my estimation, such practice should not be of concern as readability is a noble cause. (iv) In some of the papers I have studied, experimental numbers were treated with great care and are flawlessly presented. However, if negative examples are not explicitly called out here, it appears to be consistent to not explicitly cite best-practice examples either.
Overall, I would simply encourage readers of this Perspective to repeat my exercise and browse recent issues of some Medicinal Chemistry journals. It is very likely that the outcome will be similar, with some examples of good practice and a worrying number of flaws similar to those listed in Table 2.
Conclusions
In summary, this Perspective is intended to discuss a problem with the way quantitative data are often reported in the Medicinal Chemistry literature. The identification of this problem is based on two hypotheses: (i) Medicinal Chemistry is a quantitative science, and therefore, quantitative data should be treated with great care and rigor. (ii) Biological assays conducted in the field of Medicinal Chemistry often come with significant experimental uncertainties (i. e. SD values of more than 10 % of the mean value), which is perfectly acceptable in this discipline.
From these hypotheses, a general guideline is derived: For most datasets from biological assays in Medicinal Chemistry, the number of significant figures for the mean value should be limited to two. Browsing the recent Medicinal Chemistry literature, it becomes obvious that it is the exception rather than the rule that biological data are reported in such a consistent way. Far too often, data are provided with significant flaws (see selected examples in Table 2). As this is not limited to specific types of assays or to certain journals, an epidemic of superfluous digits in Medicinal Chemistry data can be identified.
The obvious argument against these statements might be that they concern a mere technicality rather than the substance of Medicinal Chemistry research. I would like to argue against this. Firstly, a quantitative science should always be rigorous with the way quantitative data are reported. Secondly, one has to wonder if authors who report data in such an unfortunate way are really aware of the inherently limited precision of their experiments. In any case, it would be no problem to do better and there is no obvious reason why we should not aim to do so.
It is a bit mysterious to me why the described phenomenon exists in the first place. One explanation might be that some authors believe that reporting numbers with many digits is an indication of precision. However, this is certainly not the case as experimental precision is part of the experiments themselves. An experiment with significant error does not become more precise by throwing many digits at the reader to whom its results are communicated. Another potential explanation would be even simpler: the scientific community of Medicinal Chemists just has not paid sufficient attention to this issue yet.
This brings me to another aspect: What can we do about this problem? I would like to propose a significant change in editorial policies in Medicinal Chemistry journals. There should be explicit guidelines for authors on the way quantitative data from biological assays are reported, and editorial offices should check if submissions follow these guidelines. In case of significant violations, submissions should be sent back to their authors before they enter the peer review process. Referees should pay attention to the issue as well and should address any remaining inconsistencies in their reports. In my estimation, all of these measures could be implemented in a relatively straightforward manner.
It is important to note that this Perspective only addresses the widespread habit to report data with superfluous digits, but that there are other discussible issues with how data are presented in the Medicinal Chemistry literature. Most notably, there are several scholars who advocate for the use of pIC50 instead of IC50 values when activity data are reported (with pIC50 being the negative decadic logarithm of IC50). This mainly results from the multiplicative nature of experimental errors in biological systems that therefore lead to log-normal distributions.6 However, this topic (despite such reasonable arguments) is not within the scope of this contribution.
Finally, I would like to point out again that this Perspective is not intended to call out anyone or to cause controversy, but to stimulate fruitful discussions on the presentation of quantitative data within our scientific community. Most of us have probably been guilty of not rounding our numbers properly here and there (including myself), but we all can do our best to improve the way we report data from biological assays. It would be very much appreciated if this contribution might help to reach this goal.
Acknowledgments
I would like to thank all students who have been involved in the aforementioned seminar on research literature and its main organiser, Dr. Matthias Engel (Saarland University). Without the thorough presentations of these students, I probably would not have been alerted to the problem discussed in this Perspective. Open Access funding enabled and organized by Projekt DEAL.
Conflict of Interests
The authors declare no conflict of interest.
Open Research
Data Availability Statement
Data sharing is not applicable to this article as no new data were created or analyzed in this study.
