3 | DISCUSSION
For the first 14 CASP experiments, from 1994 through 2020, the issue of alternative conformations was largely ignored, on the grounds that methods for determining single structures were so imperfect that such nuances were not worthwhile. But with the very high single structure accuracy achieved in CASP14 (1) and the increasing availability of experimental data on alternative conformations, it was obviously time to reconsider. A similar conclusion has been reached across the structural biology community, with greatly increased interest in this area (26). It was also clear that the successful CASP14 deep learning methods might be extendable to this problem. After extensive discussion with experimentalists in the field and help from others, the systems discussed above were identified as potentially suitable targets for CASP experiments. It is a small set, and the CASP community had limited time to prepare modeling pipelines, restricting participation and limiting methodology. Thus, the results should be regarded as a pilot for critical assessment in this area, rather than fully establishing the state of the art. Nevertheless, the results do serve to illustrate three things: some types of methods that are currently available, how these perform, and the rich and varied nature of alternative conformations.
As described above, there are notable successes among the results. Participants were able to reproduce a substantial domain swap caused by a single mutation in a dimeric enzyme, to identify alternative conformations of an ABC transporter caused by the state of ligand binding, and to identify alternative conformations of a small dimer that result from both environment and sequence differences. Possible functional motifs in alternative conformations of kinases were also identified. In all these cases, the AF2-based methods that were broadly successful in calculating the structures of single proteins (12) and multimers (13) were used. The methods vary in detail (see references in (12, 13) for specifics), but all rely on much more extensive sampling of possible conformations and/or alternative MSAs than the default AF2 protocols (for example those in (27)). A variety of methods for conformational sampling with AF2 have now been developed (28), and it is likely that the principles and best procedures will soon become clearer.
For one case, T1109, the mutation-induced domain swap, multiple groups were able to provide accurate models. That is, not only to sample the alternative conformation, but to rank them highly. For the different ligand-induced conformations of the ABC transporter, although all three distinct conformations were identified by multiple groups, different groups were successful with different conformations, and generally the appropriate conformations were not the highest ranked. This is also the case for the crystal dependent alternative conformations of the 48-residue reduced amino acid set peptide. For alternative conformations of the kinases, correct versions of functional motifs were present in the submissions. In these cases, it appears that AF2 could sometimes sample correct minor-state conformations, presumably with the help of structures in the training set and/or possible template use. This is consistent with the concept that these conformations are already populated to some degree even in the absence of the appropriate environment or ligand, with conformational selection depending on conditions (29). In one sense, this is impressive, and promising for the future. In another, it appears that to robustly achieve full sampling more extensive computation than standard would be required, and in the absence of the environmental factors (ligands, crystal environment) these are (appropriately, given missing environmental factors) not the highest scoring.
The RNA target in which multiple conformations were considered (R1156), providing an experimental uncertainty ensemble, is a nice example of the sort of data that will be needed now that calculated structures have become so accurate. Even though RNA computed structures are not yet as accurate as those of proteins, computed ensembles still allowed a model consistent with the experimental information to be identified that could otherwise have been missed. Other targets in CASP15, both RNA and protein, likely have significant flexibility, and the relatively low resolution of many of cryo-EM maps suggests that some inclusion of experimental uncertainty is desirable.
A related conclusion from this CASP is that the long-time principle of comparing computed structure coordinates with experimental ones is sometimes inappropriate. That can be the case for single conformations, but is more likely to be critical for ensembles. In these situations, direct comparison of non-structural data computed from model co-ordinates with experimental data is required, as illustrated by the kinase target example, where comparison of computed models with the experimental NMR data (e.g., nuclear Overhauser effect, NOE, data) might avoid any biases introduced by the experimental modeling process, and be more appropriate for assessment.
It is also possible to compare electron density implied by computed structures directly with maps derived from experiment, and this has already been explored in previous CASPs (30, 31) as well as the current one (23). In this CASP, the highly flexible RNA target R1156 was represented by 10 experimental atomic structures for each of four election density maps, 40 co-ordinate sets in all. A metric of electron density fit (SMOC (32)) shows a similar ranking of overall model quality as the coordinate comparisons, but the map comparison provides useful insight into the local experiment/calculation mismatches. Comparison with electron density also provides a starting point for refining models. For this target, it was possible to further refine some submitted models into specific conformation density maps so that they rival the reference structure (23).
Some types of ensemble were still beyond the state of the computational art this CASP. For the protein/RNA complex, new methods such as RosettaFold2 (24) are able to handle this sort of structure. At the moment, very large, complex molecular machines such as the Holliday junction may be too difficult. But new methods are appearing frequently, and we will see in the next CASP whether this barrier has been breached. A future goal is to include estimates of population level for each member of an ensemble under specific conditions, where those data are available.
All-in-all, this first inclusion of ensemble targets in CASP, although limited in scope, has established that it is possible to apply CASP principles to this type of structure problem and that some available data can provide stringent tests of the methods. And further, that in some cases the methods, especially those based on Alphafold2, can be remarkably effective. We plan to include a ensembles category in CASP16 in 2024. We invite discussion of the most appropriate kinds of data and suggestions on potential targets. Those interested may use the CASP15 Discord or write to casp@predictioncenter.org.