Within-study replication samples

Recall that a total of 17 studies are included in this review, but only 14 of them reported any associations whatsoever. Of these 17 studies, 4 of them (24%) followed up their initial set of associated SNPs in a subsequent replication sample. Of these 4, 3 (75%) did not replicate any of the SNPs they had initially found to be associated with intelligence, meaning they reported no associations at all. The single exception was Benyamin et al. (2014), who followed up their initial GWAS hits in their discovery sample (n = 12,441) with another GWAS in a replication sample (n = 5,548). According to the catalog, these authors reported six SNPs as associated with intelligence,\cite{catalog} but acknowledged that "no individual single-nucleotide polymorphisms (SNPs) were detected with genome-wide significance".\cite{Benyamin2014} Further examination of the catalog entry for this study shows that the p-values of the six SNPs listed as associations are all below 5*10-8 (specifically, all of them are between 5 and 9*10-6). In summary, it appears that every study analyzed here which included  a replication sample was unable to find  any genome-wide significant associations in it.

Is there a "winner's curse"?

As noted in the "Methods" section, if a "winner's curse" is present in these data, the SNPs reported as hits multiple times should, on average, display smaller associated p-values in follow-up studies than in the initial study, corresponding to a stronger association reported in the initial study than in subsequent studies. In fact, this pattern appears to be present in the studies reviewed here:  of the 288 SNPs reported more than once, the associated p-value was, on average, 1.64*108 greater in the second study reporting this SNP to be associated with intelligence than in the first. This phenomenon was evident among the 69 SNPs reported three times or more as well, albeit to a greatly reduced extent: the average p-value for such SNPs was 981 times greater in the third study than in the initial study. Similarly, for the 21 SNPs each reported  four times, the average p-value was 1.19*106 greater in the fourth study than in the initial study. Finally, the average p-value for the 6 SNPs each reported five times was 835 times greater in the fifth study than in the initial study.

Diversity

An abject lack of diversity was also evident in regard to the populations on which the studies were conducted: all 13 (100%) of studies that reported ancestry (or ancestries) stated that all of their participants were of European ancestry (this includes "British ancestry"). The only exceptions were one study which only reported ancestry information for one of the two samples they analyzed,\cite{Hill2019} and another study which did not report any ancestry information at all.\cite{Gialluisi2014} Because of these exceptions, one study (Gialluisi et al. 2014) is excluded entirely from ancestry calculations, and the other (Hill et al. 2019) is counted only with regard to the sample in which the participants' ancestry was described ("120,934 British ancestry individuals").

Discussion

This review aimed to assess the state of the genome-wide association study literature in regards to intelligence. The results indicate that most SNPs reported in such studies as being associated with intelligence have not been successfully replicated (at least not yet). Further, it appears that more than one-third of reported associations are not statistically significant by the most commonly used standard (p < 5*10-8). The literature is also deficient with regard to including replication samples within each study, which was done by only 4 of the 17 studies included here, despite the well-documented and considerable importance of replication in establishing that genotype-phenotype associations are not spurious (e.g. \cite{Kraft2009}). Consistent with past research on GWASs (e.g. \cite{Mills2019}\cite{Popejoy_2016}), the present study also revealed that the vast majority -- in this case 100% -- of studies reporting the ancestry of participants were conducted on individuals of European descent, with the only exceptions being the samples for which ancestry information was not reported. Another limitation identified in this literature was that there was consistent evidence of a "winner's curse" among SNPs reported more than once, with p-values associated with a given SNP being, on average, greater when it was reported in subsequent studies than when it was reported for the first time. On the other hand, it also appears as though the strength of reported associations is increasing over time, providing evidence that more recent SNP-intelligence associations are less likely to be false positives than those reported in earlier studies.
Some meta-analyses of intelligence GWASs have been published before, including six of the papers reviewed here. Nevertheless, I suggest that it may be better to look at SNPs that have replicable, consistent, and significant associations with intelligence (or any other phenotype for that matter), rather than entering all studies into a single meta-analysis.
Finally, I note that almost three-fifths of all SNPs identified as associated with intelligence are introns. An early study of GWASs of multiple human diseases and traits concluded that 45% of hits were introns,\cite{Hindorff2009} which is a bit lower than my estimate. Of course, it is no surprise that my figure of almost 60% is different from that of Hindorff et al. (2009), since I was only focusing on one trait (intelligence) and all the studies I included were published at least three years after their paper.
In a distant second in the current review (well behind introns) was the category of intergenic variants, comprising about 24% of all SNPs. A recent study looking specifically at type 2 diabetes reached a similar conclusion to this review, namely, that variants associated with the trait of interest tend to be intronic and intergenic.\cite{Meng_2018} Indeed, this pattern is well documented with respect to GWASs of all traits in  general: as Giral et al. (2018) have noted, "Nearly 90% of all phenotype-associated SNPs identified by GWAS lied within non-coding regions".\cite{Giral2018} Other estimates of the % of SNPs identified as "hits" in GWASs that are in noncoding regions include ~93% (from a 2012 study of many different diseases and traits).\cite{Maurano_2012} Similarly, as of 2013, 88% of all SNPs in the GWAS Catalog were in intronic or intergenic regions, meaning that these SNPs "...are likely to influence gene regulation (assuming that the same is true for the correlated candidate causal SNPs)."\cite{Edwards_2013}
The fact that GWAS hits tend to be non-coding has led to a problem for GWAS researchers looking for causal SNPs: as two researchers noted in 2015, "the GWAS field has been left with the conundrum as to how a single-nucleotide change in a non-coding region could confer increased risk for a specific disease."\cite{Tak_2015} It seems to cast doubt on the idea that GWASs of intelligence can be identifying causal variants if most of them are noncoding (i.e. intronic or intergenic variants), but it turns out that such variants can in fact have a functional effect on the phenotype of interest.\cite{Amlie-Wolf2018}\cite{Giral2018}\cite{Tak_2015} In other words, they may well be actual causal variants (or point to such variants in an indirect way through linkage disequilibrium) rather than false positives. 
How can we determine what functional role, if any, these non-coding SNPs have on intelligence? With regard to such SNPs for GWASs of diseases, it has been noted that "Intergenic SNPs are typically assigned a theoretical pathogenicity based on their proximity to nearby genes". But confusingly, this can sometimes lead to inaccurate conclusions because even functional non-coding SNPs don't necessarily interact with the genes that happen to be closest to them.\cite{Schierding_2016} We may also just have to wait for the human genome to become better annotated: "the scarce overlap of disease-associated variation with known genes is undoubtedly influenced by the incomplete annotation of the human transcriptome."\cite{Bartonicek_2017} Gene regulation may explain the functional roles of non-coding variants in at least some cases.\cite{Gallagher2018} Also, the category into which a SNP falls is only a first step in identifying whether/how it is functional (e.g. \cite{Freedman2011}).
I tried to shed light on the potential functions of the included SNPs by running as many of them as possible through Regulome. In total, 2,338 were tested in Regulome, and more than a quarter had the most common score (in this case, 6). My results can be compared to those of Savage et al., who also looked at intelligence-associated SNPs. Their results are shown in their paper's Figure 1D. They found that 31.3% of SNPs had the lowest score (7), making it the most common in their dataset, whereas 26.2% had the next-lowest score (6).\cite{Savage_2018} I found that 7.8% had a score of 7, clearly much lower than Savage et al.'s finding, and the 26.3% of SNPs in the current paper that had scores of 6 is almost identical to Savage et al.'s figure. The estimates of the % of SNPs with the coveted 1a score were similar in the current paper (0.09%) and in Savage et al. (0.02%). But we also see a relatively high number of SNPs excluded in the current paper (23.2%) vs. in Savage et al. (12.7%). 
This leads one to wonder, how many significant SNPs did each of the two papers discussed just above find? Savage et al. (p. 912) state that "12,110 variants indexed by 242 lead SNPs in approximate linkage equilibrium (r2<0.1) reached genome-wide significance (P<5×10−8)". So the answer for them is 242. Right? Not so fast: The GWAS catalog page for the same study lists 524 associations, almost all of which are less than the GWS threshold.\cite{catalog}