Nitrogen (N) is fundamental to plant growth, development, and yield. Genes underlying N utilization and assimilation are well characterized, but mechanisms underpinning plasticity of different phenotypes to varying amounts of N in the soil remain elusive. Here, using Arabidopsis thaliana accessions, we dissected the genetic architecture of plasticity in early and late rosette diameter, flowering time and yield in response to three levels of N in soil. Genome-wide association analysis identified three significant associations for phenotypic plasticity, one for early rosette diameter and two for flowering time. We confirmed that the gene At1g19880, hereafter named as PLASTICITY OF ROSETTE TO NITROGEN 1 (PROTON1), encoding for a regulator of chromatin condensation 1 (RCC1) family protein, conferred plasticity of rosette diameter in response to changes in N availability. The altered plasticities were a result of faster development under limiting N, and correlated with the plasticity in the levels of primary metabolites. By using different growth conditions for a subset of accessions, we showed that plasticities of growth and flowering-related traits in response to N availability differed between the environmental cues, indicating decoupled genetic programs regulating these traits. Our findings provide a prospective for identification of genes that stabilize performance under fluctuating environments.