Individual differences in brain function arise from genetic and environmental influences and play an important role in understanding variation in executive control, cognitive ability, and personality. By collecting task fMRI and behavioral data for monozygotic (MZ) twins, dizygotic (DZ) twins, siblings (SIB), and unrelated people, the Human Connectome Project (HCP) allows investigation of the degree to which genetics shapes these differences. This study compared activation similarity patterns in the frontoparietal and visual networks across these subject groups. Activation similarity was correlated for the N-back task under conditions of high or low working memory load and across two object stimulus categories. If heritability plays a substantial role in determining neural activation, groups of higher genetic similarity should have more similar activation patterns. Indeed, in both networks considered, MZ twins showed a higher similarity than DZ twins or siblings, and DZ twins and siblings showed a higher similarity than unrelated participants. Furthermore, this correlation is emphasized under conditions of higher cognitive load in the frontoparietal network. This provides evidence that genetic influences play a substantial role in the neural basis of individual differences, and ultimately helps lay the foundation for task-related brain activation to be considered as an endophenotype for psychiatric or neural disorders.