It is well established that boys perceive themselves to be better in mathematics than girls, even when their ability is the same. We examine the drivers of the gender gap in self-assessed mathematics ability using a longitudinal study of twins. Using measures of individual self-assessment in mathematics from childhood and adolescence, along with mathematics levels and test scores, cognitive skills, parent and teacher mathematics assessments, and characteristics of their families and siblings, we examine potential channels of the gender gap. Our results confirm that objective mathematics abilities only explain a small share of the gender gap in self-assessed mathematics abilities, and the gap is even larger within opposite-sex twin pairs. We find that the self-assessment of boys is positively correlated with the self-assessment of their male co-twins, not just in mathematics, but also in other abilities. However, this positive correlation is not observed between girls and their male co-twins; if anything, it is negative. This phenomenon might explain why men self-select into top jobs or STEM courses, that are filled with confident men, possibly making entry relatively easier for men. We also find that parents are more likely to overestimate boys’ and underestimate girls’ mathematics abilities. Gender-biased parental assessments explain a large part of the gender gap in mathematics self-assessment, highlighting the potential of the intergenerational transmission of gender stereotypes.