Approximately 1 in 10,000 people are born with isolated congenital anosmia (ICA), defined as loss of the ability to smell in the absence of other symptoms. Despite the importance of olfaction for our quality of life, the underlying mechanism for these cases of ICA remain largely enigmatic, as only two genes have been implicated, to date. In contrast, the genetic basis of other inherited sensory defects is well investigated, with almost 100 genes implicated in congenital deafness and over 200 genes implicated in congenital blindness. We examined how genetic variation associated with disease state in ten families with congenital anosmia, and determined candidate causal genes for this disorder. Our candidate list included genes for which rare variants were present in family members with congenital anosmia, but not unaffected family members, filtered by dominant or recessive inheritance pattern. There were no candidate genes common to all families, indicating that, as expected, ICA is a heterogeneous disorder with multiple genetic causes. We found several genes that have previously been linked to olfactory function, such as PLEK and CACNA1B. To identify which of the candidate genes are most likely to be involved in ICA, we conducted a targeted search in a cohort of 121 individuals with ICA (singletons). We identified a smaller list of candidate genes that have a disproportionately large number of variants in this singleton population and investigated them as to their potential role in olfaction. Historically, identification of genes related to other sensory disorders has provided a gateway to better understanding of those senses. Given how few genes have been implicated in olfactory disorders, we have the opportunity to uncover a plethora of new avenues through which to better understand basic olfactory function.