The Nanao Castle was the main fort of the Masuda clan. Located on the ridge of Mt. Nanao, the main keep stands 118m above from sea level.

This Y-shaped fort overlooks the Masuda Plain and further to the Sea of Japan.
The large scale mountain castle consisted of over 40 fortified points called kuruwa and protective ridges and moats.

The castle first appeared in documents around the Nanbokucho period (1337 – 1392), though there are various opinions on when it was first constructed.

It is believed to have been built as a small fortress for a start, then gradually expanded its size until the entire mountain was fortified by the end of 16th century.
Fujikane, the 19th lord of the Masuda clan was the last resident ruler of the castle.
He is said to have made some improvement to the castle to prepare against the Mori clan of Hiroshima.

Sumiyoshi-Jinja Shrine, sitting on the side of the castle hill, was originally erected within Myogi-ji Temple and later moved to the present location.