Never forget about Kyushu (九州)! Known as the onsen (hot spring) prefecture, the area also has some great castles. Kyushu is also easy to get around with a load of cheap, fast transportation options.
Hot Springs Hiking and Walking Shrines and Temples Castles Big cities Japanese gardens
Fukuoka and Kumamoto have the best budget accommodation, but all the towns and cities in Kyushu (with the big exceptions of Shimabara and Unzen Onsen) have good deals.