I assume that the kingdoms, as large as some of them are, are still a minority as far as the underground goes. But then, I believe the early twentieth century was a time when the idea of a hollow Earth was popular. I think Baum's contemporary Edgar Rice Burroughs used it in his Pellucidar stories, for instance.

The Easter Bunny does live underground, and there are some other underground territories mentioned in other books (the dragons' cavern that Woot fell into, the Illumi Nation, Torpedo Town, etc.). I get the impression that most of these are relatively small and not all that far underground, though. I was concentrating mostly on large and deep subterranean countries.