Well, at first it was because of all the hysteria around magic. Witches and wizards were being hanged and burned-- that is, when the non-magical community got real ones at all.

But over the years, the magical community never tried to rejoin the rest of the world. So many of them thought it was too dangerous, or they had this notion the non-magical community wouldn't understand or accept them. It's all very archaic and needs reexamining.