When you bought your house, there was already a fence between your house and the neighbors' house. You just assumed that it marked the property line, and it was clear that your neighbor thought the same thing. You didn't have a survey done and carried on with your life.
Imagine getting a foreclosure notice in the mail when you've spent years or even decades faithfully paying off that loan. You know that your payments are up to date. You've never missed one in your life. And now you're worried that you could lose your home and your entire investment anyway.