There's nothing wrong with carrying a firearm if you feel the need. On occasion I pack one along if going into some remote area due to being unfamiliar with the land, and with the owners knowledge. However that is the only time I take one along. Main reason - its just extra gear to pack in along with your MDing stuff, and after a while it becomes burdensome if out for long stretches of time. If I see an area I'd like to hunt, but find out prior it's a "bad" area or neighborhood, I just mark it off my list of places to hunt and go somewhere else. As much as I love this hobby it's just not worth the risk, even with places to hunt getting harder to find or obtain permission to do so, no matter how good the potential of a site may be. Sadly its the world we live in.
If you do decide to carry, suggest getting a concealed carry permit if allowable. There are folks in this world that see people carrying guns as a challenge or threat, and may approach you when they normally would have not. When you carry concealed you have with you the protection you feel is necessary without advertising it to others. I also recommend taking a course in defensive firearm use. Many people I know feel all they have to do is be able to hit a paper target and they're ready for anything - wrong perception. A good defensive course will prepare you for potential situations, as well as alternatives to help defuse a situaton and using the firearm only as an absolute last resort. I would also become familiar with your state laws regarding discharging a firearm in self-defense. Many states will automatically charge you and the assailant until they sort out the details of exactly what happened. This will include detaining and/or incarcerating you, as well as confiscating your firearm until they determine you were innocent. Also, many states have the "flee if you can" laws in affect, meaning you need to demonstrate you were absolutely cornered, there was no way you could safely remove yourself from the situation, and had to use your firearm as a last resort. There is no shame in running away to be able to MD another day.
My apologies if I came off as preaching as it is not my intent, and I'm sure you're a responsible person, just be cautious when introducing firearms into any activity.