Where I live and do a majority of detecting is VERY rural.
I avoid places with lots of people or go in off hours and less than ideal weather.
To answer your question yes headphones make a huge difference when detecting, headphones allow you to hear the deeper fainter targets that you may other wise may not hear
, I do understand your reluctance to wear a set of headphones or ear buds, i currently use a Neckband speaker made by a company called Avantree, they are called Avantree Torus neckband speaker, the avantree Neckband speaker can be used one of two ways as a speaker that you wear around your neck or they also have earbuds incorporated into the neckband speaker system, I use these because it allows me to keep my ears attuned to targets and also to enable me to be aware of my surroundings, I also hunt in remote areas when detecting for gold nuggets, the possibility of running into a bear is always there or a mountain lion, or two legged varmints that are up to no good LOL
the Avantree Torus neckband speaker basically moves the sound closer to your ears and allows you to hear deeper targets, but there is still a chance of not hearing the deeper more faint targets unless you use the on board earbuds, they also work great when I have my hearing aids in which is almost always, the Avantree Torus makes the tones sound very loud but when I have asked people around me if they are hearing the tones from my Legend they always respond no, they have to be fairly close to me to hear the tones coming from my Legend and when I say close I am talking less than 5 feet from me I would say within 2-3 feet if they are closer than 2-3 feet then I tell them they are invading my space and ask them if they would mind backing off a bit.