Depends on the rock, DWild. The plugs and feathers were generally used to split quarried stone into blocks or slabs. The piles of rock you are talking about could have been rubble, chunks discarded in the process, in which case they'd have sharp edges and signs of working. If the rocks in the piles are smooth and rounded, maybe cracked in half or quarters, then they'd likely be a result of clearing ground for plowing, a regular chore each spring as an new crop of rocks appeared out of the ground and farmers used stone boats to haul them to the piles along the edges of the field. Besides plugs and feathers, most stone can be split like firewood, along the grain with a sledge. As well, in the early days when they were clearing and building roads, they'd pile brush on a protruding boulder and set a good fire to burn on top and around, then pour water over the heated rock to crack it apart.
The area I am from is on the northern most point of the Appalachian mountain range that extends from the southern states into Canada. That whole range is rich in upcroppings of different rocks and minerals. My great-grandfather on my mother's side was brought in to oversee the operation of a marble quarry in the area. As a lad, my grandfather worked as a hired hand at the quarry and married the fellow's daughter. They cut giant slabs of a pearly white marble with a characteristic fine grey grain running through it. It was a fairly small operation, but the marble was shipped far and wide, used as architectural stone, for example, the steps of the Canadian parliament in Ottawa, as well as tombstones and monuments. It was abandoned in the 40's, but I could take you in there, overgrown but looks pretty much like the day it was left... piles of rubble, drill marks running down the bare face of the marble wall, plugs and feathers protruding here and there, still wedged and in place where they were pounded into the rock.
Similar marble outcropping extend down into Arkansas and Tennessee. Arkansas stone, another marble type rock, has long been prized by woodcarvers as a sharpening stone. In the early days in my area they also mined corundum used as a sharpening stone, feldspar for gunpowder, iron and then post-war, uranium. Not far to the south, there is an area of an ancient inland sea bed, lots of limestone that they quarried for building blocks and talc and such.
Whereabouts are you, DWild, any sort of relevant history of the area, the earthworks and such?