One thing that can do that is brown rot of stone fruit. Look for text & images to see if it matches.

It can be brown rot, but I'm not sure. The way you can be sure you have brown rot is some of the peaches in an advanced stage will have the brown dull powdery stuff on the outside. If that is what you have you need to spray, primarily in the early phases of fruit formation, to prevent the disease, and also perhaps a spray in July just before fruits start ripening. I use Serenade, an organic product, and it seems to work OK. You can also use propiconazole (Monterey Garden Fungi-Fighter). The peaches rotting in the bag are probably rotting due to brown rot, so it does sound like the fungus is present.

The only thing about your description that doesn't quite add up is the way the fruits are falling off before they ripen. When you have brown rot they usually don't drop early, and in fact they can "mummify" on the tree, turning all shriveled and gross. Also it sounds like the fruits falling off look fine, just hard and unripe. It makes me wonder a bit if a storm or hail or something didn't come through and blow them off. One last thing it could be is some varieties of peach can drop fruits when they are young trees. It is common for them to drop in June but is not common to drop so late.

Overall, I would spray for brown rot next year and hope for better luck..

Scott

What is brown rot on peach trees?