I suspect they would survive just fine, but I honestly don't know the answer to the quality question. I have replanted small, home grown sweet potatoes the following year. Some of them rotted away after the tops got going, some just kind of sat and got tough, dark, and corky inside, and a few of them actually grew, but were also kind of dark and funny looking flesh, not appealing so I didn't ever cook any to see what the flavor would be like.

When I lived in SD in the 80's, Murphy Canyon was just outside my back fence, and I planted some Filipino sweet potatoes there on the hillside. They naturalized, and had no problem surviving the "winter"; I did have to irrigate them in Summer, though. They spread gradually, as some of the vines rooted where they touched the ground.

I can't say whether the small tubers would get larger; I would dig up every other hill in the Fall, and the surviving hills would fill in the space again. Never had to replant.

I bought white skin purple fleshed sweet potatoes from the flea market and tried to grow but failed 2 years in a row.

My bro has a small patch growing in his yard, planted by his late mother in law.. It comes back every year. So I think you're ok leaving them in...