I tried it both ways when our tree was going strong. I do think it's better with sugar as the
fruit doesn't get as brown. But it does work fine without. And if you don't want the sugar, try it with a few peaches to start. I'd do them again, without.
I wonder if it depends on how quickly your fridge can freeze them before they start to deteriorate.
And I wonder if you placed a sheet of plastic wrap snuggling around all the slices, best you can, if that would help.