Yes, wait for the fruit to be ripe (to eat, make jam, chutneys, wine, damson vodka etc). If you have to cut it down wait until winter when the sap is low and the wood will season quicker. If it's a decent tree the wood could be useful for carving or making other things with as it's quite hard and nicely coloured and can even have a reddish purple tinge in the middle.