If you can add annotations to a drawing one at a time, you should be able to remove only certain annotation groups, instead of destroying everything in a drawing.
