Thanks for your comments. No, I'm pretty sure those were not caterpillar bites nor eaten away by slugs. If you look closely at the picture you'd see some light yellow green discoloration.
These started out as yellow green discoloration, then eventually becomes small scars then grow bigger.
I also at first suspected insect bites therefore moved by barrel cactus indoor out of reach from insects but these spots continue to surface.