With the pins removed using your drift and hammer, the backing plate can be removed. Remember the direction when mounting it again and try to align it on the outer surface of the disc during reinstallation.