Of the $500$ balls in a large bag, $80\%$ are red and the rest are blue. How many of the red balls must be removed so that $75\%$ of the remaining balls are red?