If you created more keys for a distribution than you actually needed, that means the extra keys will be unredeemed, which can be a security risk.
While you can always deactivate keys on an individual level, it's faster to deactivate all unredeemed keys from the distribution at once.
Here is how to deactivate keys from your distribution report
Visit your distributions report from the left hand navigation bar
Locate the distribution and click the "options" drop-down
Click "deactivate keys" and then click "yes" in the confirmation window
Once you've confirmed, the number of available keys will automatically drop to zero
And you're done!