Face recognition has been employed in various security-related applications such as surveillance, mugshot identification, e-passport, and access control. Despite its recent advancements, privacy concern is one of several issues preventing its wider deployment. In this paper, we address the privacy concern for a self-exclusion scenario of face recognition, through combining face recognition with a simple biometric encryption scheme called helper data system. The combined system is described in detail with focus on the key binding procedure. Experiments are carried out on the CMU PIE face database. The experimental results demonstrate that in the proposed system, the biometric encryption module tends to significantly reduce the false acceptance rate while increasing the false rejection rate.