We propose and study the use of angle diversity to combat the ambient noise in indoor optical wireless communication systems. Models for the radiation pattern of spot lamps and for the spatial distribution of the ambient light based on an isotropic and a directional noise component are derived. Performance evaluation of receivers using angle diversity is carried out and significant optical power gains are demonstrated. The optical gains are seen to increase with the relative weight of the directional noise within the cell, with the sharpness of the directional noise sources beamwidth and in environments where there are noise sources positioned outside the cell. Also the SNR of a sectored receiver is seen to be much less sensitive to the position and beamwidth of the noise sources than the SNR of a non-sectored receiver allowing for more universal transceiver designs.