On the Absence of High-redshift AGNs : Little Growth in the Supermassive Black Hole Population at High Redshifts