The development of high-throughput genotyping and phenotyping has provided access to many tools to accelerate plant breeding programs. Unmanned Aerial Systems (UAS)-based remote sensing is being broadly implemented for field-based high-throughput phenotyping due to its low cost and the capacity to rapidly cover large breeding populations. The Structure-from-Motion photogrammetry processes aerial images taken from multiple perspectives over a field to an orthomosaic photo of a complete field experiment, allowing spectral or morphological trait extraction from the canopy surface for each individual field plot. However, some phenotypic information observable in each raw aerial image seems to be lost to the orthomosaic photo, probably due to photogrammetry processes such as pixel merging and blending. To formally assess this, we introduced a set of image processing methods to extract phenotypes from orthorectified raw aerial images and compared them to the negative control of extracting the same traits from processed orthomosaic images. We predict that standard measures of accuracy in terms of the broad-sense heritability of the remote sensing spectral traits will be higher using the orthorectified photos than with the orthomosaic image. Using three case studies, we therefore compared the broad-sense heritability of phenotypes in wheat breeding nurseries including, (1) canopy temperature from thermal imaging, (2) canopy normalized difference vegetation index (NDVI), and (3) early-stage ground cover from multispectral imaging. We evaluated heritability estimates of these phenotypes extracted from multiple orthorectified aerial images via four statistical models and compared the results with heritability estimates of these phenotypes extracted from a single orthomosaic image. Our results indicate that extracting traits directly from multiple orthorectified aerial images yielded increased estimates of heritability for all three phenotypes through proper modeling, compared to estimation using traits extracted from the orthomosaic image. In summary, the image processing methods demonstrated in this study have the potential to improve the quality of the plant trait extracted from high-throughput imaging. This, in turn, can enable breeders to utilize phenomics technologies more effectively for improved selection.