The You Only Look Twice (YOLT) object detection pipeline was designed to address some of the shortcomings identified in classic object detection techniques. The pipeline was based off of the You Only Look Once framework and dramatically improved performance of object detection at varying scales over legacy techniques. In order to tailor the framework for use on remote sensing data sets such as satellite imagery, YOLT provides three major modifications:

1. Upsampling via a sliding window to look for small, densely packed objects
2. Augment training data with re-scalings and rotations
3. Define a new network architecture such that the final convolutional layer has a denser final grid

In late 2017, we released YOLT version 2 which incorporated a number of improvements to the original paper. These enhancements significantly improved the accuracy while maintaining a speed advantage over other options such as Faster R-CNN and SSD.

This project explored the utility of visible and near infrared (VNIR) multispectral imagery (MSI) for training algorithms to artificially generate spectral information and for deep learning object detection algorithms. Initially, two leading object detection algorithms were adapted to analyze multispectral data. A performance comparison using these algorithms on grayscale, 3-band RGB, and 8-band multispectral imagery indicated a  performance advantage to three-band imagery over grayscale imagery. This finding motivated the study of methods to artificially generate color images from grayscale.

To this end, a generative adversarial network (GAN) architecture was developed to artificially generate 3-band images from grayscale imagery and 8-band images from 3-band imagery. While the GAN recovered a majority of the multispectral information in the test images, some areas and objects were reconstructed with higher accuracy than others. To gauge the utility of GAN colorization, the performance of two leading object detection algorithms, Multi-task Network Cascades (MNC) and You Only Look Twice (YOLT), were tested using grayscale imagery, real 3-band imagery, and artificially generated 3-band imagery. While the best performance was achieved with real 3-band images, algorithm performance was significantly better on the artificial 3-band images than on the grayscale images. Such initial results encourage future research in this subject area. Specifically, based on these initial results, the GAN might be an effective preprocessing step for imagery collected in bandwidth-constrained environments.