Multispectral and Hyperspectral Imaging - flaviamihaela/hypespectral_imaging_ML GitHub Wiki

Spectral imaging is a fascinating technology that precisely measures the spectral content of light in every single pixel of an image. This spectral content can be used to identify targets and the materials they are made out of.

Spectral imaging, which contains spatial information and spectral information taking the form of a data cube, can be split into: hyperspectral imaging (HSI) and multispectral imaging (MSI). HSI has higher resolution spectral signature, longer acquisition time and requires more complex analysis when compared to MSI.

Data acquisition for HSI and MSI is performed in one out of four different ways: whiskbroom acquisition (registers one pixel at a time), line scanning (registers one image line at a time), staring acquisition (takes picture by picture each with a different filter) and snapshot acquisition (registers all the spectral and spatial information in one single shot). The data is then analysed using multivariate statistics (there is more than one feature).

The imaging spectrometer (MS/HS camera) measures the target at every wavelength within its spectral range and the collected spectra are used to form an image of the target where each pixel includes spectrum. Light reflectance is assessed in visual light (VIS), near-infrared (NIR) and sometimes even shortwave infrared (SWIR) wavebands. A pixel in MSI is a highly dimensional vector in which the number of entries corresponds to the number of wavebands and the entry values correspond to the reflectance, transmittance or absorbance values (depending on the data type that the MS/HS device is gathering) registered at a specific waveband.

Even though HS and MS imaging has been under continuous development in the last few years, it has still received quite limited exposure compared to other ML computer vision fields such as Face Recognition, Object Detection and GAN. That being said it is of utmost importance to delve deeper into this subject given the fact that it can provide priceless information about the surrounding world and it can open even more gates when paired with ML, where automation is concerned.

An MS/HS camera in the domain of precision agriculture can be referred to as a phenomic non-invasive tool because the pixel spectrum (spectral signature) can be later used to identify the leaf’s composition and the plant species. Each spectral range describes different plant characteristics (VIS – pigments, NIR – cell structure, SWIR - water content, other chemicals). Plant stresses result in leaf senescence and implicitly degradation of pigments (particularly chlorophyll) which alters the ratio between reflected, absorbed and transmitted radiation. Consequently the effects can be seen at pigment (biochemical) level, at physiological level and morphological level. When tissue degradation is assessed using multispectral imaging methods, spectral vegetation indexes (SVI) are usually involved. These indices maximize sensitivity to the vegetation characteristics while minimizing confounding factors. The drawback of SVIs (which help describe plant health, leaf density and pigment content to some extent) is that they only take under consideration a limited number of wavebands and ignore the rest of the spectrum. Moreover, knowing that the main source of spectral variation over time lies in the metabolic changes caused by the stress to which the plant was subjected to, regular acquisition of data samples could give an insight to stress evolution. In order to better comprehend the type of information that can be obtained from MSI, one needs to look at the healthy vegetation reflectance spectrum and (at least in broad terms) understand its peculiarities.

To emphasize again, each MS plant image in ML multivariate analysis is represented by a spectral signature, plot of the reflectance level in discrete wavelengths, which can be extracted through various methods (the most common being taking a region of interest average pixel value for each waveband). In the visual range this particular sample of healthy vegetation absorbs red and blue light for photosynthesis and reflects green light (a small portion of it) because of the chlorophyll pigment. Furthermore, there is high reflectance in the NIR region because of the cell structure and well defined dips in the SWIR region which are mainly related to water absorption. Even though different healthy plants at different stages of life can have different spectral signatures, the point remains (the plot has the same standard characteristics). Contrasting the healthy spectra to the spectral signature of dry vegetation for example, there are no water absorption dips in the dry vegetation SWIR (there is not that much moisture in the drought stressed sample) and there are also no chlorophyll absorption dips either.