Öz Bilgi Destekli Derin Öğrenme Yaklaşımları ile HSG Gürültü Giderme
Özet
Hyperspectral images (HSIs) are of critical importance, especially in areas such as environmental monitoring, target detection, and classification. However, the degradation caused by noise effects significantly hinders the effectiveness of HSIs. This situation negatively affects the accuracy and reliability of the data obtained from HSIs. Hence, noise removal is essential to extract meaningful information from HSIs accurately. Traditional denoising methods, initially adapted from two dimensional (2D) imaging techniques, overlook crucial spatial-spectral correlations, resulting in spectral distortions. Recent advancements in HSI denoising have focused on exploiting spatial-spectral correlations through complex optimization processes, sacrificing computational efficiency.
Recently, with the popularity of deep learning, Convolutional Neural Network (CNN) based approaches have created a fresh wave of HSI denoising methods, demonstrating significant improvements over the traditional methods. CNNs leverage their strong inductive bias for effective feature extraction in HSI denoising. These data-driven models automatically learn a mapping from noisy HSIs to clean HSIs. However, they encounter challenges in adapting to the specific characteristics of diverse input content and capturing long-range dependencies within the spatial-spectral domains.
This dissertation aims to use Deep Neural Networks (DNNs) to eliminate noise and perform restoration on HSIs. To address the above challenges, we propose novel self-information empowered dynamic architectures tailored for HSI denoising. Dynamic architectures, a burgeoning field within deep learning, offer a significant advantage over static models. Unlike their static counterparts, dynamic networks can adapt their structure or parameters during the inference stage for each new input. This adaptability enhances accuracy, representation power, and generality, all achieved without sacrificing computational efficiency.
Firstly, we introduce the Self-Modulating CNN (SM-CNN), equipped with Spectral Self-Modulating Residual Blocks (SSMRBs), enabling adaptive feature transformation based on spatio-spectral information. In particular, the introduction of SSMRB transforms our denoising network into a dynamic network that adapts its predicted features while denoising every input HSI with respect to its spatio-spectral characteristics. We perform noise removal by scanning band-to-band spatial information and using adjacent spectral information along with spatial information. Thanks to this architecture, test data with different spatial–spectral properties can be denoised with a single model. The qualitative and quantitative evaluation of the results show that the proposed algorithm is more efficient than other single-model algorithms on both synthetic and real data. However, our model exhibits lower performance compared to recently proposed transformer-based models due to its inability to capture global information effectively.
By investigating both local details and spatially distant areas, the unveiling of long-range dependencies contributes to a more comprehensive understanding
of the image. Transformers, with their self-attention mechanism, are increasingly recognized for their ability to capture long-range dependencies. However, training Transformers to learn local features often requires substantial data, which may not be readily available for HSIs. Additionally, the high
band count in HSIs can lead to increased memory usage and computational complexity in Transformers, particularly for self-attention calculations. To address these challenges, we present a Hybrid CNN-Transformer model, CST3D (Channel-wise and Spectral Transformer with 3D convolution network), combining CNN's local feature extraction with Transformer's long-range dependency modeling. CST3D integrates spectral and channel-wise self-attention mechanisms, augmented with learnable modulators to prioritize relevant spectral bands and enhance attention focus. This hybrid approach overcomes limitations of individual models, offering superior denoising performance and generalization to diverse datasets. Experimental results show that our approach effectively removes different types of noise from HSIs, outperforming current state-of-the-art models, including classical, CNN-based, and other hybrid models.
On the other hand, neural networks are trained in a supervised manner and then tested on unseen data. Supervised training requires a large amount of data, and it also relies on clean images. However, collecting a large number of images and obtaining clean images for HSIs are difficult and costly. Consequently, training neural networks with supervised methods becomes challenging for such cases. While dynamic networks are effective in enhancing data adaptation, performance degradation occurs in cases where HSI data exhibits significant spectral variability. In this dissertation, our proposed two-stage learning strategy leverages both pre-training and self-supervised calibration. The first stage involves supervised learning to train the model on noisy and clean data pairs. The second stage incorporates self-supervised calibration using only noisy data to adapt the model to specific noise patterns. For the latter, to estimate the middle spectral band, we leverage the information from its neighboring band as a target. To ensure the network learns meaningful relationships rather than merely copying the input, we strategically create a blind spot by excluding the target band from the input data. Therefore, our self-supervised learning technique is named as Blind Band Self-Supervised (BBSS) Learning. Our approach has been shown to improve the accuracy of the model for noisy HSIs, even when the network did not previously encounter the specific noise patterns in supervised learning.
Our contributions signify a paradigm shift towards dynamic deep learning architectures for HSI denoising. This approach holds the promise of enhanced accuracy, computational efficiency, and generalization capabilities in real-world scenarios. Additionally, it endows the models with self-calibration capabilities, reinforcing its ability to adapt and generalize to unseen and potentially noisy data, even in the absence of clean data.
