Privacy Policy Disclaimer
  Advanced SearchBrowse




Journal Article

DiverGet: a Search-Based Software Testing approach for Deep Neural Network Quantization assessment


Yahmed,  Ahmed Haj
External Organizations;

Braiek,  Houssem Ben
External Organizations;

Khomh,  Foutse
External Organizations;

Bouzidi,  Sonia
External Organizations;


Zaatour,  Rania
Potsdam Institute for Climate Impact Research;

External Ressource
No external resources are shared
Fulltext (public)
There are no public fulltexts stored in PIKpublic
Supplementary Material (public)
There is no public supplementary material available

Yahmed, A. H., Braiek, H. B., Khomh, F., Bouzidi, S., Zaatour, R. (2022): DiverGet: a Search-Based Software Testing approach for Deep Neural Network Quantization assessment. - Empirical Software Engineering, 27, 193.

Cite as: https://publications.pik-potsdam.de/pubman/item/item_27850
Quantization is one of the most applied Deep Neural Network (DNN) compression strategies, when deploying a trained DNN model on an embedded system or a cell phone. This is owing to its simplicity and adaptability to a wide range of applications and circumstances, as opposed to specific Artificial Intelligence (AI) accelerators and compilers that are often designed only for certain specific hardware (e.g., Google Coral Edge TPU). With the growing demand for quantization, ensuring the reliability of this strategy is becoming a critical challenge. Traditional testing methods, which gather more and more genuine data for better assessment, are often not practical because of the large size of the input space and the high similarity between the original DNN and its quantized counterpart. As a result, advanced assessment strategies have become of paramount importance. In this paper, we present DiverGet, a search-based testing framework for quantization assessment. DiverGet defines a space of metamorphic relations that simulate naturally-occurring distortions on the inputs. Then, it optimally explores these relations to reveal the disagreements among DNNs of different arithmetic precision. We evaluate the performance of DiverGet on state-of-the-art DNNs applied to hyperspectral remote sensing images. We chose the remote sensing DNNs as they’re being increasingly deployed at the edge (e.g., high-lift drones) in critical domains like climate change research and astronomy. Our results show that DiverGet successfully challenges the robustness of established quantization techniques against naturally-occurring shifted data, and outperforms its most recent concurrent, DiffChaser, with a success rate that is (on average) four times higher.