Abstract
Marine scientists use remote underwater image and video recording to survey fish species in their natural habitats. This helps them get a step closer towards understanding and predicting how fish respond to climate change, habitat degradation and fishing pressure. This information is essential for developing sustainable fisheries for human consumption, and for preserving the environment. However, the enormous volume of collected videos makes extracting useful information a daunting and time‐consuming task for a human being. A promising method to address this problem is the cutting‐edge deep learning (DL) technology. DL can help marine scientists parse large volumes of video promptly and efficiently, unlocking niche information that cannot be obtained using conventional manual monitoring methods. In this paper, we first provide a survey of computer visions (CVs) and DL studies conducted between 2003 and 2021 on fish classification in underwater habitats. We then give an overview of the key concepts of DL, while analysing and synthesizing DL studies. We also discuss the main challenges faced when developing DL for underwater image processing and propose approaches to address them. Finally, we provide insights into the marine habitat monitoring research domain and shed light on what the future of DL for underwater image processing may hold. This paper aims to inform marine scientists who would like to gain a high‐level understanding of essential DL concepts and survey state‐of‐the‐art DL‐based fish classification in their underwater habitat.