Journal Search Engine

ISSN : 2288-3509(Print)
ISSN : 2384-1168(Online)

Journal of Radiological Science and Technology Vol.46 No.3 pp.207-217
DOI : https://doi.org/10.17946/JRST.2023.46.3.207

A Systematic Review of Trends for Image Quality Improvement in Light Microscopy

Kyuseok Kim¹⁾, Youngjin Lee²⁾

¹⁾Department of Biomedical Engineering, Eulji University
²⁾Department of Radiological Science, Gachon University

Corresponding author: Youngjin Lee, Department of Radiological Science, Gachon University, 191, Hambangmoe-ro, Yeonsu-gu, Incheon, 21936, Republic of Korea / Tel: +82-32-820-4362 / E-mail: yj20@gachon.ac.kr

Received 27/04/2023 Review 09/05/2023 Accepted 22/05/2023

Abstract

Image noise reduction algorithm performs important functions in light microscopy. This study aims to systematically review the research trend of types and performance evaluation methods of noise reduction algorithm in light microscopic images. A systematic literature search of three databases of publications from January 1985 to May 2020 was conducted; of the 139 publications reviewed, 16 were included in this study. For each research result, the subjects were categorized into four major frameworks—1. noise reduction method, 2. imaging technique, 3. imaging type, and 4. evaluation method—and analyzed. Since 2003, related studies have been conducted and published, and the number of papers has increased over the years and begun to decrease since 2016. The most commonly used method of noise reduction algorithm for light microscopy images was wavelet-transform-based technology, which was mostly applied in basic systems. In addition, research on the real experimental image was performed more actively than on the simulation condition, with the main case being to use the comparison parameter as an evaluation method. This systematic review is expected to be extremely useful in the future method of numerically analyzing the noise reduction efficiency of light microscopy images.

Key Words : Systematic review , Light microscopy , Image quality , Noise reduction algorithm , Quantitative and comparison evaluation parameters

광학 현미경 영상 화질개선의 추세에 관한 체계적 고찰

김규석¹⁾, 이영진²⁾

¹⁾을지대학교 의료공학과
²⁾가천대학교 방사선학과

초록

키워드 : 체계적 고찰 , 광학 현미경 , 영상 화질 , 노이즈 제거 알고리즘 , 정량적 및 비교 평가 인자들

This article has been cited by 0 article in crossref

Cited-By

Funding:

Ⅰ. Introduction

Light microscopy is primarily used to examine difficult-to-observe objects, such as cells or bacteria. After magnifying the object to be observed using a convex lens, image acquisition using a light microscope began. The microscope began with a corrective lens developed by Arab scientist A. I. Firnas in the 9th century, and Z. Janssen from the Netherlands was the first to develop a compound microscope that combines two lenses (objective lens and eyepiece) like the current microscope [1]. Janssens’s microscope was developed by combining a convex and concave lens but was not widely used for severe blurring phenomena. The microscope was used for observation and research around 1660, and it was the first type of compound microscope to use light as a light source at 30 times magnification [2]. Many researchers attempted to improve the resolution and magnification based on this, and Abbe and Zeiss developed a lens that solves spherical aberration and chromatic aberration to form the basic structure of modern light microscopy [3].

The basic principle of light microscopy developed in this manner is to use the property of light refraction. Light microscopy works on the principle of magnifying the real image of the first magnification obtained by placing the objective lens with a short focal length close to the object again with the eyepiece. Fine adjustment of the focus is essential because the image is not clearly formed even if the distance between the target object and the objective lens changes slightly. The magnification of light microscopy can be expressed by multiplying the magnification of the objective lens used in the microscope and the magnification of the eyepiece. For example, when using 10x magnification with the eyepiece at the 100x objective lens, the final images can be obtained at a magnification of 1000x.

Recently, methods for maximizing image quality based on light microscopy have been actively studied. However, light microscopy has limitations in improving resolution for two main reasons: 1. the diffraction limit and 2. uncertainty in single-molecule localization. The diffraction limit is the theory that, despite excellent lens performance, it is difficult to distinguish two objects that are within half the wavelength of visible light [4,5]. This indicates that, although the resolution limit of light microscopy using visible light is 200 nm, it is possible to observe the internal organelle types of cells; however, it is difficult to confirm the detailed structure. In addition, molecules generally absorb photons and their energy levels increase, resulting in an unstable excitation state. Unstable excitation-state molecules emit light energy to return to a stable stage (ground stage). The cycle of excitation and emission takes approximately 3 ns, and even with a high-performance detector, all molecules appear to emit light. In this process, single-molecule localization uncertainty occurs, which is impossible for molecules within the limit of resolution owing to the point spread function [6].

To overcome this limitation, many studies have been conducted, and in 1989, Moerner et al. succeeded in observing a single molecule for the first time in a dense medium [7,8]. However, the noise inevitably generated in light microscopy (including additive white Gaussian noise, Poisson noise, and mixed Poisson-Gaussian noise types) degrades the resolution and deteriorates the image quality [9]. Fig. 1 shows an explanatory diagram of the three noise models in the aforementioned light microscopic image.

Software-based algorithms are widely used to reduce the noise in light microscopy. Among the noise reduction algorithms, the total variation (TV)-based approach has been introduced to estimate noiseless areas in images from noisy observations. The basic equation for the TV norm metric is as follows [10]:

{| x |}_{T V} = \sum_{p \in Ω} \sqrt{{(φ_{h o r i z o n t a l} x (p))}^{2} + {(φ_{p e r p e n d i c u l a r} x (p))}^{2}}

(1)

where φ_horizontal and φ_{perpendicular} denote derivative operators with pixel coordinates p∈Ω in the horizontal and perpendicular directions, respectively. In addition, the non-local means (NLM) approach, which outperforms, with the fusion of patches in an image, was developed by Buades et al. [11]. The basic equation for the NLM denoising procedure is as follows [9]:

N L (g (i)) = \sum_{j} \frac{1}{Z (i)} e x p (- \frac{{‖ y (N_{p}) - y (N_{j}) ‖}_{2}^{2}}{h^{2}})

(2)

where g(i) denotes the noise reduction of the pixel of interest, and h denotes the control parameter for the weight function with the L₂ norm. Here, the size of the N _p window is equal to that of the N _j window.

Another denoising method for an image is to use the wavelet transform. Since it has the characteristics of the spatial domain and performance of the frequency domain, many studies have been conducted to obtain a signal or an image, such as noise reduction and contrast enhancement of an image in image processing [12–15]. The image noise reduction method using wavelets is widely used to remove additional Gaussian noise using the threshold of wavelet coefficients. According to Nowak et al., the wavelet transform is effective in removing Rician distribution-based noise, which is primarily observed in magnetic resonance imaging [16-18]. In particular, the noise removal method based on wavelet transform in optical or microscopic images has become useful since the principle of sequential projection into binary frequency sections began. Thus, the applicability of light microscopy images to noise removal using transforms of a shape similar to a wavelet is also being studied. AlAsadi proved the usefulness of noise reduction of the contour transform method in various imaging systems and demonstrated the advantage of obtaining a multiscale and high degree of directionality in photon imaging [19].

Based on these noise reduction methods, various types of improved filters and algorithms have recently been developed and applied to light microscopy. In particular, image processing methods based on artificial intelligence or deep learning are being actively applied to light microscopy and are beginning to be introduced as approaches for noise reduction. As we mentioned above, many studies on image quality improvement of optical microscope images are being approached in various ways. Thus, this study aimed to systematically analyze the research designs, noise reduction algorithms, and evaluation methods being analyzed in previous studies and to propose future research directions that may be necessary.

Ⅱ. Materials and Methods

1. Design of study

Our systematic review of quantitative and comparative evaluations of noise reduction algorithms in light microscopic images followed the preferred reporting items for systematic reviews and meta-analysis guidelines (PRISMA checklist and statement) [20].

2. Data source and search strategy

MEDLINE-EBSCO, EMBASE, and the Cochrane Library are the most commonly used databases. The search included publications from January 1985 to May 2020 to identify relevant papers on the quantitative and comparison evaluations of noise reduction algorithms in light microscopic images. To find related papers, only English forms were applied, based on the guidelines. Table 1 provides an overview of the search strategies used in this study [21-36].

3. Descriptive analysis

All authors screened titles and abstracts, searching for valid papers, and reviewed and categorized all papers, including verification of all the data.

For descriptive analysis, the searched papers were classified in detail based on the noise reduction method. Additionally, microscope systems were classified into basic and light sheets, and papers were organized by the study type (simulation, experiment, or both). Finally, they were classified based on the measurement method to demonstrate the efficacy of the noise reduction algorithm. Finally, we classified and organized the paper using a method or parameter that evaluates the noise level and image quality.

Ⅲ. Results and Discussion

A total of 139 research papers were identified using three search engines (MEDLINE-EBSCO, EMBASE, and Cochrane libraries). A total of 95 papers were initially included, excluding 44 duplicates out of 139. While searching for papers, we set abstract, title, and keywords, as well as improved accuracy by separating abstract and title with “OR.” Subsequently, we excluded conference proceedings and reviewed the title and abstract with all authors to eliminate papers unrelated to the topic of this manuscript. During this screening process, six conference proceedings and 51 inappropriate papers (not denoising, confocal system, not microscopy, tutorial, optical coherence tomography, artifact reduction, segmentation, uniformity correction, ultrasound, and micro CT) were excluded. Among the 36 papers that compelled with the eligibility criteria, 20 papers that did not match the review papers and trials were excluded, and 16 systematic review papers were selected [21–36]. A PRSIMA flowchart (including identification, screening, and eligibility) of the study is shown in Fig. 2.

Fig. 3 shows the total number of papers for the noise reduction algorithm in light microscopic images by publication year. By displaying the graph using polynomial fitting, a Gaussian distribution was obtained, and the r-square (COD) and standard deviation values were measured to be 0.110 and 0.748, respectively.

1. Types of study

Studies on general images, including light microscopy, can be divided into simulations and experiments. Fig. 4 shows the overall publication numbers for each type of study. Among the 16 identified papers, only the results of experiments and simulations were published in six [22–24,28,32,36] and three [21,25,29] papers, respectively. Seven papers are the results of research conducted simultaneously with experimentation and simulation [26,27,30,31,33-35].

This research trend demonstrates that few simulation programs or tools can accurately model microscopes. In several reviewed studies, the Shepp-logan phantom or the phantom that can be imported from MATLAB were primarily used as simulation images. However, there is a steady increase in the flow of patterns validated by actual experiments following simulation studies. Thus, it is expected that denoising studies with improved validation accuracy can be performed when developing a simulation program or tool that can accurately simulate light microscopy systems.

2. Imaging techniques

As a method for acquiring light microscopic images, we organized basic and light sheet types. Fig. 5 shows the overall number of publications for each imaging technique. Among the 16 identified papers, 15 papers were published on the basic light microscopy system [21–33,35,36], and only one paper was related to light sheet microscopy [34]. Most studies have been conducted to reduce noise in images acquired using basic light microscopy systems. We believe that the fact that noise has been significantly reduced owing to the characteristics of recently developed equipment with improved functions has led to this research trend.

3. Noise reduction methods

Various methods have been used to reduce the noise in light microscopy images. Fig. 6 shows the overall number of publications for each noise reduction method. Among the 16 identified papers, seven transform-based modeling techniques were published and used the most, and the wavelet transform was mostly applied to light microscopic images with six papers [21,22,25-27,32,33]. Subsequently, TV- and NLM-based approaches were used for research in three papers [24,30,35], and techniques that improved conventional filtering methods were also performed in two studies [28,36]. In addition, the noise reduction method in machine learning using dictionary learning, compressed sensing (CS)-based technology, and approaches based on singular value decomposition and Gaussian Markov random field has been applied to light microscopic images [23,29,31,34].

Wavelet transform is the most widely used software technology for reducing noise in light microscopy images. The principle of wavelet transform is suitable for light microscopy systems because it is easy to analyze the characteristics of the signal and control the frequency and spatial components simultaneously. In general, a method for efficiently calculating sub-band coding using a fast wavelet transform is used. In addition, Wang et al. improved the accuracy of observing gene expression using the stationary wavelet transform, and the applicability of the mixed Poisson-Gaussian noise model was analyzed by Luisier et al. [21,26]. After such a wavelet transform has been actively used for light microscopy images, the most recent contourlet transform has been used as a denoising method [33]. The contourlet transform has the advantage of overcoming one of the biggest drawbacks of wavelets: the difficulty in expressing continuous images at edge points in a two-dimensional image. Because wavelet transform uses different sizes based on the multi-resolution structure, a smooth outline expression is not possible, which can be an issue during denoising. Yang et al. conducted a study to reduce the form of mixed Poisson-Gaussian noise generated in light microscopy images using an algorithm based on the contourlet domain. Thus, as an alternative to wavelet transform, a contourlet transform capable of preserving many edge areas generated while removing noise from a light microscopy image has been proposed [33].

4. Evaluation method

The acquired light microscopy images were divided into comparison and quantitative analysis methods. Fig. 7 shows the overall number of publications for each evaluation method. Among the 16 identified papers, 10 [21,24-27,29-33] and one [23] papers used comparison evaluation parameters and quantitative evaluation parameters for image analysis, respectively. The results of image analysis using both comparison and quantitative evaluation parameters have been published in five publications [22,28,34-36].

The easiest approach to quantitatively evaluate the noise level is to measure the standard deviation of the region of interest. However, it is important to simultaneously analyze the noise level and signal or contrast in all images, and the signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) evaluation parameters have been used in many studies. In particular, only the SNR and CNR were used in the study that measured the noise level using only quantitative evaluation methods.

However, when using software technology to reduce noise in all images, including light microscopic images, edge area preservation inevitably degrades. Since this situation implies that the noise level and edge area must be evaluated simultaneously when analyzing the resulting image, parameters for evaluating comparison or similarity are often used. Even in this systematic review, the comparison evaluation parameter was used in 10 of the most frequent studies. Among the parameters used, the mean error [24,25,27,32] and peak signal-to-noise ratio (PSNR) [25,26,29,30,33] were the most frequently applied evaluation parameters. In addition, the structural similarity index measure, which can complexly analyze the similarity of images and the overall structure, is also increasingly used as an evaluation parameter [31]. In particular, by analyzing the relationship between the imaging type and evaluation method, it was possible to derive the result that comparison evaluation parameters were mainly used in simulation research, and the quantitative evaluation method was mainly used for image evaluation using new equipment.

Recently, many studies have confirmed the quantitative evaluation results of these noise levels and have analyzed comparison evaluation results. In the results of the study using the two evaluations concurrently, the coefficient of variation (COV), which can mainly evaluate the noise level, and PSNR, which can be used as a comparison evaluation parameter, were used the most [22,36]. The COV is an evaluation parameter that can simultaneously check the noise level and signal more simply than the SNR measurement method. In addition, based on the most recently published paper by Kim et al., the credibility of the study was enhanced using no-reference-based evaluation parameters, except for the evaluation methods used in the analysis of the resulting image [36]. The no-reference-based evaluation parameter is being actively used in the medical imaging field, and we expect that it can be extended and applied to denoising studies of light microscopy images in the future.

Ⅳ. Conclusion

Noise reduction in light microscopic images is important for accurate physical and biological analysis of tissues. Software technologies for reducing such noise are widely used, and the usefulness of these algorithms has been proven through various quantitative and comparative evaluations. In this study, a systematic review of recent papers on quantitative and comparative evaluations of noise reduction algorithms in light microscopic images was conducted. Over 35 years, until 2020, 139 related papers were published, and 16 papers subject to review were analyzed in various forms.

As a result of systematic consideration, we demonstrated that studies related to image quality improvement through noise reduction of light microscopy images were steadily being validated by using simulations and actual experiments. In particular, research on the application of algorithms to images acquired with a basic light microscopic system has been the main focus, which confirms that there are many areas to be improved in terms of image quality. In addition, transform-based algorithms are widely used to improve the image quality of light microscope images, and the quality evaluation method also tends to be compared and quantitatively analyzed at the same time. In the future, we expect that light microscope image processing technology development and various evaluation methods will be presented by various researchers, and that various optical-based software technologies will be additionally considered.

Figure

Fig. 1.

Three types of noise models in light microscopic image

Fig. 2.

PRISMA flowchart showing the selection process used in this systematic review. A total of 139 recorded papers went through screening and eligibility processes, and the final 16 papers were reviewed.

Fig. 3.

Graph of the number of publications per year including the polynomial fitting curve.

Fig. 4.

Of the 16 papers included in the final classification, three major categories of imaging types were investigated. (a) Graph of the number of publications as a function of imaging type for light microscopy images. (b) Total number of papers with respect to publication year.

Fig. 5.

Of the 16 papers included in the final classification, two major categories of imaging techniques were investigated. (a) Graph of the number of publications as a function of imaging techniques in light microscopic images. (b) Total number of papers with respect to publication year.

Fig. 6.

Of the 16 papers included in the final classification, six major categories of noise reduction methods were investigated. (a) Graph of the number of publications on various noise reduction methods in light microscopy images. (b) Total number of papers with respect to publication year.

Fig. 7.

Table

Table 1.

Search strategies using the MEDLINE-EBSCO, EMBASE, and Cochrane Library search engines

Reference

Davidson MW. Pioneers in optics: Zacharias Janssen and Johannes Kepler. Microscopy Today. 2009;17: 44-6.
Davidson MW. Robert Hooke. Labmedicine. 2010;41: 180-2.
Wimmer W. Carl Zeiss, Ernst Abbe, and advances in the light microscope. Microscopy Today. 2017;25: 50-4.
Huang B, Babcock H, Zhuang X. Breaking the diffraction barrier: Super-resolution imaging of cells. Cell. 2010;143:1047-58.
Dedecker P, Hofkens J, Hotta JI. Left diffraction-unlimited optical microscopy. Materialstoday. 2008;11: 12-21.
Adhikari S, Moscatelli J, Smith EM, Banerjee C, Puchner EM. Single-molecule localization microscopy and tracking with red-shifted states of conventional BODIPY conjugates in living cells. Nature Communications. 2019;10.
Moerner WE, Kador L. Optical detection and spectroscopy of single molecules in a solid. Physics Review Letters. 1989;62L2535-8.
Moerner WE, Orrit M. Illuminating single molecules in condensed matter. Science. 1999;283:1670-6.
Meiniel W, Olivo-Marin JC, Angelini ED. Denoising of microscopy images: A review of the state-of-the-art, and a new sparsity-based method. IEEE Transactions on Image Processing. 2018;27:3842-56.
Chambolle A. An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision. 2004;20:89-97.
Buades A, Coll B, Morel JM. A review of image denoising algorithms, with a new one. Multiscale Modeling & Simulation. 2005;4:490-530.
Dautov CP, Ozerdem MS. Introduction to Wavelets and their applications in signal denoising. Bitlis Eren University Journal of Science and Technology. 2018;8:1-10.
Srivastava M, Anderson CL, Freed JH. A new wavelet denoising method for selecting decomposition levels and noise thresholds. IEEE Access. 2016;4: 3862-77.
Wahab MF, O'Haver TC. Wavelet transforms in separation science for denoising and peak overlap detection. Journal of Separation Science. 2020;43: 1998-2010.
Zhang M, Lu C, Liu C. Improved double-threshold denoising method based on the wavelet transform. OSA Continuum. 2019;2:2328-42.
Nowak RD. Wavelet-based rician noise removal for magnetic resonance imaging. IEEE Transactions on Image Processing. 1999;8:1408-19.
Zhang Y, Ding W, Pan Z, Qin J. Improved wavelet threshold for image de-noising. Frontiers in Neuroscience. 2019;13.
Donoho DL. De-noising by soft-thresholding. IEEE Transactions on Information Theory. 1995;41: 613-27.
AlAsadi AHH. Contourlet transform based method for medical image denoising. Journal of AL-Qadisiyah for Computer Science and Mathematics. 2015;7:1-7.
Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. BMJ. 2009;339:332-6.
Wang XH, Istepanian RSH, Song YH. Microarray image enhancement by denoising using stationary wavelet transform. IEEE Transactions on Nanobioscience. 2003;2:184-9.
Gopalappa C, Das TK, Enkemann S, Eschrich S. Removal of hybridization and scanning noise from microarrays. IEEE Transactions on Nanobioscience. 2009;8:210-8.
Van't Hoff M, Reuter M, Dryden DTF, Oheim M. Screening by imaging: Scaling up single-DNA-molecule analysis with a novel parabolic VA-TIRF reflector and noise-reduction techniques. Physical Chemistry Chemical Physics. 2009;11:7713-20.
Boulanger J, Kervrann C, Bouthemy P, Elbau P, Sibarita JB, Salamero J. Patch-based nonlocal functional for denoising fluorescence microscopy image sequences. IEEE Transactions on Medical Imaging. 2010;29:442-54.
Howlader T, Chaubey YP. Noise reduction of cDNA microarray images using complex wavelets. IEEE Transactions on Image Processing. 2010;19:1953-67.
Luisier F, Blu T, Unser M. Image denoising in mixed poisson–gaussian noise. IEEE Transactions on Image Processing. 2011;20:696-708.
Chang CW, Mycek MA. Total variation versus wavelet-based methods for image denoising in fluorescence lifetime imaging microscopy. Journal of Biophotonics. 2013;249:449-57.
Das DK, Chakraborty C, Mitra B, Maiti AK, Ray AK. Quantitative microscopy approach for shape-based erythrocytes characterization in anaemia. Journal of Microscopy. 2013;249:136-49.
Giryes R, Elad M. Sparsity based poisson denoising with dictionary learning. IEEE Transactions on Image Processing. 2014;23:5057-69.
Lanza A, Morigi S, Sgallari F, Wen YW. Image restoration with Poisson–Gaussian mixed noise. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization. 2014;2:12-24.
Yoon JW. Statistical denoising scheme for single molecule fluorescence microscopic images. Biomedical Signal Processing and Control. 2014;10:11-20.
Bernas T, Starosolski R, Wojcicki R. Application of detector precision characteristics for the denoising of biological micrographs in the wavelet domain. Biomedical Signal Processing and Control. 2015;19:1-28.
Yang S, Lee BU. Poisson-Gaussian noise reduction using the hidden markov model in contourlet domain for fluorescence microscopy images. PLoS One. 2015;10.
Bai C, Liu C, Jia H, Peng T, Min J, Lei M, Yu X, Yao B. Compressed blind deconvolution and denoising for complementary beam subtraction light-sheet fluorescence microscopy. IEEE Transactions on Biomedical Engineering. 2019;66:2979-89.
Xiao C, Smith ZJ, Chu K. Simultaneous recovery of both bright and dim structures from noisy fluorescence microscopy images using a modified TV constraint. Journal of Microscopy. 2019;275:24-35.
Kim JY, Lee Y. Preliminary study of improved median filter using adaptively mask size in light microscopic image. Microscopy. 2020;69:31-6.