Comparative analysis of modifications of U-Net neuronal network architectures in medical image segmentation

Cover Image

Cite item

Full Text

Abstract

Data processing methods based on neural networks are becoming increasingly popular in medical diagnostics. They are most commonly used to evaluate medical images of human organs using computed tomography, magnetic resonance imaging, ultrasound, and other non invasive diagnostic methods. Disease diagnosis involves solving the problem of medical image segmentation, i.e. finding groups (regions) of pixels that characterize specific objects in the image. The U-Net neural network architecture developed in 2015 is one of the most successful tools to solve this issue. This review evaluated various modifications of the classic U-net architecture. The papers considered were divided into several key categories, such as modifications of the encoder and decoder; use of attention blocks; combination with elements of other architectures; methods for introducing additional attributes; transfer learning; and approaches for processing small sets of real world data. Different training sets with the best parameters found in the literature were evaluated (Dice similarity score; Intersection over Union; overall accuracy, etc.). A summary table was developed showing types of images evaluated and abnormalities detected. Promising directions for further modifications to improve the quality of the segmentation are identified. The results can be used to detect diseases, especially cancer. Intelligent medical assistants can implement the presented algorithms.

About the authors

Anastasia M. Dostovalova

MIREA — Russian Technological University; Federal Research Center Computer Science and Control of the Russian Academy of Sciences

Author for correspondence.
Email: adostovalova@frccsc.ru
ORCID iD: 0009-0004-9420-4182
SPIN-code: 3784-0791
Russian Federation, Moscow; Moscow

Andrey K. Gorshenin

MIREA — Russian Technological University; Federal Research Center Computer Science and Control of the Russian Academy of Sciences

Email: agorshenin@frccsc.ru
ORCID iD: 0000-0001-8129-8985
SPIN-code: 1512-3425

Dr. Sci. (Physics and Mathematics), Assistant Professor

Russian Federation, Moscow; Moscow

Julia V. Starichkova

MIREA — Russian Technological University

Email: starichkova@mirea.ru
ORCID iD: 0000-0003-1804-9761
SPIN-code: 3001-6791

Cand. Sci. (Engineering), Assistant Professor

Russian Federation, Moscow

Kirill M. Arzamasov

MIREA — Russian Technological University; Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies

Email: ArzamasovKM@zdrav.mos.ru
ORCID iD: 0000-0001-7786-0349
SPIN-code: 3160-8062

MD, Cand. Sci. (Medicine), Head of Medical Informatics, Radiomics and Radiogenomics Department

Russian Federation, Moscow; Moscow

References

  1. Shen D, Wu G, Suk HI. Deep Learning in Medical Image Analysis. Annual Review of Biomedical Engineering. 2017;19:221–248. doi: 10.1146/annurevbioeng071516-044442
  2. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer Assisted Intervention (MICCAI) 2015. 2015:9351. doi: 10.1007/978-3-319-24574-4_28
  3. Milletari F, Navab N, Ahmadi SA. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Fourth International Conference on 3D Vision (3DV). 2016:565–571. doi: 10.48550/arXiv.1606.04797
  4. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille A. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2017;40(4):834–848. doi: 10.1109/TPAMI.2017.2699184
  5. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely Connected Convolutional Networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017:2261–2269. doi: 10.1109/CVPR.2017.243
  6. He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. IEEE International Conference on Computer Vision (ICCV). 2017:2980–2988. doi: 10.1109/ICCV.2017.322
  7. Khalal DM., Azizi H, Maalej N. Automatic segmentation of kidneys in computed tomography images using U-Net. Cancer/Radiothérapie. 2023;27(2):109–114. doi: 10.1016/j.canrad.2022.08.004
  8. Bernardo Gois FN, Lobo Marques JA. Segmentation of CT-Scan Images Using UNet Network for Patients Diagnosed with COVID-19. Computerized Systems for Diagnosis and Treatment of COVID-192023. 2023:29–44. doi: 10.1007/978-3-031-30788-1_3
  9. Sarsembayeva T, Shomanov A, Sarsembayev M, et al. UNet Model for Segmentation of COPD Lung Lesions on Computed Tomography Images. Proceedings of the 7th International Conference on Digital Technologies in Education, Science and Industry (DTESI 2022). 2022. Available at: https://ceurws.org/Vol-3382/Short5.pdf. Accessed: November 9, 2024.
  10. Çiçek Ö, Abdulkadir A, Lienkamp S, Brox T, Ronneberger O. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. Medical Image Computing and Computer-Assisted Intervention — MICCAI 2016. 2016:424–432. doi: 10.1007/978-3-319-46723-8_4
  11. Pantovic A, Ollivier I, Essert C. 2D and 3D-UNet for segmentation of SEEG electrode contacts on post operative CT scans. Medical Imaging 2022: Image Guided Procedures, Robotic Interventions, and Modeling. 2022. doi: 10.1117/12.2606538
  12. Han X, Wu X, Wang S, et al. Automated segmentation of liver segment on portal venous phase MR images using a 3D convolutional neural network. Insights Imaging. 2022;13(26). doi: 10.1186/s13244-022-01163-1
  13. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. 2018:3–11. doi: 10.1007/978-3-030-00889-5_1
  14. Yu C, Wang Y, Tang C, Feng W, Lv J. EU-Net: Automatic U-Net neural architecture search with differential evolutionary algorithm for medical image segmentation. Computers in Biology and Medicine. 2023;167:107579. doi: 10.1016/j.compbiomed.2023.107579
  15. Weng Y, Zhou T, Li Y, Qiu X. NAS-Unet: Neural Architecture Search for Medical Image Segmentation. IEEE Access. 2019;7:44247–44257. doi: 10.1109/ACCESS.2019.2908991
  16. Huang H, Lin L, Tong R, et al. UNet 3+: A Full Scale Connected UNet for Medical Image Segmentation. ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2020:1055–1059. doi: 10.1109/ICASSP40776.2020.9053405
  17. Li C, Bagher Ebadian H, Sultan RI, et al. A new architecture combining convolutional and transformer based networks for automatic 3D multi organ segmentation on CT images. Med Phys. 2023;50(11):6990–7002. doi: 10.1002/mp.16750
  18. Müller D, Soto Rey I, Kramer F. Towards a guideline for evaluation metrics in medical image segmentation. BMC Research Notes. 2022;15(210). doi: 10.1186/s13104-022-06096-y
  19. Alberg AJ, Park JW, Hager BW, Brock MV, Diener-West M. The use of «overall accuracy» to evaluate the validity of screening or diagnostic tests. Journal of General Internal Medicine. 2004;19:460–465. doi: 10.1111/j.1525-1497.2004.30091.x
  20. Soler L, Hostettler A, Agnus V, et al. 3D image reconstruction for comparison of algorithm database: A patient specific anatomical and medical image database. IRCAD. 2010. Available at: https://www.sop.inria.fr/geometrica/events/wam/abstractircad.pdf. Accessed: November 9, 2024.
  21. Löffler M, Sekuboyina A, Jakob A, et al. A Vertebral Segmentation Dataset with Fracture Grading. Radiology: Artificial Intelligence. 2020;2(4). doi: 10.1148/ryai.2020190138
  22. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing. 2004;13(4):600–612. doi: 10.1109/TIP.2003.819861
  23. Kavur AE, Gezer NS, Barıs M, et al. CHAOS Challenge – combined (CT-MR) healthy abdominal organ segmentation. Medical Image Analysis. 2021;69:101950. doi: 10.1016/j.media.2020.101950
  24. Bilic P, Christ P, Li HB, et al. The Liver Tumor Segmentation Benchmark (LiTS). Medical Image Analysis. 2023;84:102680. doi: 10.1016/j.media.2022.102680
  25. Petrusca L, Cattin P, De Luca V, et al. Hybrid ultrasound/magnetic resonance simultaneous acquisition and image fusion for motion monitoring in the upper abdomen. Investigative Radiology. 2013;48(5):333–340. doi: 10.1097/RLI.0b013e31828236c3
  26. Jun M, Cheng G, Yixin W, et al. Covid-19 CT lung and infection segmentation dataset. Zenodo. 2020. Available at: https://zenodo.org/records/3757476#.YLov8vkzaUk. Accessed: November 9, 2024.
  27. Morozov SP, Andreychenko AE, Blokhin IA, et al. MosMedData: data set of 1110 chest CT scans performed during the COVID-19 epidemic. Digital Diagnostics. 2020;1(1):49–59. doi: 10.17816/DD46826
  28. Roth HR, Oda H, Hayashi Y, et al. Hierarchical 3D fully convolutional networks for multi organ segmentation. ArXiv. 2017. Available at: https://arxiv.org/abs/1704.06382v1. Accessed: November 9, 2024.
  29. Roth H, Farag A, Turkbey EB, et al. Data from Pancreas-CT. Data From Pancreas-CT (Version 2) [Data set]. The Cancer Imaging Archive. 2016. doi: 10.7937/K9/TCIA.2016.tNB1kqBU
  30. Heimann T, Styner M, van Ginneken B. 3D Segmentation in the Clinic: A Grand Challenge. MICCAI 2007, the 10th Intel Conf. on Medical Image Computing and Computer Assisted Intervention. 2007:7–15. Available at: https://www.diagnijmegen.nl/publications/ginn07/. Accessed: November 9, 2024.
  31. Suckling J. The Mammographic Image Analysis Society Digital Mammogram Database. International Congress Series. 1994:375–378. Available at: http://peipa.essex.ac.uk/info/mias.html. Accessed: November 9, 2024.
  32. WHO Director-General’s opening remarks at the media briefing on COVID-19 — 11 March 2020 [Internet]. 2020. Available at: https://www.who.int/directorgeneral/speeches/detail/whodirectorgeneralsopeningremarksatthemediabriefingoncovid-1911march2020. Accessed: November 9, 2024.
  33. Landman B, Xu Z, Igelsias J, et al. Miccai multi atlas labeling beyond the cranial vault–workshop and challenge. Proceedings of the MICCAI Multi Atlas Labeling Beyond Cranial Vault — Workshop Challenge. 2015;5:12.
  34. Simpson AL, Antonelli M, Bakas S, et al. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. ArXiv. 2019. doi: 10.48550/arXiv.1902.09063
  35. Gutman D, Codella NCF, Celebi E, et al. Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC). ArXiv. 2016. doi: 10.48550/arXiv.1605.01397
  36. Jha D, Smedsrud PH, Riegler MA, et al. Kvasir-SEG: A Segmented Polyp Dataset. MultiMedia Modeling. 2020;11962:451–462. doi: 10.1007/978-3-030-37734-2_37
  37. Bernal J, Sánchez FJ, Fernández Esparrach G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized Medical Imaging and Graphics. 2015;43:99–111. doi: 10.1016/j.compmedimag.2015.02.007
  38. Grove O, Berglund AE, Schabath MB, et al. Quantitative Computed Tomographic Descriptors Associate Tumor Shape Complexity and Intratumor Heterogeneity with Prognosis in Lung Adenocarcinoma. PLOS ONE. 2015;10(3):e0118261. doi: 10.1371/journal.pone.0118261
  39. Heller N, Sathianathen N, Kalapara A, et al. The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes. ArXiv. 2019:13. doi: 10.48550/arXiv.1904.00445
  40. Ji Y, Bai H, Yang J, et al. AMOS: A Large Scale Abdominal Multi Organ Benchmark for Versatile Medical Image Segmentation. ArXiv. 2022. doi: 10.48550/arXiv.2206.08023
  41. Lemay A, Gros C, Zhuo Z, et al. Multiclass Spinal Cord Tumor Segmentation on MRI with Deep Learning. ArXiv. 2021. doi: 10.48550/arXiv.2012.12820
  42. Ali MAS, Misko O, Salumaa SO, et al. Evaluating Very Deep Convolutional Neural Networks for Nucleus Segmentation from Brightfield Cell Microscopy Images. SLAS Discovery. 2021;26(9):1125–1137. doi: 10.1177/24725552211023214
  43. Gibson E, Giganti F, Hu Y, et al. Automatic Multi Organ Segmentation on Abdominal CT With Dense V-Networks. IEEE Transactions on Medical Imaging. 2018;37(8):1822–1834. doi: 10.1109/TMI.2018.2806309
  44. Jimenez del Toro O, Müller H, Krenn M, et al. Cloud Based Evaluation of Anatomical Structure Segmentation and Landmark Detection Algorithms: VISCERAL Anatomy Benchmarks. IEEE Transactions on Medical Imaging. 2016;35(11):2459–2475. doi: 10.1109/TMI.2016.2578680
  45. Regan EA, Hokanson JE., Murphy JR, et al. Genetic Epidemiology of COPD (COPDGene) Study Design. COPD: Journal of Chronic Obstructive Pulmonary Disease. 2010;7(1):32–43. doi: 10.3109/15412550903499522
  46. Litjens G, Toth R, van de Ven W, et al. Evaluation of prostate segmentation algorithms for MRI: The PROMISE12 challenge. Medical Image Analysis. 2014;18(2):359–373. doi: 10.1016/j.media.2013.12.002
  47. Xiong Z, Xia Q, Hu Z, et al. A global benchmark of algorithms for segmenting the left atrium from late gadolinium enhanced cardiac magnetic resonance imaging. Medical Image Analysis. 2021;67:101832. doi: 10.1016/j.media.2020.101832
  48. Landman B, Xu Z, Igelsias J, et al. 2015 MICCAI multi atlas labeling beyond the cranial vault–workshop and challenge. MICCAI Multi Atlas Labeling Beyond Cranial Vault — Workshop Challenge. 2015;5:12.
  49. Zhuang X, Shen J. Multi scale patch and multi modality atlases for whole heart segmentation of MRI. Medical Image Analysis. 2016;31:77–87. doi: 10.1016/j.media.2016.02.006
  50. Campello VM, Gkontra P, Izquierdo C, et al. Multi Centre, Multi Vendor and Multi Disease Cardiac Segmentation: The M&Ms Challenge. IEEE Transactions on Medical Imaging. 2021;40(12):3543–3554. doi: 10.1109/TMI.2021.3090082
  51. Silva J, Histace A, Romain O, Dray X, Granado B. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. International Journal of Computer Assisted Radiology and Surgery. 2014;9:283–293. doi: 10.1007/s11548-013-0926-3
  52. Trikha S, Turnbull A, Morris R, Anderson D, Hossain P. The journey to femtosecond laser assisted cataract surgery: New beginnings or a false dawn? Eye. 2013;27(4):461–473. doi: 10.1038/eye.2012.293
  53. Xiong Z, Xia Q, Hu Z, et al. A global benchmark of algorithms for segmenting the left atrium from late gadolinium enhanced cardiac magnetic resonance imaging. Medical Image Analisys. 2021;67:101832. doi: 10.1016/j.media.2020.101832
  54. Bernard O, Lalande A, Zotti C, et al. Deep Learning Techniques for Automatic MRI Cardiac Multi Structures Segmentation and Diagnosis: Is the Problem Solved? IEEE Transactions on Medical Imaging. 2018;37(11):2514–2525. doi: 10.1109/TMI.2018.2837502
  55. Li P, Wang S, Li T, et al. A Large Scale CT and PET/CT Dataset for Lung Cancer Diagnosis (Lung-PET-CT-Dx) [Data set]. The Cancer Imaging Archive. 2020. doi: 10.7937/TCIA.2020.NNC2-0461
  56. Clark K, Vendt B, Smith K, et al. The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository. Journal of Digital Imaging. 2013;26:1045–1057. doi: 10.1007/s10278-013-9622-7
  57. Xu Z, Jia Z, Sun J, Dong W, Li Z. DO-U-Net: Improved U-Net Model for CT Image Segmentation using DBB and Octave Convolution. Proceedings of the 2023 International Conference on Computer, Vision and Intelligent Technology (ICCVIT ‘23). 2023:1–8. doi: 10.1145/3627341.3630403
  58. Ayalew Y, Fante K, Aliy M. Modified U-Net for liver cancer segmentation from computed tomography images with a new class balancing method. BMC Biomedical Engineering. 2021;3(4). doi: 10.1186/s42490-021-00050-y
  59. Guan S, Khan AA, Sikdar S, Chitnis PV. Fully Dense UNet for 2-D Sparse Photoacoustic Tomography Artifact Removal. IEEE Journal of Biomedical and Health Informatics. 2020;24(2):568–576. doi: 10.1109/JBHI.2019.2912935
  60. Özcan F, Uçan ON, Karaçam S, Tunçman D. Fully Automatic Liver and Tumor Segmentation from CT Image Using an AIM-UNet. Bioengineering. 2023;10(2). doi: 10.3390/bioengineering10020215
  61. Ansari MY, Yang Y, Meher PK, Dakua SP. Dense-PSP-UNet: A neural network for fast inference liver ultrasound segmentation. Computers in Biology and Medicine. 2023;153:106478. doi: 10.1016/j.compbiomed.2022.106478
  62. Omarov B, Tursynova A, Postolache O, et al. Modified UNet Model for Brain Stroke Lesion Segmentation on Computed Tomography Images. Computers, Materials and Continua. 2022;71(3):4701–4717. doi: 10.32604/cmc.2022.020998
  63. Mizusawa S, Sei Y, Orihara R, Ohsuga A. Computed tomography image reconstruction using stacked U-Net. Computerized Medical Imaging and Graphics. 2021;90:101920. doi: 10.1016/j.compmedimag.2021.101920
  64. Golts A, Khapun D, Shats D, Shoshan Y, Gilboa Solomon F. An Ensemble of 3D U-Net Based Models for Segmentation of Kidney and Masses in CT Scans. Kidney and Kidney Tumor Segmentation (KiTS 2021). 2022;13168:103–115. doi: 10.1007/978-3-030-98385-7_14
  65. Araújo JDL, da Cruz LB, Diniz JOB, et al. Liver segmentation from computed tomography images using cascade deep learning. Computers in Biology and Medicine. 2022;140:105095. doi: 10.1016/j.compbiomed.2021.105095
  66. Koirala CP, Mohapatra S, Gosai A, Schlaug G. Automated Ensemble Based Segmentation of Adult Brain Tumors: A Novel Approach Using the BraTS AFRICA Challenge Data. ArXiv. 2023. doi: 10.48550/arXiv.2308.07214
  67. Li Z, Zhu Q, Zhang L, et al. A deep learning based self adapting ensemble method for segmentation in gynecological brachytherapy. Radiation Oncology. 2022;17(152). doi: 10.1186/s13014-022-02121-3
  68. Woo S, Park J, Lee J-Y, Kweon IS. CBAM: Convolutional Block Attention Module. Proceedings of the European conference on computer vision (ECCV). 2018:3–19. doi: 10.48550/arXiv.1807.06521
  69. Nazir S, Zheng R, Zheng Y, Dong-Ye C. Improved 3D U-Net for COVID-19 Chest CT Image Segmentation. Scientific Programming. 2021;2021(9999368):9. doi: 10.1155/2021/9999368
  70. Salehi SSM, Erdogmus D, Gholipour A. Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks. Machine Learning in Medical Imaging. 2017;10541:379–387. doi: 10.1007/978-3-319-67389-9_44
  71. Oktay O, Schlemper J, Folgoc LL, et al. Attention U-Net: Learning Where to Look for the Pancreas. ArXiv. 2018. doi: 10.48550/arXiv.1804.03999
  72. Agarap AF. Deep Learning using Rectified Linear Units (ReLU). ArXiv. 2018:7. doi: 10.48550/arXiv.1803.08375
  73. Wu J, Zhou S, Zuo S, et al. U-Net combined with multi scale attention mechanism for liver segmentation in CT images. BMC Medical Informatics and Decision Making. 2021;21(283). doi: 10.1186/s12911-021-01649-w
  74. Zhang L, Liu Y, Li Z, Li D. Epa unet:automatic Segmentation of Liver and Tumor in Ct Images Based on Residual U-net and Efficient Multiscale Attention Methods. Research Square. 2023. doi: 10.21203/rs.3.rs-3273964/v1
  75. Zarbakhsh P. Spatial Attention Mechanism and Cascade Feature Extraction in a U-Net Model for Enhancing Breast Tumor Segmentation. Applied Sciences. 2023;13(15):8758. doi: 10.3390/app13158758
  76. Subhan Akbar A, Fatichah C, Suciati N. UNet3D with Multiple Atrous Convolutions Attention Block for Brain Tumor Segmentation. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. 2022:182–193. doi: 10.1007/978-3-031-08999-2_14
  77. Yu Z, Han S, Song Z. 3D Medical Image Segmentation based on multi scale MPU-Net. ArXiv. 2023. doi: 10.48550/arXiv.2307.05799
  78. Xingfei F, Chaobing H. CAE-UNet: An Effective Automatic Segmentation Model for CT Images of COVID-19. 2022 6th International Conference on Communication and Information Systems (ICCIS). 2022:113–117. doi: 10.1109/ICCIS56375.2022.9998131
  79. He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016:770–778. doi: 10.1109/cvpr.2016.90
  80. Hatamizadeh A, Tang Y, Nathet V, et al. U-NETR: Transformers for 3D Medical Image Segmentation. 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2022:1748–1758. doi: 10.1109/WACV51458.2022.00181
  81. Eskandari S, Lumpp J. Inter Scale Dependency Modeling for Skin Lesion Segmentation with Transformer based Networks. ArXiv. 2023. doi: 10.48550/arXiv.2310.13727
  82. Shi X, Chen Z, Wang H, et al. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Neural Information Processing Systems. 2015. doi: 10.48550/arXiv.1506.04214
  83. Pham TH, Li X, Nguyen KD. SeU Net Trans: A Simple yet Effective UNet-Transformer Model for Medical Image Segmentation. ArXiv. 2023. doi: 10.48550/arXiv.2310.09998
  84. Ghofrani F, Behnam H, Motlagh HDK. Liver Segmentation in CT Images Using Deep Neural Networks. 2020 28th Iranian Conference on Electrical Engineering (ICEE). 2020:1–6. doi: 10.1109/ICEE50131.2020.9260809
  85. Diakogiannis FI, Waldner F, Caccetta P, Wuet C, et al. ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing. 2020;16(2):94–114. doi: 10.1016/j.isprsjprs.2020.01.013
  86. Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD. Doubleu net: DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation. IEEE 33rd International symposium on computer based medical systems (CBMS). 2020:558–564. doi: 10.1109/CBMS49503.2020.00111
  87. Lee HH, Bao S, Huo Y, Landman BA. 3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation. International Conference on Learning Representations. 2023. doi: 10.48550/arXiv.2209.15076
  88. Liang J, Yang C, Zhong J, Ye X. BTSwin-U-Net: 3D U-shaped Symmetrical Swin Transformer based Network for Brain Tumor Segmentation with Self supervised Pre training. Neural Processing Letters. 2022;55:3695–3713. doi: 10.1007/s11063-022-10919-1
  89. Alalwan N, Abozeid A, ElHabshy AA, Alzahrani A. Efficient 3D Deep Learning Model for Medical Image Semantic Segmentation. Alexandria Engineering Journal. 2021;60(1):1231–1239. doi: 10.1016/j.aej.2020.10.046
  90. Lemay A, Gros C, Vincent O, et al. Benefits of Linear Conditioning with Metadata for Image Segmentation. ArXiv. 2021. doi: 10.48550/arXiv.2102.09582
  91. Du R, Vardhanabhuti V. 3D-RADNet: Extracting labels from DICOM metadata for training general medical domain deep 3D convolution neural networks. International Conference on Medical Imaging with Deep Learning. 2020;121:174–192. Available at: https://proceedings.mlr.press/v121/du20a/du20a.pdf. Accessed: November 9, 2024.
  92. Plutenko I, Papkov M, Palo K, Parts L, Fishman D. Metadata Improves Segmentation Through Multitasking Elicitation. Domain Adaptation and Representation Transfer. 2023:147–155. doi: 10.1007/978-3-031-45857-6_15
  93. Jiang J, Peng Y, Hou Q, Wang J. MDCF_Net: A Multi dimensional hybrid network for liver and tumor segmentation from CT. Biocybernetics and Biomedical Engineering. 2023;43(2):494–506. doi: 10.1016/j.bbe.2023.04.004
  94. Fu T, Yu Q, Lao H, Liu P, Wan S. Traffic Safety Oriented Multi Intersection Flow Prediction Based on Transformer and CNN. Security and Communication Networks. 2023:1–13. doi: 10.1155/2023/1363639
  95. Chen X, Wei X, Tang M, et al. Liver segmentation in CT imaging with enhanced mask region based convolutional neural networks. Annals of Translational Medicine. 2021;9(24):1768. doi: 10.21037/atm-21-5822
  96. Ernst P, Chatterjee S, Rose G, Nürnberger A. Primal Dual U-Net for Sparse View Cone Beam Computed Tomography Volume Reconstruction. ArXiv. 2022. doi: 10.48550/arXiv.2205.07866
  97. Koehler G, Wald T, Ulrichet C, et al. RecycleNet: Latent Feature Recycling Leads to Iterative Decision Refinement. ArXiv. 2023. doi: 10.48550/arXiv.2309.07513
  98. Jafari M, Auer D, Francis S, Garibaldi J, Chen X. DRU-net: An Efficient Deep Convolutional Neural Network for Medical Image Segmentation. 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). 2020:1144–1148. doi: 10.48550/arXiv.2004.13453
  99. Heker M, Ben Cohen A, Greenspan H. Hierarchical Fine Tuning for joint Liver Lesion Segmentation and Lesion Classification in CT. 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2019:895–898. doi: 10.1109/EMBC.2019.8857127
  100. Matovinovic IZ, Loncaric S, Lo J, Heisler M, Sarunic M. Transfer Learning with U-Net type model for Automatic Segmentation of Three Retinal Layers In Optical Coherence Tomography Images. 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA). 2019:49–53. doi: 10.1109/ISPA.2019.8868639
  101. Kora P, Ooi CP, Faust O, et al. Transfer learning techniques for medical image analysis: A review. Biocybernetics and Biomedical Engineering. 2022;42(1):79–107. doi: 10.1016/j.bbe.2021.11.004
  102. Humpire Mamani GE, Jacobs C, Prokop M, van Ginneken B, Lessmann N. Transfer learning from a sparsely annotated dataset of 3D medical images. ArXiv. 2023. doi: 10.48550/arXiv.2311.05032
  103. Messaoudi H, Belaid A, Salem DB, Conze P-H. Cross dimensional transfer learning in medical image segmentation with deep learning. Medical Image Analysis. 2023;88:102868. doi: 10.1016/j.media.2023.102868
  104. Tan M, Le Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. International conference on machine learning (PMLR). 2019:6105–6114. doi: 10.48550/arXiv.1905.11946
  105. Hong Y, Mao X, Hui Q. et al. Automatic liver and tumor segmentation based on deep learning and globally optimized refinement. Applied Mathematics A Journal of Chinese Universities. 2021;36:304–316. doi: 10.1007/s11766-021-4376-3
  106. Wang H, Li X. Towards Generic Semi Supervised Framework for Volumetric Medical Image Segmentation. ArXiv. 2023. doi: 10.48550/arXiv.2310.11320
  107. Wang J, Chen C. Unsupervised Adaptation of Polyp Segmentation Models via Coarse-to-Fine Self-Supervision. Information Processing in Medical Imaging. 2023:250–262. doi: 10.1007/978-3-031-34048-2_20
  108. Wang T, Huang Z, Wu J, Cai Y, Li Z. Semi Supervised Medical Image Segmentation with Co-Distribution Alignment. Bioengineering. 2023;10(7):869. doi: 10.3390/bioengineering10070869
  109. Vasilev YA, Bobrovskaya TM, Arzamasov KM, et al. Medical datasets for machine learning: fundamental principles of standartization and systematization. Manager Zdravoochranenia. 2023(4):28–41. doi: 10.21045/1811-0185-2023-4-28-41
  110. Kokina DYu, Gombolevskiy VA. Arzamasov KM, Andreychenko АE, Morozov SP Possibilities and limitations of using machine text processing tools in Russian radiology reports. Digital Diagnostics. 2022;3(4):374–383. doi: 10.17816/DD101099
  111. Ronzhin LV, Astanin PA, Kokina DYu, et al Semantic analysis methods in the system for authomated marking of the unstructured radiological chest examination protocols. Social’nye aspekty zdorov’a naselenia. 2023;69(1):12. doi: 10.21045/2071-5021-2023-69-1-12
  112. Tomashevskaya VS, Yakovlev DA. Research of unstructured data interpretation problems. Russian Technological Journal. 2021;9(1):7–17. doi: 10.32362/2500-316X-2021-9-1-7-17
  113. Protonotarios N, Katsamenis I, Sykiotis S, et al. A few-shot U-Net deep learning model for lung cancer lesion segmentation via PET/CT imaging. Biomedical Physics and Engineering Express. 2022;8:025019. doi: 10.1088/2057-1976/ac53bd
  114. Voulodimos A, Protopapadakis E, Katsamenis I, Doulamis A, Doulamis N. A Few Shot U-Net Deep Learning Model for COVID-19 Infected Area Segmentation in CT Images. Sensors. 2021;21(6):2215. doi: 10.3390/s21062215
  115. Zhao G, Zhao H. One Shot Image Segmentation with U-Net. Journal of Physics: Conference Series. 2021;1848(1):012113. doi: 10.1088/1742-6596/1848/1/012113

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Supplement 1. Ways to modify the U-Net architecture
Download (28KB)
Indexing metadata
3. Fig. 1. Classic U-Net architecture proposed in 2015 and the main categories of its modification methods.

Download (308KB)
Indexing metadata
4. Fig. 2. Segmentation tasks categorized by the availability and type of training data.

Download (126KB)
Indexing metadata
5. Fig. 3. Spatial attention block positioned between encoder elements [75].

Download (178KB)
Indexing metadata
6. Fig. 4. Architecture integrating transformer blocks into the U-Net framework [81].

Download (333KB)
Indexing metadata
7. Fig. 5. Combined U-Net and transformer-based architecture [83].

Download (342KB)
Indexing metadata
8. Fig. 6. Structural blocks of the U-Net architecture [97].

Download (105KB)
Indexing metadata
9. Fig. 7. Ratios of labeled and unlabeled data in network training and testing: (a) semi-supervised learning (SSL), (b) unsupervised domain adaptation (UDA), and (c) semi-supervised domain generalization (SemiDG) [106].

Download (196KB)
Indexing metadata
10. Fig. 8. A&D framework [106].

Download (389KB)
Indexing metadata
11. Fig. 9. Dual-network architecture trained on datasets with heterogeneous class representation [108].

Download (405KB)
Indexing metadata

Copyright (c) 2024 Eco-Vector

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Согласие на обработку персональных данных с помощью сервиса «Яндекс.Метрика»

1. Я (далее – «Пользователь» или «Субъект персональных данных»), осуществляя использование сайта https://journals.rcsi.science/ (далее – «Сайт»), подтверждая свою полную дееспособность даю согласие на обработку персональных данных с использованием средств автоматизации Оператору - федеральному государственному бюджетному учреждению «Российский центр научной информации» (РЦНИ), далее – «Оператор», расположенному по адресу: 119991, г. Москва, Ленинский просп., д.32А, со следующими условиями.

2. Категории обрабатываемых данных: файлы «cookies» (куки-файлы). Файлы «cookie» – это небольшой текстовый файл, который веб-сервер может хранить в браузере Пользователя. Данные файлы веб-сервер загружает на устройство Пользователя при посещении им Сайта. При каждом следующем посещении Пользователем Сайта «cookie» файлы отправляются на Сайт Оператора. Данные файлы позволяют Сайту распознавать устройство Пользователя. Содержимое такого файла может как относиться, так и не относиться к персональным данным, в зависимости от того, содержит ли такой файл персональные данные или содержит обезличенные технические данные.

3. Цель обработки персональных данных: анализ пользовательской активности с помощью сервиса «Яндекс.Метрика».

4. Категории субъектов персональных данных: все Пользователи Сайта, которые дали согласие на обработку файлов «cookie».

5. Способы обработки: сбор, запись, систематизация, накопление, хранение, уточнение (обновление, изменение), извлечение, использование, передача (доступ, предоставление), блокирование, удаление, уничтожение персональных данных.

6. Срок обработки и хранения: до получения от Субъекта персональных данных требования о прекращении обработки/отзыва согласия.

7. Способ отзыва: заявление об отзыве в письменном виде путём его направления на адрес электронной почты Оператора: info@rcsi.science или путем письменного обращения по юридическому адресу: 119991, г. Москва, Ленинский просп., д.32А

8. Субъект персональных данных вправе запретить своему оборудованию прием этих данных или ограничить прием этих данных. При отказе от получения таких данных или при ограничении приема данных некоторые функции Сайта могут работать некорректно. Субъект персональных данных обязуется сам настроить свое оборудование таким способом, чтобы оно обеспечивало адекватный его желаниям режим работы и уровень защиты данных файлов «cookie», Оператор не предоставляет технологических и правовых консультаций на темы подобного характера.

9. Порядок уничтожения персональных данных при достижении цели их обработки или при наступлении иных законных оснований определяется Оператором в соответствии с законодательством Российской Федерации.

10. Я согласен/согласна квалифицировать в качестве своей простой электронной подписи под настоящим Согласием и под Политикой обработки персональных данных выполнение мною следующего действия на сайте: https://journals.rcsi.science/ нажатие мною на интерфейсе с текстом: «Сайт использует сервис «Яндекс.Метрика» (который использует файлы «cookie») на элемент с текстом «Принять и продолжить».