Construction and modeling of the operation of elements of computing technology on fast neurons

Мұқаба

Дәйексөз келтіру

Толық мәтін

Аннотация

The article is devoted to the construction of fast neurons and neural networks for the implementation of two complete logical bases and modeling of computing devices on their basis. The main idea is to form a fast activation function based on semi-parabolas and its variations that have effective computational support. The constructed activation functions meet the basic requirements that allow configuring logical circuits using the backpropagation method. The main result is obtaining complete logical bases that open the way to constructing arbitrary logical functions. Models of such elements as a trigger, a half adder, and an adder, which form the basis of various specific computing devices, are presented and tested. It is shown that the new activation functions allow obtaining fast solutions with a slight decrease in quality compared to reference outputs. To standardize the outputs, it is proposed to combine the constructed circuits with a unit jump activation function.

Авторлар туралы

Mikhail Khachumov

RUDN University; Federal Research Center “Computer Science and Control” of Russian Academy of Sciences

Email: khmike@inbox.ru
ORCID iD: 0000-0001-5117-384X
Scopus Author ID: 55570238100

Candidate of Physical and Mathematical Sciences, Senior Researcher

6 Miklukho-Maklaya str., Moscow, 117198, Russian Federation; 9 60-let Octyabrya prosp., Moscow, 117312, Russian Federation

Yuliya Emelyanova

Ailamazyan Program Systems Institute of RAS

Email: yuliya.emelyanowa2015@yandex.ru
ORCID iD: 0000-0001-7735-6820
Scopus Author ID: 57202835704

Candidate of Technical Sciences, Senior Researcher

4a Peter the Great str., Pereslavl-Zalessky, Yaroslavl Region, 152021, Russian Federation

Vyacheslav Khachumov

RUDN University; Federal Research Center “Computer Science and Control” of Russian Academy of Sciences; Ailamazyan Program Systems Institute of RAS

Хат алмасуға жауапты Автор.
Email: vmh48@mail.ru
ORCID iD: 0000-0001-9577-1438
Scopus Author ID: 56042383100

Doctor of Technical Sciences, Chief Researcher

6 Miklukho-Maklaya str., Moscow, 117198, Russian Federation; 9 60-let Octyabrya prosp., Moscow, 117312, Russian Federation; 4a Peter the Great str., Pereslavl-Zalessky, Yaroslavl Region, 152021, Russian Federation

Әдебиет тізімі

  1. Limonova, E., Nikolaev, D. & Alfonso, D. Bipolar morphological neural networks: Gate-efficient architecture for confined environment, 573-580. doi: 10.1109/NAECON46414.2019.9058018 (2019).
  2. Limonova, E., Nikolaev, D. & Arlazarov, V. Bipolar morphological U-Net for document binarization, 1-9. doi: 10.1117/12.2587174 (2021).
  3. Limonova, E., Nikolaev, D., Alfonso, D. & Arlazarov, V. ResNet-like architecture with low hardware requirements, 6204-6211. doi: 10.1109/ICPR48806.2021.9413186 (2021).
  4. Dubey, S., Singh, S. & Chaudhuri, B. Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark. Neurocomputing 503, 1-18. doi: 10.1016/j.neucom.2022.06.111 (2022).
  5. Feng, J. & Lu, S. Performance Analysis of Various Activation Functions in Artificial Neural Networks. Journal of Physics Conference Series, 1-7. doi: 10.1088/1742-6596/1237/2/02203 (2019).
  6. Akgül, I. Activation functions used in artificial neural networks. In book: Academic Studies in Engineering, 41-58 (Oct. 2023).
  7. Arce, F., Zamora, E., Humberto, S. & Barrón, R. Differential evolution training algorithm for dendrite morphological neural networks. Applied Soft Computing 68, 303-313. doi: 10.1016/j.asoc. 2018.03.033 (2018).
  8. Dimitriadis, N. & Maragos, P. Advances in the training, pruning and enforcement of shape constraints of morphological neural networks using tropical algebra, 3825-3829. doi:10.48550/ arXiv.2011.07643 (2021).
  9. Limonova, E., Nikolaev, D., Alfonso, D. & Arlazarov, V. Bipolar morphological neural networks: gate-efficient architecture for computer vision. IEEE Access 9, 97569-97581. doi: 10.1109/ACCESS. 2021.3094484 (2021).
  10. Galushkin, A., Sudarikov,V. & Shabanov, E. Neuromathematics: the methods of solving problems on neurocomputers. 2, 1179-1188. doi: 10.1109/RNNS.1992.268515 (1992).
  11. Khachumov, M. & Emelyanova, Y. Parabola as an Activation Function of Artificial Neural Networks. Scientific and Technical Information Processing 51, 471-477. doi:10. 3103 / S0147688224700382 (2024).
  12. Khachumov, M., Emelyanova, Y. & Khachumov, V. Parabola-Based Artificial Neural Network Activation Functions, 249-254. doi: 10.1109/RusAutoCon58002.2023.10272855 (Sept. 2023).
  13. Hanche-Olsen, H. & Holden, H. The Kolmogorov-Riesz compactness theorem. 28, 385-394. doi: 10.1016/j.exmath.2010.03.001 (June 2009).
  14. Nabavinejad, S., Reda, S. & Ebrahimi, M. Coordinated Batching and DVFS for DNN Inference on GPU Accelerators. 33, 1-12. doi: 10.1109/TPDS.2022.3144614 (Oct. 2022).
  15. Trusov, A., Limonova, E., Slugin, D., Nikolaev, D. & Arlazarov, V. Fast Implementation of 4-bit Convolutional Neural Networks for Mobile Devices, 9897-9903. doi: 10.1109/ICPR48806.2021. 9412841 (Sept. 2020).
  16. Shashev, D. & Shatravin, V. Implementation of the sigmoid activation function using the reconfigurable computing environments. Russian. Tomsk State University Journal of Control and Computer Science, 117-127. doi: 10.17223/19988605/61/12 (2022).
  17. Hyyro, H. & Navarro, G. Bit-Parallel Computation of Local Similarity Score Matrices with Unitary Weights. International Journal of Foundations of Computer Science. doi: 10.1142/S0129054106004443.
  18. Hyyro, H. Explaining and extending the bit-parallel approximate string matching algorithm of Myers (2001).
  19. Hyyro, H. & Navarro, G. Faster bit-parallel approximate string matching, 203-224 (2002).
  20. Hyyro, H. & Navarro, G. Bit-parallel witnesses and their applications to approximate string matching. Algorithmica 41, 203-231 (2005).
  21. Zakharov, A. & Khachumov, V. Bit-parallel Representation of Activation Functions for Fast Neural Networks. 2, 568-571 (2014).
  22. Volder, J. The Birth of Cordic. The Journal of VLSI Signal Processing-Systems for Signal, Image, and Video Technology 25, 101-105. doi: 10.1023/A:1008110704586 (2000).
  23. Chetana & Sharmila, K. VLSI Implementation of Coordinate Rotation Based Design Methodology using Verilog HDL. doi: 10.1109/ICAIS56108.2023.10073928 (2023).

Қосымша файлдар

Қосымша файлдар
Әрекет
1. JATS XML
Жүктеу