Articles
| Open Access |
DOI: MACHINE LEARNING ALGORITHMS FOR ANOMALY DETECTION IN PUBLIC DATA USING GITHUB AS AN EXAMPLE
Lyamkin Ilya , Senior Full Stack Engineer at Spotify, USAAbstract
This study explores the application of machine learning algorithms for detecting anomalies in GitHub data to enhance the evaluation of technological projects. The research aims to develop a robust methodology for identifying data anomalies, such as artificial activity spikes, that can distort project assessments. Methods such as Isolation Forest, One-Class SVM, and advanced deep learning techniques like autoencoders and GANs are employed to analyze and identify irregular patterns in GitHub repositories. The findings demonstrate that these algorithms effectively detect both obvious and subtle anomalies, offering reliable insights into project authenticity. The proposed conceptual model integrates these methods into a scalable system, enhancing transparency and accuracy in technological project evaluation. The novelty of this work lies in its comprehensive approach to analyzing GitHub data, combining traditional and deep learning techniques to improve the reliability of assessments, making it a significant contribution to the field.
Zenodo DOI:- https://doi.org/10.5281/zenodo.13896665
Keywords
Machine learning, anomaly detection, GitHub data
References
Invest Boldly with the Power of Data. URL: https://www.dualspace.ai/
Github. URL: https://github.com/
Aneja N., Aneja S. Detecting fake news with machine learning //Conference Proceedings of ICDLAIR2019. – Springer International Publishing, 2021. – С. 53-64.
Dozmorov M. G. GitHub statistics as a measure of the impact of open-source bioinformatics software //Frontiers in bioengineering and biotechnology. – 2018. – Т. 6. – С. 436207.
Liu F. T., Ting K. M., Zhou Z. H. Isolation forest //2008 eighth ieee international conference on data mining. – IEEE, 2008. – С. 413-422.
Binbusayyis A., Vaiyapuri T. Unsupervised deep learning approach for network intrusion detection combining convolutional autoencoder and one-class SVM //Applied Intelligence. – 2021. – Т. 51. – №. 10. – С. 7094-7108.
Goodfellow I. et al. Generative adversarial nets //Advances in neural information processing systems. – 2014. – Т. 27.
Kingma D. P. Auto-encoding variational bayes //arXiv preprint arXiv:1312.6114. – 2013.
Chen T., Guestrin C. Xgboost: A scalable tree boosting system //Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. – 2016. – С. 785-794.
Loshchilov I., Hutter F. Sgdr: Stochastic gradient descent with warm restarts //arXiv preprint arXiv:1608.03983. – 2016.
Article Statistics
Copyright License
Copyright (c) 2024 Lyamkin Ilya
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain the copyright of their manuscripts, and all Open Access articles are disseminated under the terms of the Creative Commons Attribution License 4.0 (CC-BY), which licenses unrestricted use, distribution, and reproduction in any medium, provided that the original work is appropriately cited. The use of general descriptive names, trade names, trademarks, and so forth in this publication, even if not specifically identified, does not imply that these names are not protected by the relevant laws and regulations.