Neural Architecture Search (NAS): Automating the Design of Deep Learning Models
Abstract
Neural Architecture Search (NAS) automates the design of deep learning models, offering optimized architectures for specific tasks. This paper reviews various NAS algorithms, including reinforcement learning-based and evolutionary methods. We evaluate their performance on image classification and natural language processing tasks, discussing the computational challenges and future trends in NAS.
References
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30). https://doi.org/10.48550/arXiv.1706.03762
Zoph, B., & Le, Q. V. (2017). Neural architecture search with reinforcement learning. In International Conference on Learning Representations. https://doi.org/10.48550/arXiv.1611.01578
Zou, J., Huss, M., Abid, A., Mohammadi, P., Torkamani, A., & Telenti, A. (2019). A primer on deep learning in genomics. Nature Genetics, 51(1), 12–18. https://
Baars, H., & Kemper, H.-G. (2008). Management support with structured and unstructured data—An integrated business intelligence framework. Information Systems Management, 25(2), 132–148. https://doi.org/10.1080/10580530801941058
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828. https://doi.org/10.1109/TPAMI.2013.50
Nadella, G. S., Satish, S., Meduri, K., & Meduri, S. S. (2023). A Systematic Literature Review of Advancements, Challenges and Future Directions of AI And ML in Healthcare. International Journal of Machine Learning for Sustainable Development, 5(3), 115-130.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://doi.org/10.48550/arXiv.2005.14165
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). https://doi.org/10.1145/2939672.2939785
Maturi, M. H., Satish, S., Gonaygunta, H., & Meduri, K. (2022). The Intersection of Artificial Intelligence and Neuroscience: Unlocking the Mysteries of the Brain. International Journal of Creative Research In Computer Technology and Design, 4(4), 1-21.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (Vol. 25, pp. 1097–1105). https://doi.org/10.1145/3065386
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
Li, H., & Liu, X. (2018). Federated learning: Challenges, methods, and future directions. IEEE Communications Magazine, 56(12), 34–40. https://doi.org/10.1109/MCOM.2018.1800258
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. In Proceedings of the European Conference on Computer Vision (pp. 21–37). https://doi.org/10.1007/978-3-319-46448-0_2
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. OpenAI Research Paper. https://doi.org/10.48550/arXiv.1801.06146
Ruder, S. (2019). Transfer learning—Machine learning’s next frontier. Machine Learning Research (Vol. 1). https://doi.org/10.48550/arXiv.1810.04805
Maturi, M. H., Gonaygunta, H., Nadella, G. S., & Meduri, K. (2023). Fault Diagnosis and Prognosis using IoT in Industry 5.0. International Numeric Journal of Machine Learning and Robots, 7(7), 1-21.
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489. https://doi.org/10.1038/nature16961
Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations. https://doi.org/10.48550/arXiv.1409.1556
