Enhancing Explainability in Machine Learning Models Using Shapley Values and Local Interpretable Model-Agnostic Explanations (LIME)

Authors

  • Prof. Kummar Sharma

Abstract

The increasing complexity of machine learning models has led to challenges in understanding their decision-making processes, particularly in critical applications like healthcare, finance, and law. This paper proposes a novel framework combining Shapley Values and Local Interpretable Model-Agnostic Explanations (LIME) to enhance model explainability. Shapley Values provide a robust theoretical foundation for feature importance, while LIME offers localized insights into individual predictions. Together, they create a comprehensive explainability toolkit that ensures transparency without sacrificing predictive performance. The framework is validated using real-world datasets, demonstrating its ability to generate intuitive explanations for both global and local model behavior. By making machine learning systems more interpretable, this approach fosters trust among stakeholders and facilitates compliance with ethical and regulatory standards. The results highlight the potential of this method to bridge the gap between high-performing models and the need for interpretability in real-world applications.

 

References

Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798-1828.

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.

Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 770-778).

Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527-1554.

Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.

Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations (ICLR) (pp. 1-15).

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (pp. 1097-1105).

LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436-444.

Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations (ICLR).

Murphy, K. P. (2012). Machine learning: A probabilistic perspective. MIT Press.

Ng, A. Y., & Jordan, M. I. (2002). On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes. In Advances in Neural Information Processing Systems (pp. 841-848).

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536.

Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117.

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., ... & Hassabis, D. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484-489.

Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (pp. 5998-6008).

Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600-612.

Williams, R. J. (1992). Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8(3-4), 229-256.

Zhou, Z. H. (2012). Ensemble methods: Foundations and algorithms. Chapman & Hall/CRC.

Raghunath (2024), "Security Issues Analysis Based on Big Data in Cloud Computing," World Journal of Advanced Research and Reviews, vol. 23, no. 3, pp. 2549-2557, 2024.

Raghunath (2024), "Analysis on Addressing the Threats to Cloud Computing on the Basis of Security Safeguards for SAP Cloud Services," World Journal of Advanced Research and Reviews, vol. 23, no. 3, pp. 2539-2548, 2024.

Raghunath, V., Kunkulagunta, M., & Nadella, G. S. (2023). AI-Driven Business Analytics Framework for Data Integration Across Hybrid Cloud Systems. Transactions on Latest Trends in Artificial Intelligence, 4(4).

Raghunath, V., Kunkulagunta, M., & Nadella, G. S. (2023). Integrating AI and Cloud Computing for Scalable Business Analytics in Enterprise Systems. International Journal of Sustainable Development in Computing Science, 5(3).

Raghunath, V., Kunkulagunta, M., & Nadella, G. S. (2023). Enhancing Data Integration Using AI and ML Techniques for Real-Time Analytics. International Journal of Machine Learning for Sustainable Development, 5(3).

Ranjan, P., Dahiya, S., Singh, S. K., & Choudhary, S. K. (2023) ENHANCING STOCK PRICE PREDICTION: A COMPREHENSIVE ANALYSIS UTILIZING MACHINE LEARNING AND DEEP LEARNING APPROACHES.

Banerjee, P., Roy, R., Batchu, C., & Ranjan, P. (2023) Examining the Application of Data Federation across Cloud Databases in the Financial Services Domain.

Published

2024-12-12

How to Cite

Sharma, P. K. (2024). Enhancing Explainability in Machine Learning Models Using Shapley Values and Local Interpretable Model-Agnostic Explanations (LIME). International Journal of Sustainable Development in Computer Science Engineering, 10(10). Retrieved from https://journals.threws.com/index.php/IJSDCSE/article/view/305

Issue

Section

Articles