Technology peripherals
AI
This article will take you to understand SHAP: model explanation for machine learning
This article will take you to understand SHAP: model explanation for machine learning
Jun 01, 2024 am 10:58 AM
In the fields of machine learning and data science, the interpretability of models has always been a focus of researchers and practitioners. With the widespread application of complex models such as deep learning and ensemble methods, understanding the model's decision-making process has become particularly important. Explainable AI | XAI helps build trust and confidence in machine learning models by increasing model transparency. Improving model transparency can be achieved through methods such as the widespread use of multiple complex models, as well as the decision-making processes used to explain the models. These methods include feature importance analysis, model prediction interval estimation, local interpretability algorithms, etc. Feature importance analysis can explain the model's decision-making process by evaluating the degree of influence of the model on the input features. Model prediction interval estimates can provide deterministic information about model predictions. Local interpretability algorithms can help
XAI is a set of tools and frameworks for understanding and explaining how machine learning models make decisions. Among them, the SHAP (SHapley Additive explanations) library in Python is a very useful tool. The SHAP library quantifies the contribution of features to individual predictions and overall predictions, and provides beautiful and easy-to-use visualizations.
Next, we will outline the basics of the SHAP library to understand predictions for regression and classification models built in Scikit-learn.
SHAP and SHAP values
SHAP (Shapley Additive Explanations) is a game theory method for interpreting the output of any machine learning model. It leverages the classic game theory game value and its related extensions to combine optimal credit allocation with local interpretation (see related paper for details and citations: https://github.com/shap/shap#citations). SHAP provides optimal credit allocation and local explanation by calculating the contribution of each feature to the model output. This approach can be applied to various model types, including linear models, tree models, deep learning models, etc. The goal of SHAP is to provide an intuitive and interpretable way to help people understand the decision-making process of the machine learning model and the impact of each feature on the prediction results. By using SHAP values ??and related extensions, we can obtain a more accurate and comprehensive interpretation of feature importance, and pre-
SHAP+ values ??for the model can help us quantify the contribution of features to predictions. The closer the SHAP value is to zero, the smaller the contribution of the feature to prediction; the further the SHAP value is from zero, the greater the contribution of the feature to prediction. In addition, the SHAP value can also tell us the contribution of features to prediction. When the SHAP value is close to zero, it means that the feature contributes little to prediction; and when the SHAP value is far from zero,
Install the shap package:
pip install shap-i https://pypi.tuna.tsinghua.edu.cn/simple
us See the example below: How to get the SHAP value of a feature in a regression problem. We'll start by loading the library and sample data, then quickly build a model to predict diabetes progression:
import numpy as npnp.set_printoptions(formatter={'float':lambda x:"{:.4f}".format(x)})import pandas as pdpd.options.display.float_format = "{:.3f}".formatimport seaborn as snsimport matplotlib.pyplot as pltsns.set(style='darkgrid', context='talk', palette='rainbow')from sklearn.datasets import load_diabetesfrom sklearn.model_selection import train_test_splitfrom sklearn.ensemble import (RandomForestRegressor, RandomForestClassifier)import shapshap.initjs()# Import sample datadiabetes = load_diabetes(as_frame=True)X = diabetes['data'].iloc[:, :4] # Select first 4 columnsy = diabetes['target']# Partition dataX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)print(f"Training features shape: {X_train.shape}")print(f"Training target shape: {y_train.shape}\n")print(f"Test features shape: {X_test.shape}")print(f"Test target shape: {y_test.shape}")display(X_train.head())# Train a simple modelmodel = RandomForestRegressor(random_state=42)model.fit(X_train, y_train)
A common GET SHAP The value method is to use the Explainer object. Next create an Explainer object and extract the shap_test value for the test data:
explainer = shap.Explainer(model)shap_test = explainer(X_test)print(f"Shap values length: {len(shap_test)}\n")print(f"Sample shap value:\n{shap_test[0]}")
The length of shap_test is 89 because it contains each test Records of instances. From looking at the first test record, we can see that it contains three attributes:
shap_test[0].base_values: The base value of the target
shap_test[0].data: Each Values ??of each feature
shap_test[0].values: SHAP value of each object
- Base value: Base value (shap_test.base_values), also called expected value (explainer. expected_value), is the average of the target values ??in the training data.
print(f"Expected value: {explainer.expected_value[0]:.1f}")print(f"Average target value (training data): {y_train.mean():.1f}")print(f"Base value: {np.unique(shap_test.base_values)[0]:.1f}")
- shap_test.data contains the same value as X_test
(shap_test.data == X_test).describe()
- values: The most important attribute of shap_test is the values ??attribute because we can access the SHAP values ??through it. Let's convert the SHAP value to a DataFrame for easier manipulation:
shap_df = pd.DataFrame(shap_test.values, columns=shap_test.feature_names, index=X_test.index)shap_df
可以看到每條記錄中每個特征的 SHAP 值。如果將這些 SHAP 值加到期望值上，就會得到預(yù)測值：
np.isclose(model.predict(X_test), explainer.expected_value[0] + shap_df.sum(axis=1))
現(xiàn)在我們已經(jīng)有了 SHAP 值，可以進行自定義可視化，如下圖所示，以理解特征的貢獻：
columns = shap_df.apply(np.abs).mean()\ .sort_values(ascending=False).indexfig, ax = plt.subplots(1, 2, figsize=(11,4))sns.barplot(data=shap_df[columns].apply(np.abs), orient='h', ax=ax[0])ax[0].set_title("Mean absolute shap value")sns.boxplot(data=shap_df[columns], orient='h', ax=ax[1])ax[1].set_title("Distribution of shap values");plt.show()
左側(cè)子圖顯示了每個特征的平均絕對 SHAP 值，而右側(cè)子圖顯示了各特征的 SHAP 值分布。從這些圖中可以看出，bmi 在所使用的4個特征中貢獻最大。
Shap 內(nèi)置圖表
雖然我們可以使用 SHAP 值構(gòu)建自己的可視化圖表，但 shap 包提供了內(nèi)置的華麗可視化圖表。在本節(jié)中，我們將熟悉其中幾種選擇的可視化圖表。我們將查看兩種主要類型的圖表：
- 全局：可視化特征的整體貢獻。這種類型的圖表顯示了特征在整個數(shù)據(jù)集上的匯總貢獻。
- 局部：顯示特定實例中特征貢獻的圖表。這有助于我們深入了解單個預(yù)測。
- 條形圖/全局：對于之前顯示的左側(cè)子圖，有一個等效的內(nèi)置函數(shù)，只需幾個按鍵即可調(diào)用：
shap.plots.bar(shap_test)
這個簡單但有用的圖表顯示了特征貢獻的強度。該圖基于特征的平均絕對 SHAP 值而生成：shap_df.apply(np.abs).mean()。特征按照從上到下的順序排列，具有最高平均絕對 SHAP 值的特征顯示在頂部。
- 總結(jié)圖/全局：另一個有用的圖是總結(jié)圖：
shap.summary_plot(shap_test)
以下是解釋這張圖的指南：
- 圖的橫軸顯示了特征的 SHAP 值分布。每個點代表數(shù)據(jù)集中的一個記錄。例如，我們可以看到對于 BMI 特征，點的分布相當(dāng)散亂，幾乎沒有點位于 0 附近，而對于年齡特征，點更加集中地分布在 0 附近。
- 點的顏色顯示了特征值。這個額外的維度允許我們看到隨著特征值的變化，SHAP 值如何變化。換句話說，我們可以看到關(guān)系的方向。例如，我們可以看到當(dāng) BMI 較高時（由熱粉色點表示）SHAP 值傾向于較高，并且當(dāng) BMI 較低時（由藍色點表示）SHAP 值傾向于較低。還有一些紫色點散布在整個光譜中。
- 熱力圖/全局：熱力圖是另一種可視化 SHAP 值的方式。與將 SHAP 值聚合到平均值不同，我們看到以顏色編碼的個體值。特征繪制在 y 軸上，記錄繪制在 x 軸上：
shap.plots.heatmap(shap_test)
這個熱力圖的頂部還補充了每個記錄的預(yù)測值（即 f(x)）的線圖。
- Force plot/全局：這個交互式圖表允許我們通過記錄查看 SHAP 值的構(gòu)成。
shap.initjs()shap.force_plot(explainer.expected_value, shap_test.values, X_test)
就像熱力圖一樣，x 軸顯示每個記錄。正的 SHAP 值顯示為紅色，負的 SHAP 值顯示為藍色。例如，由于第一個記錄的紅色貢獻比藍色貢獻多，因此該記錄的預(yù)測值將高于期望值。
交互性允許我們改變兩個軸。例如，y 軸顯示預(yù)測值 f(x)，x 軸根據(jù)輸出（預(yù)測）值排序，如上面的快照所示。
- 條形圖/局部:現(xiàn)在我們將看一下用于理解個別案例預(yù)測的圖表。讓我們從一個條形圖開始：
shap.plots.bar(shap_test[0])
與“ 條形圖/全局 ”中完全相同，只是這次我們將數(shù)據(jù)切片為單個記錄。
- Force plot/局部:Force plot是單個記錄的強制圖。
shap.initjs()shap.plots.force(shap_test[0])
分類模型的SHAP values/圖表
上面示例是回歸模型，下面我們以分類模型展示SHAP values及可視化：
import numpy as npnp.set_printoptions(formatter={'float':lambda x:"{:.4f}".format(x)})import pandas as pdpd.options.display.float_format = "{:.3f}".formatimport seaborn as snsimport matplotlib.pyplot as pltsns.set(style='darkgrid', context='talk', palette='rainbow')from sklearn.datasets import load_diabetesfrom sklearn.model_selection import train_test_splitfrom sklearn.ensemble import RandomForestClassifierimport shapfrom sklearn.datasets import fetch_openml# 加載 Titanic 數(shù)據(jù)集titanic = fetch_openml('titanic', version=1, as_frame=True)df = titanic.frame# 選擇特征和目標變量features = ['pclass', 'age', 'sibsp', 'parch', 'fare']df = df.dropna(subset=features + ['survived'])# 刪除包含缺失值的行X = df[features]y = df['survived']# 分割數(shù)據(jù)集X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)# 訓(xùn)練隨機森林分類器model = RandomForestClassifier(n_estimators=100, random_state=42)model.fit(X_train, y_train)
和回歸模型一樣的，shap values 值也是包括base_values 和values 值：
explainer = shap.Explainer(model)shap_test = explainer(X_test)print(f"Length of shap_test: {len(shap_test)}\n")print(f"Sample shap_test:\n{shap_test[0]}")print(f"Expected value: {explainer.expected_value[1]:.2f}")print(f"Average target value (training data): {y_train}")print(f"Base value: {np.unique(shap_test.base_values)[0]:.2f}")shap_df = pd.DataFrame(shap_test.values[:,:,1], columns=shap_test.feature_names, index=X_test.index)shap_df我們仔細檢查一下將 shap 值之和添加到預(yù)期概率是否會給出預(yù)測概率：
np.isclose(model.predict_proba(X_test)[:,1], explainer.expected_value[1] + shap_df.sum(axis=1))
內(nèi)置圖與回歸模型是一致的，比如：
shap.plots.bar(shap_test[:,:,1])
或者瀑布圖如下：
shap.plots.waterfall(shap_test[:,:,1][0])
示例
看一個具體的用例。我們將找出模型對幸存者預(yù)測最不準確的例子，并嘗試理解模型為什么會做出錯誤的預(yù)測：
test = pd.concat([X_test, y_test], axis=1)test['probability'] = model.predict_proba(X_test)[:,1]test['order'] = np.arange(len(test))test.query("survived=='1'").nsmallest(5, 'probability')
生存概率為第一個記錄的746。讓我們看看各個特征是如何對這一預(yù)測結(jié)果產(chǎn)生貢獻的：
ind1 = test.query("survived=='1'")\ .nsmallest(1, 'probability')['order'].values[0]shap.plots.waterfall(shap_test[:,:,1][ind1])
主要是客艙等級和年齡拉低了預(yù)測值。讓我們在訓(xùn)練數(shù)據(jù)中找到類似的例子：
pd.concat([X_train, y_train], axis=1)[(X_train['pclass']==3) & (X_train['age']==29) & (X_train['fare'].between(7,8))]
所有類似的訓(xùn)練實例實際上都沒有幸存。現(xiàn)在，這就說得通了！這是一個小的分析示例，展示了 SHAP 如何有助于揭示模型為何會做出錯誤預(yù)測。
在機器學(xué)習(xí)和數(shù)據(jù)科學(xué)中，模型的可解釋性一直備受關(guān)注。可解釋人工智能（XAI）通過提高模型透明度，增強對模型的信任。SHAP庫是一個重要工具，通過量化特征對預(yù)測的貢獻，提供可視化功能。本文介紹了SHAP庫的基礎(chǔ)知識，以及如何使用它來理解回歸和分類模型的預(yù)測。通過具體用例，展示了SHAP如何幫助解釋模型錯誤預(yù)測。
The above is the detailed content of This article will take you to understand SHAP: model explanation for machine learning. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undress AI Tool
Undress images for free
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
Bytedance Cutting launches SVIP super membership: 499 yuan for continuous annual subscription, providing a variety of AI functions
Jun 28, 2024 am 03:51 AM
This site reported on June 27 that Jianying is a video editing software developed by FaceMeng Technology, a subsidiary of ByteDance. It relies on the Douyin platform and basically produces short video content for users of the platform. It is compatible with iOS, Android, and Windows. , MacOS and other operating systems. Jianying officially announced the upgrade of its membership system and launched a new SVIP, which includes a variety of AI black technologies, such as intelligent translation, intelligent highlighting, intelligent packaging, digital human synthesis, etc. In terms of price, the monthly fee for clipping SVIP is 79 yuan, the annual fee is 599 yuan (note on this site: equivalent to 49.9 yuan per month), the continuous monthly subscription is 59 yuan per month, and the continuous annual subscription is 499 yuan per year (equivalent to 41.6 yuan per month) . In addition, the cut official also stated that in order to improve the user experience, those who have subscribed to the original VIP
Context-augmented AI coding assistant using Rag and Sem-Rag
Jun 10, 2024 am 11:08 AM
Improve developer productivity, efficiency, and accuracy by incorporating retrieval-enhanced generation and semantic memory into AI coding assistants. Translated from EnhancingAICodingAssistantswithContextUsingRAGandSEM-RAG, author JanakiramMSV. While basic AI programming assistants are naturally helpful, they often fail to provide the most relevant and correct code suggestions because they rely on a general understanding of the software language and the most common patterns of writing software. The code generated by these coding assistants is suitable for solving the problems they are responsible for solving, but often does not conform to the coding standards, conventions and styles of the individual teams. This often results in suggestions that need to be modified or refined in order for the code to be accepted into the application
Seven Cool GenAI & LLM Technical Interview Questions
Jun 07, 2024 am 10:06 AM
To learn more about AIGC, please visit: 51CTOAI.x Community https://www.51cto.com/aigc/Translator|Jingyan Reviewer|Chonglou is different from the traditional question bank that can be seen everywhere on the Internet. These questions It requires thinking outside the box. Large Language Models (LLMs) are increasingly important in the fields of data science, generative artificial intelligence (GenAI), and artificial intelligence. These complex algorithms enhance human skills and drive efficiency and innovation in many industries, becoming the key for companies to remain competitive. LLM has a wide range of applications. It can be used in fields such as natural language processing, text generation, speech recognition and recommendation systems. By learning from large amounts of data, LLM is able to generate text
Can fine-tuning really allow LLM to learn new things: introducing new knowledge may make the model produce more hallucinations
Jun 11, 2024 pm 03:57 PM
Large Language Models (LLMs) are trained on huge text databases, where they acquire large amounts of real-world knowledge. This knowledge is embedded into their parameters and can then be used when needed. The knowledge of these models is "reified" at the end of training. At the end of pre-training, the model actually stops learning. Align or fine-tune the model to learn how to leverage this knowledge and respond more naturally to user questions. But sometimes model knowledge is not enough, and although the model can access external content through RAG, it is considered beneficial to adapt the model to new domains through fine-tuning. This fine-tuning is performed using input from human annotators or other LLM creations, where the model encounters additional real-world knowledge and integrates it
Five schools of machine learning you don't know about
Jun 05, 2024 pm 08:51 PM
Machine learning is an important branch of artificial intelligence that gives computers the ability to learn from data and improve their capabilities without being explicitly programmed. Machine learning has a wide range of applications in various fields, from image recognition and natural language processing to recommendation systems and fraud detection, and it is changing the way we live. There are many different methods and theories in the field of machine learning, among which the five most influential methods are called the "Five Schools of Machine Learning". The five major schools are the symbolic school, the connectionist school, the evolutionary school, the Bayesian school and the analogy school. 1. Symbolism, also known as symbolism, emphasizes the use of symbols for logical reasoning and expression of knowledge. This school of thought believes that learning is a process of reverse deduction, through existing
To provide a new scientific and complex question answering benchmark and evaluation system for large models, UNSW, Argonne, University of Chicago and other institutions jointly launched the SciQAG framework
Jul 25, 2024 am 06:42 AM
Editor |ScienceAI Question Answering (QA) data set plays a vital role in promoting natural language processing (NLP) research. High-quality QA data sets can not only be used to fine-tune models, but also effectively evaluate the capabilities of large language models (LLM), especially the ability to understand and reason about scientific knowledge. Although there are currently many scientific QA data sets covering medicine, chemistry, biology and other fields, these data sets still have some shortcomings. First, the data form is relatively simple, most of which are multiple-choice questions. They are easy to evaluate, but limit the model's answer selection range and cannot fully test the model's ability to answer scientific questions. In contrast, open-ended Q&A
A new era of VSCode front-end development: 12 highly recommended AI code assistants
Jun 11, 2024 pm 07:47 PM
In the world of front-end development, VSCode has become the tool of choice for countless developers with its powerful functions and rich plug-in ecosystem. In recent years, with the rapid development of artificial intelligence technology, AI code assistants on VSCode have sprung up, greatly improving developers' coding efficiency. AI code assistants on VSCode have sprung up like mushrooms after a rain, greatly improving developers' coding efficiency. It uses artificial intelligence technology to intelligently analyze code and provide precise code completion, automatic error correction, grammar checking and other functions, which greatly reduces developers' errors and tedious manual work during the coding process. Today, I will recommend 12 VSCode front-end development AI code assistants to help you in your programming journey.
SK Hynix will display new AI-related products on August 6: 12-layer HBM3E, 321-high NAND, etc.
Aug 01, 2024 pm 09:40 PM
According to news from this site on August 1, SK Hynix released a blog post today (August 1), announcing that it will attend the Global Semiconductor Memory Summit FMS2024 to be held in Santa Clara, California, USA from August 6 to 8, showcasing many new technologies. generation product. Introduction to the Future Memory and Storage Summit (FutureMemoryandStorage), formerly the Flash Memory Summit (FlashMemorySummit) mainly for NAND suppliers, in the context of increasing attention to artificial intelligence technology, this year was renamed the Future Memory and Storage Summit (FutureMemoryandStorage) to invite DRAM and storage vendors and many more players. New product SK hynix launched last year
