数据挖掘——Data competition: From 0 to 1: Part I

浅浅的花香味﹌ 2023-06-02 07:49 25阅读 0赞

*  [Data competition: From 0 to 1: Part I][Data competition_ From 0 to 1_ Part I]
    
     *  [1. Data competition Introduction][]
     *  [2. Example: Credit Fraud Detector][2. Example_ Credit Fraud Detector]
        
         *  [EDA(Exploratory Data Analysis)][EDA_Exploratory Data Analysis]
            
             *  [Why taking log transformation of continuous variables?][Why taking log transformation of continuous variables]
         *  [Outliers Detection][]
         *  [Unbalance][]
         *  [Metrics][]
         *  [Resampling][]
         *  [Cross-validation: evaluating estimator performance][Cross-validation_ evaluating estimator performance]
            
             *  [Cross validation iterators][]
             *  [Cross-validation iterators for grouped data][]
             *  [Cross validation of time series data][]
         *  [Modeling][]

# Data competition: From 0 to 1: Part I #

## 1. Data competition Introduction ##

A typical data science process might look like this:

*  Project Scoping / Data Collection
 *  Exploratory Analysis
 *  Data Cleaning
 *  Feature Engineering
 *  Model Training (including cross-validation to tune hyper-parameters)
 *  Project Delivery / Insights  
    ![data_20competition.png_0.44027762639143386][]

## 2. Example: Credit Fraud Detector ##

**Competition Describe**

*  The datasets contains transactions made by credit cards in September 2013 by european cardholders.
 *  Features V1, V2, ... V28 are the principal components obtained with PCA
 *  Feature 'Time' contains the seconds elapsed between each transaction and the first transaction in the dataset.
 *  The feature ‘Amount’ is the transaction Amount.
 *  Feature 'Class' is the response variable and it takes value 1 in case of fraud and 0 otherwise.

### EDA(Exploratory Data Analysis) ###

*  Gathering a basic sense of data
 *  Example:
    
     *  Shape(Traing Size, Test size)
     *  Label(Binary or Multi or Regression, Distribution)
     *  Columns(Meaning, Numerical or Time or Category)
     *  `Null` Values, how to deal with
     *  Numerical variable: Distribution
     *  Outliers

# import library import pandas as pd pd.set_option('display.max_column',100) import matplotlib.pyplot as plt from pylab import rcParams rcParams['figure.figsize'] = 14, 10 # plt.figure(figsize=(14, 10)) import numpy as np import seaborn as sns %matplotlib inline # 可以省略 plt.show() import warnings warnings.filterwarnings('ignore')

# simple ead ways data = pd.read_csv('data/creditcard.csv') data.head() data.shape data.info() data.columns data.describe() data['Class'].value_counts() data.isnull().sum() data.isnull().sum().max() # 包含缺失最多的一列有多少 data.isnull().any() data.isnull().values.any() # 数据中是否有缺失值

# Amount-histogram Fraud_transacation = data[data["Class"]==1] Normal_transacation= data[data["Class"]==0] plt.figure(figsize=(10,6)) plt.subplot(121) Fraud_transacation.Amount.plot.hist(title="Fraud Transacation") plt.yscale('log') plt.subplot(122) Normal_transacation.Amount.plot.hist(title="Normal Transaction") plt.yscale('log') # Amount-Time f, (ax1, ax2) = plt.subplots(2, 1, sharex=True) f.suptitle('Time of transaction vs Amount by class') ax1.scatter(Fraud.Time, Fraud.Amount) ax1.set_title('Fraud') ax2.scatter(Normal.Time, Normal.Amount) ax2.set_title('Normal') plt.xlabel('Time (in Seconds)') plt.ylabel('Amount') plt.show() # correlation correlation_matrix = data.corr() fig = plt.figure(figsize=(12,9)) sns.heatmap(correlation_matrix,vmax=0.8,square = True) plt.show() ## After sampling ?

data['hour'] = data['Time'].apply(lambda x: np.ceil(float(x)/3600) % 24) data.pivot_table(values='Amount',index='hour',columns='Class',aggfunc='count') # https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.pivot_table.html bins = np.arange(data['hour'].min(),data['hour'].max()+2) # plt.figure(figsize=(15,4)) sns.distplot(data[data['Class']==0.0]['hour'], norm_hist=True, bins=bins, kde=False, color='b', hist_kws={ 'alpha':.5}, label='Legit') sns.distplot(data[data['Class']==1.0]['hour'], norm_hist=True, bins=bins, kde=False, color='r', label='Fraud', hist_kws={ 'alpha':.5}) plt.xticks(range(0,24)) plt.legend()

# https://github.com/pandas-profiling/pandas-profiling import pandas_profiling profile = df.profile_report(title="Credit Fraud Detector") profile.to_file(output_file=Path("./credit_fraud_detector.html"))

**EDA Conclusion**

>  *  The data set is highly skewed, consisting of 492 frauds in a total of 284,807 observations. This resulted in only 0.172% fraud cases.
>  *  There is no missing value in the dataset.
>  *  The ‘Time’ and ‘Amount’ features are not transformed data.

#### Why taking log transformation of continuous variables? ####

Amount distribution is "power law", meaning that the vast majority of amount are small and very few are big.

\\begin\{array\}\{c\}\{\\log \\left(10^\{4\}\\right)=4 \* \\log (10)\} \\\\ \{\\log \\left(10^\{3\}\\right)=3 \* \\log (10)\} \\\\ \{10^\{4\}-10^\{3\}\} \\\\ \{4-3\}\\end\{array\}log(104)=4∗log(10)log(103)=3∗log(10)104−1034−3

which transforms a huge difference in a smaller one. **Logarithm naturally reduces the dynamic range of a variable so the differences are preserved while the scale is not that dramatically skewed.**

### Outliers Detection ###

*  **Point anomalies**: A single instance of data is anomalous if it's too far off from the rest.
 *  **Contextual anomalies**: The abnormality is context specific. This type of anomaly is common in time-series data.

![图片名称][anomaly.png]

\\mathrm\{IQR\}=Q\_\{3\}-Q\_\{1\}IQR=Q3−Q1

\\text\{Outliers\}: >Q\_3+k \\cdot IQROutliers:>Q3\+k⋅IQR

\\text\{Outliers\}: <Q\_1-k \\cdot IQROutliers:<Q1−k⋅IQR

The higher kk is (ex: 3), the less outliers will detect, and the lower kk is (ex: 1.5) the more outliers it will detect.

We want to focus more on "extreme outliers" rather than just outliers.

### Unbalance ###

count_classes = pd.value_counts(df['Class'], sort = True).sort_index() count_classes.plot(kind = 'bar') plt.title("Fraud class histogram") plt.xlabel("Class") plt.ylabel("Frequency")

![图片名称][class_distribution.png]

1.  Collect more data
2.  Using the weights parameters

LogisticRegression(class_weight='balanced') # How to choose weights?

1.  Changing the performance metric:  
    2.1 `Precision`, `Recall`  
    2.2 `F1-score`  
    2.3 `ROC` curves
2.  Resampling the dataset  
    3.1 `OVER-sampling`  
    3.2 `UNDER-sampling`  
    3.3 `SMOTE`

### Metrics ###

**Confusion Matrix**  
![图片名称][prediction.png]

*  `True Positives` : The cases in which we predicted YES and the actual output was also YES.
 *  `True Negatives` : The cases in which we predicted NO and the actual output was NO.
 *  `False Positives` : The cases in which we predicted YES and the actual output was NO.
 *  `False Negatives` : The cases in which we predicted NO and the actual output was YES.  
    ![图片名称][Confusion_20Matrix2.png]

## homework ## def find_TP(y_true, y_pred): # counts the number of true positives (y_true = 1, y_pred = 1) return sum((y_true == 1) & (y_pred == 1)) def find_FN(y_true, y_pred): # counts the number of false negatives (y_true = 1, y_pred = 0) return # your code here def find_FP(y_true, y_pred): # counts the number of false positives (y_true = 0, y_pred = 1) return # your code here def find_TN(y_true, y_pred): # counts the number of true negatives (y_true = 0, y_pred = 0) return # your code here import numpy as np def find_conf_matrix_values(y_true,y_pred): # calculate TP, FN, FP, TN TP = find_TP(y_true,y_pred) FN = find_FN(y_true,y_pred) FP = find_FP(y_true,y_pred) TN = find_TN(y_true,y_pred) return TP,FN,FP,TN def my_confusion_matrix(y_true, y_pred): TP,FN,FP,TN = find_conf_matrix_values(y_true,y_pred) return np.array([[TN,FP],[FN,TP]])

from sklearn.metrics import confusion_matrix confusion_matrix(label, prediction)

**Accuracy**  
\\text\{Accuracy\}=\\frac\{\\text\{True Positives\}+\\text\{False Negatives\}\}\{\\text\{Total Number of Predictions\}\}Accuracy=Total Number of PredictionsTrue Positives\+False Negatives

*  It works well only if there are equal number of samples belonging to each class.

# homework def my_accuracy_score(y_true, y_pred): # calculates the fraction of samples predicted correctly TP,FN,FP,TN = find_conf_matrix_values(y_true,y_pred) return # your code here

from sklearn.metrics import accuracy_score accuracy_score(label, prediction)

**Precision**  
\\text\{Precision\}=\\frac\{\\text\{True Positives\}\}\{\\text\{True Positives\}+\\text\{False Positives\}\}Precision=True Positives\+False PositivesTrue Positives

# homework def my_precision_score(y_true, y_pred): # calculates the fraction of predicted positives samples that are actually positive TP,FN,FP,TN = find_conf_matrix_values(y_true,y_pred) return # your code here

from sklearn.metrics import precision_score precision_score(label, prediction)

**Recall**  
\\text\{Recall\}=\\frac\{\\text\{True Positives\}\}\{\\text\{True Positives\}+\\text\{False Negatives\}\}Recall=True Positives\+False NegativesTrue Positives

# homework def my_recall_score(y_true, y_pred): # calculates the fraction of positive samples predicted correctly TP,FN,FP,TN = find_conf_matrix_values(y_true,y_pred) return # your code here

from sklearn.metrics import recall_score recall_score(label, prediction)

**F1 Score**  
The f1 score is the harmonic mean(调和平均) of recall and precision, with a higher score as a better model.  
F 1=\\frac\{2\}\{\\frac\{1\}\{\\text \{ precision \}\}+\\frac\{1\}\{\\text \{ recall \}\}\}=\\frac\{2 \* \\text \{ (precision \* recall) \}\}\{\\text \{ precision \}+\\text \{recall\}\}F1= precision 1\+ recall 12= precision \+recall2∗ (precision \* recall)

# homework def my_f1_score(y_true, y_pred): # calculates the F1 score recall = my_recall_score(y_true,y_pred) precision = my_precision_score(y_true,y_pred) return # your code here

from sklearn.metrics import f1_score f1_score(label, prediction)

We often assume that we defined a threshold of 0.5 for selecting which samples are predicted as positive. If we change this threshold the performance metrics will change It would be nice to be able to evaluate the performance of a model without the need to select an arbitrary threshold. This is precisely what AUC-ROC is providing.

**Area Under Curve (AUC)**\--Area Under the curve of the Receiver Operating Characteristic (AUROC)

AUC is used for binary classification problem.

*  True Positive Rate (Sensitivity) : the proportion of positive data points that are correctly considered as positive, with respect to all positive data points.\\text\{True Positive Rate\}=\\frac\{\\text\{True Positives\}\}\{\\text\{True Positives\}+\\text\{False Negatives\}\}True Positive Rate=True Positives\+False NegativesTrue Positives
 *  False Positive Rate (Specificity) : the proportion of negative data points that are mistakenly considered as positive, with respect to all negative data points.\\text\{False Positive Rate\}=\\frac\{\\text\{False Positives\}\}\{\\text\{False Positives\}+\\text\{True Negatives\}\}False Positive Rate=False Positives\+True NegativesFalse Positives

`FPR` and `TPR` bot hare computed at threshold values such as (0.00, 0.02, 0.04, …., 1.00)(0.00,0.02,0.04,….,1.00) and a graph is drawn. `AUC` is the area under the curve of plot `False Positive Rate` vs `True Positive Rate` at different points in \[0, 1\]\[0,1\]. The resulting curve is called ROC curve (Receiver Operating Characteristic curve).

from sklearn.metrics import roc_curve fpr, tpr, thresholds = roc_curve(label, prediction) import matplotlib.pyplot as plt plt.plot(fpr, tpr) plt.legend() plt.xlabel('False Positive Rate') plt.ylabel('True Positive Rate') plt.show() from sklearn.metrics import roc_auc_score auc = roc_auc_score(label, prediction)

![图片名称][auc.png]

from numba import jit @jit def fast_auc(y_true, y_prob): y_true = np.asarray(y_true) y_true = y_true[np.argsort(y_prob)] nfalse = 0 auc = 0 n = len(y_true) for i in range(n): y_i = y_true[i] nfalse += (1 - y_i) auc += y_i * nfalse auc /= (nfalse * (n - nfalse)) return auc

## test speed y_true = np.random.randint(0,2,1000000) y_pred = np.random.rand(1000000) fast_auc(y_true, y_pred) from sklearn.metrics import roc_auc_score roc_auc_score(y_true, y_pred) %timeit fast_auc(y_true, y_pred) %timeit roc_auc_score(y_true, y_pred)

You can learn more about [ROC on the Wikipedia page][] and [What does AUC stand for and what is it?][What does AUC stand for and what is it].

**Mean Absolute Error(MAE)**  
\\text \{MAE\}=\\frac\{1\}\{N\} \\sum\_\{j=1\}^\{N\}\\left|y\_\{j\}-\\hat\{y\}\_\{j\}\\right|MAE=N1j=1∑N∣yj−y^j∣

**Mean Squared Error(MSE)**  
\\text \{MSE\}=\\frac\{1\}\{N\} \\sum\_\{j=1\}^\{N\}\\left(y\_\{j\}-\\hat\{y\}\_\{j\}\\right)^\{2\}MSE=N1j=1∑N(yj−y^j)2

**Log Loss**  
`AUC` only takes into account **the order of probabilities** and hence it does not take into account the model’s capability to predict higher probability for samples more likely to be positive.

\\text \{Log Loss\}=-\\frac\{1\}\{N\} \\sum\_\{i=1\}^\{N\} \\sum\_\{j=1\}^\{M\} y\_\{i j\} \* \\log \\left(p(y\_\{i j\})\\right)Log Loss=−N1i=1∑Nj=1∑Myij∗log(p(yij))

*  y\_\{ij\}yij: whether sample ii belongs to class jj or not
 *  p(y\_\{i j\})p(yij): the probability of sample ii belonging to class jj

\\text \{Log Loss\}=-\\frac\{1\}\{N\} \\sum\_\{i=1\}^\{N\} y\_\{i\} \\cdot \\log \\left(p\\left(y\_\{i\}\\right)\\right)+\\left(1-y\_\{i\}\\right) \\cdot \\log \\left(1-p\\left(y\_\{i\}\\right)\\right)Log Loss=−N1i=1∑Nyi⋅log(p(yi))\+(1−yi)⋅log(1−p(yi))

### Resampling ###

**Under-Sampling**: samples from the majority class  
**Over-Sampling**: adding more examples from the minority class

**Under-Sampling Drawback**: Removing information that may be valuable. This could lead to underfitting and poor generalization to the test set.  
![图片名称][resampling.png]

### Create A Small Unbalanced Sample Dataset from sklearn.utils import resample from sklearn.datasets import make_classification X, y = make_classification( n_classes=2, class_sep=1.5, weights=[0.9, 0.1], n_informative=3, n_redundant=1, flip_y=0, n_features=20, n_clusters_per_class=1, n_samples=1000, random_state=10 ) df = pd.DataFrame(X) df['target'] = y df.target.value_counts().plot(kind='bar', title='Count (target)') ### Oversample minority class minority_data = df[df.target==1] magority_data = df[df.target==0] # oversample minority upsampled_data = resample(minority_data, replace=True, # sample with replacement n_samples=len(magority_data), # match number in majority class random_state=27) # reproducible results # combine majority and upsampled minority upsampled = pd.concat([upsampled_data, magority_data]) # check new class counts upsampled.target.value_counts() ### Undersample majority class downsampled_data = resample(magority_data, replace = False, # sample without replacement n_samples = len(minority_data), # match minority n random_state = 27) # reproducible results # combine minority and downsampled majority downsampled = pd.concat([minority_data, downsampled_data]) # checking counts downsampled.target.value_counts()

**Imbalanced-Learn**

def plot_2d_space(X_train, y_train,X=X,y=y ,label='Classes'): colors = ['#1F77B4', '#FF7F0E'] markers = ['o', 's'] fig,(ax1,ax2)=plt.subplots(1,2, figsize=(10,5)) for l, c, m in zip(np.unique(y), colors, markers): ax1.scatter( X_train[y_train==l, 0], X_train[y_train==l, 1], c=c, label=l, marker=m ) for l, c, m in zip(np.unique(y), colors, markers): ax2.scatter( X[y==l, 0], X[y==l, 1], c=c, label=l, marker=m ) ax1.set_title(label) ax2.set_title('original data') plt.legend(loc='upper right') plt.show() import imblearn ### Random under-sampling with imbalanced-learn from imblearn.under_sampling import RandomUnderSampler ran = RandomUnderSampler(return_indices=True) ##intialize to return indices of dropped rows X_rs,y_rs,dropped = ran.fit_sample(X,y) print("The number of removed indices are ",len(dropped)) plot_2d_space(X_rs, y_rs, X, y,'Random under sampling') ### Random over-sampling with imbalanced-learn from imblearn.over_sampling import RandomOverSampler ran=RandomOverSampler() X_ran,y_ran= ran.fit_resample(X,y) print('The new data contains {} rows '.format(X_ran.shape[0])) plot_2d_space(X_ran,y_ran,X,y,'over-sampled')

**Under-sampling: Tomek links**  
**Tomek links** are pairs of very close instances, but of opposite classes. Removing the instances of the majority class of each pair increases the space between the two classes, facilitating the classification process.

![图片名称][tomek.png]

from imblearn.under_sampling import TomekLinks tl = TomekLinks(return_indices=True, ratio='majority') X_tl, y_tl, id_tl = tl.fit_sample(X, y) #print('Removed indexes:', id_tl) plot_2d_space(X_tl, y_tl,X,y, 'Tomek links under-sampling')

**Over-sampling: SMOTE**  
**SMOTE (Synthetic Minority Oversampling TEchnique)** consists of synthesizing elements for the minority class, based on those that already exist. It works randomly picingk a point from the minority class and computing the k-nearest neighbors for this point. The synthetic points are added between the chosen point and its neighbors.

![图片名称][smote_202.png]

from imblearn.over_sampling import SMOTE smote = SMOTE(ratio='minority') X_sm, y_sm = smote.fit_sample(X, y) plot_2d_space(X_sm, y_sm,X,y, 'SMOTE over-sampling')

### Cross-validation: evaluating estimator performance ###

Trained model would have a perfect score on training data but would fail to predict anything useful on yet-unseen data. This situation is called `overfitting`. To avoid it, it is common practice when performing a (supervised) machine learning experiment to hold out part of the available data as a test set `X_test`, `y_test`. Here is a flowchart of typical cross validation workflow in model training.

![图片名称][grid_search_workflow.png]

**`k-folds`**:

*  A model is trained using `k-1` of the folds as training data;
 *  the resulting model is validated on the remaining part of the data.

The performance measure reported by `k-fold` cross-validation is then the average of the values computed in the loop. This approach can be computationally expensive, but does not waste too much data.

![图片名称][grid_search_cross_validation.png]

#### Cross validation iterators ####

**Cross-validation iterators for i.i.d. data**  
**`K-fold`**  
![图片名称][sphx_glr_plot_cv_indices_0041.png]

from sklearn.model_selection import KFold X = ["a", "b", "c", "d"] kf = KFold(n_splits=2) for train, test in kf.split(X): print("%s %s" % (train, test))

**`Random permutations cross-validation a.k.a. Shuffle & Split`**  
![图片名称][sphx_glr_plot_cv_indices_0061.png]

from sklearn.model_selection import ShuffleSplit X = np.arange(10) ss = ShuffleSplit(n_splits=5, test_size=0.25, random_state=0) for train_index, test_index in ss.split(X): print("%s %s" % (train_index, test_index))

**Cross-validation iterators with stratification based on class labels**  
Some classification problems can exhibit a large imbalance in the distribution of the target classes.

**`Stratified k-fold`**  
![图片名称][sphx_glr_plot_cv_indices_0071.png]

from sklearn.model_selection import StratifiedKFold X = np.ones(10) y = [0, 0, 0, 0, 1, 1, 1, 1, 1, 1] skf = StratifiedKFold(n_splits=3) for train, test in skf.split(X, y): print("%s %s" % (train, test))

**`Stratified Shuffle Split`**  
![图片名称][sphx_glr_plot_cv_indices_0091.png]

#### Cross-validation iterators for grouped data ####

The i.i.d. assumption is broken if the underlying generative process yield groups of dependent samples.

**`Group k-fold`**  
GroupKFold is a variation of k-fold which ensures that the same group is not represented in both testing and training sets.  
![图片名称][sphx_glr_plot_cv_indices_0051.png]

from sklearn.model_selection import GroupKFold X = [0.1, 0.2, 2.2, 2.4, 2.3, 4.55, 5.8, 8.8, 9, 10] y = ["a", "b", "b", "b", "c", "c", "c", "d", "d", "d"] groups = [1, 1, 1, 2, 2, 2, 3, 3, 3, 3] gkf = GroupKFold(n_splits=3) for train, test in gkf.split(X, y, groups=groups): print("%s %s" % (train, test))

**`Group Shuffle Split`**  
![图片名称][sphx_glr_plot_cv_indices_0081.png]

from sklearn.model_selection import GroupShuffleSplit X = [0.1, 0.2, 2.2, 2.4, 2.3, 4.55, 5.8, 0.001] y = ["a", "b", "b", "b", "c", "c", "c", "a"] groups = [1, 1, 2, 2, 3, 3, 4, 4] gss = GroupShuffleSplit(n_splits=4, test_size=0.5, random_state=0) for train, test in gss.split(X, y, groups=groups): print("%s %s" % (train, test))

**`Leave P Groups Out`**

from sklearn.model_selection import LeavePGroupsOut X = np.arange(6) y = [1, 1, 1, 2, 2, 2] groups = [1, 1, 2, 2, 3, 3] lpgo = LeavePGroupsOut(n_groups=2) for train, test in lpgo.split(X, y, groups=groups): print("%s %s" % (train, test))

#### Cross validation of time series data ####

`k-fold` 假设数据是 `iid` 的，但是时间序列存在相关性，因此不能简单使用 `k-fold`。  
![图片名称][sphx_glr_plot_cv_indices_0101.png]

from sklearn.model_selection import TimeSeriesSplit X = np.array([[1, 2], [3, 4], [1, 2], [3, 4], [1, 2], [3, 4]]) y = np.array([1, 2, 3, 4, 5, 6]) tscv = TimeSeriesSplit(n_splits=3) for train, test in tscv.split(X): print("%s %s" % (train, test))

### Modeling ###

from sklearn.preprocessing import StandardScaler data['normAmount'] = StandardScaler().fit_transform(data['Amount'].reshape(-1, 1)) data = data.drop(['Amount'],axis=1)

# train_test_split from sklearn.cross_validation import train_test_split # Whole dataset X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.3, random_state = 0) from sklearn.linear_model import LogisticRegression lr = LogisticRegression(C = c_param, penalty = 'l1') lr.fit(X_train, y_train) y_pred = lr.predict(X_test) # compare y_test with y_pred

转载于:https://www.cnblogs.com/LCharles/p/11385574.html

[Data competition_ From 0 to 1_ Part I]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#data-competition-from-0-to-1-part-i
[1. Data competition Introduction]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#1-data-competition-introduction
[2. Example_ Credit Fraud Detector]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#2-example-credit-fraud-detector
[EDA_Exploratory Data Analysis]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#edaexploratory-data-analysis
[Why taking log transformation of continuous variables]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#why-taking-log-transformation-of-continuous-variables
[Outliers Detection]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#outliers-detection
[Unbalance]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#unbalance
[Metrics]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#metrics
[Resampling]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#resampling
[Cross-validation_ evaluating estimator performance]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#cross-validation-evaluating-estimator-performance
[Cross validation iterators]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#cross-validation-iterators
[Cross-validation iterators for grouped data]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#cross-validation-iterators-for-grouped-data
[Cross validation of time series data]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#cross-validation-of-time-series-data
[Modeling]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/_E7_AC_AC_E4_BA_8C_E8_8A_82_E8_AF_BE_E8_B5_84_E6_96_99.html#modeling
[data_20competition.png_0.44027762639143386]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/data%20competition.png?0.44027762639143386
[anomaly.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/anomaly.png
[class_distribution.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/class_distribution.png
[prediction.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/prediction.png
[Confusion_20Matrix2.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/Confusion%20Matrix2.png
[auc.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/auc.png
[ROC on the Wikipedia page]: https://en.wikipedia.org/wiki/Receiver_operating_characteristic
[What does AUC stand for and what is it]: https://stats.stackexchange.com/questions/132777/what-does-auc-stand-for-and-what-is-it
[resampling.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/resampling.png
[tomek.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/tomek.png
[smote_202.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/smote%202.png
[grid_search_workflow.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/grid_search_workflow.png
[grid_search_cross_validation.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/grid_search_cross_validation.png
[sphx_glr_plot_cv_indices_0041.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/sphx_glr_plot_cv_indices_0041.png
[sphx_glr_plot_cv_indices_0061.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/sphx_glr_plot_cv_indices_0061.png
[sphx_glr_plot_cv_indices_0071.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/sphx_glr_plot_cv_indices_0071.png
[sphx_glr_plot_cv_indices_0091.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/sphx_glr_plot_cv_indices_0091.png
[sphx_glr_plot_cv_indices_0051.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/sphx_glr_plot_cv_indices_0051.png
[sphx_glr_plot_cv_indices_0081.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/sphx_glr_plot_cv_indices_0081.png
[sphx_glr_plot_cv_indices_0101.png]: https://trello-attachments.s3.amazonaws.com/5d4a82ebb300b75ed862082b/5d5b526cb799933827a7072e/x/f431f91a84c7f90f7e259efd48aa0bbf/picture/sphx_glr_plot_cv_indices_0101.png