ML之FE：基于BigMartSales数据集利用Featuretools工具(1个dataframe表结构切为2个Entity表结构)实现自动特征工程之详细攻略

亦凉 2022-11-18 01:46 122阅读 0赞

**目录**

基于BigMartSales数据集利用Featuretools工具(1个dataframe表结构切为2个Entity表结构)实现自动特征工程之详细攻略

设计思路

输出结果

1、结果输出

2、FE后的字段

3、衍生字段案例具体实现过程

T1、选择的基字段(1个)

(1)、生成的所有衍生字段

(2)、父衍生字段—outlet.COUNT(bigmart)

(3)、子衍生字段的具体字段分析

T2、选择的基字段(2个)

核心代码

--------------------

**相关文章**  
[ML之FE：基于BigMartSales数据集利用Featuretools工具(1个dataframe表结构切为2个Entity表结构)实现自动特征工程之详细攻略][ML_FE_BigMartSales_Featuretools_1_dataframe_2_Entity]  
[ML之FE：基于BigMartSales数据集利用Featuretools工具(1个dataframe表结构切为2个Entity表结构)实现自动特征工程之详细攻略实现][ML_FE_BigMartSales_Featuretools_1_dataframe_2_Entity 1]

# 基于BigMartSales数据集利用Featuretools工具(1个dataframe表结构切为2个Entity表结构)实现自动特征工程之详细攻略 #

# 设计思路 #

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70][]

# 输出结果 #

## 1、结果输出 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 1][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 2][]

## 2、FE后的字段 ##

<table style="margin-left:4.8pt;"> 
 <tbody> 
 <tr> 
 <td style="width:132px;"> Item_Identifier </td> 
 <td style="width:205px;"> Item_Weight </td> 
 <td style="width:137px;"> Item_Fat_Content </td> 
 <td style="width:164px;"> Item_Visibility </td> 
 <td style="width:113px;"> Item_Type </td> 
 <td style="width:89px;"> Item_MRP </td> 
 </tr> 
 <tr> 
 <td style="width:132px;"> Outlet_Identifier </td> 
 <td style="width:205px;"> Outlet_Establishment_Year </td> 
 <td style="width:137px;"> Outlet_Size </td> 
 <td style="width:164px;"> Outlet_Location_Type </td> 
 <td style="width:113px;"> Outlet_Type </td> 
 <td style="width:89px;">id</td> 
 </tr> 
 </tbody> 
</table>

## 3、衍生字段案例具体实现过程 ##

### T1、选择的基字段(1个) ###

\[*'id'*,*'Item\_Identifier'*,*'Item\_Weight'*,*'Outlet\_Identifier'*,*'Outlet\_Establishment\_Year'*, *'Outlet\_Size'*\]

### **(1)、生成的所有衍生字段** ###

<table style="margin-left:4.8pt;"> 
 <tbody> 
 <tr> 
 <td style="width:63.6pt;"> id </td> 
 <td style="width:260px;"> Item_Identifier </td> 
 <td style="width:312px;"> Item_Weight </td> 
 <td style="width:242px;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="width:63.6pt;">&nbsp;</td> 
 <td style="width:260px;"> Outlet_Identifier </td> 
 <td style="width:312px;"> outlet.Outlet_Establishment_Year </td> 
 <td style="width:242px;"> outlet.Outlet_Size </td> 
 </tr> 
 <tr> 
 <td style="width:63.6pt;">outlet.COUNT(bigmart)</td> 
 <td style="width:260px;"> outlet.MODE(bigmart.Item_Identifier) </td> 
 <td style="width:312px;"> outlet.NUM_UNIQUE(bigmart.Item_Identifier) </td> 
 <td style="width:242px;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="width:63.6pt;">&nbsp;</td> 
 <td style="width:260px;"> outlet.MAX(bigmart.Item_Weight) </td> 
 <td style="width:312px;"> outlet.MEAN(bigmart.Item_Weight) </td> 
 <td style="width:242px;"> outlet.MIN(bigmart.Item_Weight) </td> 
 </tr> 
 <tr> 
 <td style="width:63.6pt;">&nbsp;</td> 
 <td style="width:260px;"> outlet.SKEW(bigmart.Item_Weight) </td> 
 <td style="width:312px;"> outlet.STD(bigmart.Item_Weight) </td> 
 <td style="width:242px;">outlet.SUM(bigmart.Item_Weight)</td> 
 </tr> 
 </tbody> 
</table>

### **(2)、父衍生字段—outlet.COUNT(bigmart)** ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 3][]

outlet.COUNT(bigmart)字段意义：是根据Outlet\_Identifier字段内10个子类别的整体个数，分别进行填充设计。  
outlet.COUNT(bigmart)字段内容：根据该字段的下拉列表可知，标记为1550的个数有4650个，是因为在Outlet\_Identifier字段内，OUT035、OUT046、OUT049三个子类别均有1550个，所以，以上结果比较巧。

### **(3)、子衍生字段的具体字段分析** ###

outlet.MODE(bigmart.Item\_Identifier)    字段的分析过程

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 4][]

### T2、选择的基字段(2个) ###

\[*'Item\_Identifier'*,*'Item\_Weight'*,*'Item\_Fat\_Content'*,*'Outlet\_Identifier'*,*'Outlet\_Establishment\_Year'*, *'Outlet\_Size'*\]

<table> 
 <tbody> 
 <tr> 
 <td style="width:250px;"> id </td> 
 <td style="width:264px;"> Item_Identifier </td> 
 <td style="width:302px;"> Item_Weight </td> 
 <td style="width:77pt;"> Item_Fat_Content </td> 
 <td style="width:82.55pt;">&nbsp;</td> 
 <td style="width:73.6pt;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="width:250px;">&nbsp;</td> 
 <td style="width:264px;"> Outlet_Identifier </td> 
 <td style="width:302px;"> outlet.Outlet_Establishment_Year </td> 
 <td style="width:80.45pt;"> outlet.Outlet_Size </td> 
 <td style="width:82.55pt;">&nbsp;</td> 
 <td style="width:73.6pt;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="width:250px;">outlet.COUNT(bigmart)</td> 
 <td style="width:264px;"> outlet.MODE(bigmart.Item_Identifier) </td> 
 <td style="width:302px;"> outlet.NUM_UNIQUE(bigmart.Item_Identifier) </td> 
 <td style="width:77pt;">&nbsp;</td> 
 <td style="width:82.55pt;">&nbsp;</td> 
 <td style="width:73.6pt;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td style="width:250px;"> outlet.MAX(bigmart.Item_Fat_Content) </td> 
 <td style="width:264px;"> outlet.MEAN(bigmart.Item_Fat_Content) </td> 
 <td style="width:302px;"> outlet.MIN(bigmart.Item_Fat_Content) </td> 
 <td style="width:80.45pt;"> outlet.SKEW(bigmart.Item_Fat_Content) </td> 
 <td style="width:86pt;"> outlet.STD(bigmart.Item_Fat_Content) </td> 
 <td style="width:82.55pt;"> outlet.SUM(bigmart.Item_Fat_Content) </td> 
 </tr> 
 <tr> 
 <td style="width:250px;">outlet.MAX(bigmart.Item_Weight)</td> 
 <td style="width:264px;">outlet.MEAN(bigmart.Item_Weight)</td> 
 <td style="width:302px;">outlet.MIN(bigmart.Item_Weight)</td> 
 <td style="width:77pt;">outlet.SKEW(bigmart.Item_Weight)</td> 
 <td style="width:82.55pt;">outlet.STD(bigmart.Item_Weight)</td> 
 <td style="width:73.6pt;">outlet.SUM(bigmart.Item_Weight)</td> 
 </tr> 
 </tbody> 
</table>

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 5][]

## 核心代码 ##

#2.2、特征衍生
    #(1)、创建一个实体集EntitySet：实体集是一种包含多个数据帧及其之间关系的结构。
    ftES = ft.EntitySet(id = 'sales') 
    
    
    #(2)、规范化实体集：
    ftES.normalize_entity(base_entity_id='bigmart', 
                        new_entity_id='outlet', 
                        index = 'Outlet_Identifier', 
                        additional_variables = ['Outlet_Establishment_Year', 'Outlet_Size',
    #                                             'Outlet_Location_Type', 'Outlet_Type',
                                                ])
    
    
    
    #(3)、利用dfs函数自动创建新特征并输出
    feature_matrix, feature_names = ft.dfs(entityset=ftES,target_entity = 'bigmart', 
                                           max_depth = 2,
    #                                        verbose = 1, n_jobs = -1,
                                           )
    print('feature_names: \n',len(feature_names),feature_names)
    print('feature_matrix:',feature_matrix.shape,type(feature_matrix),'\n',feature_matrix.iloc[:,:10].head())
    feature_matrix.to_csv('%sfeature_matrix.csv'%date)
    
    feature_matrix = feature_matrix.reindex(index=data_all['id'])       # reindex()函数，重新把列/行根据columns/index参数指定字段顺序整理一遍
    feature_matrix = feature_matrix.reset_index()                       # reset_index()函数，重新设置dataframe的index，范围为0~len（df）
    feature_matrix.to_csv('%sfeature_matrix2reset_index01.csv'%date)

[ML_FE_BigMartSales_Featuretools_1_dataframe_2_Entity]: https://yunyaniu.blog.csdn.net/article/details/115601605
[ML_FE_BigMartSales_Featuretools_1_dataframe_2_Entity 1]: https://yunyaniu.blog.csdn.net/article/details/115648223
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70]: /images/20221022/6eb551de757f422c83771edf657140c8.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 1]: /images/20221022/0a8792533f6d4e8ab86cb848c2234bbf.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 2]: /images/20221022/c03f336e61c744789d51b5a6905ec40a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 3]: /images/20221022/3694be28a53f4426b625ddd31d2392e3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 4]: /images/20221022/052b5469aa1f47b48023a889f4abd7f5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMTg1ODY4_size_16_color_FFFFFF_t_70 5]: /images/20221022/beff93b6cfb94ef38e55cab7b66299c9.png