CIFAR10及cifar100 dataset 的下载与使用、转图片

蔚落 2023-06-24 08:17 9阅读 0赞

【时间】2019.12.21

【题目】CIFAR-10 dataset 的下载与使用、转图片

cifar10与cifar100下载地址：[http://www.cs.toronto.edu/~kriz/cifar.html][http_www.cs.toronto.edu_kriz_cifar.html]

## 一、cifar10介绍及转换图片 ##

具体参考：[CIFAR-10 dataset 的下载与使用、转图片][CIFAR-10 dataset].

### 解压源文件的函数： ###

import cv2
    import numpy as np
    import pickle
    import os
    
    # 解压缩，返回解压后的字典
    def unpickle(file):
        fo = open(file, 'rb')
        dict = pickle.load(fo, encoding='latin1')
        fo.close()
        return dict

### cifar10转图片 ： ###

def cifar10_to_images():
      tar_dir='../data/cifar-10-batches-py/' #原始数据库目录
      train_root_dir='../data/cifar10/train/' #图片保存目录
      test_root_dir='../data/cifar10/test/'
      if not os.path.exists(train_root_dir):
        os.makedirs(train_root_dir)
      if not os.path.exists(test_root_dir):
        os.makedirs(test_root_dir)
      # 生成训练集图片，如果需要png格式，只需要改图片后缀名即可。
      label_names = ["airplane", "automobile", "bird", "cat", "dear", "dog", "frog", "horse", "ship", "truck"]
      for j in range(1, 6):
          dataName = tar_dir+"data_batch_" + str(j) 
          Xtr = unpickle(dataName)
          print(dataName + " is loading...")
    
          for i in range(0, 10000):
              img = np.reshape(Xtr['data'][i], (3, 32, 32))  # Xtr['data']为图片二进制数据
              img = img.transpose(1, 2, 0)  # 读取image
              picName = train_root_dir + str(Xtr['labels'][i])+ '_' + label_names[Xtr['labels'][i]]+'_'+ str(i + (j - 1)*10000)+'.jpg'  # label+class+index
              cv2.imwrite(picName, img)
          print(dataName + " loaded.")
    
      print("test_batch is loading...")
    
      # 生成测试集图片
      testXtr = unpickle(tar_dir+"test_batch")
      for i in range(0, 10000):
          img = np.reshape(testXtr['data'][i], (3, 32, 32))
          img = img.transpose(1, 2, 0)
          picName = test_root_dir + str(testXtr['labels'][i])+ '_' + label_names[testXtr['labels'][i]]+'_' + str(i) + '.jpg'
          cv2.imwrite(picName, img)
      print("test_batch loaded.")

### 结果：文件名为 \# label\_class\_index.jpg ###

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu_size_16_color_FFFFFF_t_70][]

## 二、cifar100介绍及转换图片 ##

cifar100包含20个大类，共100类，train集50000张图片，test集10000张图片。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu_size_16_color_FFFFFF_t_70 1][]

### cifar100转图片 ： ###

def cifar100_to_images():
      tar_dir='../data/cifar-100-python/' #原始数据库目录
      train_root_dir='../data/cifar100/train/' #图片保存目录
      test_root_dir='../data/cifar100/test/'
      if not os.path.exists(train_root_dir):
        os.makedirs(train_root_dir)
      if not os.path.exists(test_root_dir):
        os.makedirs(test_root_dir)
    
      #获取label对应的class，分为20个coarse class，共100个 fine class
      meta_Name = tar_dir+"meta" 
      Meta_dic= unpickle(meta_Name)
      coarse_label_names=Meta_dic['coarse_label_names']
      fine_label_names=Meta_dic['fine_label_names']
      print(fine_label_names)
    
      #生成训练集图片，如果需要png格式，只需要改图片后缀名即可。
      dataName = tar_dir+"train" 
      Xtr = unpickle(dataName)
      print(dataName + " is loading...")
      for i in range(0,Xtr['data'].shape[0]):
          img = np.reshape(Xtr['data'][i], (3, 32, 32))  # Xtr['data']为图片二进制数据
          img = img.transpose(1, 2, 0)  # 读取image
          ###img_name:fine_label+coarse_label+fine_class+coarse_class+index
          picName = train_root_dir + str(Xtr['fine_labels'][i])+ '_' + str(Xtr['coarse_labels'][i]) + '_&' +fine_label_names[Xtr['fine_labels'][i]]+'&_'+coarse_label_names[ Xtr['coarse_labels'][i]]+'_'+str(i) + '.jpg' 
          cv2.imwrite(picName, img)
      print(dataName + " loaded.")
    
      print("test_batch is loading...")
      # 生成测试集图片
      testXtr = unpickle(tar_dir+"test")
      for i in range(0, testXtr['data'].shape[0]):
          img = np.reshape(testXtr['data'][i], (3, 32, 32))
          img = img.transpose(1, 2, 0)
          picName = test_root_dir +str(testXtr['fine_labels'][i])+ '_' + str(testXtr['coarse_labels'][i]) + '_&' +fine_label_names[testXtr['fine_labels'][i]]+'&_'+coarse_label_names[ testXtr['coarse_labels'][i]]+'_'+str(i) + '.jpg' 
          cv2.imwrite(picName, img)
      print("test_batch loaded.")

### 结果：文件名为 \# fine\_label+coarse\_label+fine\_class+coarse\_class+index.jpg ###

fine表示类，coarse表示大类。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu_size_16_color_FFFFFF_t_70 2][]

[http_www.cs.toronto.edu_kriz_cifar.html]: http://www.cs.toronto.edu/~kriz/cifar.html
[CIFAR-10 dataset]: https://blog.csdn.net/ctwy291314/article/details/83864405
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20191221231818979.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20191221232044672.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20191221232434230.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L0NfY2h1eGlu,size_16,color_FFFFFF,t_70