Merge branch 'master' of http://62.234.201.16/hao/Model-Transfer-Adaptability

ykl/ALL-cifar10/README.md

+# 模型整合说明
+
++ 该文件夹下实现了基于cifar10数据集的AlexNet、AlexNet_BN、VGG_16、VGG_19、Inception_BN的整合。
++ 新增了ResNet_18、ResNet_50、ResNet_152、MobileNetV2的整合
+
+### 部署说明
+
+#### cfg_table
+
+在该通用框架下，当前所有模型部署只需要提供一个cfg_table，如包含特殊结构（如Inception_BN的inception多分支结构），额外针对特殊结构提供cfg_table即可。详见`cfg.py`
+
+cfg_table书写规则说明如下：
+
++ 每项根据进行量化的合并单位给出，例如Conv后接BN或BN+ReLU或BN+ReLU6时，将被统一合并为一个量化层，则在cfg_table中表现为一项，''/'B'/'BRL'/'BRS'参数可选。
++ 针对不涉及量化，但位置需特别指定的操作，如drop、view、softmax，同样在cfg_table中单独指定。
+
+根据cfg_table，也相应简化了fold_ratio和fold_model方法，合并与量化层进行对比的全精度层的参数值、参数量、计算量。
+
+#### 训练方案
+
++ 使用了新的学习率方案，概述如下：
+
+  + 更大的batch_size(32->128)，可以更充分学习到数据分布，以期得到更好的泛化性能和量化效果
+  + 更小的初始学习率、余弦退火策略学习率调度器`CosineAnnealingLR`（周期500epochs）：以获得更平缓的参数更新，并与更大的batch_size相适应
+  + 数据增强：dataloader的transform由Resize更改为RandomCrop
+  + 验证集：从训练集切分20%的样本作为验证集，防止测试集过拟合，也避免引入测试集噪声导致量化效果的偏差。
+  + weight_decay：惩罚过大权重值，避免过拟合，同时一定程度增强量化效果。
+  + “早停”：当连续patience个epoch未取得精度提升时，如果已经达到过最小精度，则早停，从而避免过拟合
+  + “复活”：当未达到最小精度界限时，满足上述条件不停止，而是重载bestpoint。该方法最后未采用
+
++ 不同训练方式的全精度模型准确度和ptq拟合结果
+
+  + SGD优化器（初始lr=0.005,end_lr_bound=0.00005）
+  + SGD+momentum=0.9
+  + SGD+momentum=0.95+weight_decay=1e-4+nesterov。最终采用
+  + Adam优化器（初始lr=0.001,end_lr_bound=0.00001）
+
+| Model        | S_acc | S_R2                   | S_m9_acc | S_m9_R2 | S_m95_acc | S_m95_R2 | A_acc | A_R2   |
+| ------------ | ----- | ---------------------- | -------- | ------- | --------- | -------- | ----- | ------ |
+| AlexNet      | 87.33 | 0.9535                 | 88.91    | 0.9202  | 87.15     | 0.9587   | 87.75 | 0.4999 |
+| AlexNet_BN   | 88.17 | 0.9641                 | 90.15    | 0.9678  | 89.02     | 0.9508   | 89.86 | 0.8492 |
+| VGG_16       | 89.03 | 0.8411                 | 92.14    | 0.7813  | 91.67     | 0.8929   | 92.75 | 0.7334 |
+| VGG_19       | 88.67 | 0.8750                 | 92.27    | 0.7829  | 91.68     | 0.9155   | 92.26 | 0.6578 |
+| Inception_BN | 88.55 | 0.9218                 | 92.78    | 0.9725  | 92.40     | 0.9776   | 93.77 | 0.9121 |
+| ResNet_18    | 84.32 | 0.9493                 | 89.24    | 0.9699  | 87.72     | 0.9538   | 90.34 | 0.9585 |
+| ResNet_50    | 82.10 | 0.9498                 | 89.85    | 0.9693  | 89.52     | 0.9692   | 91.61 | 0.9594 |
+| ResNet_152   | 79.76 | 0.8947                 | 89.35    | 0.7978  | 88.92     | 0.8519   | 91.78 | 0.9083 |
+| MobileNetV2  | 84.61 | 0.9918（no tolerance） | 88.98    | 0.9632  | 88.93     | 0.9882   | 91.93 | 0.8082 |
+| ALL          | n     | 0.8014                 | n        | 0.6787  | n         | 0.7189   | n     | 0.6726 |
+
++ 结果展示：
+
+  + sgd-AlexNet
+
+    <img src="image/sgd-AlexNet.png" alt="sgd-AlexNet" style="zoom:50%;" />
+
+  + sgd+momentum95+...-AlexNet
+
+    <img src="image/sgd-m-AlexNet.png" alt="sgd-m-AlexNet" style="zoom:50%;" />
+
+  + adam-AlexNet
+
+    <img src="image/adam-AlexNet.png" alt="adam-AlexNet" style="zoom:50%;" />
+
+  + 最终采用SGD+momentum=0.95+weight_decay=1e-4+nesterov
+
+    以下为cifar10上拟合效果，均直接使用js_flops，未进行加权处理
+
+    + 未分段
+
+    <img src="image/sgd-ALL.png" alt="sgd-ALL" style="zoom:50%;" />
+
+    + 分段（3.5为分界）
+
+      <img src="image/sgd-sep.png" alt="sgd-sep" style="zoom:50%;" />
+
+  + SGD和Adam差别原因分析：
+
+    1. Adam优化器采用动态调整的学习率,通常会使模型更新的权重参数具有较大的变化范围。而SGD使用固定学习率,参数更新幅度较小。这会导致Adam优化器训练出的模型参数分布更加集中,而SGD训练出的模型参数分布更加分散,有利于量化。
+
+    2. Adam优化器在反向传播时会对梯度进行额外的平滑和校正,缓解梯度消失与爆炸问题；在参数更新时,会考虑一阶矩和二阶矩的影响,实现比较平稳的更新。而SGD直接使用原始梯度。这也会导致Adam优化器训练出的模型参数更加集中,不利于量化。
+
+    3. Adam优化器对小的梯度更加敏感,可以更快找到较优解。但对模型量化不利。SGD相对而言更新步骤更大,有利于量化。
+
+    所以,总体来说,Adam优化器采用的平滑逐步的梯度更新策略,会得到参数分布更加集中的模型。这种模型不利于进行有损压缩的量化。而SGD的比较激进的更新方式,会得到参数分布更加分散的模型,更加适合量化。
+
++ 如果想加快训练速度，可以缩减周期训练epoch数，并增加dataloader的num_worker以缩短数据加载实现。（num_workder的并行加载有概率出现访问共享内存错误。不进行手动指定随机种子，则每个num_worker会自动分配不同的随机种子，可以一定程度避免这个错误）
+
+### PTQ部分概述
+
+#### matlab脚本
+
++ `flops\param`、`flops\param_all`、`flops_all_weight`分别是对单个模型、全体模型、计算量加权后全体模型的拟合。
+  针对单个模型，脚本使用不同颜色标记不同量化类别的点；针对全体模型，脚本使用不同颜色标记不同模型的点。
++ 脚本内添加了拟合条件限制，以保证拟合曲线一定是单调不降的。如果想允许拟合曲线在某个位置出现略微的转折下降，可以改动tolerance为一个较小的负数，如-1e-5。
++ 拟合条件限制基于采样实现，对于fakefreeze，采样步长设为0.1，对于L2方法，采样步长设为10
++ 脚本同时在图像上输出拟合效果评估，如SSE、R方、RMSE指标。
++ 支持选择拟合模型，当前可选择poly为2/3/4分别表示rational模型分子分母的多项式次数
++ 由于每次拟合结果会具有细微差别，后续可以更改脚本，使得脚本在每次拟合完成后再进行三次符合约束的拟合，效果均不好于当前拟合才终止，否则取效果更好的拟合结果，并重复该过程。该步骤目前由人工完成
++ 新增flops_sep，对poly2提供了分段拟合。
+
+#### 拟合方式及结果简述
+
++ 使用fakefreeze方法对量化层参数rescale到和全精度层一致。
++ 将weight：bias的js散度权重比设置为1：1，即有bias的层，weight影响不变，bias额外施加影响。
++ 对模型计算量进行log10和cuberoot加权
+
+
+
+#### 拟合中发现的问题
+
+在VGG_16 VGG_19 Inception_BN的fakefreeze方式中，都观察到POT量化点扎堆（acc_loss具有略小差距，js_div相近，在图上表现为连续的一竖列点），影响了量化效果。
+
+观察这些模型的权重参数分布，可以发现出现问题的模型存在着无尖端的权重分布。而有尖无尖的分布在面对不同量化方式的分布如下：
+
+![diff1](image/diff1.png)
+
+![diff2](image/diff2.png)
+
+根据不同模型权重分布的特点，可以推测出现问题的模型POT量化散度较大且集中的重要原因是量化后分布与原分布趋势不同。基于此，我们可能需要在相似度之外额外的考虑模型参数分布与量化方式的适配性。这需要进行实验的验证，例如，直接度量全精度模型-量化模型用于衡量分布趋势的系数；度量全精度权重的尖锐程度和量化表的尖锐程度等。并将所得值作用于原先所求js散度上。
+
++ 方案一：度量全精度模型、量化模型分布趋势相似度
+
+  使用pearson相关系数或余弦相似度，并作用于js散度。例如，若POT量化的余弦相似度较小（趋势差异较大），考虑将js散度乘余弦相似度，从而矫正因趋势过大的散度。
+
++ 方案二：考虑尖锐程度
+
+  考虑到无尖端分布遇到有极大尖端的POT量化点列表会产生不同趋势的问题，从分布和量化点的角度入手。例如，衡量在均值范围内的比例，差异较大可能说明尖锐程度差异大，从而矫正js散度。或者可以考虑对原分布做bins切割，若某个bins有量化点则统计该bins内元素，考虑所有和量化点在同一bins的点数以衡量分布与量化方式的适配度。
+
+#### 后续增强拟合效果的方案
+
++ 针对POT量化点扎堆，可以考虑使用更关注趋势的Pearson相关系数、余弦相似度等对js散度进行修正，或者考虑将量化范围切分多个bins评估量化点覆盖率的方式修正。
+
++ 对weight和bias采取更合理的加权方式
+
+  + 根据对精度的影响（不易衡量，不易确定基准）
+
++ 在模型整合上，尝试更有效的加权方式
+
++ 考虑到js散度达到一定值后acc_loss不会再上升（因为最差效果是随机分类，准确度也有10%），采取分段拟合的方式。
+
+  
+

ykl/ALL/ptq_nodiv.py → ykl/ALL-cifar10/analyse.py

@@ -17,13 +17,14 @@ import torch.utils.bottleneck as bn
 import os
 import os.path as osp
 from torch.utils.tensorboard import SummaryWriter
+from analyse_utils import *
 
 
 def direct_quantize(model, test_loader,device):
     for i, (data, target) in enumerate(test_loader, 1):
         data = data.to(device)
         output = model.quantize_forward(data).cpu()
-        if i % 500 == 0:
+        if i % 100 == 0:
             break
     print('direct quantization finish')
 
@@ -52,97 +53,82 @@ def quantize_inference(model, test_loader, device):
 
 
 if __name__ == "__main__":
-    batch_size = 32
+    batch_size = 128
 
     device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
     print(device)
     train_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=True, download=True,
+    datasets.CIFAR10('../data', train=True, download=False,
                      transform=transforms.Compose([
-                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                        transforms.RandomCrop(32, padding=2),
                         transforms.RandomHorizontalFlip(),
                         transforms.ToTensor(),
-                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                        transforms.Normalize(
+                            (0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
                         ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+     batch_size=batch_size, shuffle=True, num_workers=4, pin_memory=True
     )
 
     test_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
-            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+    datasets.CIFAR10('../data', train=False, download=False, 
+                     transform=transforms.Compose([
                             transforms.ToTensor(),
-            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                            transforms.Normalize((0.4914, 0.4822, 0.4465),
+                                                (0.2023, 0.1994, 0.2010))
                             ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    batch_size=batch_size, shuffle=False, num_workers=4, pin_memory=True
     )
 
-    load_ptq = False
-    store_ptq = False
+    load_ptq = True
+    store_ptq = True
 
     append = True
     gol._init()
 
-    excel_path = 'ptq_result_nodiv.xlsx'
-    if os.path.exists(excel_path) and append:
-        workbook = openpyxl.load_workbook(excel_path)
-    else:
-        workbook = openpyxl.Workbook()
-    if 'Sheet' in workbook.sheetnames:
-        workbook.remove(workbook['Sheet'])
+    # ckpt_path = 'ckpt_sgd_5_momentum_95'
+    ckpt_path = 'ckpt_adam_1'
     
-    txt_path = 'ptq_result_nodiv.txt'
-    if os.path.exists(txt_path) and append:
-        ft = open(txt_path,'a')
-    else:
-        ft = open(txt_path,'w')
-    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
-    for model_name in model_name_list:
-        if model_name in workbook.sheetnames:
-            continue
+    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN',
+                       'MobileNetV2','ResNet_18','ResNet_50','ResNet_152']
+    # model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN',
+    #                    'MobileNetV2','ResNet_18','ResNet_50']
+    # model_name_list = ['MobileNetV2']
+    model_name = sys.argv[1]
 
     model = Model(model_name)
-        full_file = 'ckpt/cifar10_'+model_name+'.pt'
+    full_file = ckpt_path+'/cifar10_'+model_name+'.pt'
     model.load_state_dict(torch.load(full_file))
     model.to(device)
 
-        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
+    ptq_path = ckpt_path+'/'+model_name
+    if not osp.exists(ptq_path):
+        os.makedirs(ptq_path)
+    ptq_file_prefix = ptq_path+'/cifar10_'+model_name+'_ptq_'
 
     model.eval()
     full_acc = full_inference(model, test_loader, device)
     # 传入后可变
     fold_model(model)
 
-        layer, par_ratio, flop_ratio = extract_ratio(model_name)
+    Mac,Param,layer, par_ratio, flop_ratio = extract_ratio(model_name)
     par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
 
     full_names = []
     full_params = []
 
+    writer = SummaryWriter(log_dir='./log/'+model_name)
     for name, param in model.named_parameters():
         if 'conv' in name or 'fc' in name:
             full_names.append(name)
             full_params.append(param.data.cpu())
+            writer.add_histogram(tag='Full_' + name, values=param.data)
 
-        #统计每个参数对应层的参数个数
-        full_par_num=[]
-        for name in full_names:
-            prefix = name.rsplit('.',1)[0]
-            cnt = 0
-            for str in full_names:
-                sprefix = str.rsplit('.',1)[0]
-                if prefix == sprefix:
-                    cnt += 1
-            full_par_num.append(cnt)
     
     quant_type_list = ['INT','POT','FLOAT']
     # quant_type_list = ['INT']
     # quant_type_list = ['POT']
     # quant_type_list = ['INT','POT']
     title_list = []
-        js_flops_list = []
-        js_param_list = []
-        ptq_acc_list = []
-        acc_loss_list = []
 
     for quant_type in quant_type_list:
         num_bit_list = numbit_list(quant_type)
@@ -184,9 +170,7 @@ if __name__ == "__main__":
 
                 model_ptq.freeze()
                 ptq_acc = quantize_inference(model_ptq, test_loader, device)
-                    ptq_acc_list.append(ptq_acc)
                 acc_loss = (full_acc - ptq_acc) / full_acc
-                    acc_loss_list.append(acc_loss)
 
                 #将量化后分布反量化到全精度相同的scale
                 model_ptq.fakefreeze()
@@ -196,7 +180,9 @@ if __name__ == "__main__":
                 for name, param in model_ptq.named_parameters():
                     if 'conv' not in name and 'fc' not in name:
                         continue
-                            
+                    writer.add_histogram(tag=title+'_'+name,values=param.data)
+                    print('-'*89)
+                    print(name)
                     prefix = name.rsplit('.',1)[0]
                     layer_idx = layer.index(prefix)
                     name_idx = full_names.index(name)
@@ -209,46 +195,52 @@ if __name__ == "__main__":
                     js = js.item()
                     if js < 0.:
                         js = 0.
+                    js_flops_layer = js * flop_ratio[layer_idx]
+                    js_param_layer = js * par_ratio[layer_idx]
+                    print('js_flops:%f'%js_flops_layer)
+                    print('js_param:%f'%js_param_layer)
+                    
+                    if quant_type == 'INT':
+                        if 'bias' in name:
+                            plist = build_int_list(16)
+                        else:
+                            plist = build_int_list(num_bits)
+                    else:
+                        if 'bias' in name:
+                            plist = gol.get_value(True)
+                        else:
+                            plist = gol.get_value(False)
+                    list_std_mid_ratio = std_mid_ratio(plist)
+                    list_range_mid_ratio = range_mid_ratio(plist)
+                    full_std_mid_ratio = std_mid_ratio(full_params[name_idx])
+                    full_range_mid_ratio = range_mid_ratio(full_params[name_idx])
+                    ptq_std_mid_ratio = std_mid_ratio(ptq_param)
+                    ptq_range_mid_ratio = range_mid_ratio(ptq_param)
+                    print('std_mid_ratio:')
+                    print('  plist:%f'%list_std_mid_ratio)
+                    print('  full :%f'%full_std_mid_ratio)
+                    print('  ptq  :%f'%ptq_std_mid_ratio)
+                    print('range_mid_ratio:')
+                    print('  plist:%f'%list_range_mid_ratio)
+                    print('  full :%f'%full_range_mid_ratio)
+                    print('  ptq  :%f'%ptq_range_mid_ratio)
+                    cossim = cos_sim(full_params[name_idx],ptq_param)
+                    print('cossim:%f'%cossim)
+                    pearson = pearson_corr(full_params[name_idx],ptq_param)
+                    print('pearson:%f'%pearson)
+                    list_kurt = kurtosis(plist)
+                    full_kurt = kurtosis(full_params[name_idx])
+                    ptq_kurt = kurtosis(ptq_param)
+                    print('kurtosis:')
+                    print('  plist:%f'%list_kurt)
+                    print('  full :%f'%full_kurt)
+                    print('  ptq  :%f'%ptq_kurt)
                     js_flops = js_flops + js * flop_ratio[layer_idx]
                     js_param = js_param + js * par_ratio[layer_idx]
-                    js_flops_list.append(js_flops)
-                    js_param_list.append(js_param)
-
-                    print(title + ': js_flops: %f js_param: %f acc_loss: %f' % (js_flops, js_param, acc_loss))
-        
-        # 写入xlsx
-        worksheet = workbook.create_sheet(model_name)
-        worksheet.cell(row=1,column=1,value='FP32-acc')
-        worksheet.cell(row=1,column=2,value=full_acc)
-        worksheet.cell(row=3,column=1,value='title')
-        worksheet.cell(row=3,column=2,value='js_flops')
-        worksheet.cell(row=3,column=3,value='js_param')
-        worksheet.cell(row=3,column=4,value='ptq_acc')
-        worksheet.cell(row=3,column=5,value='acc_loss')
-        for i in range(len(title_list)):
-            worksheet.cell(row=i+4, column=1, value=title_list[i])
-            worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
-            worksheet.cell(row=i+4, column=3, value=js_param_list[i])
-            worksheet.cell(row=i+4, column=4, value=ptq_acc_list[i])
-            worksheet.cell(row=i+4, column=5, value=acc_loss_list[i])
-
-        if 'Sheet' in workbook.sheetnames:
-            workbook.remove(workbook['Sheet'])
-        workbook.save(excel_path)
-        
-        print(model_name,file=ft)
-        print('Full_acc: %f'%full_acc,file=ft)
-        print('title_list:',file=ft)
-        print(title_list,file=ft)
-        print('js_flops_list:',file=ft)
-        print(js_flops_list, file=ft)
-        print('js_param_list:',file=ft)
-        print(js_param_list, file=ft)
-        print('ptq_acc_list:',file=ft)
-        print(ptq_acc_list, file=ft)
-        print('acc_loss_list:',file=ft)
-        print(acc_loss_list, file=ft)
-        print("\n",file=ft)
-
-    ft.close()
+
+                print(title + ': js_flops: %f js_param: %f acc_loss: %f' % (js_flops,js_param, acc_loss))
+                print('='*89)
+        
+    writer.close()
+
 

ykl/ALL-cifar10/analyse_one.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 0-08:00:00
+### #SBATCH -t 1-06:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-short
+### #SBATCH --qos=gpu-normal             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+python analyse.py $Model
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL-cifar10/analyse_utils.py

+import torch
+
+def build_int_list(num_bits):
+    plist = [0.]
+    for i in range(0,2**(num_bits-1)): 
+        # i最高到0，即pot量化最大值为1
+        plist.append(i)
+        plist.append(-i)
+    plist = torch.Tensor(list(set(plist)))
+    # plist = plist.mul(1.0 / torch.max(plist))
+    return plist
+
+def std_mid_ratio(x):
+    x = x.view(-1)
+    std = torch.std(x)
+    max = 3*std#.item()
+    min = 3*(-std)#.item()
+    max = torch.max(torch.abs(max),torch.abs(min))
+    mid = max/2
+    cond = torch.logical_and(x>=-mid,x<=mid)
+    cnt = torch.sum(cond).item()
+    ratio = cnt/len(x)
+    return ratio
+
+def range_mid_ratio(x):
+    x = x.view(-1)
+    max = torch.max(x)
+    min = torch.min(x)
+    max = torch.max(torch.abs(max),torch.abs(min))
+    mid = max/2
+    cond = torch.logical_and(x>=-mid,x<=mid)
+    cnt = torch.sum(cond).item()
+    ratio = cnt/len(x)
+    return ratio
+
+def pearson_corr(tensor1, tensor2):
+    """
+    计算tensor1和tensor2的Pearson相关系数
+    """
+    if torch.equal(tensor1,tensor2):
+        return 1.0
+    # 将tensor1和tensor2展平为二维
+    tensor1 = tensor1.view(-1, tensor1.size(-1))
+    tensor2 = tensor2.view(-1, tensor2.size(-1))
+    
+    # 计算tensor1和tensor2的均值
+    tensor1_mean = torch.mean(tensor1, dim=0)
+    tensor2_mean = torch.mean(tensor2, dim=0)
+    
+    # 计算centered tensor
+    tensor1_c = tensor1 - tensor1_mean
+    tensor2_c = tensor2 - tensor2_mean
+    
+    # 计算covariance matrix
+    cov_mat = torch.matmul(tensor1_c.t(), tensor2_c) / (tensor1.size(0) - 1)
+    
+    # 计算相关系数
+    corr_mat = cov_mat / torch.std(tensor1, dim=0) / torch.std(tensor2, dim=0)
+    pearson = torch.mean(corr_mat)
+    return pearson.item()
+
+def cos_sim(a,b):
+    a = a.view(-1)
+    b = b.view(-1)
+    cossim = torch.cosine_similarity(a, b, dim=0, eps=1e-6)
+    # cossim = (cossim-0.97)/(1-0.97)
+    return cossim.item()
+
+def kurtosis(tensor):
+    mean = tensor.mean()
+    std = tensor.std(unbiased=False)
+    n = tensor.numel()
+    fourth_moment = ((tensor - mean)**4).sum() / n
+    second_moment = std**2
+    kurt = (fourth_moment / second_moment**2) - 3
+    return kurt.item()
\ No newline at end of file

ykl/ALL-cifar10/cfg.py

+# conv: 'C',''/'B'/'BRL'/'BRS',qi,in_ch,out_ch,kernel_size,stirde,padding,bias
+# relu: 'RL'
+# relu6: 'RS'
+# inception: 'Inc'
+# maxpool: 'MP',kernel_size,stride,padding
+# adaptiveavgpool: 'AAP',output_size
+# view: 'VW': 
+#   dafault: x = x.view(x.size(0),-1)
+# dropout: 'D'
+# MakeLayer: 'ML','BBLK'/'BTNK'/'IRES', ml_idx, blocks
+# softmax: 'SM'
+# class 10
+ResNet_18_cfg_table = [
+    ['C','BRL',True,3,16,3,1,1,True],
+    ['ML','BBLK',0,2],
+    ['ML','BBLK',1,2],
+    ['ML','BBLK',2,2],
+    ['ML','BBLK',3,2],
+    ['AAP',1],
+    ['VW'],
+    ['FC',128,10,True],
+    ['SM']
+]
+ResNet_50_cfg_table = [
+    ['C','BRL',True,3,16,3,1,1,True],
+    ['ML','BTNK',0,3],
+    ['ML','BTNK',1,4],
+    ['ML','BTNK',2,6],
+    ['ML','BTNK',3,3],
+    ['AAP',1],
+    ['VW'],
+    ['FC',512,10,True],
+    ['SM']
+]
+ResNet_152_cfg_table = [
+    ['C','BRL',True,3,16,3,1,1,True],
+    ['ML','BTNK',0,3],
+    ['ML','BTNK',1,8],
+    ['ML','BTNK',2,36],
+    ['ML','BTNK',3,3],
+    ['AAP',1],
+    ['VW'],
+    ['FC',512,10,True],
+    ['SM']
+]
+MobileNetV2_cfg_table = [
+    ['C','BRS',True,3,32,3,1,1,True],
+    ['ML','IRES',0,1],
+    ['ML','IRES',1,2],
+    ['ML','IRES',2,3],
+    ['ML','IRES',3,3],
+    ['ML','IRES',4,3],
+    ['ML','IRES',5,1],
+    ['C','',False,320,1280,1,1,0,True],
+    ['AAP',1],
+    ['VW'],
+    ['FC',1280,10,True]
+]
+AlexNet_cfg_table = [
+    ['C','',True,3,32,3,1,1,True],
+    ['RL'],
+    ['MP',2,2,0],
+    ['C','',False,32,64,3,1,1,True],
+    ['RL'],
+    ['MP',2,2,0],
+    ['C','',False,64,128,3,1,1,True],
+    ['RL'],
+    ['C','',False,128,256,3,1,1,True],
+    ['RL'],
+    ['C','',False,256,256,3,1,1,True],
+    ['RL'],
+    ['MP',3,2,0],
+    ['VW'],
+    ['D',0.5],
+    ['FC',2304,1024,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',1024,512,True],
+    ['RL'],
+    ['FC',512,10,True]
+]
+AlexNet_BN_cfg_table = [
+    ['C','BRL',True,3,32,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,32,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,64,128,3,1,1,True],
+    ['C','BRL',False,128,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['MP',3,2,0],
+    ['VW'],
+    ['D',0.5],
+    ['FC',2304,1024,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',1024,512,True],
+    ['RL'],
+    ['FC',512,10,True]
+]
+VGG_16_cfg_table = [
+    ['C','BRL',True,3,64,3,1,1,True],
+    ['C','BRL',False,64,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,64,128,3,1,1,True],
+    ['C','BRL',False,128,128,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,128,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,256,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['VW'],
+    ['FC',512,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,10,True]
+]
+VGG_19_cfg_table = [
+    ['C','BRL',True,3,64,3,1,1,True],
+    ['C','BRL',False,64,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,64,128,3,1,1,True],
+    ['C','BRL',False,128,128,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,128,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,256,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['VW'],
+    ['FC',512,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,10,True]
+]
+Inception_BN_cfg_table = [
+    ['C','',True,3,64,3,1,1,True],
+    ['RL'],
+    ['C','',False,64,64,3,1,1,True],
+    ['RL'],
+    ['Inc',0],
+    ['Inc',1],
+    ['MP',3,2,1],
+    ['Inc',2],
+    ['Inc',3],
+    ['Inc',4],
+    ['Inc',5],
+    ['Inc',6],
+    ['MP',3,2,1],
+    ['Inc',7],
+    ['Inc',8],
+    ['AAP',1],
+    ['C','',False,1024,10,1,1,0,True],
+    ['VW']
+]
+
+model_cfg_table = {
+    'AlexNet'       : AlexNet_cfg_table,
+    'AlexNet_BN'    : AlexNet_BN_cfg_table,
+    'VGG_16'        : VGG_16_cfg_table,
+    'VGG_19'        : VGG_19_cfg_table,
+    'Inception_BN'  : Inception_BN_cfg_table,
+    'ResNet_18'     : ResNet_18_cfg_table,
+    'ResNet_50'     : ResNet_50_cfg_table,
+    'ResNet_152'    : ResNet_152_cfg_table,
+    'MobileNetV2'   : MobileNetV2_cfg_table
+}
+
+#每行对应一个Inc结构(channel)的参数表
+inc_ch_table=[
+    [ 64, 64, 96,128, 16, 32, 32],#3a
+    [256,128,128,192, 32, 96, 64],#3b
+    [480,192, 96,208, 16, 48, 64],#4a
+    [512,160,112,224, 24, 64, 64],#4b
+    [512,128,128,256, 24, 64, 64],#4c
+    [512,112,144,288, 32, 64, 64],#4d
+    [528,256,160,320, 32,128,128],#4e
+    [832,256,160,320, 32,128,128],#5a
+    [832,384,192,384, 48,128,128] #5b
+]
+
+# br0,br1,br2,br3 <- br1x1,br3x3,br5x5,brM
+# 每个子数组对应Inc结构中一个分支的结构，均默认含'BRL'参数，bias为False
+# Conv层第2、3个参数是对应Inc结构(即ch_table中的一行)中的索引
+# 由于每个Inc结构操作一致，只有权重不同，使用索引而非具体值，方便复用
+# 各分支后还有Concat操作，由于只有唯一结构，未特殊说明
+# conv: 'C', ('BRL' default), in_ch_idex, out_ch_idx, kernel_size, stride, padding, (bias: True default)
+# maxpool: 'MP', kernel_size, stride, padding
+# relu: 'RL'
+inc_cfg_table = [
+    [
+        ['C',0,1,1,1,0]
+    ],
+    [
+        ['C',0,2,1,1,0],
+        ['C',2,3,3,1,1]
+    ],
+    [
+        ['C',0,4,1,1,0],
+        ['C',4,5,5,1,2]
+    ],
+    [
+        ['MP',3,1,1],
+        ['RL'],
+        ['C',0,6,1,1,0]
+    ]
+]
+
+# ml_cfg_table = []
+
+#BasicBlock
+#value: downsample,inplanes,planes,planes*expansion,stride,1(dafault stride and group) 
+bblk_ch_table = [
+    [False, 16, 16, 16,1,1], #layer1,first
+    [False, 16, 16, 16,1,1], #       other
+    [True,  16, 32, 32,2,1], #layer2
+    [False, 32, 32, 32,1,1], 
+    [True,  32, 64, 64,2,1], #layer3
+    [False, 64, 64, 64,1,1], 
+    [True,  64,128,128,2,1], #layer4
+    [False,128,128,128,1,1]  
+]
+#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
+#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
+bblk_cfg_table = [
+    [
+        ['C','BRL',1,2,3,4,1,5],
+        ['C','B'  ,2,2,3,5,1,5],
+    ],
+    # downsample, 仅当downsample传入为True时使用
+    [
+        ['C','B'  ,1,3,1,4,0,5]
+    ],
+    # 分支交汇后动作
+    [
+        ['AD',True],
+        ['RL']
+    ]
+]
+
+#BottleNeck
+#value: downsample,inplanes,planes,planes*expansion,stride,1(dafault stride and group) 
+btnk_ch_table = [
+    [True,  16, 16, 64,1,1], #layer1,first
+    [False, 64, 16, 64,1,1], #       other 
+    [True,  64, 32,128,2,1], #layer2
+    [False,128, 32,128,1,1], 
+    [True, 128, 64,256,2,1], #layer3
+    [False,256, 64,256,1,1],  
+    [True, 256,128,512,2,1], #layer4
+    [False,512,128,512,1,1]   
+]
+#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
+#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
+btnk_cfg_table = [
+    [
+        ['C','BRL',1,2,1,5,0,5],
+        ['C','BRL',2,2,3,4,1,5],
+        ['C','B'  ,2,3,1,5,0,5]
+    ],
+    # downsample, 仅当downsample传入为True时使用
+    [
+        ['C','B'  ,1,3,1,4,0,5]
+    ], 
+    # 分支交汇后动作
+    [
+        ['AD',True],
+        ['RL']
+    ]
+]
+
+#InvertedResidual
+#value: identity_flag, in_ch, out_ch, in_ch*expand_ratio, stride, 1(dafault stride and group) 
+ires_ch_table = [
+    [False, 32, 16,  32,1,1], #layer1,first
+    [ True, 16, 16,  16,1,1], #       other
+    [False, 16, 24,  96,2,1], #layer2
+    [ True, 24, 24, 144,1,1],
+    [False, 24, 32, 144,2,1], #layer3
+    [ True, 32, 32, 192,1,1],
+    [False, 32, 96, 192,1,1], #layer4
+    [ True, 96, 96, 576,1,1],
+    [False, 96,160, 576,2,1], #layer5
+    [ True,160,160, 960,1,1],
+    [False,160,320, 960,1,1], #layer6
+    [ True,320,320,1920,1,1]
+]
+
+#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
+#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
+ires_cfg_table = [
+    [
+        ['C','BRS',1,3,1,5,0,5],
+        ['C','BRS',3,3,3,4,1,3],
+        ['C','B'  ,3,2,1,5,0,5]
+    ],
+    # identity_br empty
+    [
+
+    ],
+    # 分支汇合后操作
+    [
+        ['AD',False] #有条件的相加
+    ]
+    
+]
\ No newline at end of file

ykl/ALL/extract_ratio.py → ykl/ALL-cifar10/extract_ratio.py

@@ -7,7 +7,12 @@ def extract_ratio(model_name):
     fr = open('param_flops/'+model_name+'.txt','r')
     lines = fr.readlines()
     
-    Mac = lines[1].split('Mac,')[0].split(',')[-1]
+    #跳过warning
+    for i in range(len(lines)):
+        if 'Model' in lines[i]:
+            head = i+1
+            break 
+    Mac = lines[head].split('Mac,')[0].split(',')[-1]
     if 'M' in Mac:
         Mac = Mac.split('M')[0]
         Mac = float(Mac)
@@ -16,13 +21,18 @@ def extract_ratio(model_name):
         Mac = float(Mac)
         Mac *= 1024
     
-    Param = lines[1].split('M,')[0]
+    Param = lines[head].split(',')[0]
+    if 'M' in Param:
+        Param = Param.split('M')[0]
         Param = float(Param)
+    elif 'k' in Param:
+        Param = Param.split('k')[0]
+        Param = float(Param)
+        Param /= 1024
     
     layer = []
     par_ratio = []
     flop_ratio = []
-    weight_ratio = []
     for line in lines:
         if '(' in line and ')' in line:
             layer.append(line.split(')')[0].split('(')[1])
@@ -32,34 +42,14 @@ def extract_ratio(model_name):
             r2 = line.split('%')[-2].split(',')[-1]
             r2 = float(r2)
             flop_ratio.append(r2)
-            if 'conv' in line:
-                #无论是否bias=false都计算，fold之后直接使用conv的近似计算
-                inch = line.split(',')[4]
-                # outch = line.split(',')[5]
-                klsz = line.split(',')[6].split('(')[-1]
-                inch = float(inch)
-                # outch = float(outch)
-                klsz = float(klsz)
-                wr = inch * klsz * klsz
-                wr = wr / (1+wr)
-                weight_ratio.append(wr)
-            elif 'fc' in line:
-                inch = line.split(',')[4].split('=')[-1]
-                inch = float(inch)
-                wr = inch / (1+inch)
-                weight_ratio.append(wr)
-            else:
-                weight_ratio.append(0)
-
 
-    return Mac, Param, layer, par_ratio, flop_ratio, weight_ratio
+    return Mac, Param, layer, par_ratio, flop_ratio
 
 
 if __name__ == "__main__":
-    Mac, Param, layer, par_ratio, flop_ratio, weight_ratio = extract_ratio('Inception_BN')
+    Mac, Param, layer, par_ratio, flop_ratio = extract_ratio('Inception_BN')
     print(Mac)
     print(Param)
     print(layer)
     print(par_ratio)
     print(flop_ratio)
\ No newline at end of file
-    print(weight_ratio)
\ No newline at end of file

ykl/ALL/README.md → ykl/ALL-cifar10/fit_bkp.md

-# 模型整合说明
+# fit_bkp
 
-+ 该文件夹下实现了基于cifar10数据集的AlexNet、AlexNet_BN、VGG_16、VGG_19、Inception_BN的整合。
++ 备份之前的数据拟合分析结果
 
-### 部署说明
-
-#### cfg_table
-
-在该通用框架下，当前所有模型部署只需要提供一个cfg_table，如包含特殊结构（如Inception_BN的inception多分支结构），额外针对特殊结构提供cfg_table即可。详见`cfg.py`
-
-cfg_table书写规则说明如下：
-
-+ 每项根据进行量化的合并单位给出，例如Conv后接BN或BN+ReLu时，将被统一合并为一个量化层，则在cfg_table中表现为一项，''/'B'/'BR'参数可选。
-+ 针对不涉及量化，但位置需特别指定的操作，如flatten、drop，同样在cfg_table中单独指定。
-
-根据cfg_table，也相应简化了fold_ratio和fold_model方法，合并与量化层进行对比的全精度层的参数值、参数量、计算量。
-
-#### 训练方案
-
-+ 为每个模型提供了梯度学习率的训练方案，以获得更好的全精度精确度。根据传入的模型名字会自动从epochs_cfg_table和lr_cfg_table中提取，详见`train.py`
-
-
-
-### PTQ部分概述
-
-#### matlab脚本
-
-+ `flops\param`、`flops\param_all`、`flops_all_weight`分别是对单个模型、全体模型、计算量加权后全体模型的拟合。
-  针对单个模型，脚本使用不同颜色标记不同量化类别的点；针对全体模型，脚本使用不同颜色标记不同模型的点。
-+ 脚本内添加了拟合条件限制，以保证拟合曲线一定是单调不降的。如果想允许拟合曲线在某个位置出现略微的转折下降，可以改动tolerance为一个较小的负数，如-1e-5。
-+ 拟合条件限制基于采样实现，对于fakefreeze，采样步长设为0.1，对于L2方法，采样步长设为10
-+ 脚本同时在图像上输出拟合效果评估，如SSE、R方、RMSE指标。
-+ 支持选择拟合模型，当前可选择poly为2/3/4分别表示rational模型分子分母的多项式次数
-+ 由于每次拟合结果会具有细微差别，后续可以更改脚本，使得脚本在每次拟合完成后再进行三次符合约束的拟合，效果均不好于当前拟合才终止，否则取效果更好的拟合结果，并重复该过程。该步骤目前由人工完成
+## update：2023/04/26
 
 #### 拟合方式及结果简述
 

ykl/ALL/get_param_flops.py → ykl/ALL-cifar10/get_param_flops.py

@@ -9,8 +9,8 @@ if __name__ == "__main__":
     model_name = sys.argv[1]
     model = Model(model_name)
 
-    full_file = 'ckpt/cifar10_'+model_name+'.pt'
-    model.load_state_dict(torch.load(full_file))
+    # full_file = 'ckpt/cifar10_'+model_name+'.pt'
+    # model.load_state_dict(torch.load(full_file))
 
     flops, params = get_model_complexity_info(model, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
 

ykl/ALL/get_param_flops.slurm → ykl/ALL-cifar10/get_param_flops.slurm

@@ -82,12 +82,13 @@ echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
 
 #- Job step
 # [EDIT HERE(TODO)]
-name_list="AlexNet AlexNet_BN VGG_16 VGG_19 Inception_BN"
+name_list="AlexNet AlexNet_BN VGG_16 VGG_19 Inception_BN ResNet_18 ResNet_50 ResNet_152 MobileNetV2"
+# name_list="MobileNetV2"
 for name in $name_list; do 
 	if [ -f "param_flops/$name.txt" ];then
 		echo "$name: param_flops exists"
-	elif [ ! -f "ckpt/cifar10_$name.pt" ];then
-		echo "$name: ckpt not exists"
+	# elif [ ! -f "ckpt/cifar10_$name.pt" ];then
+	# 	echo "$name: ckpt not exists"
 	else
 		python get_param_flops.py $name > param_flops/$name.txt
 	fi

ykl/ALL-cifar10/model.py

+import torch.nn as nn
+
+from cfg import *
+from module import *
+from model_deployment import *
+
+
+class Model(nn.Module):
+    def __init__(self,model_name,num_classes=10):
+        super(Model, self).__init__()
+        self.cfg_table = model_cfg_table[model_name]
+        make_layers(self,self.cfg_table)
+        # # 参数初始化
+        # for m in self.modules():
+        #     if isinstance(m, nn.Conv2d):
+        #         nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
+        #     elif isinstance(m, nn.BatchNorm2d):
+        #         nn.init.constant_(m.weight, 1)
+        #         nn.init.constant_(m.bias, 0)
+        #     elif isinstance(m, nn.Linear):
+        #         nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
+
+    def forward(self,x):
+        x = model_forward(self,self.cfg_table,x)
+        return x
+
+    def quantize(self, quant_type, num_bits=8, e_bits=3):
+        model_quantize(self,self.cfg_table,quant_type,num_bits,e_bits)
+    
+    def quantize_forward(self,x):
+        return model_utils(self,self.cfg_table,func='forward',x=x)
+
+    def freeze(self):
+        model_utils(self,self.cfg_table,func='freeze')
+    
+    def quantize_inference(self,x):
+        return model_utils(self,self.cfg_table,func='inference',x=x)
+
+    def fakefreeze(self):
+        model_utils(self,self.cfg_table,func='fakefreeze')
+
+# if __name__ == "__main__":
+#     model = Inception_BN()
+#     model.quantize('INT',8,3)
+#     print(model.named_modules)
+#     print('-------')
+#     print(model.named_parameters)
+#     print(len(model.conv0.named_parameters()))
\ No newline at end of file

ykl/ALL-cifar10/model_deployment.py

+import torch.nn as nn
+import torch.nn.functional as F
+from cfg import *
+from module import *
+
+def make_layers(model,cfg_table):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            make_inc_layers(model,cfg[1])
+        elif cfg[0] == 'ML':
+            make_ml_layers(model,cfg[1],cfg[2],cfg[3])
+        elif cfg[0] == 'C':
+            name = 'conv%d'%i
+            layer = nn.Conv2d(cfg[3],cfg[4],kernel_size=cfg[5],stride=cfg[6],padding=cfg[7],bias=cfg[8])
+            model.add_module(name,layer)
+            if 'B' in cfg[1]:
+                name = 'bn%d'%i
+                layer = nn.BatchNorm2d(cfg[4])
+                model.add_module(name,layer)
+                if 'RL' in cfg[1]:
+                    name = 'relu%d'%i
+                    layer = nn.ReLU(True)
+                    model.add_module(name,layer)
+                elif 'RS' in cfg[1]:
+                    name = 'relus%d'%i
+                    layer = nn.ReLU6(True)
+                    model.add_module(name,layer)
+        elif cfg[0] == 'RL':
+            name = 'relu%d'%i
+            layer = nn.ReLU(True)
+            model.add_module(name,layer)
+        elif cfg[0] == 'RS':
+            name = 'relus%d'%i
+            layer = nn.ReLU6(True)
+            model.add_module(name,layer)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            layer = nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
+            model.add_module(name,layer)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            layer = nn.AdaptiveAvgPool2d(cfg[1])
+            model.add_module(name,layer)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i
+            layer = nn.Linear(cfg[1],cfg[2],bias=cfg[3])
+            model.add_module(name,layer)
+        elif cfg[0] == 'D':
+            name = 'drop%d'%i
+            layer = nn.Dropout(cfg[1])
+            model.add_module(name,layer)
+        
+
+def model_forward(model,cfg_table,x):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            x = inc_forward(model,cfg[1],x)
+        elif cfg[0] == 'ML':
+            x = ml_forward(model,cfg[1],cfg[2],cfg[3],x)
+        elif cfg[0] == 'C':
+            name = 'conv%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+            if 'B' in cfg[1]:
+                name = 'bn%d'%i
+                layer = getattr(model,name)
+                x = layer(x)
+                if 'RL' in cfg[1]:
+                    name = 'relu%d'%i
+                    layer = getattr(model,name)
+                    x = layer(x)
+                elif 'RS' in cfg[1]:
+                    name = 'relus%d'%i
+                    layer = getattr(model,name)
+                    x = layer(x)
+        elif cfg[0] == 'RL':
+            name = 'relu%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'RS':
+            name = 'relus%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'D':
+            name = 'drop%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'VW':
+            if len(cfg) == 1: #default
+                x = x.view(x.size(0),-1)
+        elif cfg[0] == 'SM':
+            x = F.softmax(x,dim=1)
+    return x
+
+
+def model_quantize(model,cfg_table,quant_type,num_bits,e_bits):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            inc_quantize(model,cfg[1],quant_type,num_bits,e_bits)
+        elif cfg[0] == 'ML':
+            ml_quantize(model,cfg[1],cfg[2],cfg[3],quant_type,num_bits,e_bits)
+        elif cfg[0] == 'C':
+            conv_name = 'conv%d'%i
+            conv_layer = getattr(model,conv_name)
+            qname = 'q_'+conv_name
+            if 'B' in cfg[1]:
+                bn_name = 'bn%d'%i
+                bn_layer = getattr(model,bn_name)
+                if 'RL' in cfg[1]:
+                    qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+                elif 'RS' in cfg[1]:
+                    qlayer = QConvBNReLU6(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+                else:
+                    qlayer = QConvBN(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+            else:
+                qlayer = QConv2d(quant_type,conv_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'RL':
+            name = 'relu%d'%i
+            qname = 'q_'+name
+            qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'RS':
+            name = 'relus%d'%i
+            qname = 'q_'+name
+            qlayer = QReLU6(quant_type,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            qname = 'q_'+name
+            qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            qname = 'q_'+name
+            qlayer = QAdaptiveAvgPool2d(quant_type,output_size=cfg[1],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i 
+            layer = getattr(model,name)
+            qname = 'q_'+name
+            qlayer = QLinear(quant_type,layer,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+
+
+# 增加了func='fakefreeze'
+def model_utils(model,cfg_table,func,x=None):
+    last_qo = None
+    # 表示已经经过反量化，用于区别反量化不再最后，而是在softmax前的情形
+    done_flag = False
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            x,last_qo = inc_utils(model,cfg[1],func,x,last_qo)
+        elif cfg[0] == 'ML':
+            x,last_qo = ml_utils(model,cfg[1],cfg[2],cfg[3],func,x,last_qo)
+        elif cfg[0] == 'C':
+            qname = 'q_conv%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                # cfg[2]为True表示起始层，需要量化
+                if cfg[2]:
+                    x = qlayer.qi.quantize_tensor(x)
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
+            last_qo = qlayer.qo
+        elif cfg[0] == 'RL':
+            qname = 'q_relu%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'RS':
+            qname = 'q_relus%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'MP':
+            qname = 'q_pool%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'AAP':
+            qname = 'q_aap%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            last_qo = qlayer.qo
+        elif cfg[0] == 'FC':
+            qname = 'q_fc%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
+            last_qo = qlayer.qo
+        elif cfg[0] == 'D':
+            if func == 'forward':
+                name = 'drop%d'%i
+                layer = getattr(model,name)
+                x = layer(x)
+        elif cfg[0] == 'VW':
+            if func == 'inference' or func == 'forward':
+                if len(cfg) == 1: #default
+                    x = x.view(x.size(0),-1)
+        elif cfg[0] == 'SM':
+            if func == 'inference':
+                done_flag = True
+                x = last_qo.dequantize_tensor(x)
+                x = F.softmax(x,dim=1)
+            elif func == 'forward':
+                x = F.softmax(x,dim=1)
+
+    if func == 'inference' and not done_flag:
+        x = last_qo.dequantize_tensor(x)
+
+    return x
+
+
+def make_inc_layers(model,inc_idx):
+    inc_name = 'inc%d'%inc_idx
+    ch = inc_ch_table[inc_idx]
+    for i in range(4): # branch
+        prefix = inc_name+'_br%d_'%i
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                layer =nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
+                model.add_module(name,layer)
+            elif cfg[0] == 'RL':
+                name=prefix+'relu%d'%j
+                layer=nn.ReLU(True)
+                model.add_module(name,layer)
+            elif cfg[0] == 'C': # 'BRL' default
+                name=prefix+'conv%d'%j
+                layer=nn.Conv2d(ch[cfg[1]],ch[cfg[2]],kernel_size=cfg[3],stride=cfg[4],padding=cfg[5],bias=False)
+                model.add_module(name,layer)
+                name=prefix+'bn%d'%j
+                layer=nn.BatchNorm2d(ch[cfg[2]])
+                model.add_module(name,layer)
+                name=prefix+'relu%d'%j
+                layer=nn.ReLU(True)
+                model.add_module(name,layer)
+
+
+def inc_forward(model,inc_idx,x):
+    inc_name = 'inc%d'%inc_idx
+    outs = []
+    for i in range(4):
+        prefix = inc_name+'_br%d_'%i
+        tmp = x
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+            elif cfg[0] == 'RL':
+                name=prefix+'relu%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+            elif cfg[0] == 'C': # 'BRL' default
+                name=prefix+'conv%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                name=prefix+'bn%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                name=prefix+'relu%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+        outs.append(tmp)
+    
+    out = torch.cat(outs,1)
+    return out
+
+
+def inc_quantize(model,inc_idx,quant_type,num_bits,e_bits):
+    inc_name = 'inc%d'%inc_idx
+    for i in range(4):
+        prefix = inc_name+'_br%d_'%i
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                qname = 'q_'+name
+                qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
+                model.add_module(qname,qlayer)
+            elif cfg[0] == 'RL':
+                name = prefix+'relu%d'%j
+                qname = 'q_'+name
+                qlayer = QReLU(quant_type, num_bits=num_bits, e_bits=e_bits)
+                model.add_module(qname,qlayer)
+            elif cfg[0] == 'C': # 'BRL' default
+                conv_name=prefix+'conv%d'%j
+                conv_layer=getattr(model,conv_name)
+                bn_name=prefix+'bn%d'%j
+                bn_layer=getattr(model,bn_name)
+                qname='q_'+conv_name
+                qlayer=QConvBNReLU(quant_type, conv_layer, bn_layer, num_bits=num_bits, e_bits=e_bits)
+                model.add_module(qname,qlayer)
+    qname = 'q_'+inc_name+'_concat'
+    qlayer = QConcat(quant_type,4,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
+    model.add_module(qname,qlayer)
+    
+
+def inc_utils(model,inc_idx,func,x=None,qo=None):
+    inc_name = 'inc%d'%inc_idx
+    outs=[]
+    qos=[]
+    for i in range(4): 
+        qprefix = 'q_'+inc_name+'_br%d_'%i
+        tmp = x
+        last_qo = qo
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                qname = qprefix+'pool%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+            elif cfg[0] == 'RL':
+                qname = qprefix+'relu%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+            elif cfg[0] == 'C': # 'BRL' default
+                qname = qprefix+'conv%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
+                last_qo = qlayer.qo
+        
+        outs.append(tmp)
+        qos.append(last_qo)
+
+    qname = 'q_'+inc_name+'_concat'
+    qlayer = getattr(model,qname)
+    out = None
+    if func == 'forward':
+        out = qlayer(outs)
+    elif func == 'inference':
+        out = qlayer.quantize_inference(outs)
+    elif func == 'freeze':
+        qlayer.freeze(qos)
+    last_qo = qlayer.qo
+    return out,last_qo
+
+
+
+def make_ml_layers(model,blk_type,ml_idx,blocks):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("Make_ml_layers: Illegal blk_type")
+    
+    #一个makelayer对应两行，分别表示第一个blk和其余的特征
+    make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0)
+    for i in range(1,blocks):
+        make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i)
+
+
+# ma表示主分支，ds表示downsample
+# 当cfgtable含有三个元素时，第3个表示分支合并后需经过的层。
+# BasicBlock和BottleNeck合并后需经过relu层，InvertedResidual合并后无需经过relu层
+def make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    for i in range(2):
+        if i == 0:
+            prefix = blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                prefix = blk_name+'_ds_'
+            else:
+                continue
+        
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                name = prefix+'conv%d'%j
+                layer = nn.Conv2d(ch[cfg[2]],ch[cfg[3]],kernel_size=cfg[4],stride=ch[cfg[5]],padding=cfg[6],groups=ch[cfg[7]])
+                model.add_module(name,layer)
+                if 'B' in cfg[1]:
+                    name = prefix+'bn%d'%j
+                    layer=nn.BatchNorm2d(ch[cfg[3]])
+                    model.add_module(name,layer)
+                    if 'RL' in cfg[1]:
+                        name = prefix+'relu%d'%j
+                        layer = nn.ReLU(True)
+                        model.add_module(name,layer)
+                    elif 'RS' in cfg[1]:
+                        name = prefix+'relus%d'%j
+                        layer = nn.ReLU6(True)
+                        model.add_module(name,layer)
+
+    #分支汇总
+    prefix = blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'RL': #当前没有blk出现汇总处有RS
+            name = prefix+'relu%d'%j
+            layer = nn.ReLU(True)
+            model.add_module(name,layer)
+
+
+def ml_forward(model,blk_type,ml_idx,blocks,x):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("ml_forward: Illegal blk_type")
+    
+    x = blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,x)
+    for i in range(1,blocks):
+        x = blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,x)
+
+    return x
+
+
+def blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,x):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    outs = []
+    for i in range(2):
+        tmp=x
+        if i == 0:
+            prefix = blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                prefix = blk_name+'_ds_'
+            else:
+                outs.append(tmp)
+                continue
+       
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                name = prefix+'conv%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                if 'B' in cfg[1]:
+                    name = prefix+'bn%d'%j
+                    layer = getattr(model,name)
+                    tmp = layer(tmp)
+                    if 'RL' in cfg[1]:
+                        name = prefix+'relu%d'%j
+                        layer = getattr(model,name)
+                        tmp = layer(tmp)
+                    elif 'RS' in cfg[1]:
+                        name = prefix+'relus%d'%j
+                        layer = getattr(model,name)
+                        tmp = layer(tmp)
+        
+        outs.append(tmp)
+    
+    #分支汇总
+    prefix = blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'AD':
+            if cfg[1] or ch[0]: #无条件加或flag为true
+                out = outs[0] + outs[1]
+            else:
+                out = outs[0]
+        elif cfg[0] == 'RL':
+            name = prefix+'relu%d'%j
+            layer = getattr(model,name)
+            out = layer(out)
+
+    return out
+
+
+def ml_quantize(model,blk_type,ml_idx,blocks,quant_type,num_bits,e_bits):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("ml_quantize: Illegal blk_type")
+    
+    blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,quant_type,num_bits,e_bits)
+    for i in range(1,blocks):
+        blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,quant_type,num_bits,e_bits)
+
+
+def blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,quant_type,num_bits,e_bits):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    for i in range(2):
+        if i == 0:
+            prefix = blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                prefix = blk_name+'_ds_'
+            else:
+                continue
+
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                conv_name = prefix+'conv%d'%j
+                conv_layer = getattr(model,conv_name)
+                qname = 'q_'+conv_name
+                if 'B' in cfg[1]:
+                    bn_name = prefix+'bn%d'%j
+                    bn_layer = getattr(model,bn_name)
+                    if 'RL' in cfg[1]:
+                        qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
+                    elif 'RS' in cfg[1]:
+                        qlayer = QConvBNReLU6(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
+                    else:
+                        qlayer = QConvBN(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
+                else:
+                    qlayer = QConv2d(quant_type,conv_layer,num_bits=num_bits,e_bits=e_bits)
+                model.add_module(qname,qlayer)
+
+    #分支汇总
+    prefix = blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'AD':
+            if cfg[1] or ch[0]: #无条件加或flag为true
+                qname = 'q_'+prefix+'add%d'%j
+                qlayer = QElementwiseAdd(quant_type,2,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
+                model.add_module(qname,qlayer)
+        elif cfg[0] == 'RL':
+            qname = 'q_'+prefix+'relu%d'%j
+            qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+
+
+def ml_utils(model,blk_type,ml_idx,blocks,func,x=None,qo=None):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("ml_quantize: Illegal blk_type")
+
+    last_qo = qo
+    x,last_qo = blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,func,x,last_qo)
+    for i in range(1,blocks):
+        x,last_qo = blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,func,x,last_qo)
+    
+    return x, last_qo
+
+def blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,func,x=None,qo=None):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    outs = []
+    qos = []
+    for i in range(2):
+        tmp=x
+        last_qo = qo
+        if i == 0:
+            qprefix = 'q_'+blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                qprefix = 'q_'+blk_name+'_ds_'
+            else:
+                outs.append(tmp)
+                qos.append(last_qo)
+                continue
+       
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                qname = qprefix+'conv%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
+                last_qo = qlayer.qo
+        
+        outs.append(tmp)
+        qos.append(last_qo)
+    
+    #分支汇总
+    qprefix = 'q_'+blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'AD':
+            if cfg[1] or ch[0]: #无条件加或flag为true
+                qname = qprefix+'add%d'%j
+                qlayer = getattr(model,qname)
+                out = None
+                if func == 'forward':
+                    out = qlayer(outs)
+                elif func == 'inference':
+                    out = qlayer.quantize_inference(outs)
+                elif func == 'freeze':
+                    qlayer.freeze(qos)
+                last_qo = qlayer.qo
+            else:
+                out = outs[0]
+                last_qo = qos[0]
+        elif cfg[0] == 'RL':
+            qname = qprefix+'relu%d'%j
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                out = qlayer(out)
+            elif func == 'inference':
+                out = qlayer.quantize_inference(out)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+
+    return out,last_qo
\ No newline at end of file

ykl/ALL/module.py → ykl/ALL-cifar10/module.py

@@ -38,15 +38,9 @@ def get_nearest_val(quant_type, x, is_bias=False, block_size=1000000):
         start_idx = i * block_size
         end_idx = min(start_idx + block_size, xhard.numel())
         block_size_i = end_idx - start_idx
-        # print(x.numel())
-        # print(block_size_i)
-        # print(start_idx)
-        # print(end_idx)
         xblock = xhard[start_idx:end_idx]
-        # xblock = xblock.view(shape[start_idx:end_idx])
-        plist_block = plist.unsqueeze(1) #.expand(-1, block_size_i)
+        plist_block = plist.unsqueeze(1) 
         idx = (xblock.unsqueeze(0) - plist_block).abs().min(dim=0)[1]
-        # print(xblock.shape)
         xhard_block = plist[idx].view(xblock.shape)
         xout[start_idx:end_idx] = (xhard_block - xblock).detach() + xblock
 
@@ -245,7 +239,8 @@ class QConv2d(QModule):
         tmp_wgt = FakeQuantize.apply(self.conv_module.weight, self.qw)
         x = F.conv2d(x, tmp_wgt, self.conv_module.bias,
                      stride=self.conv_module.stride,
-                     padding=self.conv_module.padding, dilation=self.conv_module.dilation,
+                     padding=self.conv_module.padding, 
+                     dilation=self.conv_module.dilation,
                      groups=self.conv_module.groups)
 
         if hasattr(self, 'qo'):
@@ -358,6 +353,37 @@ class QReLU(QModule):
         x[x < self.qi.zero_point] = self.qi.zero_point
         return x
 
+class QReLU6(QModule):
+
+    def __init__(self,quant_type, qi=False, qo=False, num_bits=8, e_bits=3):
+        super(QReLU6, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+
+    def freeze(self, qi=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.relu6(x)
+
+        return x
+
+    def quantize_inference(self, x):
+        x = x.clone()
+        upper = torch.tensor(6)
+        qupper = self.qi.quantize_tensor(upper)
+        x.clamp_(min=0,max=qupper.item())
+        return x
+
 
 class QMaxPooling2d(QModule):
 
@@ -432,6 +458,126 @@ class QAdaptiveAvgPool2d(QModule):
         x = x+self.qo.zero_point
         return x
     
+class QConvBN(QModule):
+
+    def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConvBN, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.bn_module = bn_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def fold_bn(self, mean, std):
+        if self.bn_module.affine:
+            gamma_ = self.bn_module.weight / std
+            weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
+            else:
+                bias = self.bn_module.bias - gamma_ * mean
+        else:
+            gamma_ = 1 / std
+            weight = self.conv_module.weight * gamma_
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean
+            else:
+                bias = -gamma_ * mean
+            
+        return weight, bias
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(self.bn_module.running_mean, std)
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+        
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                        bias.data, scale=self.qi.scale * self.qw.scale,
+                                                        zero_point=0., qmax=self.bias_qmax,is_bias=True)
+        else:
+            bias = quantize_tensor(self.quant_type,
+                                    bias, scale=self.qi.scale * self.qw.scale,
+                                    zero_point=0., qmax=self.bias_qmax,is_bias=True)
+            self.conv_module.bias = torch.nn.Parameter(bias)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        if self.training:
+            y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias, 
+                            stride=self.conv_module.stride,
+                            padding=self.conv_module.padding,
+                            dilation=self.conv_module.dilation,
+                            groups=self.conv_module.groups)
+            y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
+            y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
+            # mean = y.mean(1)
+            # var = y.var(1)
+            mean = y.mean(1).detach()
+            var = y.var(1).detach()
+            self.bn_module.running_mean = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_mean + \
+                self.bn_module.momentum * mean
+            self.bn_module.running_var = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_var + \
+                self.bn_module.momentum * var
+        else:
+            mean = Variable(self.bn_module.running_mean)
+            var = Variable(self.bn_module.running_var)
+
+        std = torch.sqrt(var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(mean, std)
+
+        self.qw.update(weight.data)
+
+        x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
+                stride=self.conv_module.stride,
+                padding=self.conv_module.padding, 
+                dilation=self.conv_module.dilation, 
+                groups=self.conv_module.groups)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+    
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x) 
+        x = x + self.qo.zero_point        
+        return x
+
 
 class QConvBNReLU(QModule):
 
@@ -535,7 +681,8 @@ class QConvBNReLU(QModule):
 
         x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
                 stride=self.conv_module.stride,
-                padding=self.conv_module.padding, dilation=self.conv_module.dilation, 
+                padding=self.conv_module.padding, 
+                dilation=self.conv_module.dilation, 
                 groups=self.conv_module.groups)
 
         x = F.relu(x)
@@ -555,10 +702,10 @@ class QConvBNReLU(QModule):
         x.clamp_(min=0)
         return x
 
-class QConvBN(QModule):
+class QConvBNReLU6(QModule):
 
     def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
-        super(QConvBN, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        super(QConvBNReLU6, self).__init__(quant_type, qi, qo, num_bits, e_bits)
         self.conv_module = conv_module
         self.bn_module = bn_module
         self.qw = QParam(quant_type, num_bits,e_bits)
@@ -657,9 +804,12 @@ class QConvBN(QModule):
 
         x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
                 stride=self.conv_module.stride,
-                padding=self.conv_module.padding, dilation=self.conv_module.dilation, 
+                padding=self.conv_module.padding, 
+                dilation=self.conv_module.dilation, 
                 groups=self.conv_module.groups)
 
+        x = F.relu6(x)
+
         if hasattr(self, 'qo'):
             self.qo.update(x)
             x = FakeQuantize.apply(x, self.qo)
@@ -667,14 +817,20 @@ class QConvBN(QModule):
         return x
     
     def quantize_inference(self, x):
+        upper = torch.tensor(6)
+        qupper = self.qo.quantize_tensor(upper)
+
         x = x - self.qi.zero_point
         x = self.conv_module(x)
         x = self.M * x
         x = get_nearest_val(self.quant_type,x) 
         x = x + self.qo.zero_point        
+        x.clamp_(min=0,max=qupper.item())
         return x
     
 
+
+
 # 作为具体量化层的父类，qi和qo分别为量化输入/输出
 # 用于处理多个层结果或qo以array形式传入
 class QModule_array(nn.Module):
@@ -697,6 +853,60 @@ class QModule_array(nn.Module):
     def quantize_inference(self, x):
         raise NotImplementedError('quantize_inference should be implemented.')
     
+
+class QElementwiseAdd(QModule_array):
+    def __init__(self, quant_type, len, qi_array=False, qo=True, num_bits=8, e_bits=3):
+        super(QElementwiseAdd,self).__init__(quant_type,len,qi_array,qo,num_bits,e_bits)
+        for i in range(len):
+            self.register_buffer('M%d'%i,torch.tensor([], requires_grad=False))
+
+    def freeze(self, qi_array=None, qo=None):
+        if qi_array is None:
+            raise ValueError('qi_array should be provided')
+        elif len(qi_array) != self.len:
+            raise ValueError('qi_array len no match')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+        
+        for i in range(self.len):
+            self.add_module('qi%d'%i,qi_array[i])
+        if qo is not None:
+            self.qo = qo
+
+        for i in range(self.len):
+            getattr(self,'M%d'%i).data = (getattr(self,'qi%d'%i).scale / self.qo.scale).data
+        
+    def forward(self,x_array):
+        outs=[]
+        for i in range(self.len):
+            x = x_array[i]
+            if hasattr(self,'qi%d'%i):
+                qi = getattr(self,'qi%d'%i)
+                qi.update(x)
+                x = FakeQuantize.apply(x,qi)
+            outs.append(x)
+        out = outs[0]+outs[1]
+        if hasattr(self,'qo'):
+            self.qo.update(x)
+            out = FakeQuantize.apply(out,self.qo)
+        return out
+
+    def quantize_inference(self, x_array):
+        outs=[]
+        for i in range(self.len):
+            qi = getattr(self,'qi%d'%i)
+            x = x_array[i] - qi.zero_point
+            x = getattr(self,'M%d'%i) * x
+            outs.append(x)
+        out = outs[0]+outs[1]
+        out = get_nearest_val(self.quant_type,out)
+        out = out + self.qo.zero_point
+        return out
+
+
 class QConcat(QModule_array):
     def __init__(self, quant_type, len, qi_array=False, qo=True, num_bits=8, e_bits=3):
         super(QConcat,self).__init__(quant_type, len, qi_array, qo, num_bits, e_bits)

ykl/ALL-cifar10/param_flops/MobileNetV2.txt

+Model(
+  2.03 M, 100.000% Params, 59.12 MMac, 100.000% MACs, 
+  (conv0): Conv2d(896, 0.044% Params, 917.5 KMac, 1.552% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(64, 0.003% Params, 65.54 KMac, 0.111% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relus0): ReLU6(0, 0.000% Params, 32.77 KMac, 0.055% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(1.06 k, 0.052% Params, 1.08 MMac, 1.829% MACs, 32, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(64, 0.003% Params, 65.54 KMac, 0.111% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relus0): ReLU6(0, 0.000% Params, 32.77 KMac, 0.055% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(320, 0.016% Params, 327.68 KMac, 0.554% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32)
+  (ml0_blk0_ma_bn1): BatchNorm2d(64, 0.003% Params, 65.54 KMac, 0.111% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relus1): ReLU6(0, 0.000% Params, 32.77 KMac, 0.055% MACs, inplace=True)
+  (ml0_blk0_ma_conv2): Conv2d(528, 0.026% Params, 540.67 KMac, 0.915% MACs, 32, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn2): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.055% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_conv0): Conv2d(1.63 k, 0.081% Params, 1.67 MMac, 2.827% MACs, 16, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(192, 0.009% Params, 196.61 KMac, 0.333% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relus0): ReLU6(0, 0.000% Params, 98.3 KMac, 0.166% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(960, 0.047% Params, 245.76 KMac, 0.416% MACs, 96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96)
+  (ml1_blk0_ma_bn1): BatchNorm2d(192, 0.009% Params, 49.15 KMac, 0.083% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relus1): ReLU6(0, 0.000% Params, 24.58 KMac, 0.042% MACs, inplace=True)
+  (ml1_blk0_ma_conv2): Conv2d(2.33 k, 0.115% Params, 595.97 KMac, 1.008% MACs, 96, 24, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn2): BatchNorm2d(48, 0.002% Params, 12.29 KMac, 0.021% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_conv0): Conv2d(3.6 k, 0.178% Params, 921.6 KMac, 1.559% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(288, 0.014% Params, 73.73 KMac, 0.125% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(1.44 k, 0.071% Params, 368.64 KMac, 0.624% MACs, 144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144)
+  (ml1_blk1_ma_bn1): BatchNorm2d(288, 0.014% Params, 73.73 KMac, 0.125% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relus1): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml1_blk1_ma_conv2): Conv2d(3.48 k, 0.172% Params, 890.88 KMac, 1.507% MACs, 144, 24, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn2): BatchNorm2d(48, 0.002% Params, 12.29 KMac, 0.021% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_conv0): Conv2d(3.6 k, 0.178% Params, 921.6 KMac, 1.559% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(288, 0.014% Params, 73.73 KMac, 0.125% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(1.44 k, 0.071% Params, 92.16 KMac, 0.156% MACs, 144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144)
+  (ml2_blk0_ma_bn1): BatchNorm2d(288, 0.014% Params, 18.43 KMac, 0.031% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relus1): ReLU6(0, 0.000% Params, 9.22 KMac, 0.016% MACs, inplace=True)
+  (ml2_blk0_ma_conv2): Conv2d(4.64 k, 0.229% Params, 296.96 KMac, 0.502% MACs, 144, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn2): BatchNorm2d(64, 0.003% Params, 4.1 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_conv0): Conv2d(6.34 k, 0.313% Params, 405.5 KMac, 0.686% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(384, 0.019% Params, 24.58 KMac, 0.042% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relus0): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(1.92 k, 0.095% Params, 122.88 KMac, 0.208% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
+  (ml2_blk1_ma_bn1): BatchNorm2d(384, 0.019% Params, 24.58 KMac, 0.042% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relus1): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk1_ma_conv2): Conv2d(6.18 k, 0.305% Params, 395.26 KMac, 0.669% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn2): BatchNorm2d(64, 0.003% Params, 4.1 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_conv0): Conv2d(6.34 k, 0.313% Params, 405.5 KMac, 0.686% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn0): BatchNorm2d(384, 0.019% Params, 24.58 KMac, 0.042% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relus0): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk2_ma_conv1): Conv2d(1.92 k, 0.095% Params, 122.88 KMac, 0.208% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
+  (ml2_blk2_ma_bn1): BatchNorm2d(384, 0.019% Params, 24.58 KMac, 0.042% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relus1): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk2_ma_conv2): Conv2d(6.18 k, 0.305% Params, 395.26 KMac, 0.669% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn2): BatchNorm2d(64, 0.003% Params, 4.1 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_conv0): Conv2d(6.34 k, 0.313% Params, 405.5 KMac, 0.686% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(384, 0.019% Params, 24.58 KMac, 0.042% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relus0): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(1.92 k, 0.095% Params, 122.88 KMac, 0.208% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
+  (ml3_blk0_ma_bn1): BatchNorm2d(384, 0.019% Params, 24.58 KMac, 0.042% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relus1): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml3_blk0_ma_conv2): Conv2d(18.53 k, 0.915% Params, 1.19 MMac, 2.006% MACs, 192, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn2): BatchNorm2d(192, 0.009% Params, 12.29 KMac, 0.021% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_conv0): Conv2d(55.87 k, 2.758% Params, 3.58 MMac, 6.049% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(1.15 k, 0.057% Params, 73.73 KMac, 0.125% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(5.76 k, 0.284% Params, 368.64 KMac, 0.624% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
+  (ml3_blk1_ma_bn1): BatchNorm2d(1.15 k, 0.057% Params, 73.73 KMac, 0.125% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relus1): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk1_ma_conv2): Conv2d(55.39 k, 2.734% Params, 3.55 MMac, 5.997% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn2): BatchNorm2d(192, 0.009% Params, 12.29 KMac, 0.021% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_conv0): Conv2d(55.87 k, 2.758% Params, 3.58 MMac, 6.049% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn0): BatchNorm2d(1.15 k, 0.057% Params, 73.73 KMac, 0.125% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk2_ma_conv1): Conv2d(5.76 k, 0.284% Params, 368.64 KMac, 0.624% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
+  (ml3_blk2_ma_bn1): BatchNorm2d(1.15 k, 0.057% Params, 73.73 KMac, 0.125% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relus1): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk2_ma_conv2): Conv2d(55.39 k, 2.734% Params, 3.55 MMac, 5.997% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn2): BatchNorm2d(192, 0.009% Params, 12.29 KMac, 0.021% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk0_ma_conv0): Conv2d(55.87 k, 2.758% Params, 3.58 MMac, 6.049% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk0_ma_bn0): BatchNorm2d(1.15 k, 0.057% Params, 73.73 KMac, 0.125% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk0_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml4_blk0_ma_conv1): Conv2d(5.76 k, 0.284% Params, 92.16 KMac, 0.156% MACs, 576, 576, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=576)
+  (ml4_blk0_ma_bn1): BatchNorm2d(1.15 k, 0.057% Params, 18.43 KMac, 0.031% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk0_ma_relus1): ReLU6(0, 0.000% Params, 9.22 KMac, 0.016% MACs, inplace=True)
+  (ml4_blk0_ma_conv2): Conv2d(92.32 k, 4.557% Params, 1.48 MMac, 2.499% MACs, 576, 160, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk0_ma_bn2): BatchNorm2d(320, 0.016% Params, 5.12 KMac, 0.009% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk1_ma_conv0): Conv2d(154.56 k, 7.630% Params, 2.47 MMac, 4.183% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk1_ma_bn0): BatchNorm2d(1.92 k, 0.095% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk1_ma_relus0): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk1_ma_conv1): Conv2d(9.6 k, 0.474% Params, 153.6 KMac, 0.260% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
+  (ml4_blk1_ma_bn1): BatchNorm2d(1.92 k, 0.095% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk1_ma_relus1): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk1_ma_conv2): Conv2d(153.76 k, 7.590% Params, 2.46 MMac, 4.161% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk1_ma_bn2): BatchNorm2d(320, 0.016% Params, 5.12 KMac, 0.009% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk2_ma_conv0): Conv2d(154.56 k, 7.630% Params, 2.47 MMac, 4.183% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk2_ma_bn0): BatchNorm2d(1.92 k, 0.095% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk2_ma_relus0): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk2_ma_conv1): Conv2d(9.6 k, 0.474% Params, 153.6 KMac, 0.260% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
+  (ml4_blk2_ma_bn1): BatchNorm2d(1.92 k, 0.095% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk2_ma_relus1): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk2_ma_conv2): Conv2d(153.76 k, 7.590% Params, 2.46 MMac, 4.161% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk2_ma_bn2): BatchNorm2d(320, 0.016% Params, 5.12 KMac, 0.009% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml5_blk0_ma_conv0): Conv2d(154.56 k, 7.630% Params, 2.47 MMac, 4.183% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
+  (ml5_blk0_ma_bn0): BatchNorm2d(1.92 k, 0.095% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml5_blk0_ma_relus0): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml5_blk0_ma_conv1): Conv2d(9.6 k, 0.474% Params, 153.6 KMac, 0.260% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
+  (ml5_blk0_ma_bn1): BatchNorm2d(1.92 k, 0.095% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml5_blk0_ma_relus1): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml5_blk0_ma_conv2): Conv2d(307.52 k, 15.181% Params, 4.92 MMac, 8.323% MACs, 960, 320, kernel_size=(1, 1), stride=(1, 1))
+  (ml5_blk0_ma_bn2): BatchNorm2d(640, 0.032% Params, 10.24 KMac, 0.017% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (conv7): Conv2d(410.88 k, 20.283% Params, 6.57 MMac, 11.120% MACs, 320, 1280, kernel_size=(1, 1), stride=(1, 1))
+  (aap8): AdaptiveAvgPool2d(0, 0.000% Params, 20.48 KMac, 0.035% MACs, output_size=1)
+  (fc10): Linear(12.81 k, 0.632% Params, 12.81 KMac, 0.022% MACs, in_features=1280, out_features=10, bias=True)
+)

ykl/ALL-cifar10/param_flops/ResNet_152.txt

+Model(
+  3.69 M, 100.000% Params, 240.02 MMac, 100.000% MACs, 
+  (conv0): Conv2d(448, 0.012% Params, 458.75 KMac, 0.191% MACs, 3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(272, 0.007% Params, 278.53 KMac, 0.116% MACs, 16, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(2.32 k, 0.063% Params, 2.38 MMac, 0.990% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn1): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk0_ma_conv2): Conv2d(1.09 k, 0.030% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn2): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ds_conv0): Conv2d(1.09 k, 0.030% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ds_bn0): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.027% MACs, inplace=True)
+  (ml0_blk1_ma_conv0): Conv2d(1.04 k, 0.028% Params, 1.06 MMac, 0.444% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk1_ma_conv1): Conv2d(2.32 k, 0.063% Params, 2.38 MMac, 0.990% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn1): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk1_ma_conv2): Conv2d(1.09 k, 0.030% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn2): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.027% MACs, inplace=True)
+  (ml0_blk2_ma_conv0): Conv2d(1.04 k, 0.028% Params, 1.06 MMac, 0.444% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk2_ma_conv1): Conv2d(2.32 k, 0.063% Params, 2.38 MMac, 0.990% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk2_ma_bn1): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk2_ma_conv2): Conv2d(1.09 k, 0.030% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn2): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.027% MACs, inplace=True)
+  (ml1_blk0_ma_conv0): Conv2d(2.08 k, 0.056% Params, 2.13 MMac, 0.887% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(64, 0.002% Params, 65.54 KMac, 0.027% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml1_blk0_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk0_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ds_conv0): Conv2d(8.32 k, 0.226% Params, 2.13 MMac, 0.887% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2))
+  (ml1_blk0_ds_bn0): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk1_ma_conv0): Conv2d(4.13 k, 0.112% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk1_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk2_ma_conv0): Conv2d(4.13 k, 0.112% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk2_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk2_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk2_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk3_ma_conv0): Conv2d(4.13 k, 0.112% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk3_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk3_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk3_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk4_ma_conv0): Conv2d(4.13 k, 0.112% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk4_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk4_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk4_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk4_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk4_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk4_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk4_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk4_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk5_ma_conv0): Conv2d(4.13 k, 0.112% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk5_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk5_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk5_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk5_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk5_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk5_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk5_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk5_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk6_ma_conv0): Conv2d(4.13 k, 0.112% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk6_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk6_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk6_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk6_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk6_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk6_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk6_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk6_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk7_ma_conv0): Conv2d(4.13 k, 0.112% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk7_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk7_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk7_ma_conv1): Conv2d(9.25 k, 0.251% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk7_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk7_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk7_ma_conv2): Conv2d(4.22 k, 0.115% Params, 1.08 MMac, 0.451% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk7_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk7_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml2_blk0_ma_conv0): Conv2d(8.26 k, 0.224% Params, 2.11 MMac, 0.881% MACs, 128, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(128, 0.003% Params, 32.77 KMac, 0.014% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml2_blk0_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk0_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ds_conv0): Conv2d(33.02 k, 0.896% Params, 2.11 MMac, 0.881% MACs, 128, 256, kernel_size=(1, 1), stride=(2, 2))
+  (ml2_blk0_ds_bn0): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk1_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk1_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk2_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk2_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk2_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk2_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk3_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk3_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk3_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk3_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk4_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk4_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk4_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk4_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk5_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk5_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk5_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk5_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk6_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk6_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk6_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk6_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk6_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk6_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk6_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk6_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk6_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk7_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk7_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk7_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk7_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk7_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk7_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk7_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk7_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk7_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk8_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk8_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk8_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk8_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk8_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk8_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk8_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk8_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk8_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk9_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk9_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk9_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk9_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk9_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk9_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk9_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk9_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk9_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk10_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk10_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk10_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk10_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk10_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk10_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk10_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk10_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk10_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk11_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk11_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk11_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk11_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk11_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk11_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk11_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk11_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk11_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk12_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk12_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk12_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk12_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk12_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk12_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk12_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk12_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk12_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk13_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk13_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk13_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk13_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk13_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk13_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk13_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk13_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk13_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk14_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk14_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk14_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk14_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk14_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk14_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk14_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk14_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk14_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk15_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk15_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk15_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk15_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk15_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk15_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk15_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk15_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk15_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk16_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk16_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk16_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk16_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk16_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk16_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk16_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk16_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk16_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk17_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk17_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk17_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk17_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk17_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk17_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk17_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk17_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk17_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk18_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk18_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk18_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk18_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk18_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk18_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk18_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk18_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk18_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk19_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk19_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk19_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk19_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk19_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk19_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk19_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk19_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk19_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk20_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk20_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk20_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk20_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk20_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk20_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk20_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk20_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk20_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk21_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk21_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk21_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk21_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk21_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk21_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk21_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk21_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk21_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk22_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk22_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk22_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk22_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk22_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk22_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk22_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk22_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk22_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk23_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk23_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk23_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk23_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk23_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk23_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk23_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk23_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk23_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk24_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk24_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk24_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk24_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk24_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk24_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk24_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk24_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk24_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk25_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk25_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk25_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk25_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk25_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk25_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk25_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk25_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk25_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk26_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk26_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk26_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk26_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk26_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk26_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk26_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk26_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk26_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk27_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk27_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk27_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk27_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk27_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk27_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk27_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk27_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk27_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk28_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk28_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk28_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk28_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk28_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk28_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk28_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk28_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk28_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk29_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk29_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk29_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk29_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk29_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk29_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk29_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk29_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk29_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk30_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk30_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk30_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk30_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk30_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk30_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk30_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk30_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk30_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk31_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk31_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk31_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk31_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk31_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk31_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk31_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk31_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk31_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk32_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk32_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk32_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk32_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk32_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk32_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk32_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk32_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk32_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk33_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk33_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk33_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk33_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk33_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk33_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk33_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk33_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk33_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk34_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk34_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk34_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk34_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk34_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk34_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk34_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk34_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk34_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk35_ma_conv0): Conv2d(16.45 k, 0.446% Params, 1.05 MMac, 0.439% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk35_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk35_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk35_ma_conv1): Conv2d(36.93 k, 1.002% Params, 2.36 MMac, 0.985% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk35_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk35_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk35_ma_conv2): Conv2d(16.64 k, 0.451% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk35_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk35_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml3_blk0_ma_conv0): Conv2d(32.9 k, 0.892% Params, 2.11 MMac, 0.877% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(256, 0.007% Params, 16.38 KMac, 0.007% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(147.58 k, 4.004% Params, 2.36 MMac, 0.984% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml3_blk0_ma_bn1): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk0_ma_conv2): Conv2d(66.05 k, 1.792% Params, 1.06 MMac, 0.440% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.028% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ds_conv0): Conv2d(131.58 k, 3.570% Params, 2.11 MMac, 0.877% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2))
+  (ml3_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.028% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml3_blk1_ma_conv0): Conv2d(65.66 k, 1.781% Params, 1.05 MMac, 0.438% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(147.58 k, 4.004% Params, 2.36 MMac, 0.984% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn1): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk1_ma_conv2): Conv2d(66.05 k, 1.792% Params, 1.06 MMac, 0.440% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.028% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml3_blk2_ma_conv0): Conv2d(65.66 k, 1.781% Params, 1.05 MMac, 0.438% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn0): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk2_ma_conv1): Conv2d(147.58 k, 4.004% Params, 2.36 MMac, 0.984% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk2_ma_bn1): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk2_ma_conv2): Conv2d(66.05 k, 1.792% Params, 1.06 MMac, 0.440% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.028% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (aap5): AdaptiveAvgPool2d(0, 0.000% Params, 8.19 KMac, 0.003% MACs, output_size=1)
+  (fc7): Linear(5.13 k, 0.139% Params, 5.13 KMac, 0.002% MACs, in_features=512, out_features=10, bias=True)
+)

ykl/ALL-cifar10/param_flops/ResNet_18.txt

+Model(
+  702.67 k, 100.000% Params, 35.65 MMac, 100.000% MACs, 
+  (conv0): Conv2d(448, 0.064% Params, 458.75 KMac, 1.287% MACs, 3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(32, 0.005% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(2.32 k, 0.330% Params, 2.38 MMac, 6.664% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(32, 0.005% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(2.32 k, 0.330% Params, 2.38 MMac, 6.664% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn1): BatchNorm2d(32, 0.005% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk1_ma_conv0): Conv2d(2.32 k, 0.330% Params, 2.38 MMac, 6.664% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn0): BatchNorm2d(32, 0.005% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk1_ma_conv1): Conv2d(2.32 k, 0.330% Params, 2.38 MMac, 6.664% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn1): BatchNorm2d(32, 0.005% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml1_blk0_ma_conv0): Conv2d(4.64 k, 0.660% Params, 1.19 MMac, 3.332% MACs, 16, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(9.25 k, 1.316% Params, 2.37 MMac, 6.641% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk0_ma_bn1): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ds_conv0): Conv2d(544, 0.077% Params, 139.26 KMac, 0.391% MACs, 16, 32, kernel_size=(1, 1), stride=(2, 2))
+  (ml1_blk0_ds_bn0): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml1_blk1_ma_conv0): Conv2d(9.25 k, 1.316% Params, 2.37 MMac, 6.641% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(9.25 k, 1.316% Params, 2.37 MMac, 6.641% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn1): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml2_blk0_ma_conv0): Conv2d(18.5 k, 2.632% Params, 1.18 MMac, 3.321% MACs, 32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(36.93 k, 5.255% Params, 2.36 MMac, 6.630% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk0_ma_bn1): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ds_conv0): Conv2d(2.11 k, 0.301% Params, 135.17 KMac, 0.379% MACs, 32, 64, kernel_size=(1, 1), stride=(2, 2))
+  (ml2_blk0_ds_bn0): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml2_blk1_ma_conv0): Conv2d(36.93 k, 5.255% Params, 2.36 MMac, 6.630% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(36.93 k, 5.255% Params, 2.36 MMac, 6.630% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn1): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml3_blk0_ma_conv0): Conv2d(73.86 k, 10.511% Params, 1.18 MMac, 3.315% MACs, 64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(147.58 k, 21.003% Params, 2.36 MMac, 6.624% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk0_ma_bn1): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ds_conv0): Conv2d(8.32 k, 1.184% Params, 133.12 KMac, 0.373% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2))
+  (ml3_blk0_ds_bn0): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (ml3_blk1_ma_conv0): Conv2d(147.58 k, 21.003% Params, 2.36 MMac, 6.624% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(147.58 k, 21.003% Params, 2.36 MMac, 6.624% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn1): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (aap5): AdaptiveAvgPool2d(0, 0.000% Params, 2.05 KMac, 0.006% MACs, output_size=1)
+  (fc7): Linear(1.29 k, 0.184% Params, 1.29 KMac, 0.004% MACs, in_features=128, out_features=10, bias=True)
+)

ykl/ALL-cifar10/param_flops/ResNet_50.txt

+Model(
+  1.49 M, 100.000% Params, 84.77 MMac, 100.000% MACs, 
+  (conv0): Conv2d(448, 0.030% Params, 458.75 KMac, 0.541% MACs, 3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(272, 0.018% Params, 278.53 KMac, 0.329% MACs, 16, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(2.32 k, 0.156% Params, 2.38 MMac, 2.803% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn1): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk0_ma_conv2): Conv2d(1.09 k, 0.073% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn2): BatchNorm2d(128, 0.009% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ds_conv0): Conv2d(1.09 k, 0.073% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ds_bn0): BatchNorm2d(128, 0.009% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.077% MACs, inplace=True)
+  (ml0_blk1_ma_conv0): Conv2d(1.04 k, 0.070% Params, 1.06 MMac, 1.256% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk1_ma_conv1): Conv2d(2.32 k, 0.156% Params, 2.38 MMac, 2.803% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn1): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk1_ma_conv2): Conv2d(1.09 k, 0.073% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn2): BatchNorm2d(128, 0.009% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.077% MACs, inplace=True)
+  (ml0_blk2_ma_conv0): Conv2d(1.04 k, 0.070% Params, 1.06 MMac, 1.256% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk2_ma_conv1): Conv2d(2.32 k, 0.156% Params, 2.38 MMac, 2.803% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk2_ma_bn1): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk2_ma_conv2): Conv2d(1.09 k, 0.073% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn2): BatchNorm2d(128, 0.009% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.077% MACs, inplace=True)
+  (ml1_blk0_ma_conv0): Conv2d(2.08 k, 0.140% Params, 2.13 MMac, 2.513% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(64, 0.004% Params, 65.54 KMac, 0.077% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(9.25 k, 0.620% Params, 2.37 MMac, 2.793% MACs, 32, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml1_blk0_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk0_ma_conv2): Conv2d(4.22 k, 0.283% Params, 1.08 MMac, 1.276% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ds_conv0): Conv2d(8.32 k, 0.558% Params, 2.13 MMac, 2.513% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2))
+  (ml1_blk0_ds_bn0): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk1_ma_conv0): Conv2d(4.13 k, 0.277% Params, 1.06 MMac, 1.247% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(9.25 k, 0.620% Params, 2.37 MMac, 2.793% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk1_ma_conv2): Conv2d(4.22 k, 0.283% Params, 1.08 MMac, 1.276% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk2_ma_conv0): Conv2d(4.13 k, 0.277% Params, 1.06 MMac, 1.247% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn0): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk2_ma_conv1): Conv2d(9.25 k, 0.620% Params, 2.37 MMac, 2.793% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk2_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk2_ma_conv2): Conv2d(4.22 k, 0.283% Params, 1.08 MMac, 1.276% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk3_ma_conv0): Conv2d(4.13 k, 0.277% Params, 1.06 MMac, 1.247% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn0): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk3_ma_conv1): Conv2d(9.25 k, 0.620% Params, 2.37 MMac, 2.793% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk3_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk3_ma_conv2): Conv2d(4.22 k, 0.283% Params, 1.08 MMac, 1.276% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml2_blk0_ma_conv0): Conv2d(8.26 k, 0.554% Params, 2.11 MMac, 2.493% MACs, 128, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(128, 0.009% Params, 32.77 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(36.93 k, 2.477% Params, 2.36 MMac, 2.788% MACs, 64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml2_blk0_ma_bn1): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk0_ma_conv2): Conv2d(16.64 k, 1.116% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn2): BatchNorm2d(512, 0.034% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ds_conv0): Conv2d(33.02 k, 2.215% Params, 2.11 MMac, 2.493% MACs, 128, 256, kernel_size=(1, 1), stride=(2, 2))
+  (ml2_blk0_ds_bn0): BatchNorm2d(512, 0.034% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk1_ma_conv0): Conv2d(16.45 k, 1.103% Params, 1.05 MMac, 1.242% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(36.93 k, 2.477% Params, 2.36 MMac, 2.788% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn1): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk1_ma_conv2): Conv2d(16.64 k, 1.116% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn2): BatchNorm2d(512, 0.034% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk2_ma_conv0): Conv2d(16.45 k, 1.103% Params, 1.05 MMac, 1.242% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn0): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk2_ma_conv1): Conv2d(36.93 k, 2.477% Params, 2.36 MMac, 2.788% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk2_ma_bn1): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk2_ma_conv2): Conv2d(16.64 k, 1.116% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn2): BatchNorm2d(512, 0.034% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk3_ma_conv0): Conv2d(16.45 k, 1.103% Params, 1.05 MMac, 1.242% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn0): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk3_ma_conv1): Conv2d(36.93 k, 2.477% Params, 2.36 MMac, 2.788% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk3_ma_bn1): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk3_ma_conv2): Conv2d(16.64 k, 1.116% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn2): BatchNorm2d(512, 0.034% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk4_ma_conv0): Conv2d(16.45 k, 1.103% Params, 1.05 MMac, 1.242% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn0): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk4_ma_conv1): Conv2d(36.93 k, 2.477% Params, 2.36 MMac, 2.788% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk4_ma_bn1): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk4_ma_conv2): Conv2d(16.64 k, 1.116% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn2): BatchNorm2d(512, 0.034% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk5_ma_conv0): Conv2d(16.45 k, 1.103% Params, 1.05 MMac, 1.242% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn0): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk5_ma_conv1): Conv2d(36.93 k, 2.477% Params, 2.36 MMac, 2.788% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk5_ma_bn1): BatchNorm2d(128, 0.009% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk5_ma_conv2): Conv2d(16.64 k, 1.116% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn2): BatchNorm2d(512, 0.034% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml3_blk0_ma_conv0): Conv2d(32.9 k, 2.207% Params, 2.11 MMac, 2.484% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(256, 0.017% Params, 16.38 KMac, 0.019% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(147.58 k, 9.899% Params, 2.36 MMac, 2.786% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml3_blk0_ma_bn1): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk0_ma_conv2): Conv2d(66.05 k, 4.430% Params, 1.06 MMac, 1.247% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.069% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ds_conv0): Conv2d(131.58 k, 8.826% Params, 2.11 MMac, 2.484% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2))
+  (ml3_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.069% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml3_blk1_ma_conv0): Conv2d(65.66 k, 4.405% Params, 1.05 MMac, 1.239% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(147.58 k, 9.899% Params, 2.36 MMac, 2.786% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn1): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk1_ma_conv2): Conv2d(66.05 k, 4.430% Params, 1.06 MMac, 1.247% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.069% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml3_blk2_ma_conv0): Conv2d(65.66 k, 4.405% Params, 1.05 MMac, 1.239% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn0): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk2_ma_conv1): Conv2d(147.58 k, 9.899% Params, 2.36 MMac, 2.786% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk2_ma_bn1): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk2_ma_conv2): Conv2d(66.05 k, 4.430% Params, 1.06 MMac, 1.247% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.069% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (aap5): AdaptiveAvgPool2d(0, 0.000% Params, 8.19 KMac, 0.010% MACs, output_size=1)
+  (fc7): Linear(5.13 k, 0.344% Params, 5.13 KMac, 0.006% MACs, in_features=512, out_features=10, bias=True)
+)

ykl/ALL/ptq_weightratio.py → ykl/ALL-cifar10/ptq.py

@@ -23,7 +23,7 @@ def direct_quantize(model, test_loader,device):
     for i, (data, target) in enumerate(test_loader, 1):
         data = data.to(device)
         output = model.quantize_forward(data).cpu()
-        if i % 500 == 0:
+        if i % 100 == 0:
             break
     print('direct quantization finish')
 
@@ -52,28 +52,30 @@ def quantize_inference(model, test_loader, device):
 
 
 if __name__ == "__main__":
-    batch_size = 32
+    batch_size = 128
 
     device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
     print(device)
     train_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=True, download=True,
+    datasets.CIFAR10('../data', train=True, download=False,
                      transform=transforms.Compose([
-                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                        transforms.RandomCrop(32, padding=2),
                         transforms.RandomHorizontalFlip(),
                         transforms.ToTensor(),
-                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                        transforms.Normalize(
+                            (0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
                         ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+     batch_size=batch_size, shuffle=True, num_workers=4, pin_memory=True
     )
 
     test_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
-            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+    datasets.CIFAR10('../data', train=False, download=False, 
+                     transform=transforms.Compose([
                             transforms.ToTensor(),
-            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                            transforms.Normalize((0.4914, 0.4822, 0.4465),
+                                                (0.2023, 0.1994, 0.2010))
                             ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    batch_size=batch_size, shuffle=False, num_workers=4, pin_memory=True
     )
 
     load_ptq = False
@@ -82,7 +84,12 @@ if __name__ == "__main__":
     append = True
     gol._init()
 
-    excel_path = 'ptq_result_weightratio.xlsx'
+    # ckpt_path = 'ckpt_sgd_5_momentum_95'
+    # excel_path = 'ptq_result_sgd_mom95.xlsx'
+    optim_type = 'sgd_5_momentum_95_ntb'
+    ckpt_path = 'ckpt_'+optim_type
+    
+    excel_path = 'ptq_result_'+optim_type+'.xlsx'
     if os.path.exists(excel_path) and append:
         workbook = openpyxl.load_workbook(excel_path)
     else:
@@ -90,29 +97,34 @@ if __name__ == "__main__":
     if 'Sheet' in workbook.sheetnames:
         workbook.remove(workbook['Sheet'])
 
-    txt_path = 'ptq_result_weightratio.txt'
+    # txt_path = 'ptq_result_sgd_mom95.txt'
+    txt_path = 'ptq_result_'+optim_type+'.txt'
     if os.path.exists(txt_path) and append:
         ft = open(txt_path,'a')
     else:
         ft = open(txt_path,'w')
-    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
+    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN',
+                       'MobileNetV2','ResNet_18','ResNet_50','ResNet_152']
+    # model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN',
+    #                    'MobileNetV2','ResNet_18','ResNet_50']
+    # model_name_list = ['MobileNetV2']
     for model_name in model_name_list:
         if model_name in workbook.sheetnames:
             continue
 
         model = Model(model_name)
-        full_file = 'ckpt/cifar10_'+model_name+'.pt'
+        full_file = ckpt_path+'/cifar10_'+model_name+'.pt'
         model.load_state_dict(torch.load(full_file))
         model.to(device)
 
-        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
+        ptq_file_prefix = ckpt_path+'/cifar10_'+model_name+'_ptq_'
 
         model.eval()
         full_acc = full_inference(model, test_loader, device)
         # 传入后可变
         fold_model(model)
     
-        Mac,Param,layer, par_ratio, flop_ratio,weight_ratio = extract_ratio(model_name)
+        Mac,Param,layer, par_ratio, flop_ratio = extract_ratio(model_name)
         par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
 
         full_names = []
@@ -123,24 +135,14 @@ if __name__ == "__main__":
                 full_names.append(name)
                 full_params.append(param.data.cpu())
     
-        #统计每个参数对应层的参数个数
-        full_par_num=[]
-        for name in full_names:
-            prefix = name.rsplit('.',1)[0]
-            cnt = 0
-            for str in full_names:
-                sprefix = str.rsplit('.',1)[0]
-                if prefix == sprefix:
-                    cnt += 1
-            full_par_num.append(cnt)
-    
         quant_type_list = ['INT','POT','FLOAT']
         # quant_type_list = ['INT']
         # quant_type_list = ['POT']
         # quant_type_list = ['INT','POT']
         title_list = []
         js_flops_list = []
-        js_flops_weighted_list = []
+        js_flops_wt_log_list = []
+        js_flops_wt_cbrt_list = []
         js_param_list = []
         ptq_acc_list = []
         acc_loss_list = []
@@ -206,23 +208,20 @@ if __name__ == "__main__":
                         ptq_param = param.data.cpu()
 
                         js = js_div(ptq_param,full_params[name_idx])
-                        if full_par_num[name_idx] != 1:
-                            if 'weight' in name:
-                                js *= weight_ratio[name_idx]
-                            else:
-                                js *= 1-weight_ratio[name_idx]
 
                         js = js.item()
                         if js < 0.:
                             js = 0.
                         js_flops = js_flops + js * flop_ratio[layer_idx]
                         js_param = js_param + js * par_ratio[layer_idx]
-                    js_flops_weighted = js_flops * torch.log10(torch.tensor(Mac)).item()
+                    js_flops_wt_log = js_flops * torch.log10(torch.tensor(Mac)).item()
+                    js_flops_wt_cbrt = js_flops * torch.pow(torch.tensor(Mac),1/3).item()
                     js_flops_list.append(js_flops)
-                    js_flops_weighted_list.append(js_flops_weighted)
+                    js_flops_wt_log_list.append(js_flops_wt_log)
+                    js_flops_wt_cbrt_list.append(js_flops_wt_cbrt)
                     js_param_list.append(js_param)
 
-                    print(title + ': js_flops: %f js_flops_weighted: %f js_param: %f acc_loss: %f' % (js_flops,js_flops_weighted, js_param, acc_loss))
+                    print(title + ': js_flops: %f js_flops_wt_log: %f js_flops_wt_cbrt: %f js_param: %f acc_loss: %f' % (js_flops,js_flops_wt_log, js_flops_wt_cbrt, js_param, acc_loss))
         
         # 写入xlsx
         worksheet = workbook.create_sheet(model_name)
@@ -235,17 +234,19 @@ if __name__ == "__main__":
 
         worksheet.cell(row=3,column=1,value='title')
         worksheet.cell(row=3,column=2,value='js_flops')
-        worksheet.cell(row=3,column=3,value='js_flops_weighted')
-        worksheet.cell(row=3,column=4,value='js_param')
-        worksheet.cell(row=3,column=5,value='ptq_acc')
-        worksheet.cell(row=3,column=6,value='acc_loss')
+        worksheet.cell(row=3,column=3,value='js_flops_wt_log')
+        worksheet.cell(row=3,column=4,value='js_flops_wt_cbrt')
+        worksheet.cell(row=3,column=5,value='js_param')
+        worksheet.cell(row=3,column=6,value='ptq_acc')
+        worksheet.cell(row=3,column=7,value='acc_loss')
         for i in range(len(title_list)):
             worksheet.cell(row=i+4, column=1, value=title_list[i])
             worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
-            worksheet.cell(row=i+4, column=3, value=js_flops_weighted_list[i])
-            worksheet.cell(row=i+4, column=4, value=js_param_list[i])
-            worksheet.cell(row=i+4, column=5, value=ptq_acc_list[i])
-            worksheet.cell(row=i+4, column=6, value=acc_loss_list[i])
+            worksheet.cell(row=i+4, column=3, value=js_flops_wt_log_list[i])
+            worksheet.cell(row=i+4, column=4, value=js_flops_wt_cbrt_list[i])
+            worksheet.cell(row=i+4, column=5, value=js_param_list[i])
+            worksheet.cell(row=i+4, column=6, value=ptq_acc_list[i])
+            worksheet.cell(row=i+4, column=7, value=acc_loss_list[i])
 
         if 'Sheet' in workbook.sheetnames:
             workbook.remove(workbook['Sheet'])
@@ -257,8 +258,10 @@ if __name__ == "__main__":
         print(title_list,file=ft)
         print('js_flops_list:',file=ft)
         print(js_flops_list, file=ft)
-        print('js_flops_weighted_list:',file=ft)
-        print(js_flops_weighted_list, file=ft)
+        print('js_flops_wt_log_list:',file=ft)
+        print(js_flops_wt_log_list, file=ft)
+        print('js_flops_wt_cbrt_list:',file=ft)
+        print(js_flops_wt_cbrt_list, file=ft)
         print('js_param_list:',file=ft)
         print(js_param_list, file=ft)
         print('ptq_acc_list:',file=ft)
@@ -266,6 +269,7 @@ if __name__ == "__main__":
         print('acc_loss_list:',file=ft)
         print(acc_loss_list, file=ft)
         print("\n",file=ft)
+        ft.flush()
 
     ft.close()
 

ykl/ALL/ptq_L2.slurm → ykl/ALL-cifar10/ptq.slurm

@@ -5,7 +5,7 @@
 # (TODO)
 # Please modify job name
 
-#SBATCH -J ALL-L2            # The job name
+#SBATCH -J PTQ            # The job name
 #SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
 #SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
 
@@ -36,7 +36,8 @@
 ###
 
 # set constraint for RTX8000 to meet my cuda
-#SBATCH --constraint="Ampere|RTX8000|T4"
+### #SBATCH --constraint="Ampere|RTX8000|T4"
+#SBATCH --constraint="Ampere"
 
 #- Log information
 
@@ -82,7 +83,7 @@ echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
 
 #- Job step
 # [EDIT HERE(TODO)]
-python ptq_L2.py
+python ptq.py
 
 #- End
 echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL-cifar10/ptq_result_adam_5_ntb.txt

+AlexNet
+Full_acc: 88.160000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[1020.2421664266105, 315.1754872551783, 65.98671127536508, 18.720867379403302, 4.794779615320007, 1.1058867065321465, 0.25846980786744894, 0.06333175175253605, 0.015733811822498115, 0.003952638802166622, 0.000946705441078194, 0.0003151697974351115, 4.3000903376745556e-05, 5.000616512642608e-07, 1.010108597132575e-05, 1017.2183970747138, 182.26342551863434, 42.05032345529016, 41.83623863149336, 41.82916294692661, 41.81217192872177, 41.85164269080257, 201.5582567833174, 73.16670443469069, 34.102893786681285, 40.5924514155745, 11.312823243330202, 11.766481596852811, 29.68094460511102, 5.6109603403933965, 3.041349998214461, 11.734209276980751, 25.626946280495872, 3.8682222427574002, 0.7719464331031887, 3.0291643439940055, 11.734214309275503, 23.915842487979326, 3.247502904647542, 0.19717417402039264, 0.7646126076876036, 3.0291723490300084, 11.73421395000948]
+js_flops_wt_log_list:
+[1882.9528703857611, 581.686003511264, 121.78468160990566, 34.551121418185666, 8.849216721904016, 2.0410179238910633, 0.477030339117118, 0.11688470411556386, 0.02903822945982242, 0.007294966649149128, 0.00174723392773654, 0.0005816755024132115, 7.936221135224777e-05, 9.229108074564328e-07, 1.8642504152066264e-05, 1877.3722196658637, 336.3842933964257, 77.60782670658604, 77.21271302991248, 77.19965418858702, 77.16829566651022, 77.24114267009631, 371.9947191453377, 135.03603425302816, 62.94009780932869, 74.91718674661217, 20.878879249623377, 21.716148406937045, 54.77897471705289, 10.355555010599929, 5.613097456110281, 21.65658680544988, 47.29693953651396, 7.139167949508616, 1.4246997426960955, 5.590607685203928, 21.656596093023122, 44.13893656067954, 5.993572033304165, 0.363903482089952, 1.4111644781563533, 5.590622459250733, 21.65659542996389]
+js_flops_wt_cbrt_list:
+[4206.310220810214, 1299.422741988443, 272.0536233543201, 77.18341624344251, 19.768179718637207, 4.55941021633801, 1.0656334646622438, 0.26110761098176005, 0.0648682201727437, 0.01629615550096014, 0.003903128986377076, 0.0012993992836869838, 0.00017728647700495547, 2.061681534017834e-06, 4.164530986920985e-05, 4193.843659096926, 751.4456222919315, 173.3673740989079, 172.48473347485796, 172.45556145541812, 172.38550995599095, 172.5482421733283, 830.9954082429545, 301.6556919663227, 140.6012762602684, 167.35677949393525, 46.64112658792185, 48.511494067303374, 122.37022225130795, 23.13317426491138, 12.53904416375722, 48.37843998132832, 105.65617616585554, 15.948118291429472, 3.18262298729789, 12.488804481864548, 48.378460728748436, 98.60154383622108, 13.38898264494778, 0.8129204719764432, 3.152386690643676, 12.488837485464934, 48.37845924754682]
+js_param_list:
+[608.7232888383022, 216.61867458682656, 51.38856054484552, 20.836830211827625, 8.633429909933053, 1.6947684929135065, 0.1938079797698649, 0.04092757127916033, 0.00945392039774901, 0.0023297188232831835, 0.0005704133874965861, 0.00015561897220565203, 3.5493880702613275e-05, 1.991719870531483e-06, 4.811550279890886e-06, 601.6512576879582, 121.15442250417354, 25.993434019493115, 26.19574909903115, 26.039704956850215, 25.719511449596787, 26.113258574327514, 119.43090897701401, 42.01142331061515, 21.614453786553806, 23.077559679923798, 7.319097535101606, 6.992341669037263, 16.869079166757345, 3.711538186376228, 1.8151614749087326, 6.976071177341839, 14.564079968690514, 2.594863819112129, 0.46007988005470524, 1.8090518505476396, 6.976078322774788, 13.59433327554591, 2.190108218432631, 0.11716545611255333, 0.4563926401541665, 1.8090234738065607, 6.976057248815575]
+ptq_acc_list:
+[10.0, 10.0, 10.04, 10.54, 10.16, 17.32, 23.05, 30.0, 42.04, 58.75, 64.75, 71.18, 82.41, 86.21, 87.64, 10.0, 10.0, 12.83, 40.71, 41.49, 40.15, 39.06, 10.01, 10.28, 10.03, 10.01, 13.1, 27.1, 15.49, 19.19, 53.52, 78.17, 18.84, 26.1, 59.9, 86.35, 76.88, 21.23, 38.02, 73.25, 87.74, 86.5, 76.4]
+acc_loss_list:
+[0.8865698729582577, 0.8865698729582577, 0.8861161524500908, 0.8804446460980038, 0.8847549909255898, 0.8035390199637025, 0.7385435571687841, 0.6597096188747731, 0.5231397459165155, 0.33359800362976405, 0.26553992740471866, 0.19260435571687828, 0.06522232304900182, 0.022118874773139778, 0.005898366606170554, 0.8865698729582577, 0.8865698729582577, 0.8544691470054446, 0.5382259528130672, 0.5293784029038112, 0.5445780399274047, 0.5569419237749546, 0.8864564428312159, 0.8833938294010889, 0.8862295825771325, 0.8864564428312159, 0.8514065335753177, 0.6926043557168784, 0.8242967332123412, 0.7823275862068966, 0.3929219600725952, 0.11331669691470049, 0.7862976406533575, 0.7039473684210527, 0.3205535390199637, 0.02053085299455538, 0.12794918330308533, 0.7591878402903811, 0.5687386569872958, 0.16912431941923772, 0.004764065335753196, 0.018829401088929183, 0.13339382940108882]
+
+
+AlexNet_BN
+Full_acc: 89.020000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[34.2949873689207, 10.092186239889795, 1.920683097007928, 0.42702710806280536, 0.10402053427158794, 0.02498501779286633, 0.006135765512561582, 0.0015438911858557665, 0.0003923436730160432, 0.00010034247199691392, 2.4272874113013817e-05, 3.0455731501405126e-05, 1.902388113629261e-05, 7.821244778092451e-05, 6.281829142910155e-05, 33.487462600063864, 5.909637171216701, 1.4854993207894585, 1.481572310394753, 1.4824694548122934, 1.4878883877421758, 1.4764487160370043, 6.556862351758255, 2.348684691211474, 1.0544934369192567, 1.3219531378289433, 0.35457769868795475, 0.3852662301379704, 0.9837751580795154, 0.17788519578974077, 0.09899956368330899, 0.3849574763042947, 0.8551483167479819, 0.12347912134407507, 0.02530344317086967, 0.09885665443905302, 0.3849582230785851, 0.7991699097536786, 0.10385222190352592, 0.0064204938045799055, 0.025239433599125193, 0.09885736840438641, 0.3849844948486603]
+js_flops_wt_log_list:
+[63.3328336564329, 18.637322868353536, 3.5469411835900524, 0.788594452909413, 0.19209557137338248, 0.04614003669855903, 0.011330968353529676, 0.0028511164796650983, 0.0007245442697493831, 0.00018530326368963534, 4.482491514069393e-05, 5.624284844646022e-05, 3.5131556881587765e-05, 0.00014443556697911757, 0.00011600705254278737, 61.841571061427345, 10.913375296172877, 2.743283744888099, 2.7360317026750622, 2.7376884666082484, 2.7476956543687145, 2.726569919071188, 12.10861133042318, 4.337335228025426, 1.9473416541546686, 2.4412616712491544, 0.6548015360879181, 0.7114742981034365, 1.8167456302504463, 0.3285020459025818, 0.18282330392389834, 0.7109041198736382, 1.579209390386283, 0.22802990326259195, 0.04672807544836619, 0.1825593922537447, 0.7109054989478277, 1.4758336083693706, 0.19178474754673625, 0.011856778419056004, 0.046609868448722086, 0.18256071073936506, 0.7109540152404898]
+js_flops_wt_cbrt_list:
+[141.514197897946, 41.644209557836575, 7.925471001500873, 1.7620767148306786, 0.42922839755000725, 0.10309771263037677, 0.025318508668905747, 0.006370683868689331, 0.0016189596336610838, 0.0004140513098018752, 0.00010015913619766613, 0.00012567196390695712, 7.849978922424193e-05, 0.0003227343895563839, 0.00025921222915990507, 138.18204271391872, 24.385416887789837, 6.129736762929142, 6.113532420289633, 6.117234380324816, 6.139594964359613, 6.0923905158447775, 27.056114832606543, 9.691568817814579, 4.351242101623005, 5.454882835970067, 1.46312281974474, 1.5897554050289238, 4.059431511373348, 0.7340221627524827, 0.40850995791840705, 1.5884813689533155, 3.5286681061162266, 0.5095219492626352, 0.10441165718658121, 0.4079202598714145, 1.5884844504288556, 3.2976798488473946, 0.4285338765255094, 0.026493406196360158, 0.10414752928048902, 0.4079232059646845, 1.5885928577721664]
+js_param_list:
+[267.07975718896716, 73.31117500931612, 13.782238309168012, 3.198213663053448, 0.8060788800911768, 0.195319431148546, 0.04769994097402521, 0.011892087169172746, 0.002941950693702072, 0.0007351702059170222, 0.00018647865057610423, 1.905370821184249e-05, 2.013237293458836e-05, 2.1831433795970412e-05, 1.8440481565737743e-05, 266.1899524598974, 41.547306390870396, 10.816220864937437, 10.794111912777911, 10.78117685176176, 10.778609337126428, 10.787103885740638, 51.631211374329304, 19.148736390878614, 7.704084073453693, 10.988627711536209, 2.5332146079171207, 3.01905240704633, 8.244575117821462, 1.2397064537040177, 0.7758990294930821, 3.015448661977383, 7.17844871153941, 0.8461019635302609, 0.19625192710899095, 0.7744569410119928, 3.0154544231373426, 6.71498607498684, 0.7070517434747303, 0.04960356958793626, 0.19542472736392935, 0.7744716732454135, 3.015461540734204]
+ptq_acc_list:
+[10.0, 10.0, 15.01, 45.7, 75.6, 87.04, 88.28, 88.75, 88.9, 89.0, 89.03, 89.02, 89.01, 89.02, 89.02, 10.0, 10.0, 60.71, 60.66, 62.9, 59.38, 58.06, 14.09, 40.87, 50.53, 67.2, 82.65, 84.95, 77.82, 85.57, 88.06, 84.68, 79.82, 86.46, 88.53, 88.17, 84.95, 81.27, 87.07, 88.71, 88.54, 88.14, 85.02]
+acc_loss_list:
+[0.8876656931026735, 0.8876656931026735, 0.8313862053471129, 0.4866322174792181, 0.15075263985621212, 0.02224219276567052, 0.0083127387104021, 0.003033026286227769, 0.0013480116827678087, 0.0002246686137946082, -0.00011233430689738392, 0.0, 0.00011233430689722428, 0.0, 0.0, 0.8876656931026735, 0.8876656931026735, 0.31801842282633114, 0.3185800943608178, 0.29341720961581663, 0.33295888564367554, 0.3477870141541226, 0.841720961581667, 0.5408896877106268, 0.43237474724780944, 0.24511345764996623, 0.07155695349359684, 0.04572006290721179, 0.12581442372500565, 0.038755335879577656, 0.01078409346214327, 0.048753089193439555, 0.10334756234554036, 0.028757582565715596, 0.005504381037968939, 0.009548416086272684, 0.04572006290721179, 0.087059087845428, 0.02190518984497869, 0.0034823635138171456, 0.005392046731071555, 0.009885419006964677, 0.04493372275893058]
+
+
+VGG_16
+Full_acc: 91.620000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[18.64627424415145, 8.950773501728182, 3.1577337118782656, 1.291229820857623, 0.4789011306356713, 0.14884709650924563, 0.03660000104636694, 0.00907020348434925, 0.0022016183680255426, 0.0005511358846565404, 0.00014285205554687727, 5.304420653463737e-05, 2.496477512431461e-05, 1.7847573583002367e-05, 2.7326568761432746e-05, 19.268543044714693, 6.6835752775673445, 1.1676176263706077, 1.1120337010166463, 1.0741925360183047, 1.12124141309498, 1.0922721230191454, 5.168913822399699, 2.0470270628982745, 1.4613368519934489, 0.9229217713710606, 0.49835971321068334, 0.2309757144480617, 0.48900035271068815, 0.22359999467550903, 0.05905535132253138, 0.21709072882450575, 0.3339168380042632, 0.14689327525180002, 0.01722549896150749, 0.05369812608193352, 0.21708762808008178, 0.2876402356305345, 0.12150274696853358, 0.006011328692611109, 0.013885819742118121, 0.05369504290347704, 0.28831968062413815]
+js_flops_wt_log_list:
+[47.042936194148524, 22.582026908788297, 7.966688872046093, 3.257661089782565, 1.2082261065566942, 0.37552834266421914, 0.09233863512814396, 0.022883338418988434, 0.005554492605587239, 0.001390468139464132, 0.0003604033731512098, 0.00013382594242712426, 6.298396708622707e-05, 4.502788354884885e-05, 6.894256803352052e-05, 48.61286652431557, 16.862082001717198, 2.945798220990142, 2.8055647877793826, 2.7100948034157453, 2.8287950484804445, 2.7557080367384224, 13.040722235246042, 5.164472121706751, 3.68682640758908, 2.3284517557933806, 1.257318426369162, 0.5827317379817135, 1.233705569824514, 0.564123435320127, 0.14899154049824473, 0.5477011209161895, 0.8424432837736179, 0.3705990207232628, 0.043458443115666254, 0.13547572485211504, 0.5476932980063588, 0.7256914209486821, 0.30654091526339733, 0.015166062046805845, 0.035032721474468356, 0.13546794625970004, 0.7274056018656316]
+js_flops_wt_cbrt_list:
+[129.2894871179865, 62.06284967146606, 21.895085673320033, 8.953125922327056, 3.3206033950471645, 1.0320756047400705, 0.25377699061177217, 0.06289095296960516, 0.015265575626768568, 0.0038214645417387824, 0.000990507205545307, 0.0003677977792048671, 0.00017310069183664893, 0.00012375145858270682, 0.00018947689032130355, 133.60417288421036, 46.34255661139238, 8.096024014595123, 7.71061633974704, 7.4482333698024, 7.774460749442518, 7.573593562409076, 35.840201013065204, 14.193670843480628, 10.132613605643341, 6.399352541326853, 3.455525263524745, 1.6015387989408707, 3.3906293544035826, 1.5503970526577797, 0.4094778390625377, 1.5052631222880275, 2.3153116896353807, 1.0185282040643244, 0.11943811921479448, 0.37233192483521266, 1.5052416223545282, 1.994439106291337, 0.8424754260983508, 0.04168133501571243, 0.09628146026192716, 0.3723105467005724, 1.999150240193889]
+js_param_list:
+[59.477399729797796, 41.82124701293296, 20.63092545394198, 10.400490652026322, 4.178046691701964, 1.3370779021640422, 0.32882320005205184, 0.08123180981254484, 0.019816423414074125, 0.0049696886762884175, 0.0011959118896011952, 0.00030425019248398377, 9.393444047836174e-05, 1.6395485807107146e-05, 1.1144858390484956e-05, 60.18983086467781, 33.63585392935613, 3.5325323712043573, 2.67639267266417, 2.6307965399368056, 2.6767203761245746, 2.659835861647592, 25.677909930320446, 12.139196280530062, 9.086893458396096, 5.0182711955566575, 3.079032502226496, 0.8056084287953912, 1.9098258144726512, 1.217500933198603, 0.22572374753397093, 0.6706444688654639, 0.8015857777774394, 0.7376182387491544, 0.07567521597006992, 0.17273121233493913, 0.6706443152397759, 0.5210219776284817, 0.59100643479658, 0.03576451251441776, 0.04353254583448526, 0.17271000328019337, 0.7532189692833622]
+ptq_acc_list:
+[10.0, 10.01, 27.36, 12.1, 14.31, 16.55, 30.26, 57.41, 84.31, 90.52, 91.32, 91.54, 91.58, 91.62, 91.63, 10.0, 9.88, 26.06, 51.19, 61.53, 43.63, 57.97, 11.42, 10.4, 10.45, 11.3, 12.73, 73.29, 14.47, 22.11, 88.07, 88.19, 44.12, 65.9, 91.31, 90.95, 88.32, 73.78, 78.98, 91.48, 91.47, 90.9, 87.76]
+acc_loss_list:
+[0.8908535254311286, 0.8907443789565597, 0.7013752455795678, 0.8679327657716657, 0.843811394891945, 0.8193625845885179, 0.6697227679545951, 0.3733900895001092, 0.07978607290984503, 0.012006112202575948, 0.0032743942370662667, 0.0008731717965509528, 0.00043658589827555395, 0.0, -0.00010914647456877215, 0.8908535254311286, 0.892163283125955, 0.7155642872735211, 0.4412791966819472, 0.32842174197773416, 0.5237939314560139, 0.3672778869242524, 0.8753547260423489, 0.8864876664483736, 0.8859419340755293, 0.8766644837371753, 0.8610565378738266, 0.2000654878847413, 0.842065051298843, 0.7586771447282253, 0.03874699847194948, 0.037437240777122974, 0.5184457542021393, 0.2807247325911373, 0.0033835407116350386, 0.007312813796114404, 0.03601833660772769, 0.19471731063086664, 0.13796114385505348, 0.0015280506439642062, 0.0016371971185331334, 0.00785854616895873, 0.042130539183584365]
+
+
+Inception_BN
+Full_acc: 93.160000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[17.68117799721503, 8.051702629177601, 1.7940944579483746, 0.4147412235914379, 0.1022648346795652, 0.024943354369008828, 0.006098844512810525, 0.0015095388132713892, 0.0003632829690227437, 0.00010140678037741721, 3.464970585803773e-05, 1.1452024976970843e-05, 7.610709558928563e-06, 1.332874101948134e-05, 1.8655884020139345e-05, 19.128810633782667, 5.675615313896114, 1.7382098360159648, 1.6205105881643667, 1.762281841376144, 1.8335119049463995, 1.7333966678048034, 3.81876660423519, 1.1332241317934173, 0.9326973316627165, 0.5394221311451176, 0.3340239074002578, 0.1996416212974325, 0.3660842459603092, 0.18337958008054614, 0.05084175551828808, 0.19913711570832673, 0.30717963212937327, 0.13426369146139225, 0.012949091416406164, 0.050655073881440006, 0.19913649690002924, 0.2840584209713243, 0.11541738853200562, 0.0033196689003825094, 0.01284006813516262, 0.050655952332081235, 0.8001437771464878]
+js_flops_wt_log_list:
+[56.13160149898979, 25.561360416175496, 5.695627021059705, 1.3166593929151593, 0.324655345277197, 0.07918649015975188, 0.019361713900018576, 0.004792261642700157, 0.0011532973002006087, 0.00032193132077157577, 0.00011000078623646765, 3.635620332904606e-05, 2.416136052437338e-05, 4.231412519610452e-05, 5.9225954718332436e-05, 60.72733252355984, 18.018108132351838, 5.518212748734308, 5.144558500234726, 5.594633008310449, 5.820763730126782, 5.502932610724325, 12.123258149448132, 3.5975931798710454, 2.9609902093802223, 1.7124779870403855, 1.0604099378604719, 0.6337928350161288, 1.1621903819156678, 0.5821665000914881, 0.1614049222688171, 0.6321912049275591, 0.9751886838087446, 0.42624060603208985, 0.041108869515039496, 0.16081227288491376, 0.6321892404260555, 0.9017868657227537, 0.36641013738751194, 0.010538796219019505, 0.04076275844835384, 0.16081506166071463, 2.5401786944149003]
+js_flops_wt_cbrt_list:
+[202.17562756769212, 92.06728376914835, 20.514593145805307, 4.742363159909301, 1.1693483959442057, 0.28521506451617706, 0.06973730579543172, 0.017260838444078965, 0.004153963172498819, 0.0011595369644292212, 0.0003962024491800336, 0.00013094830768654964, 8.702474357501388e-05, 0.00015240763826512754, 0.00021332091448860723, 218.72859914185048, 64.89788678675649, 19.875579811266864, 18.529746445277883, 20.150831425795808, 20.965312384376244, 19.820543585522532, 43.66572944785703, 12.957864009755145, 10.664938070830074, 6.1680283918494485, 3.8194001051229005, 2.2828043516556127, 4.185994404972833, 2.0968558594751974, 0.5813506221232607, 2.277035576854243, 3.5124489392930105, 1.5352396817133769, 0.14806653063250041, 0.5792160088587447, 2.2770285010838442, 3.248069191702665, 1.3197414200772046, 0.037958791170899266, 0.14681990270372924, 0.5792260535124271, 9.149252969143523]
+js_param_list:
+[9.05252973770469, 3.754294442247411, 0.7944823222744769, 0.1791782416971616, 0.043270364795606836, 0.01049832202883462, 0.0025660653640806916, 0.0006454284890096805, 0.00014696385492221297, 4.6284276858831826e-05, 2.1739416635154558e-05, 8.815165768416014e-06, 1.0390602260868296e-05, 1.5675540756955762e-05, 1.4391626926263224e-05, 10.941616897157827, 2.9312588289202726, 1.6254179977424212, 1.4024790941539542, 1.7173496495227005, 1.8585902198880357, 1.6748221655262017, 1.835732925394922, 0.5639643754126217, 0.4138714514179044, 0.2800674333574201, 0.14713096849824134, 0.0994284015381885, 0.1961314542715952, 0.0796834028283601, 0.02555642359633782, 0.09927991988634685, 0.16657844425898266, 0.05794621567065086, 0.006488352055608574, 0.025494946367431763, 0.09928001394158392, 0.15466530755417235, 0.049673420150651175, 0.0016581711245878244, 0.006464979091786633, 0.02549445991334841, 0.8887528076447424]
+ptq_acc_list:
+[10.0, 10.57, 40.6, 83.37, 92.12, 93.05, 93.04, 93.22, 93.2, 93.23, 93.18, 93.16, 93.17, 93.16, 93.16, 10.0, 10.03, 46.43, 38.63, 32.54, 40.74, 36.46, 10.0, 15.09, 39.62, 50.29, 85.89, 85.33, 74.66, 90.76, 92.11, 86.79, 73.45, 91.82, 92.92, 91.3, 81.72, 80.22, 92.0, 93.18, 92.95, 91.57, 54.47]
+acc_loss_list:
+[0.892657793044225, 0.8865392872477459, 0.5641906397595534, 0.10508802060970365, 0.011163589523400515, 0.001180764276513519, 0.0012881064834691966, -0.0006440532417346745, -0.0004293688278231672, -0.0007513954486905044, -0.00021468441391165986, 0.0, -0.00010734220695582993, 0.0, 0.0, 0.892657793044225, 0.8923357664233577, 0.5016101331043367, 0.5853370545298411, 0.6507084585659081, 0.5626878488621726, 0.6086303134392443, 0.892657793044225, 0.8380206097037355, 0.5747101760412194, 0.4601760412194075, 0.07803778445684839, 0.08404894804637182, 0.19858308286818377, 0.025762129669385913, 0.011270931730356346, 0.06837698583082857, 0.2115714899098325, 0.014383855732073889, 0.0025762129669385453, 0.019965650493774148, 0.1227994847574066, 0.13890081580077285, 0.012451696006869866, -0.00021468441391165986, 0.002254186346071208, 0.017067410905968264, 0.4153069987118935]
+
+
+MobileNetV2
+Full_acc: 91.660000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[15.920392862809823, 7.423070167252823, 3.4227171982067026, 1.0649398682629783, 0.24960008595025393, 0.060582319916002, 0.014884040503795439, 0.0036991368989103286, 0.0009153808787656726, 0.00021548692701953652, 6.13927591150154e-05, 3.00046055812091e-05, 1.1648894244138173e-05, 6.792471915958498e-06, 1.0651051727246186e-05, 15.332036260906044, 5.598481868316186, 0.8134119554123899, 0.7946692823358026, 0.7968873048617099, 0.796469865854626, 0.7986185317517084, 5.164892103844858, 1.7985382812784843, 1.0235148882676417, 0.6437759404114258, 0.3274904698005964, 0.17845003854169048, 0.3595186256731896, 0.1660374686962875, 0.04591977779066673, 0.17629647852300465, 0.2768361963752948, 0.1170826607066467, 0.011916712198098636, 0.04510130493125089, 0.17629263460706654, 0.24886489551925106, 0.09951344777101664, 0.0032111025662945143, 0.011431299327778578, 0.04510807483755414, 0.2183616255599933]
+js_flops_wt_log_list:
+[28.20670890792902, 13.15170933375311, 6.064146061967197, 1.886790679563705, 0.4422250775129617, 0.10733578483666292, 0.026370567704086575, 0.0065538883755741455, 0.0016218118616622243, 0.00038178561774709, 0.00010877166790632932, 5.3160194149694916e-05, 2.0638747540659878e-05, 1.203445667137322e-05, 1.8870835551774448e-05, 27.16429723210124, 9.919023339857617, 1.441150004667488, 1.4079429646036243, 1.4118727115815173, 1.4111331205010023, 1.4149399859484608, 9.150817405664249, 3.1865322795078046, 1.8133966143346503, 1.1406000285038482, 0.5802261559364564, 0.31616608554380904, 0.6369715439014976, 0.29417430761183994, 0.08135765344609214, 0.31235054901237547, 0.4904802333475866, 0.20743938652625277, 0.021113249843007373, 0.07990753686332157, 0.31234373861398473, 0.44092251527986287, 0.17631140625043976, 0.005689221122963388, 0.020253227125523774, 0.07991953133971894, 0.3868788202595814]
+js_flops_wt_cbrt_list:
+[62.02002270967336, 28.917564052327393, 13.333653270419614, 4.148615890528319, 0.9723505652384409, 0.23600654138204502, 0.05798277329031753, 0.014410483237039815, 0.003565988815619464, 0.000839458185646221, 0.00023916371582872822, 0.00011688695973630318, 4.537982773350503e-05, 2.6460984104648148e-05, 4.1492598561988455e-05, 59.72800076486951, 21.809635955892055, 3.1687552173943394, 3.0957406241071754, 3.104381227929754, 3.1027550383675773, 3.1111254541532234, 20.120528955146373, 7.006446763614702, 3.987239332663333, 2.5079154006009343, 1.27578298784832, 0.6951760260107305, 1.4005529587707968, 0.6468211976888978, 0.17888664469161003, 0.6867865445188794, 1.0784524812929217, 0.45611118635759706, 0.046423148443604303, 0.17569817404483337, 0.6867715700295042, 0.9694865324460563, 0.3876679642180053, 0.012509280129003507, 0.0445321575930491, 0.17572454712157137, 0.8506569588359881]
+js_param_list:
+[15.84578449347336, 9.165277641074987, 5.397935202669104, 1.7701799445493756, 0.4144348376139199, 0.10059026641140113, 0.024739082887076987, 0.006144469677149246, 0.0015210578424430959, 0.0003578447131488114, 9.710997407497191e-05, 4.426117907441296e-05, 1.6615057415404865e-05, 4.043816332111038e-06, 1.0627932495119162e-05, 15.927904227733018, 7.730310358467607, 0.8140974687865655, 0.7804405356604674, 0.7849037707708275, 0.7838008812093467, 0.7861818708556347, 7.090579624620935, 2.4898513759480143, 1.4293208818533012, 0.7462933158874986, 0.44720326187076337, 0.18173166078811112, 0.34402384573257955, 0.22518511206063657, 0.04689016541209239, 0.17798404940751167, 0.23747407882863283, 0.15907341002647438, 0.012402179080473546, 0.045446775959800394, 0.1779760423082919, 0.20552491239433243, 0.1354811304255961, 0.003519758857807126, 0.011529218244190684, 0.04545835463932422, 0.25450369629930714]
+ptq_acc_list:
+[10.0, 10.02, 27.75, 81.62, 89.75, 91.28, 91.39, 91.54, 91.59, 91.68, 91.64, 91.68, 91.67, 91.68, 91.67, 10.0, 10.0, 13.64, 16.61, 13.53, 12.02, 13.52, 10.44, 14.88, 40.3, 15.17, 78.75, 70.82, 33.74, 86.46, 88.67, 71.15, 39.12, 88.52, 90.83, 88.83, 73.32, 44.82, 88.49, 91.45, 91.02, 89.05, 78.24]
+acc_loss_list:
+[0.8909011564477417, 0.8906829587606372, 0.6972507091424831, 0.1095352389264673, 0.020837879118481308, 0.004145756054985768, 0.002945668775910932, 0.0013091861226269949, 0.000763691904865734, -0.00021819768710462832, 0.00021819768710447328, -0.00021819768710462832, -0.00010909884355231416, -0.00021819768710462832, -0.00010909884355231416, 0.8909011564477417, 0.8909011564477417, 0.8511891773947196, 0.8187868208596989, 0.8523892646737944, 0.8688631900501855, 0.8524983635173468, 0.8861008073314423, 0.8376609207942396, 0.5603316604843989, 0.8344970543312241, 0.1408466070259655, 0.22736198996290644, 0.6319005018546803, 0.05673139864717437, 0.03262055422212519, 0.22376172812568176, 0.5732053240235654, 0.034257036875409125, 0.009055204014837424, 0.030874972725289093, 0.20008727907484186, 0.511018983198778, 0.03458433340606592, 0.002291075714597357, 0.0069823259873445405, 0.028474798167139423, 0.14641064804713072]
+
+
+ResNet_18
+Full_acc: 90.090000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[25.418874552189077, 7.240082861902716, 1.3442524340774453, 0.2918200209718998, 0.0678502396360393, 0.016523380831439394, 0.00408867071195187, 0.0010175605923691132, 0.00023689020544294514, 5.224164563567157e-05, 2.7994430245779972e-05, 2.3149984164633426e-05, 1.3840717723447858e-05, 1.680395769938725e-05, 8.238124267023085e-06, 24.57741908116388, 4.650292488262814, 1.6009155451781116, 1.619244665751234, 1.606410846557817, 1.6229878288983595, 1.5428129216016386, 4.416529002891841, 1.5366189717926932, 0.7434215699872327, 0.8539510965099725, 0.2532675586781974, 0.2641782484368439, 0.6354692462403534, 0.1285977735943963, 0.06856854042838523, 0.2640548711019812, 0.5511661459453607, 0.09005283511668381, 0.017239550170039763, 0.06851662897023013, 0.26405330140122735, 0.5148933200872232, 0.07602101309987143, 0.004294139576458291, 0.017192240755745897, 0.068517247588803, 0.42167401492185647]
+js_flops_wt_log_list:
+[39.45160652140348, 11.237039612579702, 2.086359802664632, 0.4529220449478799, 0.10530761112212304, 0.025645270707278744, 0.006345860348471653, 0.0015793146159720614, 0.000367667701208457, 8.108214403503617e-05, 4.3449022283808184e-05, 3.5930153570122774e-05, 2.148161786149007e-05, 2.6080742709414783e-05, 1.278605928794084e-05, 38.145617537508464, 7.217530779331184, 2.484716230558908, 2.513164116843154, 2.4932452654421615, 2.518973728882531, 2.3945374998486284, 6.854715933811864, 2.384924121011807, 1.1538345334071642, 1.3253829385271156, 0.3930863283932535, 0.4100203684252564, 0.9862861004193525, 0.19959140019980676, 0.10642245671316057, 0.40982887945674723, 0.8554426700958003, 0.13976736105549858, 0.026756808155215136, 0.10634188703975184, 0.40982644318773104, 0.7991450849986903, 0.1179891379540366, 0.00666476025799612, 0.02668338112784429, 0.10634284717260405, 0.6544631739238365]
+js_flops_wt_cbrt_list:
+[83.65839190648246, 23.828501464644987, 4.424192610117312, 0.9604356648639527, 0.22330815342615895, 0.0543816157705366, 0.013456599586843969, 0.0033489870942250877, 0.0007796511055225431, 0.00017193727658773185, 9.213503972773084e-05, 7.61910384308096e-05, 4.555245690354457e-05, 5.530504806217807e-05, 2.7113247169535594e-05, 80.88900054643828, 15.305004580911241, 5.268920140925611, 5.329244792556397, 5.287006231854765, 5.341564260473404, 5.077693261703423, 14.535644110901528, 5.057307784781464, 2.4467429872251483, 2.810516860379614, 0.8335521164635502, 0.8694612893077226, 2.0914511827556166, 0.42323994005212967, 0.2256722191155912, 0.869055230832902, 1.8139935089734933, 0.29638115397334064, 0.05673866643684457, 0.2255013685493374, 0.8690500646466267, 1.6946126813540296, 0.25019973618340924, 0.014132837032218109, 0.05658296207968524, 0.22550340454172182, 1.3878100670698097]
+js_param_list:
+[35.41507388942745, 11.4273788772453, 2.176867648151144, 0.4728655209205628, 0.11052086491930233, 0.026782211415444297, 0.006581219010531642, 0.001636301723997944, 0.0003701171276225904, 7.243992484020456e-05, 5.279392893607949e-05, 3.939463490095397e-05, 4.416957763601402e-06, 7.930991311447145e-06, 1.0184300651134127e-05, 38.022060150368254, 7.150966520521558, 2.401348438517191, 2.3754591639266587, 2.282702932996093, 2.3383491324149075, 2.248717807702022, 6.762903036744426, 2.322778806883924, 1.1819698959396938, 1.2792627348647096, 0.406626593617735, 0.39414494602860783, 0.9487296030099757, 0.20952531772555974, 0.10229901099441852, 0.3939558519817304, 0.8230072467884747, 0.14763659574732413, 0.02587870817146058, 0.10219357562621037, 0.3939505040364, 0.7689372642247968, 0.12499745065607068, 0.006491944497656732, 0.025823633610068553, 0.1021893028247024, 0.7917628764929623]
+ptq_acc_list:
+[10.0, 10.12, 51.92, 80.74, 88.01, 89.48, 89.93, 89.96, 90.04, 90.08, 90.06, 90.09, 90.09, 90.09, 90.09, 10.0, 12.8, 20.13, 28.78, 35.0, 36.05, 30.08, 9.91, 18.93, 52.13, 51.72, 79.88, 80.89, 67.91, 85.39, 88.33, 82.34, 74.74, 87.2, 89.49, 88.53, 80.28, 76.5, 87.77, 89.93, 89.82, 88.23, 77.24]
+acc_loss_list:
+[0.888999888999889, 0.8876678876678876, 0.4236874236874237, 0.10378510378510387, 0.023088023088023067, 0.006771006771006764, 0.001776001776001738, 0.0014430014430015501, 0.0005550005550005234, 0.00011100011100016779, 0.00033300033300034563, 0.0, 0.0, 0.0, 0.0, 0.888999888999889, 0.8579198579198579, 0.7765567765567766, 0.6805416805416805, 0.6114996114996115, 0.5998445998445999, 0.6661116661116662, 0.88999888999889, 0.7898767898767898, 0.4213564213564214, 0.4259074259074259, 0.11333111333111341, 0.10212010212010215, 0.24619824619824626, 0.0521700521700522, 0.01953601953601959, 0.08602508602508602, 0.17038517038517048, 0.03207903207903209, 0.0066600066600067545, 0.01731601731601734, 0.10889110889110891, 0.15084915084915088, 0.025752025752025832, 0.001776001776001738, 0.0029970029970031106, 0.020646020646020637, 0.14263514263514274]
+
+
+ResNet_50
+Full_acc: 91.230000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[33.95336814507865, 8.917976654960306, 1.542390515502202, 0.33314443388077053, 0.07742898398091703, 0.018765920823191406, 0.0046484834855213385, 0.0011551740717369672, 0.0002951371673827428, 8.058066824489261e-05, 2.7771983548555028e-05, 1.5169364887497982e-05, 1.55548844326665e-05, 1.5152166043721802e-05, 1.5124459254047317e-05, 31.225529242175682, 5.367642838499628, 1.9570706802528577, 1.9661837347191395, 1.975742271855932, 1.956524170928336, 2.011146332799624, 5.897958763998223, 2.11665932105902, 0.8923507430913189, 1.2008762567461062, 0.2921336974132803, 0.38348190002379756, 0.899924800628914, 0.14177139836399033, 0.09733843221909155, 0.3833128730944512, 0.7824217510777883, 0.09599938881312278, 0.024638826365294648, 0.0972804618610991, 0.3833126997043221, 0.7312111019735751, 0.07998796250053494, 0.006119104148075691, 0.02460623831136035, 0.09727898775991689, 0.48635463317251015]
+js_flops_wt_log_list:
+[65.47031751266837, 17.1960189833315, 2.974102491009983, 0.6423831582939319, 0.1493018349241066, 0.03618524057649393, 0.008963402053342076, 0.0022274554010625683, 0.0005690959428759484, 0.00015537904554369778, 5.355111083858831e-05, 2.9250209622987472e-05, 2.999358468144043e-05, 2.921704608651398e-05, 2.9163620685254985e-05, 60.21038340735776, 10.350115471006895, 3.773706286911044, 3.791278463163732, 3.809709637903995, 3.772652484565216, 3.877977242499722, 11.372693021371365, 4.081431839803183, 1.720668365895895, 2.3155802831298202, 0.5633045253146548, 0.7394459850828805, 1.7352729833149019, 0.27336959401038946, 0.18769207332654483, 0.7391200602236674, 1.5086986437693257, 0.18511007331472193, 0.04750962491799674, 0.1875802924355336, 0.7391197258855021, 1.4099521087405569, 0.1542361653113488, 0.011799114884738443, 0.04744678725697054, 0.18757745001141732, 0.9378095310457994]
+js_flops_wt_cbrt_list:
+[149.15240679887032, 39.17542660804519, 6.775506236315715, 1.4634569952720942, 0.34013471851742855, 0.08243607069013954, 0.020420139082160657, 0.005074518449392568, 0.0012964963788818591, 0.0003539796275536538, 0.00012199844710976076, 6.663690249853878e-05, 6.833043604679223e-05, 6.656135037857196e-05, 6.643963831905074e-05, 137.16939127036406, 23.57930573420833, 8.597138315938345, 8.63717069214112, 8.67916001153168, 8.594737576761538, 8.83468510929642, 25.908909569501198, 9.298188938436002, 3.9199722534779564, 5.275281768688429, 1.2833025545525816, 1.6845824575622725, 3.953244030997595, 0.6227819635117853, 0.42759414551956754, 1.683839945855684, 3.437069536264954, 0.4217119147507374, 0.10823492495297025, 0.42733948982883835, 1.6838391841770497, 3.2121083031116258, 0.3513759539529458, 0.02688037037264381, 0.10809177018091869, 0.4273330143081109, 2.1364880145469747]
+js_param_list:
+[37.41214182255085, 11.73358698644466, 2.2728765666665853, 0.49334701898145333, 0.11585499912652744, 0.028007944256143443, 0.006901480144685807, 0.0017303390663871595, 0.000441410638529141, 0.00010844341676261181, 3.466996901401665e-05, 2.0062866046214545e-05, 1.7774724767754936e-05, 1.865638100000858e-05, 2.231428277235614e-05, 37.39366363694135, 7.225607516490855, 2.1656178390756393, 2.2544380267122466, 2.3017804026693915, 2.1915483516204985, 2.2004211232257656, 7.155003695171174, 2.4616588800878008, 1.22882182031447, 1.355558785430177, 0.42028742325608176, 0.4235775769262994, 1.0050276305189831, 0.21467846884549674, 0.1098593545276636, 0.4232871311733021, 0.8716719692751755, 0.15053168066876388, 0.027710425387683417, 0.10975903011220567, 0.4232873048283283, 0.8143136692414167, 0.12717234676817643, 0.0070225470444649, 0.02765189964933404, 0.1097570231282434, 0.6622049035685255]
+ptq_acc_list:
+[10.0, 10.15, 15.72, 73.09, 88.84, 90.79, 91.13, 91.11, 91.21, 91.2, 91.23, 91.23, 91.21, 91.23, 91.23, 10.0, 11.76, 14.9, 13.41, 14.07, 14.07, 10.45, 10.61, 20.91, 27.53, 22.09, 78.23, 74.17, 38.92, 85.99, 89.0, 71.29, 66.68, 89.3, 90.5, 88.63, 74.95, 54.2, 88.39, 90.83, 90.53, 89.0, 77.08]
+acc_loss_list:
+[0.8903869341225474, 0.8887427381343855, 0.8276882604406446, 0.198838101501699, 0.026197522744711176, 0.004822974898607889, 0.0010961306587746193, 0.0013153567905294808, 0.00021922613175501734, 0.0003288391976323702, 0.0, 0.0, 0.00021922613175501734, 0.0, 0.0, 0.8903869341225474, 0.8710950345281157, 0.8366765318425956, 0.8530088786583361, 0.8457744163104242, 0.8457744163104242, 0.8854543461580621, 0.8837005371040229, 0.7707990792502467, 0.698235229639373, 0.7578647374767072, 0.14249698564068836, 0.18699989038693413, 0.5733859476049545, 0.057437246519785254, 0.02444371369067197, 0.21856845335964042, 0.2691000767291461, 0.021155321714348425, 0.008001753809054083, 0.028499397128137767, 0.17845007124849283, 0.40589718294420696, 0.031130110709196572, 0.004384522635098166, 0.007672914611421712, 0.02444371369067197, 0.15510248821659547]
+
+

ykl/ALL-cifar10/ptq_result_sgd.txt

+AlexNet
+Full_acc: 87.330000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[9.77361085558411, 2.479563595916126, 0.44430045512672134, 0.0958056759769912, 0.02245261585245248, 0.0053918210118606, 0.0012884719676490438, 0.0003892981409782301, 7.42283057092763e-05, 1.2211032407095422e-05, 1.4114504686203013e-05, 9.672525006213742e-06, 9.431765981603251e-06, 5.91721715332687e-05, 4.7400273659832236e-06, 9.745560382060907, 1.3869853742370126, 0.4188167146248724, 0.41858764464746395, 0.41849686169738753, 0.4185104626312336, 0.41853267045144643, 1.8243750242955095, 0.681160018324127, 0.2557060099824116, 0.39484584618745944, 0.08392454868101795, 0.11527118974461374, 0.30025781656803335, 0.039722100368376916, 0.030512137719105395, 0.11522814516699598, 0.2628058818210259, 0.02666592884758961, 0.007520376004815472, 0.030573360227075866, 0.11522710862872872, 0.2459087651283948, 0.022142621303734202, 0.0018608540274093388, 0.007568314026438258, 0.03057435988336355, 0.11522753429840206]
+js_flops_wt_log_list:
+[18.03811802742162, 4.576267815500212, 0.8199982757277453, 0.17681838539566022, 0.04143841419054442, 0.009951112770067501, 0.0023779961951522384, 0.0007184863320815593, 0.00013699531924728345, 2.2536608736576685e-05, 2.6049645846382502e-05, 1.7851554585436096e-05, 1.7407211162492295e-05, 0.00010920780761862545, 8.748166296386058e-06, 17.98634824042959, 2.5598119520484603, 0.772965635924013, 0.7725428657370179, 0.7723753172646516, 0.7724004190672119, 0.7724414056880269, 3.3670556870719466, 1.2571448758952013, 0.4719294901892361, 0.7287251437985457, 0.15489064756874038, 0.2127438217561804, 0.5541540392735854, 0.07331087203379852, 0.05631301978139514, 0.21266437893990883, 0.48503297139969576, 0.049214479578837476, 0.013879561196970102, 0.056426011677739696, 0.21266246591105176, 0.4538477533949648, 0.04086629009636251, 0.0034343811181140453, 0.013968035324373983, 0.05642785663743479, 0.2126632515244675]
+js_flops_wt_cbrt_list:
+[40.29517754598936, 10.222880449238106, 1.8317862239078426, 0.3949928824105683, 0.09256887301069011, 0.022229694652205825, 0.005312182720041966, 0.0016050196739804483, 0.0003060325197809922, 5.034431246897984e-05, 5.819205211953967e-05, 3.987841527583627e-05, 3.888580079733613e-05, 0.00024395826608468964, 1.9542444149596472e-05, 40.179529519117004, 5.718339182354399, 1.726720464360927, 1.7257760421257349, 1.7254017572124587, 1.7254578318866212, 1.7255493915027762, 7.521633161039411, 2.8083237895560447, 1.0542387275354654, 1.627892056529932, 0.34600872078270795, 0.4752463675226859, 1.237919353139309, 0.16376844857996942, 0.12579711069594773, 0.47506890098313065, 1.0835104675830318, 0.10993975033581649, 0.03100541762960615, 0.12604952220126886, 0.4750646274864538, 1.0138461104477305, 0.09129081052554881, 0.007672030790829696, 0.031203059127823767, 0.12605364363890253, 0.4750663824606713]
+js_param_list:
+[3.6876790792391945, 0.7354808749246111, 0.13279509416192287, 0.02878319222837831, 0.006701915710043522, 0.0016354941350858782, 0.0003357735418399548, 0.00010818453039652098, 3.311944011332813e-05, 5.232918183272872e-06, 1.94368299872083e-05, 6.096077263682887e-06, 8.109164503877803e-06, 2.9776915021149546e-05, 1.370526710559261e-05, 3.6828958645359724, 0.4102143750455689, 0.14872265773144863, 0.1486344355433121, 0.14862066308175462, 0.1485973572392031, 0.1486065195646579, 0.7174121093715675, 0.2817493275376938, 0.07990256381590628, 0.16849032768981526, 0.02485125279540051, 0.04261276271395425, 0.12940432997257098, 0.011162834589360841, 0.010657878676352936, 0.04260455438761678, 0.11333632014273526, 0.007192081618058183, 0.0026876830044347527, 0.010716117109474694, 0.04260446712068692, 0.10618139847609967, 0.005866707553528726, 0.0007273759519408429, 0.002689605584193657, 0.010716744478476632, 0.042604591796474835]
+ptq_acc_list:
+[10.0, 10.07, 44.64, 76.89, 84.07, 86.45, 86.88, 87.2, 87.2, 87.24, 87.32, 87.34, 87.33, 87.33, 87.33, 10.0, 13.78, 63.83, 64.07, 64.13, 64.49, 64.22, 22.64, 57.95, 56.88, 70.7, 77.77, 81.67, 78.55, 82.92, 86.02, 82.15, 80.03, 84.56, 86.79, 86.17, 82.12, 80.94, 84.87, 87.18, 86.88, 86.15, 82.09]
+acc_loss_list:
+[0.8854918126646055, 0.8846902553532576, 0.48883545173479903, 0.11954654757815182, 0.03732966907133866, 0.010076720485514662, 0.005152868430092784, 0.0014886064353600763, 0.0014886064353600763, 0.0010305736860185895, 0.00011450818733545307, -0.00011450818733545307, 0.0, 0.0, 0.0, 0.8854918126646055, 0.8422077178518264, 0.269094240238177, 0.2663460437421276, 0.2656589946181152, 0.26153669987404105, 0.26462842093209665, 0.7407534638726669, 0.3364250543913889, 0.34867743043627614, 0.19042711553876096, 0.10946982709263715, 0.06481163403183324, 0.10053818848047637, 0.05049811061490893, 0.015000572540936704, 0.059315241039734255, 0.08359097675483794, 0.031718767891904226, 0.006183442116111211, 0.01328294973090572, 0.05965876560174045, 0.07317073170731708, 0.02816901408450697, 0.0017176228100308196, 0.005152868430092784, 0.013511966105576464, 0.06000229016374665]
+
+
+AlexNet_BN
+Full_acc: 88.170000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[30.676258885657155, 6.752506122292226, 1.1975613044403228, 0.2615889354922754, 0.06116750619356366, 0.0146795884576702, 0.0035961755433752242, 0.0009166885208440013, 0.00018741605563638242, 6.254568783795732e-05, 1.9828084062587516e-05, 1.5661980339161772e-05, 1.1503601232601968e-06, 8.208689703707249e-06, 2.3369941704032284e-06, 27.5985199480491, 4.238250800158914, 1.5985014742617267, 1.6112567135292075, 1.6181915384122534, 1.5964866129564175, 1.6271977252477536, 5.17056338972317, 1.9449676584957654, 0.6901332412050594, 1.1364631578308386, 0.22364418269246758, 0.3149602437537294, 0.873149308088429, 0.10602173484161402, 0.08058353858373231, 0.31500891278496074, 0.7642346896998464, 0.07057668275257911, 0.020437484931548008, 0.08055274772894576, 0.31501348637081195, 0.7156834611837101, 0.058401530068076996, 0.005087429925172956, 0.020364291675189855, 0.0805515168596381, 0.3749233715416447]
+js_flops_wt_log_list:
+[56.65009817054611, 12.469908281544793, 2.2115462551892664, 0.4830784265839787, 0.11295853394733917, 0.027108916062090326, 0.00664108678734536, 0.0016928561885982045, 0.00034610276272970474, 0.00011550363326152735, 3.661668500276645e-05, 2.892310718415266e-05, 2.124379447868114e-06, 1.5159054410770757e-05, 4.315746247637678e-06, 50.96641250311523, 7.82681241527326, 2.951965745895732, 2.9755209505699707, 2.988327548398097, 2.9482448850446334, 3.004959347284496, 9.54852169935892, 3.5917876819014185, 1.2744747008020387, 2.0987158081531057, 0.41300554125664624, 0.5816396580487286, 1.612451968316158, 0.19579120483290893, 0.14881427975667677, 0.5817295355467442, 1.4113184517775557, 0.13033453725201358, 0.03774207057147311, 0.14875741803331882, 0.5817379816251408, 1.3216585009996034, 0.10785058321481131, 0.00939499845046532, 0.03760690398634398, 0.14875514497690168, 0.6923740565443238]
+js_flops_wt_cbrt_list:
+[126.58194400291832, 27.86341564782524, 4.941594688705776, 1.079415717141053, 0.25240045967313096, 0.06057358073097671, 0.014839191863419279, 0.003782606459480844, 0.0007733501255234361, 0.0002580873627725847, 8.181823721891708e-05, 6.462730431557032e-05, 4.746824612756004e-06, 3.3872184489153165e-05, 9.643329270227302e-06, 113.88201930518872, 17.488639258644287, 6.596026746865996, 6.648659728896767, 6.677275461287335, 6.5877126606577345, 6.714438423105477, 21.335716584641705, 8.025678364264595, 2.8477529681305507, 4.689480433130111, 0.922841195064778, 1.2996460906978686, 3.6029470619154575, 0.43748611435432355, 0.3325184145362742, 1.2998469176829637, 3.1535238067085203, 0.2912263107896678, 0.084332857628062, 0.33239135972587763, 1.2998657900425388, 2.953182920546032, 0.2409869872436926, 0.02099267620303298, 0.08403083433662177, 0.3323862806896641, 1.5470768257226857]
+js_param_list:
+[9.247066205382364, 1.8291626132331038, 0.32484792383860195, 0.07078788606864135, 0.016559910946962644, 0.004002268657306018, 0.0009703928966982574, 0.00024542904958567565, 5.708190570910254e-05, 1.912911498100378e-05, 5.325805429046148e-06, 3.5924029933595005e-06, 1.4781240274808206e-07, 7.547117344749665e-06, 4.714974095097802e-06, 8.421436952611495, 1.1475635118022187, 0.4578410447572586, 0.4618981303871528, 0.4631248195552776, 0.45697519348414195, 0.46811381739834057, 1.5932687004441444, 0.6142567408123271, 0.1910668453117496, 0.36379568525081657, 0.060467040826875956, 0.0944652986898777, 0.2783494276404343, 0.02798978196957205, 0.02385603799422852, 0.09452078104335693, 0.24372357070150708, 0.01831664019479303, 0.0060155525475752356, 0.023864569761761305, 0.09452214101594421, 0.2282378529304508, 0.015064360638921925, 0.0015050323116371372, 0.006025602263673135, 0.023864085776068605, 0.11150396621668866]
+ptq_acc_list:
+[10.0, 13.44, 30.12, 69.95, 84.49, 87.75, 88.05, 88.12, 88.13, 88.18, 88.11, 88.18, 88.16, 88.18, 88.16, 10.0, 13.47, 47.92, 47.1, 48.51, 46.26, 43.22, 11.7, 27.76, 60.48, 51.34, 82.14, 77.59, 65.05, 85.14, 86.95, 77.65, 68.56, 85.9, 87.7, 86.84, 77.57, 70.97, 86.41, 87.99, 87.83, 86.75, 75.36]
+acc_loss_list:
+[0.8865827378927073, 0.8475671997277986, 0.6583872065328342, 0.20664625155948735, 0.0417375524554838, 0.004763525008506314, 0.0013610071452875643, 0.0005670863105364314, 0.0004536690484292418, -0.00011341726210735075, 0.0006805035726437822, -0.00011341726210735075, 0.00011341726210735075, -0.00011341726210735075, 0.00011341726210735075, 0.8865827378927073, 0.8472269479414767, 0.45650447998185323, 0.46580469547465125, 0.449812861517523, 0.47533174549166385, 0.5098105931722808, 0.8673018033344675, 0.6851536803901553, 0.3140523987750936, 0.4177157763411591, 0.06839060905069753, 0.11999546330951569, 0.2622207099920608, 0.03436543041850971, 0.013836905977089699, 0.11931495973687191, 0.22241125099240103, 0.025745718498355403, 0.005330611319042745, 0.015084495860269914, 0.12022229783373038, 0.19507769082454351, 0.01996143813088358, 0.0020415107179313466, 0.0038561869116479916, 0.016105251219235586, 0.14528751275944202]
+
+
+VGG_16
+Full_acc: 89.030000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[23.605684125304947, 2.8799485365176007, 0.515920049858074, 0.11131222030615666, 0.026339050646610164, 0.006392935911682317, 0.0015768483515481544, 0.0004155211929455521, 0.00013754723300831985, 2.9380204477488903e-05, 3.859661932568326e-05, 1.2108336980641176e-05, 3.6251534332109926e-05, 1.5495158638189566e-05, 1.608919364914052e-05, 23.140325405181265, 2.1778594855013904, 1.9516372839527187, 1.9916956523170342, 1.9286636368427774, 1.9757525247996541, 1.9893314480927673, 4.262649748125066, 1.6876677840214565, 0.37797035017480535, 1.0114076804137628, 0.10772749068300758, 0.26905387361946526, 0.7735102085747884, 0.041674583106026626, 0.06690853178052192, 0.26900786929275045, 0.6757229714149876, 0.023089172229154575, 0.01713271448244418, 0.06685480542031387, 0.26900779859284374, 0.6315635475439138, 0.017525207198115504, 0.004399907848984724, 0.017116331399230172, 0.06685365675362268, 1.1339683220199694]
+js_flops_wt_log_list:
+[59.555098116947214, 7.265860915261593, 1.3016216359881063, 0.28083109842353554, 0.06645114529362729, 0.01612882403469327, 0.003978251924134126, 0.0010483240089201873, 0.0003470197649871868, 7.41237132152051e-05, 9.737593025145307e-05, 3.054828629779566e-05, 9.145948376578968e-05, 3.909294423057186e-05, 4.059164315301323e-05, 58.38103834009739, 5.494551001168407, 4.92381196484366, 5.024875761413918, 4.86585144141744, 4.984652630470184, 5.018911078763871, 10.754296407602595, 4.257839761503218, 0.9535864824111144, 2.551694046380344, 0.27178713582130304, 0.6787996381335536, 1.9514993135381844, 0.10514136648984851, 0.1688044351478158, 0.6786835731986522, 1.7047905770862648, 0.05825198330398441, 0.04322435590493289, 0.168668888190707, 0.6786833948289063, 1.593380172394353, 0.044214581058667594, 0.011100586717201374, 0.04318302280384211, 0.16866599020405534, 2.860903938260118]
+js_flops_wt_cbrt_list:
+[163.67703025644096, 19.968979557905303, 3.577285079401766, 0.7718163792351882, 0.18262963982432134, 0.04432732214366175, 0.01093354693624378, 0.002881139750448261, 0.0009537246410350189, 0.00020371638422659994, 0.00026762113716475834, 8.395675498390099e-05, 0.0002513607930285949, 0.00010744029004999416, 0.00011155920844035658, 160.45032719164266, 15.100836349858726, 13.532257446107836, 13.810014054896865, 13.372962832439608, 13.699467639415216, 13.793621141889403, 29.556349567880346, 11.701946424504744, 2.6207698159987136, 7.01289590379714, 0.7469605904397956, 1.8655650384568958, 5.36336304173629, 0.2889631142974368, 0.4639302009855668, 1.8652460537777966, 4.685326154928134, 0.16009564143478044, 0.11879477043886581, 0.4635576732910225, 1.8652455635589702, 4.379133658295318, 0.1215162353943975, 0.03050806942518644, 0.11868117345973661, 0.4635497086689522, 7.862706556940782]
+js_param_list:
+[14.255871522650548, 1.5517057130410357, 0.2785646605227516, 0.060468507399652975, 0.014140065140259894, 0.0035224558177102596, 0.0008530366767683901, 0.00023753768584110712, 0.00010257568787997683, 1.9484122848915324e-05, 2.3987978783849505e-05, 1.0892382568622682e-05, 1.5862635260569302e-05, 2.1041012081489313e-05, 1.372793177490017e-05, 15.037268736964243, 1.3818109412276671, 1.448594555648352, 1.4334786771443346, 1.4356920885598843, 1.4378959956504607, 1.4403338823951508, 2.706965452711975, 1.058579364343309, 0.24818892010939028, 0.6307193938807106, 0.06588402498262305, 0.20044912273755333, 0.4807576663444964, 0.02295622597307408, 0.04782130040212223, 0.20048264085311596, 0.41931325326297403, 0.01134926934871423, 0.012181421811300077, 0.047749383272991795, 0.20048080393566348, 0.39143611089797836, 0.00798991163104407, 0.003077857310389632, 0.012206108425538893, 0.047750538346391445, 0.8952793427245047]
+ptq_acc_list:
+[10.0, 10.79, 22.37, 81.84, 87.79, 88.9, 88.87, 88.92, 89.01, 88.97, 88.97, 89.0, 89.0, 89.01, 89.01, 10.0, 10.09, 46.5, 41.49, 43.36, 44.65, 43.57, 11.84, 49.48, 73.42, 76.76, 86.23, 85.0, 81.83, 87.53, 88.11, 84.81, 82.59, 88.11, 88.64, 88.18, 84.85, 83.29, 88.15, 88.76, 88.63, 88.16, 67.08]
+acc_loss_list:
+[0.8876783106817927, 0.8788048972256544, 0.7487363809951701, 0.08075929461979106, 0.013927889475457653, 0.0014601819611366443, 0.001797147029091279, 0.0012355385825002744, 0.00022464337863637, 0.0006739301359092696, 0.0006739301359092696, 0.0003369650679546348, 0.0003369650679546348, 0.00022464337863637, 0.00022464337863637, 0.8876783106817927, 0.8866674154779287, 0.47770414467033584, 0.5339773110187577, 0.512973155116253, 0.49848365719420423, 0.5106143996405706, 0.8670111198472424, 0.4442322812535101, 0.17533415702572167, 0.13781871279344038, 0.03145007300909802, 0.04526564079523757, 0.08087161630910931, 0.016848253397731102, 0.010333595417275096, 0.04739975289228349, 0.07233516792092551, 0.010333595417275096, 0.004380545883410093, 0.00954734359204756, 0.04695046613501075, 0.06447264966865096, 0.009884308660002195, 0.0030326856115915537, 0.004492867572728358, 0.00977198697068409, 0.24654610805346516]
+
+
+VGG_19
+Full_acc: 88.670000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[22.64613813841908, 2.5744148465912473, 0.4685501548079211, 0.10137851372072057, 0.02347341095891982, 0.005717900069014295, 0.0014105304829670328, 0.0003459162217224385, 0.0001025663353431254, 3.415296185252337e-05, 1.9245433204116313e-05, 2.7662369299138115e-05, 4.449214890142153e-06, 1.9435948302376125e-05, 2.3054035766334464e-05, 22.300813632591403, 2.0678458733764074, 2.0597369835580177, 2.0527029260316754, 2.060098033221477, 2.0875206773142403, 2.0957662630186036, 3.9658918604423565, 1.5526936546675005, 0.3583156130135812, 0.9256710841230973, 0.10479981367005915, 0.2640462074681324, 0.7055723695476808, 0.03922003000484049, 0.0687705964560354, 0.26409684102672143, 0.615722693401841, 0.02081742974265003, 0.0179431293060982, 0.06873522574441217, 0.2640951228922021, 0.5759049446194897, 0.015374132403389001, 0.0045473284958226145, 0.017965617471264295, 0.068735512915018, 1.196962338984946]
+js_flops_wt_log_list:
+[59.36892479128787, 6.749064254340029, 1.2283471350263377, 0.26577305674592033, 0.06153769624194253, 0.014989998607555406, 0.0036978348205436986, 0.0009068510500995986, 0.0002688870398954968, 8.953511681456746e-05, 5.045366541028699e-05, 7.251943410533345e-05, 1.1664024240186298e-05, 5.09531181854603e-05, 6.0438265773235254e-05, 58.463624978588626, 5.421047305553049, 5.399789108379373, 5.381348682479536, 5.400735633132726, 5.472626605658391, 5.494243163614115, 10.396948660908404, 4.070528592752809, 0.9393571897567062, 2.426731508872203, 0.2747423078446818, 0.6922213110590295, 1.8497225746151567, 0.10281889995712873, 0.18028841579500296, 0.692354051569284, 1.6141734212443066, 0.05457479828062142, 0.047039556492278076, 0.1801956882356576, 0.6923495473223155, 1.5097875467800244, 0.040304695873935086, 0.011921238041543604, 0.047098511276457145, 0.18019644107969643, 3.1379463750883096]
+js_flops_wt_cbrt_list:
+[169.3784061228655, 19.254951142195377, 3.504450865966846, 0.7582454440649026, 0.1755658695618289, 0.04276617912245385, 0.010549851967358448, 0.002587228689027178, 0.0007671295783897695, 0.0002554419745917557, 0.00014394334174475085, 0.00020689655749831035, 3.327723791068452e-05, 0.00014536827095291654, 0.00017242921547743693, 166.79560308445124, 15.466144205225243, 15.405494975565434, 15.352884793418431, 15.408195392570494, 15.613298961210614, 15.674970587322669, 29.66229553452092, 11.613140166203527, 2.679968083309355, 6.9234185477657135, 0.7838345457803999, 1.974894151505069, 5.277223102140032, 0.29334035364912736, 0.5143593995870247, 1.9752728576402043, 4.605205875926097, 0.15570085494634128, 0.13420295434687987, 0.5140948496349951, 1.9752600071098132, 4.3073949739995685, 0.11498852590632992, 0.03401106396294776, 0.13437115121776264, 0.5140969974846287, 8.952500948601932]
+js_param_list:
+[14.313417339246161, 1.4583549161788254, 0.26215754555671145, 0.0569093314993998, 0.013342465446140553, 0.00319847300629592, 0.0007885907619884734, 0.0001804209733083404, 4.9177236808971345e-05, 1.9220158853538117e-05, 1.275090042723459e-05, 1.2731826937989241e-05, 1.76798824695433e-06, 6.543163985868865e-06, 1.265077465191911e-05, 15.007427567094211, 1.4419143034838962, 1.5754876911940663, 1.5752595373917164, 1.5847132233946344, 1.5884108572838913, 1.5787871651815366, 2.5010110409460626, 0.959856634411802, 0.2487824460945377, 0.5637721068909868, 0.07116750446958721, 0.20705761155611535, 0.42688919680221465, 0.023510235435859617, 0.05536111482159561, 0.20710659356356156, 0.3711832625727504, 0.010776413972281268, 0.0140759452315166, 0.05533250653153942, 0.2071092167844784, 0.34620498218426937, 0.007176265187113025, 0.0035567621854581117, 0.01406414896489224, 0.055333456435466936, 0.8839351374174705]
+ptq_acc_list:
+[10.0, 12.29, 40.65, 84.11, 87.59, 88.45, 88.52, 88.64, 88.6, 88.66, 88.71, 88.67, 88.69, 88.69, 88.69, 10.0, 11.0, 47.91, 45.77, 44.0, 43.35, 40.08, 10.87, 33.74, 74.22, 63.86, 84.99, 84.83, 74.6, 86.97, 88.04, 85.13, 78.4, 87.82, 88.51, 87.91, 85.14, 78.64, 88.02, 88.43, 88.47, 87.92, 71.58]
+acc_loss_list:
+[0.8872222848765084, 0.8613961881132287, 0.5415585880230067, 0.05142663809631219, 0.012179993233337074, 0.002481109732716802, 0.001691665726852438, 0.00033833314537048763, 0.0007894440058645245, 0.0001127777151235493, -0.00045111086049387663, 0.0, -0.00022555543024693831, -0.00022555543024693831, -0.00022555543024693831, 0.8872222848765084, 0.8759445133641592, 0.45968196684335183, 0.4838163978797789, 0.503778053456637, 0.5111086049396639, 0.5479869177850457, 0.8774106236607646, 0.6194879891733394, 0.16296379835344538, 0.2798015112213827, 0.04150219916544499, 0.043306642607420814, 0.15867824517875276, 0.019172211570993605, 0.007104996052779919, 0.0399233111537161, 0.11582271343182582, 0.009586105785496882, 0.001804443441975827, 0.008571106349385418, 0.039810533438592546, 0.11311604826886208, 0.007330551483027018, 0.0027066651629637406, 0.002255554302469864, 0.00845832863426187, 0.1927371151460472]
+
+
+Inception_BN
+Full_acc: 88.550000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[37.00856475254243, 3.8201358871556654, 0.5569503269575472, 0.12098860071538146, 0.028351791991631275, 0.006857406739649816, 0.0016963071300445834, 0.000413318819129965, 0.00010967275526530565, 3.372666440893336e-05, 4.502127574749881e-05, 1.7739837528982555e-05, 3.0305344175810765e-05, 2.7041498738468365e-05, 2.804244019906132e-05, 31.755206143862214, 2.53131668037521, 2.541549044744631, 2.52474576705468, 2.6019677251819666, 2.665918755165287, 2.686623910331779, 5.440759100541038, 2.0846010598188744, 0.5028294845415828, 1.2216849482404493, 0.1505418957152067, 0.4180739862916387, 0.9234280536861618, 0.04991507691434201, 0.11846023130208055, 0.4180162500506496, 0.8020425089347685, 0.022446261988629226, 0.030771630814681, 0.1184630203537209, 0.4180146276583736, 0.7477717683676754, 0.014748894551371534, 0.00775192934039569, 0.030756068490420608, 0.11846239688087609, 1.248436272122401]
+js_flops_wt_log_list:
+[117.48934426577661, 12.127605147325966, 1.768123923215822, 0.3840968018276769, 0.09000709625270238, 0.02176988560870428, 0.005385186205271797, 0.0013121437525876675, 0.00034817292121701516, 0.0001070704501018313, 0.0001429269197806155, 5.631782514663423e-05, 9.620894619319369e-05, 8.584737008824998e-05, 8.90250117136366e-05, 100.81175457123409, 8.036051624668172, 8.06853582901628, 8.015191256198918, 8.260344162937104, 8.463366480287176, 8.529098159418696, 17.27252119353558, 6.6178846224247865, 1.5963090384971095, 3.878425559809881, 0.47791825298741114, 1.3272397574026293, 2.9315634699601856, 0.15846303943032444, 0.37607010675272234, 1.3270564648826533, 2.5462065086312395, 0.07125908880540155, 0.09768924438381707, 0.37607896102336313, 1.3270513143504477, 2.3739157493251035, 0.04682261960366036, 0.02460968430112263, 0.09763983940729797, 0.37607698171354137, 3.9633517254730877]
+js_flops_wt_cbrt_list:
+[423.17484759235873, 43.681386528726215, 6.368454742914153, 1.38344550810031, 0.3241888826343797, 0.07841109406251075, 0.019396442851168226, 0.004726098660185433, 0.001254054346737168, 0.00038564792140631505, 0.0005147962810838951, 0.00020284637063977673, 0.00034652679693426033, 0.00030920631977586146, 0.00032065159610223396, 363.1052598242048, 28.94436889374873, 29.061370978576726, 28.86923371189149, 29.752229055801795, 30.48347782341867, 30.72023115175071, 62.21242078835336, 23.836394134120514, 5.749609364983835, 13.969370204825752, 1.7213729903203254, 4.780471672280673, 10.558948408133135, 0.5707545052618243, 1.3545348397648869, 4.77981148658732, 9.17096403901335, 0.2566620337635164, 0.3518585567216947, 1.354566731207063, 4.779792935324695, 8.550404648997386, 0.16864639970950634, 0.08863952274569957, 0.3516806091508794, 1.3545596020998456, 14.275258516959257]
+js_param_list:
+[31.420076652786115, 3.895922983453959, 0.4456515003293517, 0.09679968963085979, 0.022687161390963863, 0.005473259998868628, 0.001348886186021119, 0.00033306966278065807, 8.430234387160484e-05, 2.1275207710751863e-05, 3.905622507112506e-05, 1.921521208724247e-05, 2.2071602077168343e-05, 1.1833716297854345e-05, 2.3988188792299883e-05, 26.37758982238287, 2.1096990464756384, 2.5350296098798304, 2.6226263524521922, 2.696714319551626, 2.847237389431771, 2.8113059707874193, 4.3712238160418835, 1.6565341194865733, 0.4103636282662783, 0.9638706840230874, 0.12528582484375167, 0.34477876635694493, 0.7261184152241175, 0.04099939033910127, 0.10041466500920988, 0.3447177775272593, 0.6297363167544182, 0.01795686243435546, 0.026175838102123174, 0.10043509449239672, 0.34471656262939643, 0.5868003753135255, 0.011563241498306528, 0.006601666575454745, 0.026170708777066523, 0.1004344510621425, 1.4524937060928087]
+ptq_acc_list:
+[10.0, 12.1, 20.93, 66.84, 83.9, 87.7, 88.1, 88.37, 88.41, 88.43, 88.5, 88.55, 88.56, 88.56, 88.57, 10.0, 10.0, 10.8, 18.3, 17.11, 13.28, 11.31, 9.73, 14.9, 31.19, 46.02, 62.06, 52.58, 57.54, 80.49, 82.21, 49.08, 57.4, 84.25, 87.36, 82.92, 69.14, 59.56, 84.79, 88.17, 86.36, 85.5, 24.4]
+acc_loss_list:
+[0.8870694522868436, 0.8633540372670808, 0.7636363636363637, 0.2451722190852625, 0.05251270468661764, 0.009599096555618232, 0.005081874647092071, 0.002032749858836732, 0.0015810276679841962, 0.0013551665725577678, 0.0005646527385657499, 0.0, -0.00011293054771321418, -0.00011293054771321418, -0.00022586109542626788, 0.8870694522868436, 0.8870694522868436, 0.8780350084697911, 0.7933370976849238, 0.8067758328627894, 0.8500282326369283, 0.87227555053642, 0.8901185770750988, 0.8317334839073969, 0.6477696216826652, 0.48029361942405413, 0.2991530208921513, 0.4062111801242236, 0.350197628458498, 0.09102202145680409, 0.0715979672501412, 0.4457368718238284, 0.3517786561264822, 0.04856013551665723, 0.013438735177865587, 0.063579898362507, 0.21919819311123656, 0.3273856578204404, 0.04246188594014671, 0.004291360813099892, 0.024731789949181227, 0.034443817052512676, 0.7244494635798985]
+
+
+MobileNetV2
+Full_acc: 84.610000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[20.199695400098356, 2.342790399605169, 0.4090258650763467, 0.08759231541582455, 0.020585522100895776, 0.005036308713529635, 0.0011856790838455176, 0.0003237396068461287, 8.637934886645483e-05, 2.101893433109226e-05, 2.245353869550648e-05, 1.2182943826256843e-05, 1.2746905921235451e-05, 1.6312513350994315e-05, 2.2701872597252205e-05, 19.46437607016121, 1.5351081487052831, 1.0121378644139682, 1.0122502429366667, 1.011692454556928, 1.0125805271815815, 1.0121422806497415, 3.552402773738667, 1.3747820151871235, 0.33942906058385536, 0.8174602752660846, 0.09899190226144897, 0.2644486847550292, 0.6195063981685683, 0.03510565742159185, 0.06998656741141959, 0.264433125987863, 0.5392255904222267, 0.017708359436120872, 0.017736717497933617, 0.07001566690910287, 0.2644330133287734, 0.5034802676846727, 0.012590556475658909, 0.00449580075877099, 0.017742769747582425, 0.0700156389578681, 0.29339623872327997]
+js_flops_wt_log_list:
+[35.78849674686029, 4.150802521231921, 0.7246852267680864, 0.15519032506272018, 0.03647207921445894, 0.0089230017799903, 0.0021007085104272896, 0.0005735806227249393, 0.00015304127040885855, 3.723997060505688e-05, 3.9781708617037984e-05, 2.15849415972407e-05, 2.2584132684121952e-05, 2.890144229562652e-05, 4.02216903523427e-05, 34.48570613907564, 2.7198040315870426, 1.7932395489378048, 1.7934386538408416, 1.7924504009380962, 1.7940238298244664, 1.7932473733349787, 6.293914467188684, 2.4357486934147943, 0.6013781688230805, 1.4483225525225165, 0.17538736609028904, 0.46853285193700306, 1.0976008437932676, 0.06219790355962072, 0.1239976143459724, 0.46850528593280927, 0.955364568940763, 0.031374510927959766, 0.031424753883748484, 0.12404917092925474, 0.46850508633081617, 0.8920333482876934, 0.022307122981076517, 0.007965365201943795, 0.03143547686310918, 0.12404912140708844, 0.5198202313010137]
+js_flops_wt_cbrt_list:
+[78.69061889603884, 9.126653785469024, 1.5934150406638645, 0.3412275964604256, 0.08019365848505254, 0.01961961513623767, 0.004618971676149028, 0.0012611709988338794, 0.000336502322807555, 8.188207387725235e-05, 8.74707673238953e-05, 4.7460289409074874e-05, 4.965727928485078e-05, 6.354758059041765e-05, 8.843818529897744e-05, 75.8260839605916, 5.980219399416437, 3.9429186124490965, 3.9433563980357897, 3.941183458694613, 3.944643064548783, 3.942935816487044, 13.838860799462333, 5.355647472866916, 1.322291367250745, 3.184525989599885, 0.38563621383180247, 1.0301952706433084, 2.413370148230212, 0.13675879039462796, 0.2726420470670629, 1.0301346593819347, 2.1006255091698245, 0.0689852859689694, 0.06909575859692428, 0.2727554080577564, 1.030134220503362, 1.961374816862698, 0.049048199078799225, 0.01751399400504067, 0.06911933594604489, 0.27275529916997765, 1.1429643442441433]
+js_param_list:
+[16.32697281784347, 1.692197232592246, 0.29820042842024075, 0.06454944478201839, 0.015060550723462442, 0.00366768598303814, 0.0008801230876300767, 0.00025354129220515637, 7.235440043255049e-05, 2.0248561525279206e-05, 3.0133216351318382e-05, 1.2662763463374916e-05, 1.3118701301500071e-05, 1.867692215485625e-05, 2.037370454139592e-05, 15.700086298758913, 1.2747372151086926, 0.926718217515368, 0.926883268722745, 0.9264741858598228, 0.9270823912959991, 0.9268008990409753, 2.7558812080947015, 1.066972080595087, 0.27260221438750915, 0.6286149830016252, 0.0799023949143092, 0.22416943190674513, 0.476829940182954, 0.026581949666080833, 0.06149757827877358, 0.22414164033060022, 0.414994222657401, 0.012342519397345843, 0.01560580826910591, 0.061539133267352615, 0.22414172086791606, 0.38726291430344134, 0.008343637073036581, 0.003898858394887285, 0.015616816746903253, 0.06153915878337477, 0.2750791411080558]
+ptq_acc_list:
+[10.0, 9.84, 34.44, 71.05, 81.65, 83.81, 84.34, 84.51, 84.6, 84.56, 84.59, 84.54, 84.59, 84.6, 84.58, 10.0, 16.5, 11.96, 12.04, 11.84, 11.45, 11.15, 10.13, 12.23, 46.95, 16.14, 73.63, 55.09, 23.05, 80.66, 80.55, 54.03, 30.37, 82.04, 83.72, 80.49, 51.9, 32.51, 82.63, 84.28, 83.67, 81.18, 58.1]
+acc_loss_list:
+[0.8818106606784069, 0.8837016901075523, 0.5929559153764331, 0.1602647441200804, 0.03498404443919151, 0.009455147145727422, 0.0031911121616829694, 0.0011818933932158646, 0.00011818933932165366, 0.0005909466966079323, 0.00023637867864313936, 0.0008273253752510717, 0.00023637867864313936, 0.00011818933932165366, 0.000354568017964793, 0.8818106606784069, 0.8049875901193713, 0.8586455501713746, 0.8577000354568017, 0.8600638222432336, 0.8646732064767758, 0.8682188866564235, 0.8802741992672262, 0.8554544380096915, 0.44510105188511995, 0.8092424063349486, 0.12977189457510938, 0.34889492967734304, 0.7275735728637277, 0.04668478903202934, 0.04798487176456687, 0.36142299964543195, 0.6410589764803214, 0.03037466020564937, 0.010518851199621801, 0.04869400780049645, 0.38659732892093135, 0.6157664578655005, 0.0234014891856755, 0.003900248197612555, 0.011109797896229734, 0.040538943387306375, 0.31331993854154355]
+
+
+ResNet_18
+Full_acc: 84.320000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[23.977648658006217, 2.5703435285908354, 0.44873454950721126, 0.09689630468080668, 0.022652440304365563, 0.005533391768453214, 0.0013547212148875528, 0.00033512278505034665, 7.909558165277274e-05, 2.247525467652256e-05, 9.074229703141868e-06, 1.1583516331141077e-05, 1.637496420931184e-05, 1.9876633191019955e-05, 2.0394259621628437e-05, 18.429421282892605, 1.8435169128412672, 1.1881884486026975, 1.2518352166921287, 1.2656761302958621, 1.3173620442453833, 1.2313652271142637, 3.4302405230232442, 1.3509932274595458, 0.3219788958204206, 0.8075808650445306, 0.0948279125706275, 0.22439870558068833, 0.6183634039618191, 0.03691081052646126, 0.058276202219240986, 0.2243656164293477, 0.5409057913635112, 0.020475371999194487, 0.014928776323051686, 0.058277637737427335, 0.22436340829306797, 0.5059803953496315, 0.015503407066963295, 0.0037695860912099312, 0.014941526631541309, 0.05827880598480743, 0.47577493622052686]
+js_flops_wt_log_list:
+[37.21473813570765, 3.989326172038112, 0.6964627345467244, 0.1503888332190537, 0.035157935879598104, 0.00858815343415702, 0.0021026079737011274, 0.0005201305126636516, 0.00012276105138093131, 3.488293323711061e-05, 1.408374469916704e-05, 1.7978306926695664e-05, 2.5414919274315406e-05, 3.084971799252439e-05, 3.1653105027757457e-05, 28.603558956832043, 2.8612479955255394, 1.8441392065296416, 1.9429227795736737, 1.964404701453028, 2.044624316827778, 1.9111521371455409, 5.323937498107358, 2.096821915281251, 0.4997304141071521, 1.253413578816677, 0.1471785655301667, 0.3482801497881098, 0.9597368148715466, 0.05728777528234899, 0.09044813509720231, 0.3482287934553918, 0.8395179889734277, 0.031778996266458095, 0.02317034958152221, 0.09045036310688566, 0.34822536629643264, 0.7853116952087371, 0.02406221070453758, 0.005850622021586007, 0.02319013881933942, 0.09045217629636676, 0.7384310244727802]
+js_flops_wt_cbrt_list:
+[78.91504103806501, 8.459493586454819, 1.4768714770426432, 0.31890432499808496, 0.0745535261496308, 0.01821145370488076, 0.00445864737584649, 0.0011029533676234038, 0.00026031873104346345, 7.397037426130263e-05, 2.986503053847872e-05, 3.812357415337591e-05, 5.389314819838306e-05, 6.541781249447726e-05, 6.712142036681198e-05, 60.65476050595326, 6.067367776809044, 3.9105560982989847, 4.120029820570771, 4.1655829221539085, 4.335691179164107, 4.052659158391743, 11.28957953741749, 4.446377854132052, 1.059694306955124, 2.657903534059226, 0.3120968498121018, 0.7385391834021967, 2.0351525746370736, 0.12148055753879489, 0.19179815983075554, 0.738430280658095, 1.7802247139410667, 0.06738837676006043, 0.04913346646224812, 0.19180288439646265, 0.7384230132580419, 1.6652785363981029, 0.051024686464021606, 0.012406430894341352, 0.04917543017320485, 0.19180672932267948, 1.5658665765039048]
+js_param_list:
+[25.019965453487124, 2.234282434447261, 0.4030157762041818, 0.08760092547989393, 0.02046930149314366, 0.004964000130850394, 0.0012380860917145943, 0.0002929583694304512, 6.615616263627544e-05, 1.1569818518820438e-05, 7.894867032746332e-06, 1.5262947016171314e-05, 2.2188288562469575e-05, 3.0270781258863287e-05, 2.074052848019181e-05, 19.848037676391975, 1.6913738148518316, 1.3876737056985733, 1.5002883788119872, 1.517203298217617, 1.6205635163882692, 1.3995765300151835, 3.7537800430114965, 1.491068738307882, 0.3139510374792983, 0.8954109520400352, 0.08829294447216889, 0.24015678854002387, 0.6855674923874414, 0.03237130139013715, 0.06051411543292261, 0.2401406521011821, 0.598906505329306, 0.016670960148347402, 0.015666452794268194, 0.06053434630319065, 0.24013972500633615, 0.5597453631404266, 0.01209810345400558, 0.003948155083131268, 0.01569997958390292, 0.06053472996433339, 0.8580477415395857]
+ptq_acc_list:
+[10.0, 11.97, 28.56, 72.45, 81.54, 83.6, 84.09, 84.03, 84.28, 84.28, 84.33, 84.35, 84.3, 84.34, 84.34, 10.0, 14.43, 26.16, 19.97, 20.39, 15.88, 18.64, 12.03, 16.08, 38.73, 33.46, 73.51, 64.56, 47.25, 78.19, 81.53, 65.26, 54.81, 81.27, 83.15, 81.86, 65.06, 58.96, 81.3, 83.87, 82.85, 81.7, 57.89]
+acc_loss_list:
+[0.881404174573055, 0.8580407969639469, 0.6612903225806451, 0.14077324478178357, 0.03296963946869055, 0.008538899430740024, 0.002727703984819613, 0.00343927893738131, 0.00047438330170768557, 0.00047438330170768557, -0.00011859582542700565, -0.0003557874762808484, 0.00023719165085384278, -0.0002371916508540113, -0.0002371916508540113, 0.881404174573055, 0.8288662239089183, 0.689753320683112, 0.7631641366223909, 0.7581831119544592, 0.8116698292220115, 0.7789373814041746, 0.8573292220113852, 0.8092979127134725, 0.5406783681214421, 0.6031783681214421, 0.12820208728652738, 0.23434535104364318, 0.43963472485768496, 0.07269924098671722, 0.03308823529411756, 0.226043643263757, 0.34997628083491455, 0.03617172675521819, 0.013875711574952414, 0.02917457305502839, 0.22841555977229594, 0.30075901328273236, 0.03581593927893734, 0.005336812144212389, 0.0174335863377609, 0.03107210626185947, 0.3134487666034155]
+
+
+ResNet_50
+Full_acc: 82.100000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[32.43454990142351, 4.214043863283937, 0.5388255214190923, 0.11540611251969539, 0.02705915578990609, 0.006509442202278693, 0.0016039070460832647, 0.0003991510572820047, 9.85893471520507e-05, 3.197663217543934e-05, 1.4193172350008603e-05, 9.494402295860181e-06, 1.4965900384283742e-05, 1.1006413972908124e-05, 9.19979977724061e-06, 27.97930767247182, 2.468904213217743, 2.002059016208805, 2.103427360082547, 2.1176935160244597, 2.0745701738918623, 2.0688235336534557, 4.81856416779809, 1.8450247546811167, 0.504675683635617, 1.0805896157556816, 0.1492571288789468, 0.4201372520873196, 0.8164550600708292, 0.0489459586267891, 0.11690259957670662, 0.420099260792996, 0.7090480603755034, 0.02218520887551022, 0.029685969242844282, 0.11690145812467295, 0.420099664402911, 0.6608826611924046, 0.014722341213935834, 0.007446566935912076, 0.02968041538493898, 0.11690123534737493, 0.6443664633800725]
+js_flops_wt_log_list:
+[62.54166807113872, 8.125697237536077, 1.0389861123792599, 0.2225309370570455, 0.05217660626834425, 0.01255178119568444, 0.0030927212616783566, 0.0007696599154495928, 0.0001901041402971097, 6.165869178475788e-05, 2.7367873970458382e-05, 1.830750723306581e-05, 2.885788078035027e-05, 2.1223031965581613e-05, 1.77394422225007e-05, 53.950881964765536, 4.760645307913557, 3.8604546951048095, 4.0559174141726135, 4.083426018185369, 4.000273769796785, 3.989192855543097, 9.291358803430924, 3.557654604149022, 0.9731369538175249, 2.083638505085131, 0.2878038955363636, 0.8101263820285742, 1.5743231065990642, 0.09437966326537438, 0.22541652656196548, 0.8100531256113771, 1.3672163965051112, 0.04277845611540752, 0.057241738837094706, 0.22541432556597737, 0.8100539038690502, 1.2743418409037999, 0.028388239708287414, 0.014358784659900266, 0.057231029653872396, 0.22541389599738867, 1.2424946111899446]
+js_flops_wt_cbrt_list:
+[142.48045026238654, 18.511706463970096, 2.3669853023392213, 0.5069629430595606, 0.11886709426779113, 0.028595070958482575, 0.007045739768225618, 0.0017534148781806179, 0.00043308924020809105, 0.00014046888161168392, 6.234862494552844e-05, 4.170758398676049e-05, 6.574311133700011e-05, 4.834964022624979e-05, 4.041343624526502e-05, 122.90919304629132, 10.845551580739786, 8.79476579598787, 9.240062780881415, 9.30273201255859, 9.113297190045397, 9.088053001636661, 21.167279778281802, 8.10493620510138, 2.216969831843561, 4.746879345317246, 0.6556657327503118, 1.8456043025519484, 3.5865731120501874, 0.21501276400826458, 0.5135368018102701, 1.8454374120990906, 3.1147491550527016, 0.09745652581443902, 0.13040631909648226, 0.5135317875708184, 1.8454391851011087, 2.9031652797243033, 0.06467318989945364, 0.03271172910252679, 0.13038192177395289, 0.5135308089412451, 2.830611928187878]
+js_param_list:
+[33.03456091879867, 3.9984768996296913, 0.49915617472803026, 0.10781748223103156, 0.025283552892414147, 0.006117779274258625, 0.0015045989680227618, 0.00036642736274082035, 8.705934353284482e-05, 2.2445667334022087e-05, 1.1461111601173797e-05, 1.0002359152822217e-05, 2.114519755323751e-05, 1.358607891960662e-05, 8.62114865466085e-06, 28.339961742851614, 2.5108256813922205, 2.416008469752967, 2.5536338667635565, 2.5895696403940964, 2.5412725386390154, 2.5119317288745378, 4.668675643992857, 1.770042795036069, 0.4997155235783285, 1.025712051693073, 0.15168070213072746, 0.42727341005119857, 0.7717747429312491, 0.04781233863688248, 0.12432027178557617, 0.42721741424280274, 0.6692060893714656, 0.02037538716385666, 0.03146633038851121, 0.12432571514681283, 0.42721749408936366, 0.6235907633174202, 0.012870451493143545, 0.007864776888036686, 0.031451836616023056, 0.12432633116858423, 0.9692708078086846]
+ptq_acc_list:
+[10.0, 10.03, 16.75, 66.79, 78.99, 81.26, 81.8, 82.23, 82.08, 82.21, 82.1, 82.15, 82.11, 82.1, 82.07, 10.0, 10.23, 13.72, 11.43, 10.47, 16.49, 12.78, 9.95, 12.75, 16.68, 26.39, 59.63, 53.6, 31.92, 74.35, 76.66, 58.33, 33.49, 77.58, 80.77, 76.83, 51.55, 39.04, 77.84, 81.59, 80.94, 75.42, 49.87]
+acc_loss_list:
+[0.8781973203410475, 0.8778319123020706, 0.7959805115712546, 0.18647990255785615, 0.03788063337393422, 0.010231425091351878, 0.0036540803897685404, -0.0015834348355665002, 0.00024360535931785654, -0.0013398294762484706, 0.0, -0.000609013398294901, -0.00012180267965901482, 0.0, 0.0003654080389768714, 0.8781973203410475, 0.8753958587088916, 0.8328867235079171, 0.8607795371498173, 0.8724725943970767, 0.7991473812423874, 0.8443361753958587, 0.8788063337393422, 0.8447015834348356, 0.7968331303288672, 0.6785627283800243, 0.27369062119366616, 0.34713763702801453, 0.6112058465286235, 0.09439707673568819, 0.06626065773447014, 0.28952496954933005, 0.5920828258221681, 0.05505481120584648, 0.016199756394640663, 0.06419001218026792, 0.37210718635809986, 0.5244823386114494, 0.051887941534713655, 0.006211936662606467, 0.014129110840438448, 0.08136419001218019, 0.3925700365408039]
+
+
+ResNet_152
+Full_acc: 79.760000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[34.395640743617925, 3.503029669404365, 0.5665010081777374, 0.12284888808096202, 0.028694268658336224, 0.006941139055980186, 0.0017113686732310866, 0.0004272219889055042, 0.00011062316111393365, 2.88482591167858e-05, 1.828076430432278e-05, 1.5860396688294654e-05, 1.604485607656334e-05, 1.7353923347356936e-05, 1.0451333534658155e-05, 31.237778002381113, 2.981360635005943, 3.143829524004256, 3.2218300205350014, 3.2042351535053677, 3.1900783907814976, 3.0823922750431167, 5.244018540125816, 2.0048524207135237, 0.5612364336540991, 1.168937773772066, 0.16609668819390203, 0.47514408968838134, 0.8825127368217848, 0.05347642422798172, 0.13349439025341864, 0.4751185395366517, 0.7666849446225685, 0.023341455498657303, 0.033972227602133837, 0.13348612813137076, 0.47511821269455784, 0.7150214634868213, 0.015116210388604199, 0.008415063623769014, 0.033963959689082585, 0.13348561574672796, 1.5660946994139746]
+js_flops_wt_log_list:
+[81.8701328865851, 8.338077103360835, 1.3484125260408606, 0.2924107408588912, 0.06829945706353073, 0.01652162787526515, 0.004073480757045212, 0.0010168940088797359, 0.0002633104865886916, 6.866599244487472e-05, 4.3512740873738457e-05, 3.7751667258752014e-05, 3.819072622969262e-05, 4.1306630137871354e-05, 2.487675899694503e-05, 74.35363845085854, 7.096375764305536, 7.4830919075335345, 7.668752383051109, 7.626872247350913, 7.593175650329031, 7.3368560582255675, 12.482061257122474, 4.772044670581215, 1.3358815364583634, 2.782361043604784, 0.3953512026659835, 1.1309604624906184, 2.1005986071391796, 0.1272871172971714, 0.3177496692422843, 1.1308996468095567, 1.8248998111785566, 0.05555843770103233, 0.08086230487670391, 0.3177300033481372, 1.1308988688445274, 1.701927946880549, 0.03598031978765312, 0.02002993292848804, 0.08084262519853637, 0.31772878374594676, 3.727692765188247]
+js_flops_wt_cbrt_list:
+[213.75644761905772, 21.770060445086415, 3.520598554430379, 0.7634613381932741, 0.1783244853948795, 0.04313666484935379, 0.010635536371692504, 0.0026550299025940856, 0.0006874828737851819, 0.0001792814803120775, 0.00011360832806053685, 9.856662008972728e-05, 9.971296836857466e-05, 0.00010784834725524751, 6.495125198866898e-05, 194.13147459800393, 18.528076367596558, 19.53776165939733, 20.022506490151166, 19.913160765190888, 19.825181613059467, 19.155951412357304, 32.589675614453014, 12.45943154967126, 3.487881130824671, 7.264524823704857, 1.0322307496541274, 2.9528483994828516, 5.484496974710366, 0.3323366050816797, 0.8296192779295041, 2.9526896145452324, 4.7646692040747265, 0.1450586906300643, 0.21112508832345694, 0.8295679318338777, 2.9526875833400155, 4.443599383584156, 0.09394177181391433, 0.05229657211834265, 0.21107370623884245, 0.8295647475489308, 9.73271124899402]
+js_param_list:
+[34.573598806192116, 3.294925147777751, 0.554545115952515, 0.120654473923552, 0.028210152042126846, 0.006827639313272994, 0.001684914469636476, 0.0004225569496713925, 0.000108978856732773, 2.6407277518448736e-05, 2.1345037242811895e-05, 1.6695036439651395e-05, 1.8612698936728237e-05, 2.3916595609392664e-05, 1.3332924498732996e-05, 31.764417239538936, 3.081480872130916, 3.478690608554047, 3.5561727044199496, 3.5401642449640187, 3.5336853318116317, 3.4012489500320964, 5.234487884218976, 1.9908348386800327, 0.5570415039094814, 1.1565608134760599, 0.166702769588833, 0.4756209119461034, 0.8712121130896631, 0.05303711351404139, 0.13588938195327838, 0.4755935859575454, 0.755901865994144, 0.02273258776515624, 0.03458845336385294, 0.13587320889509194, 0.4755932029757784, 0.7045100320786581, 0.014470452325467596, 0.008611736782138136, 0.0345804892826189, 0.1358728514479267, 1.9126512380740057]
+ptq_acc_list:
+[10.0, 9.49, 17.94, 55.47, 77.37, 79.45, 79.58, 79.72, 79.85, 79.8, 79.81, 79.74, 79.68, 79.73, 79.75, 10.0, 9.84, 10.0, 10.0, 10.0, 10.04, 10.0, 10.14, 10.13, 11.88, 10.06, 27.02, 10.12, 12.51, 34.71, 31.75, 12.2, 14.27, 52.93, 31.75, 40.67, 10.16, 15.0, 48.75, 32.89, 35.42, 28.89, 22.8]
+acc_loss_list:
+[0.8746238716148446, 0.8810180541624876, 0.7750752256770311, 0.30453861584754266, 0.02996489468405216, 0.003886659979939848, 0.0022567703109328836, 0.0005015045135407003, -0.0011283851554662638, -0.000501504513540522, -0.0006268806419257416, 0.0002507522567704392, 0.0010030090270812223, 0.00037612838515548063, 0.0001253761283852196, 0.8746238716148446, 0.876629889669007, 0.8746238716148446, 0.8746238716148446, 0.8746238716148446, 0.8741223671013039, 0.8746238716148446, 0.8728686058174524, 0.8729939819458375, 0.8510531594784354, 0.8738716148445336, 0.66123370110331, 0.8731193580742226, 0.8431544633901704, 0.5648194583751254, 0.6019307923771314, 0.8470411233701103, 0.8210882647943832, 0.3363841524573722, 0.6019307923771314, 0.4900952858575727, 0.8726178535606821, 0.8119358074222668, 0.38879137412236714, 0.5876379137412236, 0.5559177532597793, 0.6377883650952859, 0.7141424272818456]
+
+

ykl/ALL-cifar10/ptq_result_sgd_5_momentum_95_ntb.txt

+AlexNet
+Full_acc: 87.150000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[35.311066408555135, 13.37020247855369, 2.6267547846436083, 0.5578432905598815, 0.12965500929031754, 0.031283100415830635, 0.00774254634109718, 0.0019494966934114131, 0.0005390533862208429, 0.00012997932001559677, 9.000539737556857e-05, 2.9654329656310097e-05, 6.822028105535338e-07, 1.0656474235427815e-05, 8.138780646021359e-06, 35.06702479199253, 7.781307201901994, 1.478647471541299, 1.4766887961222073, 1.4761457947368435, 1.476575513875985, 1.4759444833181186, 6.90416992739758, 2.263254291505926, 1.3778596363166196, 1.1942125307927134, 0.4771081087085134, 0.4182496835260322, 0.87096164970216, 0.2500989487081769, 0.10701005151658648, 0.41798840195733483, 0.7521799335808195, 0.1784904279165476, 0.026758174844390838, 0.1067945643296612, 0.4179872822004957, 0.7011191504194255, 0.15173383768452395, 0.006798474619081248, 0.026702464878614877, 0.10679308823196294, 0.4179863676140317]
+js_flops_wt_log_list:
+[65.16989400981979, 24.675966121659457, 4.847922997407303, 1.0295522570530076, 0.23929051350443017, 0.057735903946086804, 0.014289597447364879, 0.0035979794949306523, 0.0009948737214329924, 0.00023988905944887024, 0.00016611342572923237, 5.472985432594478e-05, 1.2590694470276166e-06, 1.9667525426899814e-05, 1.5020885122344502e-05, 64.7194922547055, 14.361134261380982, 2.7289829733062847, 2.725368053609866, 2.7243658934847113, 2.7251589805704373, 2.723994354328417, 12.74229492783807, 4.177048650645559, 2.5429695444544116, 2.2040315394752277, 0.8805478859235525, 0.771919126702679, 1.6074416371621014, 0.4615811312719333, 0.19749711420869692, 0.7714369069945084, 1.3882188088179317, 0.3294208714770094, 0.049384728241531776, 0.19709941233897962, 0.7714348403779503, 1.293981331303511, 0.2800390677866317, 0.01254722429585507, 0.049281910260253525, 0.1970966880618527, 0.7714331524226156]
+js_flops_wt_cbrt_list:
+[145.5823964444013, 55.12340225734819, 10.829728334898544, 2.299906837717651, 0.5345487656790905, 0.12897567788108677, 0.03192139364688421, 0.008037491623830086, 0.0022224387921200377, 0.0005358858516801899, 0.0003710791764615545, 0.00012226049268429764, 2.812623070410215e-06, 4.393509498953364e-05, 3.355501011705595e-05, 144.5762483162116, 32.08119904440247, 6.096248691919503, 6.088173357743274, 6.085934641924294, 6.08770631150092, 6.085104664193502, 28.464889501224093, 9.331068614814752, 5.680715090790501, 4.923564756872221, 1.9670474129892874, 1.7243826775245696, 3.5908483393775223, 1.0311216285538294, 0.44118689487239854, 1.7233054515785275, 3.101128581649285, 0.7358901013587634, 0.1103200671781725, 0.4402984725089198, 1.7233008349837697, 2.8906123913151784, 0.6255765673071519, 0.02802912310154681, 0.11009038308340002, 0.44029238677257265, 1.7232970642766592]
+js_param_list:
+[9.7618765851903, 3.697932937367412, 0.7234039218116686, 0.1537534865810826, 0.03571757028915834, 0.008586101603981603, 0.002131463912393883, 0.0005424596042225236, 0.0001383080386174278, 4.349068328362193e-05, 2.4367527335995125e-05, 7.315387928591655e-06, 3.464621295883319e-07, 3.973864595849364e-06, 7.90752313079679e-06, 9.70930449543552, 2.134130400180493, 0.3939161449930689, 0.39360981436146547, 0.3933654329461897, 0.393495507200394, 0.3934062600986581, 1.878273033228151, 0.6128464456412978, 0.37380890960380914, 0.3242360482004804, 0.13027057923471536, 0.10956590779968875, 0.23630108208585549, 0.0686196246177969, 0.028478968562719444, 0.10950021968697857, 0.20374638886956414, 0.04905350480680959, 0.007196386380628369, 0.028388299271820374, 0.1095005097462983, 0.1900653796647475, 0.04170031250612274, 0.0018140344958256095, 0.007191483406997917, 0.028387723281176534, 0.10949937847867021]
+ptq_acc_list:
+[10.0, 10.15, 42.94, 73.91, 84.72, 86.17, 86.99, 87.04, 87.22, 87.19, 87.18, 87.15, 87.16, 87.18, 87.14, 10.0, 10.94, 75.3, 74.89, 75.18, 75.01, 75.29, 15.83, 53.85, 52.79, 71.94, 79.37, 83.82, 77.89, 84.4, 86.06, 83.87, 81.78, 85.28, 86.99, 86.07, 84.12, 83.68, 85.85, 87.11, 87.12, 86.0, 84.02]
+acc_loss_list:
+[0.8852553069420539, 0.8835341365461847, 0.5072862880091796, 0.1519219736087207, 0.02788296041308097, 0.011244979919678759, 0.001835915088927261, 0.0012621916236374001, -0.0008032128514055441, -0.00045897877223169295, -0.00034423407917385124, 0.0, -0.0001147446930578417, -0.00034423407917385124, 0.00011474469305800477, 0.8852553069420539, 0.874469305794607, 0.1359724612736662, 0.14067699368904193, 0.1373493975903614, 0.13930005737234652, 0.13608720596672402, 0.8183591508892714, 0.38209982788296043, 0.39426276534710275, 0.17452667814113607, 0.08927137119908204, 0.03820998278829618, 0.10625358577165811, 0.03155479059093517, 0.01250717154331616, 0.03763625932300632, 0.06161790017211709, 0.021457257601835967, 0.001835915088927261, 0.012392426850258318, 0.03476764199655767, 0.03981640849110727, 0.014916810097533118, 0.000458978772231856, 0.00034423407917385124, 0.013195639701663863, 0.035915088927137226]
+
+
+AlexNet_BN
+Full_acc: 89.020000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[28.34542972445257, 7.592466006852456, 1.4017616658844982, 0.3052162499058463, 0.07160364821881604, 0.017347743622537954, 0.004268773101890292, 0.001066633404989284, 0.00022743536067713335, 9.348804661186706e-05, 6.307638079988287e-05, 2.4777847101481232e-05, 1.46496363750227e-05, 6.29750966431044e-05, 4.2512986232933206e-05, 27.313727386335778, 4.399390204398832, 1.1949816547358039, 1.1859390759435553, 1.1935124511994109, 1.185799451193582, 1.1899367528386484, 5.2698238726936415, 1.9218104244147758, 0.7895114207333976, 1.101757983469671, 0.2625945914999009, 0.31581190266998116, 0.8326758629682507, 0.1294327485236109, 0.08150750091885518, 0.31561155593181206, 0.7266312917702542, 0.08858189371868416, 0.02059589181172277, 0.08140615033197927, 0.315616259161768, 0.6805719784744779, 0.07410786319607146, 0.005203136505344816, 0.020565529277841495, 0.08140615308380574, 0.31659599456416987]
+js_flops_wt_log_list:
+[52.34573689581621, 14.021069069991055, 2.588644732725007, 0.5636453449183924, 0.1322310427775711, 0.03203622002661253, 0.007883178199507858, 0.0019697606324765625, 0.00042000674064715423, 0.0001726451402720128, 0.00011648366829459837, 4.575745288842849e-05, 2.7053603306101474e-05, 0.00011629662601392104, 7.850907936967453e-05, 50.44048375021752, 8.12438987096499, 2.206782394982214, 2.1900833907716555, 2.2040692047956463, 2.189825544603956, 2.197465933388905, 9.731826845064354, 3.549023028332148, 1.45799719770347, 2.0346254788233673, 0.48493557975803864, 0.5832124235346804, 1.537709326199836, 0.2390245152669009, 0.1505205685575928, 0.5828424415747637, 1.341875949280828, 0.1635849075991014, 0.038034601852625714, 0.15033340359991124, 0.5828511270679888, 1.2568178387204936, 0.1368555970568415, 0.009608674738377298, 0.03797853111298657, 0.15033340868173167, 0.5846604124483901]
+js_flops_wt_cbrt_list:
+[116.96405391196215, 31.329410489908575, 5.784203261480199, 1.2594386557490715, 0.29546396198516334, 0.07158340656825635, 0.017614585916709818, 0.0044013362400303424, 0.0009384850413743564, 0.00038576733640419803, 0.00026027720433832437, 0.00010224284734339044, 6.044998701433125e-05, 0.0002598592672779578, 0.00017542479553300677, 112.70685657646047, 18.1535618986634, 4.930950070153634, 4.893636941242157, 4.924887576012292, 4.893060796271272, 4.910132880814274, 21.74530319505284, 7.930122784341097, 3.25782523942859, 4.546273647257964, 1.0835654373829489, 1.3031603602066897, 3.435938190889044, 0.5340888857024177, 0.3363310355276824, 1.3023336531533096, 2.9983578450185626, 0.36552267837371705, 0.08498650483161113, 0.3359128243513976, 1.3023530604739066, 2.8082995514648856, 0.30579730808201816, 0.021470125682993252, 0.08486121743663014, 0.33591283570648306, 1.3063958224126024]
+js_param_list:
+[8.799590634835157, 2.4123049418725535, 0.4389000122886768, 0.09579143454709549, 0.022400024625311737, 0.00542295589265722, 0.00135018134861399, 0.0003273817262759041, 6.90366041088355e-05, 2.8363758177682995e-05, 1.7424378512274077e-05, 8.08438003345624e-06, 1.0055043617189839e-05, 1.9116609365309454e-05, 1.3836154751436515e-05, 8.514886270326267, 1.3786185615774904, 0.3619246475186809, 0.3592875045623714, 0.3614364700726146, 0.3596284490221087, 0.36088308631200383, 1.6170141291532427, 0.5885179776721339, 0.24599012825838476, 0.33633085189864675, 0.08195515661265507, 0.09642318855808786, 0.25363269503024666, 0.0406417591471982, 0.024888481859801963, 0.09639696720320688, 0.2212880694350999, 0.027914325339466754, 0.006247855044243767, 0.024817722597610874, 0.09640005199235512, 0.2070776173579275, 0.023366797560355507, 0.0015862183494694477, 0.006259983186041011, 0.024815790286256967, 0.09667671734939992]
+ptq_acc_list:
+[10.0, 14.86, 34.63, 77.96, 87.87, 88.77, 88.97, 89.1, 89.05, 89.06, 89.05, 89.03, 89.03, 89.01, 89.02, 10.0, 17.7, 65.76, 67.91, 67.93, 70.75, 68.94, 14.61, 56.05, 72.54, 76.93, 84.66, 83.86, 79.98, 87.17, 88.11, 83.84, 81.24, 87.84, 88.74, 88.33, 83.71, 81.42, 88.17, 89.08, 88.85, 88.33, 83.82]
+acc_loss_list:
+[0.8876656931026735, 0.8330712199505729, 0.6109862952145585, 0.12424174342844307, 0.012918445293192447, 0.0028083576724331613, 0.0005616715344866003, -0.0008986744551785924, -0.00033700292069199214, -0.000449337227589376, -0.00033700292069199214, -0.00011233430689738392, -0.00011233430689738392, 0.00011233430689722428, 0.0, 0.8876656931026735, 0.8011682767917322, 0.26128959784318123, 0.23713772186025614, 0.23691305324646136, 0.20523477870141538, 0.2255672882498315, 0.835879577623006, 0.3703662098404853, 0.18512693776679387, 0.13581217703886755, 0.048977757807234325, 0.05796450235902041, 0.10155021343518301, 0.02078184677600533, 0.010222421927656669, 0.058189170972815014, 0.08739609076612, 0.013255448213884437, 0.0031453605931251534, 0.0077510671759154996, 0.05964951696248037, 0.08537407324196804, 0.009548416086272684, -0.0006740058413839843, 0.0019096832172545688, 0.0077510671759154996, 0.058413839586609784]
+
+
+VGG_16
+Full_acc: 91.670000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[10.108746086343771, 2.7305314005276338, 0.5008592253870917, 0.1087524357305037, 0.025494288381742166, 0.006187239508539377, 0.001516890225839242, 0.0004162398944452334, 9.70535268306172e-05, 2.6961578727649812e-05, 3.3646192785583554e-05, 2.7548545267455978e-05, 1.65688613487372e-05, 4.2442754164580034e-05, 2.7545475486856677e-05, 10.237148229284443, 1.7249460896828313, 0.7883360585516611, 0.7763803409983743, 0.7847340811616329, 0.7901838519638318, 0.7852284214036602, 1.8164917764084012, 0.6581522729845122, 0.28044916658629526, 0.37592537653739005, 0.09335334704262917, 0.11079867284433677, 0.284301808727179, 0.046333175538657824, 0.02861032057320222, 0.11077512347720783, 0.24819652915908222, 0.03175691713446556, 0.007305592113557693, 0.028544052172150034, 0.11077628910954862, 0.23228109792960727, 0.02663686938424192, 0.0018098529256600424, 0.007245119116744774, 0.02854547222359787, 0.4272937582331487]
+js_flops_wt_log_list:
+[25.50349152415138, 6.888894412321031, 1.263624479272366, 0.2743729834732538, 0.06431988320850192, 0.015609869811360727, 0.003826982761988782, 0.0010501372306045544, 0.0002448576487896433, 6.802172976599712e-05, 8.488643252067849e-05, 6.950259554747422e-05, 4.180180324671657e-05, 0.00010707939558995434, 6.949485075669339e-05, 25.8274390183509, 4.351889700471808, 1.9889036499409274, 1.9587404092500456, 1.9798161727155503, 1.9935654728043328, 1.9810633503639852, 4.5828515453471335, 1.6604612255853481, 0.7075489760942111, 0.9484271909041895, 0.23552241541304544, 0.27953546262183443, 0.7172688587924572, 0.1168945920239818, 0.07218136275355079, 0.2794760496066254, 0.626178363138795, 0.08011995355844939, 0.018431376647072863, 0.07201417331283107, 0.27947899039614266, 0.5860251075324968, 0.0672025161942364, 0.004566102299461968, 0.018278808509691134, 0.07201775597973697, 1.0780251722954692]
+js_flops_wt_cbrt_list:
+[70.09199692100829, 18.932951414919014, 3.4728563744534795, 0.7540673517037225, 0.17677222946280263, 0.04290106496669746, 0.01051780944252476, 0.002886123081008996, 0.0006729494880650498, 0.00018694612338882784, 0.00023329588269278183, 0.00019101603043263107, 0.00011488512706924142, 0.00029428945675485306, 0.0001909947451960175, 70.98231136068017, 11.96042664186497, 5.466162481157777, 5.383263958356269, 5.441187099841942, 5.478974706239926, 5.444614754906789, 12.59518588275328, 4.56349449254692, 1.9445776907406496, 2.6065903831914246, 0.6472931912215468, 0.7682555452004607, 1.9712911306440786, 0.3212648502056459, 0.19837816524396115, 0.768092258660845, 1.7209444385120076, 0.22019602817175543, 0.05065549530632601, 0.19791867357972942, 0.7681003409192917, 1.6105900634783026, 0.18469465460091208, 0.012549153436964416, 0.05023618780070556, 0.19792851992870875, 2.962768332553546]
+js_param_list:
+[4.961402155931791, 1.1296616883250303, 0.20746860154818111, 0.0449109423953304, 0.01051816136090789, 0.002567053669443602, 0.0005934493496828104, 0.00016344114889739816, 4.210806121108668e-05, 2.6227902230950837e-05, 2.9154426936399688e-05, 1.822315129499491e-05, 6.341555326732112e-06, 1.6399926998313174e-05, 1.9389907310355726e-05, 5.38646261044325, 0.7712023944167686, 0.4611057705681646, 0.4598777854260451, 0.45751136551795996, 0.46100941465204115, 0.46040841192715953, 0.9204522499690955, 0.3538231585652671, 0.11748794146457685, 0.209772963625879, 0.03916606378595014, 0.05477834272648655, 0.16120963248928666, 0.0180170573196551, 0.01478428688833779, 0.054843193208805416, 0.1414841547778738, 0.011993081651207774, 0.003787380955311598, 0.014631386839481363, 0.05484340504941017, 0.13263181423447634, 0.009855733382986048, 0.0009732370319088145, 0.0037284030959092427, 0.014641616365210611, 0.26652530596383184]
+ptq_acc_list:
+[10.0, 11.47, 63.43, 88.66, 91.54, 91.54, 91.73, 91.67, 91.69, 91.67, 91.67, 91.67, 91.67, 91.67, 91.67, 10.0, 18.01, 79.6, 79.05, 79.87, 80.62, 80.35, 13.94, 47.09, 82.93, 79.71, 90.08, 89.84, 84.8, 90.94, 91.38, 89.68, 86.2, 91.02, 91.7, 91.48, 89.86, 86.93, 91.1, 91.6, 91.78, 91.37, 84.85]
+acc_loss_list:
+[0.8909130577069925, 0.8748772771899204, 0.30806152503545325, 0.032835169630195324, 0.0014181302498090483, 0.0014181302498090483, -0.00065452165375807, 0.0, -0.00021817388458597164, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.8909130577069925, 0.8035344169302934, 0.13166793934766016, 0.13766772117377554, 0.12872259190574886, 0.12054107123377328, 0.12348641867568459, 0.8479328024435475, 0.4863095887422275, 0.09534198756408852, 0.1304679829824371, 0.017344823824588233, 0.019962910439620358, 0.07494272935529622, 0.007963346787389592, 0.0031635213264972865, 0.021708301516308443, 0.0596705574342751, 0.0070906512490455514, -0.000327260826879035, 0.0020726519035671183, 0.019744736555034385, 0.051707210646885514, 0.00621795571070151, 0.0007636085960511333, -0.0011999563652230767, 0.0032726082687901947, 0.07439729464383121]
+
+
+VGG_19
+Full_acc: 91.680000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[9.288979856351963, 2.5405749051785684, 0.4667742391484616, 0.10159864850876543, 0.023836559565255072, 0.005760752546001823, 0.001439205246085693, 0.0003712385101332156, 0.00012256112699033926, 4.390461553542525e-05, 3.897290542604717e-05, 3.9004312406667404e-05, 4.483688876210791e-05, 3.712789243871969e-05, 1.757598261487988e-05, 9.42295564558682, 1.6129817061069969, 0.6740627568793178, 0.6636322793327664, 0.6582334722357649, 0.6870966507950584, 0.6620322183057537, 1.679151367441454, 0.6080843763901622, 0.2602089968686309, 0.3479214306009024, 0.08732030839119877, 0.10157579991505712, 0.263167325917142, 0.043224937799345005, 0.02652390436452618, 0.10154155583782759, 0.22947957938676722, 0.029746898957403352, 0.00661922168147049, 0.026498099971811056, 0.10154069317845658, 0.21469545290594133, 0.024929119587572757, 0.001683971434164398, 0.0066504993433871675, 0.02649951684381438, 0.3330723012600656]
+js_flops_wt_log_list:
+[24.35191126667066, 6.660349749271167, 1.2236914095080316, 0.26635016025008074, 0.0624897235662395, 0.01510234029989353, 0.0037730083377497795, 0.0009732357478795828, 0.0003213052170816128, 0.00011509996988365251, 0.0001021710402451203, 0.00010225337652069042, 0.00011754400951378956, 9.733416975470469e-05, 4.607704782236354e-05, 24.703141066043987, 4.228579240057331, 1.7671172397330457, 1.7397727877468645, 1.7256193205841437, 1.8012867861206687, 1.7355780812501733, 4.402049066268707, 1.5941488737728307, 0.6821617121067625, 0.9121078887874445, 0.22891818419292942, 0.2662902605694745, 0.6899172425956883, 0.11331813246179186, 0.06953484403229891, 0.26620048658535583, 0.6016017303469668, 0.07798421948067327, 0.01735289574682057, 0.06946719545393976, 0.26619822504482277, 0.5628437890246892, 0.06535396970827397, 0.004414685312546742, 0.01743489300156787, 0.06947090991363661, 0.8731795365154669]
+js_flops_wt_cbrt_list:
+[69.47553675419447, 19.001850357207815, 3.491168170172565, 0.7598919093166524, 0.17828198529647835, 0.04308668782050205, 0.01076432057314469, 0.002776622961206618, 0.0009166776346306863, 0.0003283780110928294, 0.0002914920222906435, 0.00029172692610894893, 0.0003353508093792905, 0.00027769252335800424, 0.0001314569355337939, 70.47758864935423, 12.06405563791265, 5.041551662767246, 4.963538464616784, 4.923158893695436, 5.139036724614385, 4.951571047267379, 12.558961731923723, 4.54807622525491, 1.946194308232511, 2.602226349188998, 0.6530991980644529, 0.7597210166743481, 1.9683206882784625, 0.3232944630321928, 0.19838157914426577, 0.7594648932159097, 1.7163582221701328, 0.2224874855795013, 0.04950747943550488, 0.19818857904498433, 0.7594584410840541, 1.6057825573951785, 0.18645362471873309, 0.012595012700700297, 0.04974141603389156, 0.19819917632740555, 2.4911645052361733]
+js_param_list:
+[4.909673099947618, 1.0558669980235038, 0.1960863245154005, 0.04249322736691104, 0.010035105011089741, 0.002444954603366522, 0.0006176232362627835, 0.0001495496363756992, 5.677266015724936e-05, 3.0652245177395615e-05, 3.2402104063774646e-05, 1.3187157644885763e-05, 2.600921518682889e-05, 2.3665910773343016e-05, 1.110912920095726e-05, 5.282912152138583, 0.7646712313134155, 0.4330661310603931, 0.4235658590421957, 0.41802322270770037, 0.42891972424501923, 0.425718941879512, 0.9240047496676567, 0.36047045211156675, 0.11126228877308444, 0.215995949774722, 0.036711716058334246, 0.05393049458281086, 0.16642594454902734, 0.016840971141331696, 0.014931042136526948, 0.05397800859585413, 0.14640072762629258, 0.011070804538263647, 0.0037402674153204324, 0.01495949439491221, 0.05397842419908961, 0.13746081134837645, 0.008875208900147072, 0.0008116802231543937, 0.0036908954513075478, 0.014960180176422908, 0.23379051509469487]
+ptq_acc_list:
+[10.0, 11.16, 60.49, 90.17, 91.39, 91.51, 91.67, 91.63, 91.65, 91.7, 91.7, 91.69, 91.68, 91.68, 91.68, 10.0, 18.66, 81.0, 79.84, 76.67, 77.94, 76.95, 10.14, 43.55, 82.5, 81.24, 89.72, 89.76, 85.9, 91.1, 91.12, 89.71, 87.4, 91.09, 91.55, 91.17, 89.78, 88.03, 91.39, 91.61, 91.62, 91.07, 86.0]
+acc_loss_list:
+[0.8909249563699826, 0.8782722513089005, 0.3402050610820245, 0.01647033158813269, 0.003163176265270574, 0.0018542757417103152, 0.00010907504363007325, 0.0005453752181502112, 0.00032722513089006474, -0.00021815008725999149, -0.00021815008725999149, -0.00010907504362991824, 0.0, 0.0, 0.0, 0.8909249563699826, 0.7964659685863875, 0.1164921465968587, 0.12914485165794068, 0.16372164048865623, 0.14986910994764407, 0.16066753926701574, 0.8893979057591623, 0.524978184991274, 0.10013089005235609, 0.11387434554973834, 0.021378708551483504, 0.02094240837696337, 0.0630453752181501, 0.006326352530541148, 0.006108202443281001, 0.02148778359511358, 0.04668411867364748, 0.006435427574171066, 0.0014179755671903322, 0.005562827225130945, 0.020724258289703375, 0.03981239092495643, 0.003163176265270574, 0.0007635253054102027, 0.0006544502617801295, 0.006653577661431213, 0.06195462478184998]
+
+
+Inception_BN
+Full_acc: 92.400000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[18.789850068761115, 3.101083954207525, 0.5270291119382463, 0.11470128598901126, 0.026858685743949652, 0.0064974962892005, 0.0015982231257645, 0.0003737921480898302, 0.00011351786008651563, 2.2655216611807476e-05, 1.8362926203087765e-05, 3.0616330837651924e-05, 1.6694227335252057e-05, 1.2745124690585837e-05, 2.5749707479101948e-05, 18.535780039416682, 2.332323640080497, 1.877631670132598, 1.8811187556254851, 1.7308942310989472, 1.8512670140834868, 1.7347132595577326, 3.4403096526250145, 1.3758215020221167, 0.3382668083070035, 0.8311818646800002, 0.1022104507691941, 0.20660033043470866, 0.6404598237627288, 0.04308417758468814, 0.05235230231790444, 0.2065721098885874, 0.5617643922111636, 0.026200473670156092, 0.013181386197805206, 0.0523259864355059, 0.2065713765125279, 0.5263077233047716, 0.02081638800250542, 0.003314874769111333, 0.01315967066980172, 0.052327272263839, 1.1630908524482078]
+js_flops_wt_log_list:
+[59.6512504116859, 9.844864904357951, 1.673134453730614, 0.3641367604336629, 0.08526700230060723, 0.02062729488408599, 0.005073803544990788, 0.0011866602951078563, 0.0003603797940611208, 7.192244718805345e-05, 5.8295915359792904e-05, 9.719622087459323e-05, 5.299837580185605e-05, 4.0461345974775167e-05, 8.174638133048424e-05, 58.844666277854735, 7.4043178091579955, 5.960828666866334, 5.971898952642004, 5.494988242992057, 5.877130037327073, 5.50711233243879, 10.92178872271362, 4.36775560414646, 1.0738796751672661, 2.6387138463711954, 0.32448269524305406, 0.6558843205666102, 2.0332375823163606, 0.13677730564537857, 0.16620038391820469, 0.6557947301304293, 1.7834069090241595, 0.0831774074877603, 0.041846324796686436, 0.16611684012807248, 0.6557924019160577, 1.6708442952747418, 0.06608480476747819, 0.010523576516684632, 0.041777385534583884, 0.1661209221866096, 3.6924096486685656]
+js_flops_wt_cbrt_list:
+[214.8527507699419, 35.45937915905603, 6.026320275116372, 1.311553137539949, 0.30711594254533015, 0.07429569399136036, 0.018274900207188194, 0.004274130498083355, 0.001298021240821608, 0.00025905132773934125, 0.00020997108505288787, 0.00035008277739737324, 0.00019089032918473696, 0.00014573427082480278, 0.00029443531817234834, 211.94758417770788, 26.668980748822896, 21.46980033286383, 21.509673450934226, 19.791929445020433, 21.1683333783897, 19.835598168647074, 39.33825920174589, 15.731846353012168, 3.867915602996626, 9.504158328196933, 1.1687265425128002, 2.3623737891168184, 7.323345018161035, 0.4926465104753894, 0.5986229863984753, 2.3620511009665566, 6.423501225900469, 0.2995896092256381, 0.15072270790855344, 0.5983220771468541, 2.362042715169802, 6.018071548718395, 0.23802522143954458, 0.03790397262325812, 0.15047440146071606, 0.5983367799653105, 13.29937536113243]
+js_param_list:
+[17.412627503405528, 2.7331820793627695, 0.4496146283570642, 0.09788688762966251, 0.022947737939707107, 0.005541195791473513, 0.001360578988816906, 0.0003334226851268109, 0.00011347696173331826, 2.477935675392428e-05, 1.1512300006804475e-05, 1.7051820370420416e-05, 1.6836825376600026e-05, 2.245958949732943e-05, 1.1883649172800078e-05, 18.28581989228255, 2.253524725918134, 2.218227390427663, 2.2624667985078473, 2.0171870362461704, 2.1110807209001337, 1.9603054460242182, 3.317151345513068, 1.3336641898528143, 0.3034191686428555, 0.8077038944489773, 0.0893867203502547, 0.2004593324554517, 0.6223058017048475, 0.03613890314523507, 0.05042565131296881, 0.20041934830710176, 0.5456019656206686, 0.02109090850766096, 0.012775770579106957, 0.05039946307915701, 0.20041843839153567, 0.5109477981522788, 0.016408153112313736, 0.003214700782942353, 0.012768745713897798, 0.05039996977099034, 1.372684757768232]
+ptq_acc_list:
+[10.0, 12.05, 70.21, 89.14, 92.03, 92.24, 92.53, 92.44, 92.42, 92.4, 92.39, 92.4, 92.39, 92.4, 92.4, 10.0, 12.37, 27.55, 26.83, 30.53, 31.2, 35.4, 10.23, 13.31, 81.38, 33.3, 90.18, 87.56, 62.92, 91.41, 90.98, 87.2, 69.23, 92.04, 91.95, 90.82, 86.95, 69.9, 91.91, 92.34, 92.01, 91.25, 30.55]
+acc_loss_list:
+[0.8917748917748918, 0.8695887445887446, 0.24015151515151525, 0.03528138528138534, 0.004004329004329054, 0.0017316017316018485, -0.0014069264069263576, -0.00043290043290034676, -0.00021645021645017338, 0.0, 0.00010822510822516359, 0.0, 0.00010822510822516359, 0.0, 0.0, 0.8917748917748918, 0.8661255411255411, 0.7018398268398269, 0.7096320346320346, 0.6695887445887446, 0.6623376623376623, 0.6168831168831169, 0.8892857142857142, 0.8559523809523809, 0.11926406926406936, 0.6396103896103896, 0.024025974025974013, 0.05238095238095242, 0.3190476190476191, 0.010714285714285812, 0.015367965367965385, 0.05627705627705631, 0.25075757575757573, 0.0038961038961038896, 0.0048701298701299, 0.017099567099567232, 0.05898268398268401, 0.2435064935064935, 0.005303030303030401, 0.000649350649350674, 0.0042207792207792265, 0.012445887445887507, 0.6693722943722944]
+
+
+MobileNetV2
+Full_acc: 88.930000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[12.831799711320127, 1.993271107564635, 0.35270115251888184, 0.07584820428274724, 0.017820575420393413, 0.00432754213409143, 0.0010585941112774115, 0.0002608063708472059, 5.7748640027636795e-05, 2.2079139706249858e-05, 1.8510216315203774e-05, 6.598093548300765e-06, 7.339584010540023e-06, 1.3159617416375918e-05, 6.234160995256087e-06, 12.57092879322021, 1.187412264130166, 0.583586694614265, 0.5873596916976219, 0.5892050999505944, 0.5843521934401908, 0.5859612625360875, 2.3705075655847287, 0.9287890646333746, 0.24106359886782278, 0.5542247166772163, 0.07309526703177732, 0.1553423866764422, 0.4234005047186028, 0.029852747687546522, 0.04043661294761285, 0.15534149384783863, 0.3700909930850026, 0.017542559489415382, 0.010349635478253204, 0.040430403892965844, 0.15534110238349375, 0.34610529077749097, 0.013684366045501894, 0.0025962286491194248, 0.010354257221157665, 0.040430568434830665, 0.1553424046493608]
+js_flops_wt_log_list:
+[22.734541938820872, 3.531547141465321, 0.6248927916741961, 0.13438287847722086, 0.031573327856063746, 0.007667255595699619, 0.0018755476831354906, 0.00046207963881377234, 0.0001023152564842449, 3.911837301995897e-05, 3.279518840552046e-05, 1.1690069815988018e-05, 1.3003794031628833e-05, 2.331534786874075e-05, 1.10452779647875e-05, 22.272347939414136, 2.10377924569088, 1.0339606666351948, 1.0406454156459632, 1.0439149890702146, 1.0353169272964935, 1.0381677704881038, 4.199909979777402, 1.6455676068258478, 0.42710068904432597, 0.9819390380379469, 0.12950540463916646, 0.27522546207267534, 0.7501532714036341, 0.052891142284026596, 0.07164294125557184, 0.2752238802174572, 0.6557029717388916, 0.031080757446247855, 0.01833680599178001, 0.07163194045938981, 0.2752231866465812, 0.6132066760274543, 0.024245063106243935, 0.004599827805456096, 0.018344994493023466, 0.07163223198388448, 0.2752254939159149]
+js_flops_wt_cbrt_list:
+[49.98799441445422, 7.76505457013427, 1.373994578993595, 0.29547683858309576, 0.06942243836545511, 0.016858519996726346, 0.004123895144266191, 0.0010160061489786882, 0.0002249675618453719, 8.601224591576993e-05, 7.210902683888234e-05, 2.5703757139190327e-05, 2.8592332547056773e-05, 5.1265052191067716e-05, 2.428600913516044e-05, 48.971736813007404, 4.6257235120829865, 2.2734401320955824, 2.288138347573666, 2.295327382589588, 2.2764222352997723, 2.2826905794750005, 9.234629717866756, 3.618222199504919, 0.9390955364665081, 2.1590566144180356, 0.28475240280473607, 0.6051570732248515, 1.6494133746709783, 0.11629537697231265, 0.1575262416527201, 0.6051535950914444, 1.4417390320433585, 0.06833939007000837, 0.04031839119357117, 0.1575020534487272, 0.6051520700896577, 1.3482995161571414, 0.05330928075852274, 0.010113956430939035, 0.04033639581207292, 0.15750269444359968, 0.605157143240763]
+js_param_list:
+[11.64306990393129, 1.8111948895845984, 0.3234632671815244, 0.07008450271665352, 0.016401300088192267, 0.003968574916230168, 0.000980312895789833, 0.0002411791224606334, 5.841659499907292e-05, 2.2969269213277244e-05, 2.1441167455345407e-05, 7.149071618997471e-06, 4.001053660470322e-06, 9.233829469391413e-06, 5.531670003141448e-06, 11.488326973180685, 1.0704022893653384, 0.5087476800631758, 0.5115176278659856, 0.5151777317759163, 0.5107807068971318, 0.511207269897487, 2.1723823401168723, 0.8517660834442466, 0.21703483248183933, 0.5087028498117648, 0.06586611649811078, 0.13765610714237508, 0.3893204918329101, 0.027210609441983842, 0.035735526223220127, 0.13765631650142093, 0.3405504207720941, 0.01614329098121896, 0.009145250003614254, 0.03572979707798319, 0.13765600942836523, 0.3186385336290934, 0.012647476530902785, 0.0023084517617242032, 0.009138672692023779, 0.03573015773675336, 0.13765851036927695]
+ptq_acc_list:
+[10.0, 11.35, 33.25, 79.93, 87.2, 88.7, 88.84, 88.96, 88.87, 88.89, 88.88, 88.94, 88.93, 88.94, 88.94, 10.0, 12.79, 14.81, 14.26, 14.87, 12.75, 13.63, 9.95, 10.56, 55.94, 20.36, 79.57, 71.76, 29.54, 84.64, 86.41, 72.23, 39.52, 85.82, 88.54, 86.71, 71.01, 46.36, 86.71, 88.78, 88.64, 86.77, 72.19]
+acc_loss_list:
+[0.8875520071966716, 0.8723715281682223, 0.6261104239289329, 0.1012031935229956, 0.019453502754975868, 0.0025863038344765992, 0.0010120319352299944, -0.0003373439784098383, 0.0006746879568199963, 0.00044979197121338413, 0.0005622399640167701, -0.00011244799280322617, 0.0, -0.00011244799280322617, -0.00011244799280322617, 0.8875520071966716, 0.856179017204543, 0.8334645226582705, 0.8396491622624536, 0.8327898347014505, 0.8566288091757562, 0.8467333858090633, 0.8881142471606882, 0.8812549195996852, 0.37096592825818064, 0.7710558866524233, 0.10525132126391558, 0.19307320364331498, 0.6678286292589678, 0.048240188912627976, 0.028336894186438887, 0.18778814798155855, 0.5556055324412459, 0.0349713257618353, 0.004385471719329816, 0.024963454402339064, 0.2015068031035646, 0.4786911053637693, 0.024963454402339064, 0.0016867198920499907, 0.0032609917912965955, 0.024288766445519066, 0.18823793995277194]
+
+
+ResNet_18
+Full_acc: 87.720000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[17.132639553663317, 3.308895598329574, 0.5969859721989359, 0.1296859127412463, 0.03001997911872445, 0.007394401042947096, 0.00181295555481838, 0.00046620035198360736, 0.00013848358136903994, 5.1418622719264556e-05, 1.1630965815506488e-05, 3.529150979065299e-05, 2.2553957490436983e-05, 1.6323174006368734e-05, 2.0846042808995603e-05, 15.86387656996309, 2.09194566490344, 1.0067098365080311, 1.0114686434065223, 1.0227339732911203, 1.0028485780921184, 1.0088029481993872, 2.980140440745267, 1.1406015989849505, 0.3655457247955589, 0.6706065397624208, 0.1160780841779645, 0.188962558981352, 0.5115645448947981, 0.052565932886952396, 0.048786617224968055, 0.18893002583131316, 0.4468136786569628, 0.03417551495071177, 0.012085716024163274, 0.04875248505850929, 0.18892985360382042, 0.4181034086772375, 0.02793278111985615, 0.0030658456267059504, 0.012060601315533916, 0.048753518217516126, 0.3693456183633073]
+js_flops_wt_log_list:
+[26.59087651408034, 5.135602951172363, 0.9265577681511328, 0.20128025693396, 0.046592794717997085, 0.011476550616308907, 0.00281381494851934, 0.0007235706997504785, 0.00021493476238017064, 7.980476347322088e-05, 1.8051951351156087e-05, 5.47745241414394e-05, 3.5005084689579684e-05, 2.5334537796214163e-05, 3.235429942975194e-05, 24.62168083237961, 3.246824207990701, 1.562473596917016, 1.569859548521016, 1.5873440111406518, 1.5564806939901266, 1.565722230867476, 4.625355375408584, 1.7702815830200722, 0.5673487262626922, 1.0408212717312657, 0.1801600969132216, 0.2932811407091893, 0.7939786277340253, 0.08158545715423707, 0.07571973425959604, 0.29323064732359116, 0.6934814286353692, 0.053042433713325385, 0.01875775074682243, 0.0756667591053228, 0.29323038001626955, 0.6489213804696711, 0.043353339170327236, 0.004758374926153147, 0.018718771223920627, 0.07566836262960662, 0.5732463872921068]
+js_flops_wt_cbrt_list:
+[56.38680309114762, 10.890210114312508, 1.9647953461649206, 0.4268212147072706, 0.09880150960194208, 0.024336392199209172, 0.005966784485388187, 0.0015343547832185934, 0.0004557760296978478, 0.00016922855030060107, 3.827973950800843e-05, 0.00011615112820892563, 7.422940031837991e-05, 5.372269670629064e-05, 6.860832549583026e-05, 52.21106074233106, 6.884994452537129, 3.3132751753341, 3.3289373216539087, 3.366013683178355, 3.3005670332349126, 3.320164007403356, 9.80821383009477, 3.7539386482721313, 1.2030810979415458, 2.2070947556434284, 0.3820352407017765, 0.6219120277121923, 1.683657043682318, 0.17300456813528742, 0.16056611535723694, 0.6218049548750251, 1.4705495226200787, 0.11247817512464464, 0.039776409673213976, 0.16045377985031425, 0.6218043880413104, 1.3760585170181832, 0.09193214062900647, 0.010090286036745609, 0.03969375234970352, 0.1604571801747825, 1.2155872765546443]
+js_param_list:
+[20.786070261472137, 3.8243340754773674, 0.6913034612539486, 0.15047848475908562, 0.03519389620659456, 0.00857323746402218, 0.002098806560194455, 0.0005585881712651304, 0.00017491804563323177, 7.238506060690497e-05, 2.3566719856301874e-05, 8.156965721694642e-05, 2.77128079482547e-05, 2.498257132773363e-05, 4.04262486266973e-05, 20.84375053153798, 2.4612026942605736, 1.4760387225423475, 1.4948973001010017, 1.4996030129654683, 1.4510430756203132, 1.4879770246827977, 3.9001480781366054, 1.5283178272873799, 0.42936798576685126, 0.9135918339152062, 0.13363340554224395, 0.238449785974722, 0.7010061595913828, 0.058970222119830114, 0.061078493213794556, 0.23841129362895874, 0.6143271316547569, 0.037240379174542565, 0.015461384466492747, 0.0610395868818998, 0.23841018837515324, 0.575452359230672, 0.0300949004306533, 0.0038842069111853735, 0.015426270994102766, 0.06104015135699442, 0.7022645451361362]
+ptq_acc_list:
+[10.0, 9.73, 53.99, 79.63, 86.56, 87.49, 87.76, 87.82, 87.68, 87.72, 87.72, 87.73, 87.72, 87.72, 87.72, 10.0, 13.51, 19.57, 21.67, 22.33, 21.76, 25.87, 9.88, 21.23, 50.37, 51.52, 80.87, 81.73, 68.14, 85.64, 86.37, 81.76, 72.22, 86.59, 87.58, 86.38, 81.32, 74.02, 86.32, 87.74, 87.57, 86.23, 75.94]
+acc_loss_list:
+[0.886000911992704, 0.889078887368901, 0.3845189238486092, 0.09222526219790246, 0.013223894208846291, 0.002621979024167852, -0.00045599635202925505, -0.0011399908800728946, 0.00045599635202909307, 0.0, 0.0, -0.00011399908800735427, 0.0, 0.0, 0.0, 0.886000911992704, 0.8459872321021431, 0.7769037847697219, 0.7529639762881897, 0.7454400364797081, 0.7519379844961239, 0.7050843593251254, 0.8873689010487916, 0.7579799361605106, 0.42578659370725036, 0.41267669858641126, 0.07808937528499765, 0.06828545371637021, 0.22321021431828544, 0.023711810305517538, 0.015389876880984888, 0.06794345645234831, 0.1766985864113087, 0.01288189694482439, 0.0015959872321021497, 0.015275877792977696, 0.07295941632466947, 0.15617875056999547, 0.015959872321021498, -0.00022799817601454653, 0.001709986320109504, 0.016985864113087036, 0.13429092567259462]
+
+
+ResNet_50
+Full_acc: 89.520000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[29.7677753915303, 4.062077691426077, 0.6963673542361583, 0.15067134586507205, 0.03532845168247791, 0.008582309848246072, 0.0020944201540720317, 0.0005308624271256868, 0.000129461777807359, 3.455474378865376e-05, 1.4683295087827954e-05, 1.4334196140826647e-05, 1.5821202706575886e-05, 9.923054396377083e-06, 1.5187357995324608e-05, 28.500446892296846, 2.5130823354184835, 1.6073710380844661, 1.5778466391534316, 1.6039559283183118, 1.5148962192809852, 1.5973463878725747, 5.306755183948775, 2.0870301990571187, 0.5241027793573747, 1.2451286157807306, 0.15415646961125287, 0.3729572825322485, 0.9516476438335548, 0.05960446109256321, 0.09652126955592964, 0.37292768706107793, 0.830958510676339, 0.032827149345821885, 0.02488237339593698, 0.09651782668580297, 0.37292798157640267, 0.7765255245892075, 0.024776426789531195, 0.00627665006472412, 0.024879365104048583, 0.09651755665839988, 0.5624834423951455]
+js_flops_wt_log_list:
+[57.39948090574831, 7.832669650988351, 1.3427649237180703, 0.2905308484182635, 0.068121811627046, 0.016548772079279368, 0.004038549339379836, 0.001023631337867791, 0.0002496336641061883, 6.662991541061519e-05, 2.8312949319923004e-05, 2.7639801996044727e-05, 3.0507110817574302e-05, 1.9134052305218882e-05, 2.92849046929262e-05, 54.955764603932934, 4.845831427748187, 3.0994006772583065, 3.0424704851157722, 3.092815518467126, 2.921086828596199, 3.080070723613601, 10.232709325642674, 4.024299716215401, 1.0105970997393612, 2.4009095495608266, 0.29725101112078084, 0.719151973427634, 1.835007152626073, 0.1149318375787476, 0.18611638578980005, 0.7190949061909995, 1.6022892721975617, 0.06329869489010496, 0.04797924258176694, 0.18610974710231004, 0.7190954740878791, 1.4973292909945275, 0.04777495186365207, 0.012102901570693252, 0.04797344186637804, 0.1861092264240745, 1.0846043141250659]
+js_flops_wt_cbrt_list:
+[130.76568208855215, 17.84414028356169, 3.0590445830493884, 0.6618781905634846, 0.15519295683432627, 0.03770088918100717, 0.009200495382169712, 0.002332004540655938, 0.0005687075186744611, 0.00015179416605663184, 6.450166571780904e-05, 6.296812277344748e-05, 6.95003350494001e-05, 4.359059282989802e-05, 6.671594370960997e-05, 125.19848489451272, 11.039619904858322, 7.060956601557251, 6.931260038287369, 7.0459545010608355, 6.654727630873298, 7.016919712491294, 23.31183476691597, 9.168032341333824, 2.3023103515714154, 5.469676204855689, 0.6771878527010864, 1.638349282025914, 4.180455261338257, 0.2618340775477382, 0.4240044640059107, 1.6382192732522165, 3.6502847460611965, 0.14420508481969202, 0.10930479306248467, 0.4239893399580255, 1.6382205670169274, 3.4111682363395577, 0.10883938440935563, 0.027572447593055105, 0.1092915780561386, 0.42398815376533616, 2.4709112468392886]
+js_param_list:
+[30.69524926956487, 3.8836694182220084, 0.6824733904919843, 0.14882558983807073, 0.03484222157902474, 0.008430327210360945, 0.002082452633129232, 0.0005276637670812718, 0.00012606606751967495, 3.234656861447193e-05, 1.857721120485012e-05, 1.9004037739922423e-05, 2.0749932611249685e-05, 1.6430572176630244e-05, 2.4097736501868746e-05, 30.273973315857234, 2.4709641293182507, 1.80897656364437, 1.750042804512283, 1.792667065720219, 1.636262254419038, 1.7987184857704557, 5.526061234093648, 2.1433093163903214, 0.5228266005917082, 1.2685707061461515, 0.15466299502148095, 0.3789765355265202, 0.9649675859701402, 0.058633262301952184, 0.10005960600695782, 0.37895863929590123, 0.8408741213489173, 0.03159662988857441, 0.025931983759404996, 0.10005216395970003, 0.37895872370755523, 0.7851965965469262, 0.023526393725487368, 0.006566531891578675, 0.0259316744717014, 0.10005129655776644, 0.7550371985096369]
+ptq_acc_list:
+[10.0, 9.84, 29.25, 81.68, 88.0, 89.3, 89.64, 89.53, 89.52, 89.52, 89.5, 89.53, 89.53, 89.53, 89.52, 10.0, 10.81, 10.06, 10.03, 9.95, 18.85, 10.33, 10.0, 10.53, 58.89, 23.38, 83.91, 79.27, 43.52, 86.35, 87.73, 79.28, 50.28, 88.15, 89.32, 87.35, 79.99, 50.99, 88.3, 89.55, 89.08, 87.32, 55.17]
+acc_loss_list:
+[0.8882931188561215, 0.8900804289544235, 0.6732573726541555, 0.0875781948168006, 0.01697944593386948, 0.002457551385165314, -0.0013404825737265923, -0.00011170688114393561, 0.0, 0.0, 0.0002234137622877125, -0.00011170688114393561, -0.00011170688114393561, -0.00011170688114393561, 0.0, 0.8882931188561215, 0.8792448614834674, 0.8876228775692583, 0.8879579982126898, 0.8888516532618409, 0.789432529043789, 0.8846067917783735, 0.8882931188561215, 0.8823726541554959, 0.3421581769436997, 0.7388293118856122, 0.06266756032171582, 0.11449955317247543, 0.5138516532618409, 0.03541108132260949, 0.019995531724754156, 0.11438784629133149, 0.43833780160857905, 0.015303842716711242, 0.0022341376228776012, 0.024240393208221647, 0.10645665773011619, 0.43040661304736366, 0.01362823949955316, -0.0003351206434316481, 0.004915102770330628, 0.024575513851653293, 0.38371313672922247]
+
+
+ResNet_152
+Full_acc: 88.920000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[28.686310703302077, 3.3932087208092927, 0.6061009590823148, 0.1319442589971631, 0.03085864771272258, 0.007477060772068394, 0.0018380342604652654, 0.00046042016469972056, 0.00011334558121989824, 3.720126570205286e-05, 1.650623354577907e-05, 1.3288177717338037e-05, 1.2759841369322621e-05, 1.819689317249517e-05, 1.4257175535155987e-05, 28.4749996590581, 2.810808173571048, 2.342710816108272, 2.300945594626436, 2.3248148944700535, 2.322196853397037, 2.382205501867068, 4.903655309128148, 1.8918837213720505, 0.5068719460450458, 1.11117095127199, 0.15390591395741046, 0.3892249568043773, 0.8422381938332751, 0.054463263181182754, 0.10930544224205266, 0.38919825259307567, 0.7330977874276017, 0.02741513107343332, 0.02774185047542788, 0.10929008681094653, 0.3891977646490629, 0.6842361988504659, 0.01957097252956391, 0.006951054299062933, 0.027736067483432606, 0.10928992835429206, 1.1241558566258438]
+js_flops_wt_log_list:
+[68.2805151621134, 8.076676080997915, 1.4426702044201056, 0.31405997342044284, 0.07345121458188518, 0.017797254122207198, 0.004374976185966498, 0.0010959138789883925, 0.00026979051984377166, 8.854821427163992e-05, 3.92889187302227e-05, 3.162914985791866e-05, 3.037157866326484e-05, 4.331310683408624e-05, 3.393560435030332e-05, 67.77754260807347, 6.690418719170394, 5.576231222417695, 5.476819664475138, 5.533634502281853, 5.527402916943814, 5.670238343712172, 11.671912576968667, 4.503151222831526, 1.2064806086477613, 2.6448617171706768, 0.36633414454017693, 0.9264516737418941, 2.004735232736783, 0.12963603810485305, 0.2601739897299123, 0.9263881111135892, 1.744954069119811, 0.06525479321101454, 0.06603246620327835, 0.2601374400056446, 0.9263869496861437, 1.6286514021720526, 0.0465837555886324, 0.01654522860638216, 0.06601870125188046, 0.260137062839612, 2.6757690037877166]
+js_flops_wt_cbrt_list:
+[178.27502958706097, 21.087563031509564, 3.7666979044714792, 0.8199857736970838, 0.19177531718531698, 0.0464672241806022, 0.01142271711254798, 0.0028613445393266897, 0.0007044017285643495, 0.0002311923904158988, 0.00010258026220869086, 8.258121095522658e-05, 7.929779193879914e-05, 0.00011308709935801336, 8.86031813796565e-05, 176.9618079931691, 17.46815460132523, 14.559099089990498, 14.299543367667258, 14.447882419692986, 14.43161224279295, 14.804544255282197, 30.474441428222423, 11.757372005458254, 3.150025533524954, 6.905524947531651, 0.9564695038049168, 2.4188921122692686, 5.234205279234327, 0.3384694516996775, 0.6792937283317385, 2.4187261552694603, 4.555937188842502, 0.17037510866417857, 0.17240555139554528, 0.6791982998898213, 2.4187231228741757, 4.252280115635909, 0.12162650481067067, 0.043198284493379195, 0.1723696121951468, 0.6791973151391137, 6.986221430606312]
+js_param_list:
+[29.170052782219717, 3.303806319397033, 0.5959659622681815, 0.12970443743012838, 0.03039701895628507, 0.007358429601290915, 0.0018106872688447033, 0.00045290392546223007, 0.00011216616155640409, 3.701588742186668e-05, 1.912862706028629e-05, 1.460306354877511e-05, 1.3361649062109465e-05, 2.1714641450158115e-05, 1.5708731184005286e-05, 29.45276476186047, 2.917225716451529, 2.5405618880115255, 2.4932447850064565, 2.5226851356723863, 2.517664913087458, 2.5850642529566183, 5.038639900459697, 1.9410390226978973, 0.5058390817667794, 1.14034915320807, 0.15266149239541862, 0.39424332023458886, 0.8643002355414382, 0.05346738233877521, 0.11041236732489446, 0.3942130421288897, 0.7521360232175813, 0.026447411864721156, 0.028189150987024562, 0.11039354223296356, 0.39421289139832993, 0.7019246938177399, 0.018641393353090733, 0.007078942183055831, 0.028183960437794454, 0.11039351593506737, 1.3402905976736923]
+ptq_acc_list:
+[10.0, 9.69, 24.61, 83.49, 87.96, 88.81, 88.87, 88.9, 88.9, 88.91, 88.92, 88.89, 88.91, 88.91, 88.94, 10.0, 10.2, 10.72, 10.0, 10.05, 10.03, 10.04, 10.18, 10.37, 15.46, 10.0, 32.64, 20.77, 10.02, 54.09, 37.21, 17.35, 10.04, 37.63, 43.98, 33.22, 25.65, 10.05, 40.35, 41.28, 44.64, 28.09, 23.51]
+acc_loss_list:
+[0.8875393612235718, 0.8910256410256411, 0.7232343679712101, 0.061066126855600616, 0.010796221322537202, 0.0012370670265407044, 0.0005623031938821093, 0.00022492127755281176, 0.00022492127755281176, 0.00011246063877648578, 0.0, 0.00033738191632929753, 0.00011246063877648578, 0.00011246063877648578, -0.00022492127755281176, 0.8875393612235718, 0.8852901484480431, 0.8794421952316689, 0.8875393612235718, 0.8869770580296896, 0.8872019793072424, 0.887089518668466, 0.8855150697255961, 0.8833783175888439, 0.826135852451642, 0.8875393612235718, 0.6329284750337382, 0.7664192532613586, 0.887314439946019, 0.39170040485829954, 0.5815339631129105, 0.804880791722897, 0.887089518668466, 0.5768106162843005, 0.5053981106612686, 0.6264057579847053, 0.7115384615384616, 0.8869770580296896, 0.546221322537112, 0.5357624831309041, 0.4979757085020243, 0.684098065677013, 0.7356050382366172]
+
+

ykl/ALL-cifar10/ptq_result_sgd_mom9.txt

+AlexNet
+Full_acc: 88.910000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[34.08698085638661, 9.529156204897387, 1.8124827750519978, 0.38184765825259287, 0.08884375060304205, 0.021438695299264742, 0.005264174533104677, 0.0013424482298871076, 0.00032480434874334214, 6.424055987152854e-05, 5.101576475215808e-06, 2.165563416065552e-07, 9.595217569001947e-06, 2.5811229939423754e-05, 1.8684019643020523e-05, 33.86696275210702, 5.422747233995497, 1.471489918367148, 1.4705240648560032, 1.4703524023146604, 1.4710983025028217, 1.470734617369435, 6.605847076675886, 2.413660453619744, 0.9884972330329851, 1.382904855470519, 0.3296553670623794, 0.4012132051362278, 1.0395468697425083, 0.16294763611696944, 0.10459875773286055, 0.4010959237640444, 0.9063474823327556, 0.11075149980082473, 0.027116228352962648, 0.1045984295584822, 0.4010924809671434, 0.8450929088040069, 0.09248655883280817, 0.006667217057420956, 0.027067618290722385, 0.10459837134590665, 0.40109353455222907]
+js_flops_wt_log_list:
+[62.91072899988264, 17.586953978986045, 3.3451074226448423, 0.7047357655046707, 0.16396949735939811, 0.039567128451929244, 0.009715529188563578, 0.0024776144634990876, 0.0005994569729676505, 0.00011856199497120492, 9.415439180626215e-06, 3.9967509523400443e-07, 1.770887643941322e-05, 4.763705236066707e-05, 3.4483115455216915e-05, 62.50466489606197, 10.008190021582257, 2.715773035766249, 2.7139904622743574, 2.7136736429097046, 2.7150502718577503, 2.7143790567400434, 12.19171205001454, 4.454637368300557, 1.8243646102444553, 2.552280970898227, 0.6084099835064305, 0.7404767035790562, 1.9185815159336415, 0.3007351874346324, 0.19304684475226716, 0.7402602497765546, 1.672749519266348, 0.2044023088948578, 0.050045549665996475, 0.1930462390755876, 0.7402538957710838, 1.5596984429182799, 0.1706926425480815, 0.012304976121234205, 0.04995583522442863, 0.19304613163880463, 0.7402558402614475]
+js_flops_wt_cbrt_list:
+[140.53566956065907, 39.287326538115806, 7.472603145237761, 1.5743024161860733, 0.36628987559452253, 0.08838862588279423, 0.021703426766099886, 0.005534719007018134, 0.0013391211389232483, 0.0002648544948152939, 2.1033058597356846e-05, 8.928303328917993e-07, 3.955968794423856e-05, 0.00010641595091697333, 7.703149837995773e-05, 139.62856688322717, 22.3571989727856, 6.066739140106062, 6.062757066408831, 6.062049326691789, 6.06512456479546, 6.063625143830456, 27.23494773113733, 9.951171368642365, 4.075430472661565, 5.701515796340247, 1.3591211826463696, 1.654143752359736, 4.285900707794081, 0.671809428987148, 0.431245480939874, 1.6536602183020783, 3.7367389860925844, 0.45661234255188954, 0.11179626977228563, 0.43124412792459266, 1.6536460241506608, 3.484195279133497, 0.3813086446608754, 0.02748796724528175, 0.1115958575482432, 0.4312438879226014, 1.6536503679288932]
+js_param_list:
+[9.961239275836823, 2.705316687985323, 0.5131537184726842, 0.10722989038659306, 0.024873288631771857, 0.006035844170212003, 0.001484872148965179, 0.00037341058356369315, 9.14765900298704e-05, 1.5320182111956624e-05, 1.409228414943442e-05, 2.84718938011963e-07, 2.653078695479039e-06, 1.6908174716258054e-05, 5.9373453871407475e-06, 9.91127328741361, 1.5057648843708138, 0.4077517570388736, 0.4074022563199942, 0.40747726648881144, 0.4074480170569041, 0.4074052074141073, 1.8972020452892577, 0.7006123912714691, 0.27415786727356534, 0.40446242879491423, 0.0909882520546512, 0.11314812323988674, 0.3061930232966958, 0.044502158126589864, 0.0294023563704407, 0.1131398216596534, 0.26741498515138346, 0.03030842278832954, 0.0074250792085407795, 0.02935361540861882, 0.11313883037981877, 0.25029743385359177, 0.025288839691646737, 0.0018812273808870542, 0.007421758626773649, 0.029354114068612724, 0.11313930294887717]
+ptq_acc_list:
+[10.0, 10.02, 38.71, 78.22, 86.89, 88.17, 88.74, 88.83, 88.92, 88.97, 88.87, 88.93, 88.93, 88.91, 88.91, 10.0, 11.77, 68.79, 69.06, 69.54, 69.21, 68.97, 23.13, 69.03, 57.91, 80.51, 82.29, 84.44, 84.02, 86.11, 87.69, 84.13, 85.05, 87.07, 88.72, 87.97, 84.41, 84.42, 87.62, 88.9, 88.68, 87.83, 84.42]
+acc_loss_list:
+[0.8875267124058036, 0.8873017658306153, 0.5646159037228657, 0.1202339444381959, 0.022719604094027623, 0.008323023281970475, 0.0019120458891013577, 0.0008997863007535519, -0.00011247328759425392, -0.0006748397255652038, 0.00044989315037669605, -0.00022494657518850784, -0.00022494657518850784, 0.0, 0.0, 0.8875267124058036, 0.8676189405016309, 0.22629625463952302, 0.22325947587447975, 0.21786075806995828, 0.2215723765605669, 0.22427173546282755, 0.7398492857946238, 0.22359689573726235, 0.34866719154200876, 0.09447756157912486, 0.07445731638735789, 0.05027555955460577, 0.054999437633562034, 0.031492520526374954, 0.013721741086491946, 0.05376223147002589, 0.043414689011359794, 0.020695084917332175, 0.0021369924642897055, 0.010572489033854434, 0.05061297941738837, 0.05050050612979412, 0.014509054099651244, 0.0001124732875940941, 0.0025868856146664017, 0.01214711506017319, 0.05050050612979412]
+
+
+AlexNet_BN
+Full_acc: 90.150000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[25.285092945372888, 5.60860519311973, 1.0204605089615462, 0.22066907725177062, 0.052055574667772185, 0.012646543777735535, 0.003044880729065153, 0.0007528694549592104, 0.00013611946336894223, 6.784174317417345e-05, 3.840394103950117e-05, 3.912801899393803e-06, 1.3534021294374837e-05, 6.177665369762054e-07, 3.332203311372433e-06, 22.82241801378766, 3.399609952373232, 1.1478114768670011, 1.2281287194799113, 1.1485961742680921, 1.1878158092494089, 1.151043949191644, 4.3401284894116605, 1.6335118078373596, 0.5899444822750272, 0.9540445465065329, 0.19274530418147298, 0.26494500130849585, 0.722946910114741, 0.09137445900594629, 0.06898014983845337, 0.2649354598770586, 0.6314795689015886, 0.06118844438163122, 0.017030771111870294, 0.0689780205143233, 0.2649335763726143, 0.5909410612118904, 0.05083688870236846, 0.004316473917450096, 0.017101115292008588, 0.06897847434498293, 0.2926378562616656]
+js_flops_wt_log_list:
+[46.69418793686359, 10.357457080225085, 1.8844927677561758, 0.40751138970735123, 0.09613145547654398, 0.023354475824357995, 0.0056230061449831755, 0.001390330179831221, 0.00025137292625920674, 0.00012528390196481324, 7.092087200505446e-05, 7.225803268539919e-06, 2.499338781258859e-05, 1.1408345162500615e-06, 6.153607107603573e-06, 42.146345999668476, 6.278087502826325, 2.1196728417296433, 2.267995350539096, 2.1211219488379016, 2.1935491695164764, 2.1256422748084014, 8.014950777225046, 3.0166196152425737, 1.0894553002901595, 1.7618418667062121, 0.35594432960329037, 0.48927610077444506, 1.335071971544802, 0.16874196075414, 0.12738620686222088, 0.48925848053497506, 1.1661584844589756, 0.11299720067048692, 0.031450864298796476, 0.12738227462193513, 0.4892550022519135, 1.0912956907634057, 0.09388089813718513, 0.007971267685724231, 0.03158076947156916, 0.12738311271472597, 0.5404167225030374]
+js_flops_wt_cbrt_list:
+[104.33593715745599, 23.143244133393935, 4.21080926080445, 0.9105647753194368, 0.21480115493211369, 0.05218446298392169, 0.012564339197251287, 0.0031066265134994104, 0.0005616808214488905, 0.0002799409069907393, 0.00015846930788638032, 1.6145712968772547e-05, 5.584653369905735e-05, 2.549140346020509e-06, 1.374994175589311e-05, 94.17400113229542, 14.028087265369734, 4.7363079256909595, 5.067727501487072, 4.73954588644929, 4.901381058643025, 4.749646339358483, 17.909025460371364, 6.740492736043911, 2.43433593706837, 3.9367516688498077, 0.7953406375812051, 1.0932641246930426, 2.9831546810948937, 0.3770458678256741, 0.2846384070727263, 1.0932247531078567, 2.6057255458572692, 0.252486858622865, 0.07027545709679854, 0.2846296206692473, 1.0932169810501562, 2.438448201854331, 0.20977239183538537, 0.017811417674669316, 0.07056572401312582, 0.2846314933476387, 1.2075354062084704]
+js_param_list:
+[8.360216678687635, 1.8570109677817088, 0.3333807431540638, 0.0725281408182254, 0.017012654076048763, 0.004110735113433596, 0.0010220934798935851, 0.00024139679695401383, 5.199705611840244e-05, 1.7795754961684394e-05, 1.156257793403137e-05, 7.149903936472302e-06, 8.418665385323183e-06, 5.16164228426419e-06, 2.0073634046824657e-06, 7.694125822174103, 1.0882545841027151, 0.362805687759871, 0.38593277658512304, 0.36282039813499795, 0.374535373080802, 0.3639321077001404, 1.4448951770175773, 0.5443081315109297, 0.19089445786617618, 0.3183779568451513, 0.062180055212836115, 0.08661770572930828, 0.24215678408107494, 0.02961121954891416, 0.02232756441454233, 0.08661743556154042, 0.2117682760759882, 0.019791635633312356, 0.005586465366161675, 0.02231621540081869, 0.0866164227122127, 0.1983056267930965, 0.01638151171031815, 0.0014116120198250856, 0.005639933403755662, 0.022316637482142368, 0.09446733495030342]
+ptq_acc_list:
+[10.0, 11.65, 42.95, 80.46, 87.53, 89.88, 89.89, 90.09, 90.12, 90.14, 90.15, 90.14, 90.15, 90.15, 90.15, 10.0, 18.93, 58.93, 53.07, 59.04, 56.75, 55.55, 15.18, 57.36, 72.86, 73.97, 85.88, 86.28, 79.63, 87.99, 88.98, 86.45, 82.47, 88.68, 89.94, 89.09, 86.37, 82.58, 88.64, 89.84, 89.87, 88.98, 86.56]
+acc_loss_list:
+[0.8890737659456461, 0.8707709373266777, 0.5235718247365502, 0.10748752079866901, 0.02906267332224076, 0.0029950083194676676, 0.002884082085413257, 0.0006655574043261483, 0.00033277870216307415, 0.00011092623405441059, 0.0, 0.00011092623405441059, 0.0, 0.0, 0.0, 0.8890737659456461, 0.7900166389351081, 0.3463117027176928, 0.41131447587354414, 0.3450915141430949, 0.37049362174154193, 0.3838047698280644, 0.8316139767054908, 0.3637271214642263, 0.19179145867997788, 0.1794786466999446, 0.047365501941209205, 0.04292845257903499, 0.11669439822518037, 0.02396006655574055, 0.01297836938435942, 0.04104270660011095, 0.08519134775374383, 0.016306156405990004, 0.0023294509151415193, 0.011758180809761534, 0.04193011647254577, 0.08397115917914594, 0.01674986134220749, 0.0034387132556849944, 0.0031059345535219204, 0.01297836938435942, 0.03982251802551307]
+
+
+VGG_16
+Full_acc: 92.140000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[13.641211824971252, 3.169712474264897, 0.5755907792663381, 0.1260512239343909, 0.028980261751639622, 0.007135236122326657, 0.001744593998903511, 0.0004684411944800341, 0.00014830649614421966, 2.519718073686583e-05, 2.1314622597865098e-05, 3.890285764919899e-05, 2.5651149605125267e-05, 2.8391795393574226e-05, 1.774567592027045e-05, 14.60086144348995, 2.4601868383791516, 1.9075639147595784, 1.8948428468226037, 1.944390173591926, 1.9011108764232212, 1.8892560675864127, 2.4221807979274215, 0.9145159521397388, 0.334387431561167, 0.5352458137349727, 0.10819431484357288, 0.1527237718128384, 0.40661633535055164, 0.05148536027950881, 0.03907713476530778, 0.15266662587828553, 0.3550581403765451, 0.03434281252675792, 0.009794518150671592, 0.039085618306572124, 0.15266766479118357, 0.3326012484411065, 0.02847960761924577, 0.0024560402837546383, 0.009785229043448966, 0.03908577597995974, 1.4117296982000362]
+js_flops_wt_log_list:
+[34.4155968688634, 7.996910252857111, 1.452165722139351, 0.3180163290046971, 0.07311469233054536, 0.01800158322449214, 0.004401459675026202, 0.0011818365928811217, 0.00037416445472899655, 6.35703063333739e-05, 5.3774948160929765e-05, 9.81485430385825e-05, 6.471563049969399e-05, 7.163004263740056e-05, 4.4770804564411296e-05, 36.836709804434115, 6.206838478730163, 4.812618668656655, 4.78052451518424, 4.905528132597565, 4.796338211406013, 4.766429554673758, 6.110952527866615, 2.307244601346285, 0.8436305505906201, 1.3503788657199003, 0.2729648927775619, 0.3853088589520533, 1.025857823870325, 0.12989310823428268, 0.09858822911987981, 0.3851646847048163, 0.8957809600052332, 0.08664394383937686, 0.024710721637004517, 0.09860963236028984, 0.3851673057921463, 0.8391241651622052, 0.07185158528314536, 0.006196377080272093, 0.02468728602339867, 0.09861003015660111, 3.5616718517716466]
+js_flops_wt_cbrt_list:
+[94.58539853190959, 21.978143984326138, 3.99102982541023, 0.8740136436738066, 0.20094326241016836, 0.04947428138415894, 0.012096661262938358, 0.0032480763173510186, 0.0010283271913565269, 0.00017471214525928178, 0.00014779127388710314, 0.00026974453164371916, 0.00017785987339929908, 0.00019686295593047962, 0.00012304492083792042, 101.23941452280762, 17.058437003747727, 13.226661635173645, 13.138456327902656, 13.482007451424408, 13.181917574996476, 13.099718732786831, 16.794910820819744, 6.341068294136561, 2.3185746899976225, 3.7112860099410803, 0.7501974545716584, 1.058955685723631, 2.819395272369621, 0.35698905515724233, 0.2709529338512097, 1.0585594473935243, 2.461901196199993, 0.23812610281468893, 0.06791320409037391, 0.27101175700722735, 1.0585666510050336, 2.306189658196223, 0.19747182810896982, 0.017029685634241395, 0.06784879530324187, 0.27101285028254607, 9.788647653659925]
+js_param_list:
+[8.009374619790865, 1.321144393616359, 0.240979567038955, 0.05221474460419919, 0.012251316426940318, 0.0029463550153740335, 0.0007292732627541021, 0.0001987700560922061, 0.00010187036981475034, 6.2562519844568955e-06, 1.3131870914994346e-05, 1.6785325208226396e-05, 1.3474710531836242e-05, 1.9863718775907645e-05, 7.25516235615164e-06, 8.987281444571327, 1.2089481377532683, 1.0191620116037927, 1.0362185795943388, 1.0155824900700519, 1.0312461080540078, 1.0028005708398315, 1.5016587541191626, 0.5860505516145746, 0.1608998314630143, 0.34912362397775093, 0.04942858276119339, 0.09972931830649734, 0.26741277979249006, 0.02046316582204935, 0.026154169438368347, 0.09983267280518633, 0.23414609672219786, 0.012321641145594106, 0.006541855309547202, 0.026144727744760975, 0.09983132347641265, 0.2193265994823164, 0.009723986855427426, 0.001665293169888134, 0.006537799295047079, 0.026145442233956713, 0.6492463233003857]
+ptq_acc_list:
+[10.0, 11.84, 61.06, 89.22, 91.66, 91.86, 92.16, 92.1, 92.11, 92.14, 92.12, 92.14, 92.14, 92.14, 92.14, 10.0, 20.11, 73.81, 74.28, 73.59, 72.66, 78.11, 13.93, 54.63, 79.28, 79.79, 90.12, 89.07, 85.12, 91.22, 91.5, 89.25, 87.24, 91.47, 92.03, 91.47, 89.03, 87.52, 91.56, 92.05, 92.06, 91.48, 81.17]
+acc_loss_list:
+[0.8914695029303235, 0.8714998914695029, 0.3373127848925548, 0.03169090514434558, 0.005209463859344519, 0.0030388539179509563, -0.00021706099413930998, 0.0004341219882787742, 0.00032559149120904205, 0.0, 0.00021706099413930998, 0.0, 0.0, 0.0, 0.0, 0.8914695029303235, 0.7817451703928804, 0.19893640112871716, 0.19383546776644237, 0.20132407206425001, 0.21141740829173003, 0.15226828738875625, 0.8488170175819406, 0.4070978945083568, 0.13957021923160406, 0.13403516388105052, 0.021923160408074627, 0.03331886260039079, 0.07618840894291291, 0.009984805730410263, 0.006945951812459307, 0.03136531365313654, 0.05317994356414159, 0.007271543303668349, 0.0011938354677664361, 0.007271543303668349, 0.03375298458866941, 0.05014108964619063, 0.006294768830041223, 0.0009767744736271261, 0.0008682439765573941, 0.007163012806598617, 0.1190579552854352]
+
+
+VGG_19
+Full_acc: 92.270000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[13.300130641135755, 2.766255624757825, 0.500419521326824, 0.10889609318331517, 0.02531930023370591, 0.006178066906245539, 0.001520363870459331, 0.0003820634902518314, 0.00011398771334963874, 4.7228083667157684e-05, 1.411225743952449e-05, 3.196493566071779e-05, 3.113180789123023e-05, 3.224161726796637e-05, 3.4888009154587396e-05, 14.240838472772179, 2.0696096124329135, 1.43356526884602, 1.4712603991762299, 1.4565978024341697, 1.4845232152142436, 1.4679831577711817, 2.443270091040176, 0.940647908218263, 0.29665481811870226, 0.5580085033609947, 0.0953604189031691, 0.1462424640710768, 0.4278998386206458, 0.04389343963971811, 0.0378407174693863, 0.14623836025786988, 0.3751251987139573, 0.028966056335861693, 0.009396054005880386, 0.03787723802057925, 0.14623816003116025, 0.3513479572255884, 0.023785332330103538, 0.002338465396360128, 0.009383665608766157, 0.03787655113895168, 0.960121119124881]
+js_flops_wt_log_list:
+[34.86751034200901, 7.251992420778777, 1.3118955975695772, 0.28548107967677694, 0.06637686400017076, 0.01619636810791092, 0.003985773103077633, 0.0010016144244822396, 0.0002988292281722347, 0.00012381274591429626, 3.69965751088062e-05, 8.379900579958175e-05, 8.161488506403222e-05, 8.452435196816343e-05, 9.146211062373463e-05, 37.333662061372195, 5.425671108991726, 3.7582226209745797, 3.857043856802661, 3.818604516818792, 3.891813543495762, 3.8484522683689084, 6.405256269000815, 2.4659946246354063, 0.7777077697845356, 1.462870387279669, 0.24999607011684405, 0.3833880106912782, 1.1217786088735207, 0.11507067124970191, 0.09920290584455474, 0.3833772521691783, 0.9834250578902705, 0.07593716904810999, 0.02463261595422441, 0.09929864781362063, 0.38337672725652466, 0.9210908420942255, 0.06235542667851463, 0.006130501165141504, 0.024600138636808797, 0.0992968470905376, 2.517045430149948]
+js_flops_wt_cbrt_list:
+[99.47633965014415, 20.689795575123206, 3.7428130305055993, 0.8144720562799047, 0.18937192255556848, 0.046207928217351375, 0.011371334376359564, 0.0028575867823930788, 0.0008525540684571532, 0.0003532353943478569, 0.00010555052068003838, 0.00023907695965376134, 0.0002328457049987744, 0.0002411463583897768, 0.00026093965104712015, 106.51222330394745, 15.479335827937323, 10.722137206180042, 11.00407230058331, 10.894405599328884, 11.10326955123544, 10.979560663222713, 18.27407354042323, 7.035435465551714, 2.2187853820590564, 4.173541215927996, 0.7132340031798076, 1.0937986565495985, 3.200412900548347, 0.328294422650937, 0.2830239916551954, 1.093767962691963, 2.8056928676462123, 0.2166472898754421, 0.07027638185516635, 0.28329714166046255, 1.0937664651262087, 2.627854543041718, 0.17789883884620886, 0.017490202487858004, 0.07018372469018751, 0.28329200423197204, 7.18108243658479]
+js_param_list:
+[8.3387907115147, 1.2190752591059653, 0.22485935766170984, 0.0486448508444762, 0.011336327947276856, 0.0027981550393282005, 0.0006956795596520919, 0.00018611873168182226, 4.8133517648193625e-05, 3.430532808847619e-05, 8.674319196086524e-06, 2.128742393333305e-05, 2.2659909379671383e-05, 2.083387209285803e-05, 2.9691968975044337e-05, 9.139225010632934, 1.0828044205957803, 0.8980029317192395, 0.8927713498976024, 0.8863165983950725, 0.8923838108497576, 0.8903497655990152, 1.6296236542404483, 0.6480972104618413, 0.15922780489786742, 0.38964452977132497, 0.04625922091891728, 0.10715956636657878, 0.2992250684701924, 0.01813946543607426, 0.02670481779878899, 0.10720786355946778, 0.26199597771698346, 0.010695520549743303, 0.006529405363552598, 0.026711195481184686, 0.10720810961983916, 0.24502473452465054, 0.00828989678757824, 0.001627246400466536, 0.00655433396965262, 0.026709219886149528, 0.5227248410063146]
+ptq_acc_list:
+[10.0, 10.48, 48.35, 89.0, 91.56, 92.0, 92.21, 92.21, 92.26, 92.27, 92.24, 92.28, 92.27, 92.27, 92.28, 10.0, 20.98, 77.86, 79.16, 76.42, 78.8, 77.53, 11.45, 54.5, 81.99, 82.08, 89.7, 89.57, 85.4, 91.33, 91.52, 89.52, 87.06, 91.24, 92.03, 91.53, 89.52, 87.21, 91.41, 92.21, 92.05, 91.58, 84.82]
+acc_loss_list:
+[0.8916224124850981, 0.8864202882843828, 0.4759943643654492, 0.03543947111737288, 0.007694808713557969, 0.0029261948629023087, 0.0006502655250894361, 0.0006502655250894361, 0.00010837758751480335, 0.0, 0.0003251327625447181, -0.00010837758751495736, 0.0, 0.0, -0.00010837758751495736, 0.8916224124850981, 0.7726238213937358, 0.15617210360897363, 0.14208301723203642, 0.1717784762111195, 0.14598461038257288, 0.15974856399696538, 0.8759076622954373, 0.4093421480437845, 0.11141215996531918, 0.11043676167768504, 0.02785303999132972, 0.02926194862902355, 0.07445540262273752, 0.010187493226400756, 0.008128319063617644, 0.02980383656659803, 0.05646472309526383, 0.01116289151403491, 0.0026010621003575906, 0.008019941476102688, 0.02980383656659803, 0.054839059282540396, 0.00932047252628156, 0.0006502655250894361, 0.00238430692532783, 0.007478053538528208, 0.08074130269860197]
+
+
+Inception_BN
+Full_acc: 92.780000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[24.848217490903977, 3.2708952819836266, 0.5567268020661914, 0.12104854006868221, 0.028348168357583057, 0.006883730500960081, 0.0017000974081696713, 0.0004051360296128354, 0.00010574818135927088, 3.926831057021473e-05, 1.4112278110129494e-05, 2.691068416781206e-05, 2.625966576984522e-05, 1.8099734553551073e-05, 1.1095893727718466e-05, 22.95526257408921, 2.6051774030951287, 2.385504720387069, 2.4016519290510616, 2.411961486072148, 2.383570724888951, 2.462496521502844, 4.246467237196871, 1.7071033702313791, 0.38276944131367524, 1.034152021323611, 0.11239825980700917, 0.25956483202623115, 0.7966014992207354, 0.044649091844730746, 0.06576091078258045, 0.2595283601253564, 0.6982188687000102, 0.02560519898553945, 0.016714138298419872, 0.06572526025523454, 0.25952840381589637, 0.653743804723548, 0.019767112613876217, 0.0042080288206791175, 0.016722842439297152, 0.06572532556111359, 1.7745617389559578]
+js_flops_wt_log_list:
+[78.88446360187883, 10.383956914078336, 1.7674143092902868, 0.38428708846435433, 0.08999559247267117, 0.02185345441164575, 0.005397219022390729, 0.0012861662368136633, 0.0003357137616190757, 0.00012466325268672965, 4.4801583426481904e-05, 8.543207924329056e-05, 8.336532185365617e-05, 5.746037324874225e-05, 3.522561025614557e-05, 72.87498894680007, 8.270533775958933, 7.573149275450822, 7.62441105688818, 7.657140321105184, 7.567009511198784, 7.817571513584551, 13.481059167799524, 5.419460519570768, 1.2151600843108055, 3.2830736254934476, 0.3568254518822619, 0.8240282246320648, 2.5289331918268765, 0.14174536510604538, 0.20876806052401253, 0.8239124390092892, 2.2166024969104217, 0.08128761703462414, 0.05306158619712342, 0.2086548824770224, 0.823912577711585, 2.07540960986624, 0.0627537197013978, 0.013359030540602479, 0.05308921882246047, 0.20865508980051517, 5.63361741972846]
+js_flops_wt_cbrt_list:
+[284.1272208194104, 37.40108868579932, 6.365898845043888, 1.3841308853060308, 0.32414744815032615, 0.07871209340563615, 0.01943978282878264, 0.004632532461929831, 0.0012091787625134614, 0.00044901393641909445, 0.0001613670018917652, 0.0003077105191045712, 0.0003002664419515789, 0.00020696161719254292, 0.00012687612093375484, 262.4822066509735, 29.788930153799104, 27.27708040640242, 27.4617158444724, 27.57960058953924, 27.254966113248038, 28.157444017390354, 48.556277119157386, 19.519892580336766, 4.376781457859674, 11.825022854497066, 1.2852191589032067, 2.9679969749567396, 9.108748752582219, 0.5105405400849312, 0.7519438698589741, 2.9675799365982605, 7.983791463513528, 0.2927829341864249, 0.19111800131577733, 0.751536223505297, 2.9675804361782285, 7.475240847034763, 0.2260272702724415, 0.048116752615563266, 0.19121752891199173, 0.7515369702461014, 20.291246052019066]
+js_param_list:
+[22.376322125614507, 3.02586645089218, 0.4836367204225824, 0.10510893380080222, 0.024645269341281507, 0.005969294795109833, 0.001496105381208345, 0.0003536383194705052, 8.500224584797425e-05, 3.44891489528618e-05, 1.8608716687978036e-05, 2.9195713892187336e-05, 2.3259451942551137e-05, 1.8678130940779195e-05, 1.8570740922767502e-05, 22.361218819280907, 2.624674226813863, 2.882002824420018, 2.8292324368810373, 3.0166989859863693, 2.9144326525822404, 3.024674433282705, 4.056468489415542, 1.6303346980246423, 0.34962596163763965, 0.986652555227066, 0.10077209863047813, 0.25092707578327306, 0.7590834123809459, 0.038549749038723456, 0.06343597319305173, 0.25091135136131804, 0.6647809214983476, 0.02118492111664206, 0.01617983438017046, 0.06341866967569981, 0.25090912280263694, 0.6220858854981818, 0.01597187243967527, 0.004084708080905273, 0.016177643729275073, 0.06341890690885256, 2.332671606843097]
+ptq_acc_list:
+[10.0, 12.85, 57.18, 87.24, 91.89, 92.47, 92.72, 92.75, 92.71, 92.76, 92.73, 92.76, 92.74, 92.76, 92.78, 10.0, 13.52, 24.25, 23.13, 14.28, 18.13, 12.53, 12.04, 18.99, 68.57, 41.99, 89.05, 84.47, 57.3, 91.51, 91.01, 82.12, 67.88, 92.13, 92.34, 91.22, 81.67, 70.2, 92.19, 92.61, 92.38, 90.61, 25.95]
+acc_loss_list:
+[0.892218150463462, 0.8615003233455487, 0.38370338435007545, 0.05971114464324215, 0.009592584608751892, 0.003341237335632704, 0.0006466910972192527, 0.00032334554860962637, 0.000754472946755846, 0.0002155636990730332, 0.0005389092476826596, 0.0002155636990730332, 0.00043112739814621956, 0.0002155636990730332, 0.0, 0.892218150463462, 0.8542789394266006, 0.7386290148738952, 0.7507005820219875, 0.8460875188618237, 0.8045915067902566, 0.8649493425307179, 0.8702306531580083, 0.7953222677301143, 0.2609398577279587, 0.5474240137960767, 0.040202629877128734, 0.08956671696486314, 0.38241000215563703, 0.013688294891140288, 0.019077387367967193, 0.11489545160594952, 0.2683768053459798, 0.007005820219875034, 0.004742401379607649, 0.01681396852769996, 0.11974563483509376, 0.2433714162535029, 0.006359129122655781, 0.0018322914421211652, 0.004311273981461583, 0.023388661349428776, 0.7203061004526837]
+
+
+MobileNetV2
+Full_acc: 88.980000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[16.257379508308613, 2.0872636706319843, 0.3673569039704757, 0.07968336516522288, 0.018337585272396234, 0.004493122602240293, 0.0010984497350990364, 0.0002694257873428869, 6.388189765335462e-05, 1.3099535396595918e-05, 1.10230860963334e-05, 1.2672575818299502e-05, 9.33353813857396e-06, 1.688814875686345e-05, 7.241302087656968e-06, 15.17292863895635, 1.3862298709085517, 0.8238612142634946, 0.8237263833222047, 0.823783901504274, 0.8236382344883858, 0.8236706411154064, 2.7748006955848035, 1.0847080062116685, 0.2753605921395851, 0.6455869889525454, 0.08282928548961851, 0.1966944980866937, 0.49264531599876277, 0.031141350974544027, 0.05282158451941949, 0.19661441355706435, 0.43113995208418615, 0.016957700971581377, 0.01333679763105676, 0.05280339096658972, 0.1966144830802525, 0.4031867580589643, 0.012775418917760356, 0.0033427705774578946, 0.01334780100863115, 0.05280348996445171, 0.21436596992758317]
+js_flops_wt_log_list:
+[28.80375976574097, 3.6980770059477583, 0.6508588917939833, 0.14117776523405268, 0.032489332035335064, 0.007960620219685995, 0.0019461612659263166, 0.0004773509561829447, 0.00011318176047046787, 2.3208898482903436e-05, 1.9529981669776462e-05, 2.2452439478139052e-05, 1.6536551304007493e-05, 2.9921315389862627e-05, 1.2829664560480558e-05, 26.882400772884356, 2.456031254073747, 1.4596633168235733, 1.4594244321964465, 1.4595263391426208, 1.4592682558685424, 1.4593256718068888, 4.91621005664466, 1.9218145710954924, 0.4878658544501161, 1.1438087256420213, 0.14675150073181503, 0.348490423517311, 0.8728366908390031, 0.0551742051536023, 0.09358582237472975, 0.34834853499516527, 0.763865517127118, 0.030044543446566756, 0.023629264163571733, 0.09355358822993869, 0.3483486581717946, 0.714339879555814, 0.022634650143081744, 0.005922501877739683, 0.023648759226973712, 0.09355376362786383, 0.37979957942105924]
+js_flops_wt_cbrt_list:
+[63.3327993218331, 8.13121518854432, 1.4310880217049553, 0.31041722146661277, 0.07143651949012658, 0.017503560887573957, 0.004279158064963306, 0.0010495842403871636, 0.00024886048838990957, 5.103099463587363e-05, 4.29419082755363e-05, 4.936771641339245e-05, 3.6360047914910196e-05, 6.579004541224335e-05, 2.8209462152985254e-05, 59.10817571333743, 5.400244128108922, 3.2094617047800025, 3.2089364527895006, 3.209160522571503, 3.2085930572010635, 3.208719301557034, 10.809607755157737, 4.22562531957135, 1.0727040673562915, 2.5149705827573334, 0.32267257544205513, 0.7662497617569168, 1.9191658113809233, 0.12131530366708894, 0.20577355720335788, 0.7659377817455499, 1.679563428473676, 0.06606099538021748, 0.05195528144815347, 0.2057026817815391, 0.765938052582441, 1.5706680172114325, 0.049768355481728206, 0.013022210501567764, 0.0519981465792421, 0.20570306744096986, 0.835091347158088]
+js_param_list:
+[14.378767704512425, 1.7642008979547863, 0.3172581244353037, 0.06872861944752165, 0.01604360877251897, 0.0038832136474442366, 0.0009495657272641881, 0.00023202358015285537, 5.295823140423917e-05, 1.588461536902086e-05, 1.3103930608435511e-05, 9.670751775829211e-06, 4.134339961343103e-06, 1.742488683173134e-05, 5.714308287624648e-06, 13.626820593353402, 1.19882863080523, 0.7288842053003567, 0.7288302575953983, 0.7291628958821641, 0.729005146686173, 0.7288945267461153, 2.4862192507339054, 0.9774023520806081, 0.23736847125525187, 0.582285561982151, 0.07230331940344965, 0.1713579859361879, 0.4446992495955345, 0.026967966696905668, 0.04730208296466469, 0.1713030264699067, 0.3884133880408106, 0.014487887085049132, 0.012000210156688437, 0.047273661143663374, 0.17130304887218364, 0.36307559697994757, 0.010763739690967786, 0.0030059266028757145, 0.012011478563072811, 0.04727356569791278, 0.20347025929202792]
+ptq_acc_list:
+[10.0, 11.4, 43.33, 83.74, 87.95, 88.62, 88.69, 88.8, 88.94, 89.0, 88.97, 88.96, 88.94, 88.96, 88.98, 10.0, 12.5, 10.71, 10.38, 10.15, 10.47, 10.76, 9.84, 11.0, 63.26, 13.64, 83.58, 70.08, 21.39, 86.92, 85.8, 74.76, 28.28, 87.49, 88.5, 85.89, 74.4, 34.97, 88.02, 88.89, 88.48, 85.47, 74.52]
+acc_loss_list:
+[0.8876151944257137, 0.8718813216453135, 0.5130366374466172, 0.05888963812092615, 0.011575634974151507, 0.004045853000674302, 0.0032591593616543746, 0.002022926500337231, 0.0004495392222972157, -0.000224769611148528, 0.00011238480557434384, 0.0002247696111486877, 0.0004495392222972157, 0.0002247696111486877, 0.0, 0.8876151944257137, 0.8595189930321421, 0.8796358732299394, 0.8833445718138908, 0.8859294223420993, 0.8823331085637222, 0.8790739492020678, 0.8894133513149022, 0.876376713868285, 0.2890537199370646, 0.8467071251966735, 0.060687795010114697, 0.21240728253540128, 0.7596089008766015, 0.023151269948303015, 0.03573836817262314, 0.15981119352663517, 0.6821757698359182, 0.01674533603056877, 0.005394470667565789, 0.03472690492245452, 0.16385704652730948, 0.6069903349067206, 0.010788941335131579, 0.0010114632501686154, 0.005619240278714317, 0.03944706675657457, 0.16250842886041816]
+
+
+ResNet_18
+Full_acc: 89.240000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[19.634024657363646, 3.0915857482955906, 0.5385152042628397, 0.11826505943160853, 0.02757539761660052, 0.006614136254944384, 0.0016376958204389971, 0.0003862673113095914, 0.00011132333867250922, 2.8336162705276282e-05, 2.1814901994788646e-05, 6.138414172118801e-06, 2.198147723790611e-05, 1.4520397742061373e-05, 1.4539346800832022e-05, 17.305264927922686, 2.2250078126861483, 1.4844314552301077, 1.5958173575037944, 1.5781102433330383, 1.5195455894264163, 1.515195586210641, 3.1729814190630723, 1.2331539262978317, 0.35046401625194956, 0.7327472839305194, 0.1078662219246328, 0.2042766114270692, 0.559948333635334, 0.046576590736551175, 0.0521588702023082, 0.2042360173068515, 0.49063541280574663, 0.0288203774519327, 0.013254343200123003, 0.05214674678935153, 0.20423487905192614, 0.4589766187125356, 0.02310853773423418, 0.0033246759695705346, 0.013244511952799711, 0.05214811326315113, 0.7563499709273178]
+js_flops_wt_log_list:
+[30.473175105509785, 4.798325127200901, 0.8358076554786383, 0.1835544127001643, 0.04279865869780145, 0.010265533215220302, 0.002541801407338161, 0.0005995098621136274, 0.000172780448832993, 4.39794114046939e-05, 3.385802656346418e-05, 9.527184222362902e-06, 3.411656125720857e-05, 2.2536521712553423e-05, 2.2565931779825902e-05, 26.858801371528603, 3.4533445827003737, 2.30392598852057, 2.476803539787077, 2.449321044468099, 2.3584252151802025, 2.3516737512283563, 4.924656053814952, 1.9139283047618687, 0.5439410167706241, 1.1372674059939605, 0.16741479783393928, 0.3170494617690429, 0.8690731482049148, 0.07228964158401795, 0.08095367163473435, 0.31698645727784786, 0.7614953687952211, 0.044730941517395324, 0.02057152969370392, 0.08093485537610413, 0.316984690638442, 0.7123590356761794, 0.03586582624278692, 0.005160095026762956, 0.020556271012592105, 0.08093697622478896, 1.1739001813095515]
+js_flops_wt_cbrt_list:
+[64.6193412739358, 10.17500171427277, 1.7723568332391129, 0.3892329957590859, 0.0907559229678002, 0.02176839111436422, 0.005389971081866601, 0.0012712798138979028, 0.0003663864612054905, 9.32597468017994e-05, 7.179702691928843e-05, 2.020269848852122e-05, 7.23452579962398e-05, 4.7789414218528366e-05, 4.785177920568148e-05, 56.95494631020095, 7.322927504382867, 4.885548657356098, 5.252141027373784, 5.1938635181241715, 5.0011160084605, 4.986799313475732, 10.442890479787394, 4.058546110506578, 1.153444302206545, 2.411611864310128, 0.35500842685678236, 0.6723135117963845, 1.8428973732282072, 0.15329249426468686, 0.17166484675885316, 0.672179908760133, 1.614775255427943, 0.0948533904133413, 0.04362258587107343, 0.17162494628134417, 0.6721761625448377, 1.5105800913934917, 0.07605463027826351, 0.010942146342999405, 0.0435902293503387, 0.17162944360873064, 2.4892928345973884]
+js_param_list:
+[22.67168893393059, 3.5166976269264856, 0.6338906597128334, 0.13813099283110955, 0.03235524374072725, 0.007761652301977761, 0.0019051573235626, 0.00045399778788963194, 0.00011969297933960909, 5.575635729216976e-05, 1.2642954050159805e-05, 7.295257896923202e-06, 3.241961021277539e-05, 6.6367596214680345e-06, 9.89624185452531e-06, 22.537005058367622, 2.6374999308870173, 2.2483874006459956, 2.4822964090248867, 2.4568183913775723, 2.3605886118658823, 2.3523085356979863, 4.164380176479744, 1.6526508796222692, 0.41459521978182506, 0.9943441201153862, 0.12545056403489277, 0.2560131083644279, 0.7641929606925943, 0.05256780741359665, 0.06525352104407976, 0.2559191123325789, 0.6696051326825595, 0.03175708109824689, 0.01651696205480994, 0.06525656209641477, 0.25591907886111154, 0.6270376914669819, 0.02514405941058533, 0.004171133503944475, 0.01650378165959455, 0.06526099213168313, 1.386806619038747]
+ptq_acc_list:
+[10.0, 11.43, 32.42, 80.79, 87.98, 88.83, 89.05, 89.28, 89.2, 89.22, 89.24, 89.28, 89.25, 89.25, 89.24, 10.0, 14.29, 16.93, 29.52, 15.74, 21.24, 17.47, 10.49, 18.81, 57.96, 43.02, 83.4, 78.58, 57.97, 86.57, 86.9, 77.9, 64.36, 87.1, 88.78, 86.46, 78.51, 66.59, 87.35, 88.98, 88.62, 87.11, 58.91]
+acc_loss_list:
+[0.8879426266248319, 0.871918422232183, 0.636709995517705, 0.0946884805020169, 0.014119229045271077, 0.004594352308381853, 0.002129090094128168, -0.00044822949350074246, 0.0004482294935005832, 0.0002241147467502916, 0.0, -0.00044822949350074246, -0.00011205737337522542, -0.00011205737337522542, 0.0, 0.8879426266248319, 0.8398700134468847, 0.8102868668758405, 0.6692066337965038, 0.8236216943074854, 0.761990138951143, 0.8042357687135814, 0.8824518153294487, 0.7892200806813088, 0.35051546391752575, 0.5179291797400268, 0.06544150605109804, 0.11945316001792915, 0.3504034065441506, 0.0299193186911699, 0.026221425369789213, 0.12707306140744049, 0.2787987449574182, 0.02398027790228598, 0.0051546391752576625, 0.031151949798296743, 0.12023756163155525, 0.25380995069475565, 0.021178843567906776, 0.0029134917077542686, 0.006947557149260313, 0.02386822052891075, 0.3398700134468848]
+
+
+ResNet_50
+Full_acc: 89.850000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[31.83704611388645, 3.8516944444240644, 0.6048486478821427, 0.13039055342800818, 0.030715211770088274, 0.007454410332335868, 0.001827429986037497, 0.0004525573498987203, 0.00011202413892903265, 3.127425135003992e-05, 1.6877799561591186e-05, 1.3263596996545557e-05, 1.8906519930125092e-05, 9.720838570247528e-06, 6.058470455608143e-06, 28.648168482850167, 2.572851346998403, 2.2067399717784575, 2.2432599639066675, 2.1568859465365597, 2.2084178277876516, 2.282073260714797, 5.098990965166972, 1.967784148923632, 0.5007878736140631, 1.159351606696317, 0.14699214314458967, 0.38393925748951574, 0.8798900526261768, 0.05245931982472846, 0.10324849363251126, 0.3839183569134797, 0.766648509418362, 0.026667422307958245, 0.02610411253488137, 0.10324001583825021, 0.38391861879027256, 0.7165367631175784, 0.01919721016553306, 0.0065989746125180776, 0.02609118215501123, 0.10323966615833328, 0.9098904757948152]
+js_flops_wt_log_list:
+[61.38953604942225, 7.426999794563097, 1.1662946914352523, 0.25142456845836053, 0.059226367718926934, 0.01437390862792751, 0.0035237276286354986, 0.0008726401829697578, 0.0002160096728380127, 6.030433143213481e-05, 3.2544485468747605e-05, 2.557542753972047e-05, 3.645634970869e-05, 1.8744131214438748e-05, 1.1682198439782121e-05, 55.24060760966001, 4.9610805585217665, 4.255129152516006, 4.325548542721352, 4.1589985168769985, 4.258364465289487, 4.400389979803051, 9.832089589971833, 3.794364449380834, 0.9656403144428252, 2.235510700290623, 0.28343645444608934, 0.7403278810517697, 1.6966411366179601, 0.10115427461417865, 0.19908810318734207, 0.7402875796789153, 1.4782840134671367, 0.05142124923472442, 0.050335051554902255, 0.19907175594662926, 0.7402880846408003, 1.381656429204863, 0.03701687088964243, 0.012724421367964275, 0.05031011865068963, 0.19907108167905466, 1.7544892187867938]
+js_flops_wt_cbrt_list:
+[139.85569952773784, 16.919956046332796, 2.6570156808946837, 0.5727875003305968, 0.13492763784937437, 0.032746184048177175, 0.008027644842458008, 0.0019880212668001943, 0.0004921064051566876, 0.00013738342068921917, 7.4141817545872e-05, 5.826513017477415e-05, 8.305370294103513e-05, 4.2702286932492766e-05, 2.66140150253335e-05, 125.84740522174036, 11.30217682972784, 9.693900662922456, 9.854327890610847, 9.474898889028678, 9.701271250163282, 10.024829285641681, 22.399155555075104, 8.644201088343793, 2.199891225108176, 5.0928697773377385, 0.6457159665827559, 1.6865915650678318, 3.8652341796200007, 0.23044646932984908, 0.4535562203385999, 1.6864997517547626, 3.367779887400924, 0.1171462637626534, 0.11467172256047463, 0.4535189785718044, 1.6865009021429591, 3.1476459808700747, 0.08433066456848748, 0.028988374339067622, 0.11461492120662324, 0.45351744247674025, 3.9970218509186894]
+js_param_list:
+[31.956057828244052, 3.6781639668772703, 0.6141538527740996, 0.13330863275702742, 0.03123701650588441, 0.007585242497140086, 0.0018671523361780372, 0.0004621250002368864, 0.00011846044173816939, 2.6062206353594685e-05, 2.747291246795935e-05, 1.416447670351295e-05, 3.3991025341785555e-05, 1.2811940759827942e-05, 6.1681174538745776e-06, 30.93497336748605, 2.6118116888082588, 2.7362817349561666, 2.7646933782712653, 2.5991994314260247, 2.676614353664408, 2.810640990555061, 5.318147614534668, 2.0406562989800165, 0.5038261286110767, 1.1993098035371026, 0.15193232234543597, 0.3897359095174924, 0.9088494117138247, 0.053981933311736154, 0.10871032464814381, 0.3897054267805748, 0.7906229272787243, 0.026929365934415236, 0.028108390887867356, 0.1087211208636762, 0.3897057522419193, 0.7377119053553417, 0.01906486880376661, 0.00713978154647066, 0.028086525808140382, 0.10872103282474936, 1.3247868721360945]
+ptq_acc_list:
+[8.93, 9.55, 37.51, 83.33, 88.58, 89.49, 89.68, 89.71, 89.77, 89.86, 89.87, 89.88, 89.85, 89.86, 89.85, 10.0, 11.89, 11.46, 11.27, 15.6, 18.19, 12.68, 10.61, 10.03, 56.58, 12.2, 83.96, 69.28, 31.0, 87.29, 86.91, 68.55, 34.74, 87.97, 89.29, 87.11, 68.77, 42.91, 88.18, 89.6, 89.25, 86.85, 63.29]
+acc_loss_list:
+[0.9006121313299944, 0.8937117417918754, 0.5825264329437952, 0.07256538675570391, 0.014134668892598732, 0.004006677796327206, 0.0018920422927099334, 0.0015581524763494778, 0.0008903728436282504, -0.00011129660545359061, -0.00022259321090718122, -0.0003338898163606137, 0.0, -0.00011129660545359061, 0.0, 0.8887033945464663, 0.8676683361157485, 0.8724540901502503, 0.8745687256538676, 0.8263772954924875, 0.7975514746800223, 0.8588759042849192, 0.8819143016138008, 0.8883695047301057, 0.3702838063439065, 0.8642181413466888, 0.06555370061213134, 0.2289371174179187, 0.6549805230940456, 0.02849193099610449, 0.032721202003338876, 0.2370617696160267, 0.613355592654424, 0.02092376182526428, 0.0062326099053977525, 0.03049526989426817, 0.23461324429604896, 0.5224262659988871, 0.018586533110739985, 0.002782415136338342, 0.006677796327211957, 0.033388981636060105, 0.2956037840845854]
+
+
+ResNet_152
+Full_acc: 89.350000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[30.627356547056827, 3.508123437462832, 0.5771908401215131, 0.12472600494350392, 0.02910628348229935, 0.007052040796604915, 0.0017338477327802524, 0.00042984751015978683, 0.00011139901508254095, 3.3235367203821945e-05, 1.5142626350115095e-05, 1.4694906490850088e-05, 1.1186725752392825e-05, 1.0949788302823225e-05, 9.67040989068593e-06, 29.582794760903646, 3.4372029592546154, 3.6959787092144736, 3.7615447526297228, 3.8768585308416044, 3.823413459382687, 3.814715615625443, 4.872341466063144, 1.8678059168695311, 0.5058669970869822, 1.0926263549786188, 0.1550712216029171, 0.40243989565498695, 0.8270144396298036, 0.05305877312425442, 0.1159220308389062, 0.4024146458476917, 0.7188630030540977, 0.02530931001141556, 0.029623528885693976, 0.11591286884283702, 0.40241544262426465, 0.6707128920562425, 0.017493582101068165, 0.007370448938047219, 0.029620082838057393, 0.11591281031972718, 2.36403248413144]
+js_flops_wt_log_list:
+[72.90068439668833, 8.350201531306475, 1.373856970245162, 0.29687874330505787, 0.0692801542582868, 0.016785601449997432, 0.0041269864790689295, 0.0010231433988994452, 0.0002651572109449532, 7.910839486298072e-05, 3.60431963101562e-05, 3.497751229958022e-05, 2.6627174377733344e-05, 2.6063204640211563e-05, 2.3017967559307827e-05, 70.41436896857765, 8.181393250670965, 8.797343544923597, 8.953406946308265, 9.227882261835354, 9.100669771883805, 9.079966752290705, 11.597377884853731, 4.445840092362056, 1.2040885815489155, 2.600720990902971, 0.36910786498058135, 0.9579064969810354, 1.9684989320682982, 0.1262930043624174, 0.2759231072334371, 0.9578463961940191, 1.7110717612727944, 0.060242418199080716, 0.07051132626134801, 0.2759012994165314, 0.957848292719349, 1.5964625869509097, 0.04163905251692631, 0.01754349158631911, 0.07050312381557752, 0.27590116011705407, 5.626982066323527]
+js_flops_wt_cbrt_list:
+[190.33792637446095, 21.801716368384643, 3.5870319876371792, 0.7751269395203039, 0.1808850082760557, 0.043825878993884904, 0.01077523558389394, 0.002671346566111044, 0.0006923045251519011, 0.00020654576787095048, 9.410593744569017e-05, 9.13235207040352e-05, 6.95214482307271e-05, 6.804896781073914e-05, 6.009809442601459e-05, 183.84635325939544, 21.360971286813626, 22.969168832941342, 23.37663804188408, 24.09327140178674, 23.761129642807454, 23.707075694594874, 30.27983723681431, 11.60773717250694, 3.1437801397869696, 6.790276999230642, 0.9637114686964429, 2.501018170198093, 5.139595161568077, 0.3297410547634758, 0.7204134296439638, 2.5008612517923354, 4.467473160421456, 0.15728819358405088, 0.18409950108928919, 0.7203564911572689, 2.500866203470109, 4.168237662641855, 0.10871627581906577, 0.04580467025162074, 0.18407808515153673, 0.7203561274573408, 14.69160553311559]
+js_param_list:
+[30.28208224346576, 3.173931046714681, 0.5485285595463477, 0.11917701675799117, 0.027903458577392418, 0.006751971372850573, 0.001656986563142275, 0.00041002456395797607, 0.00010731725879695602, 3.00861834384405e-05, 1.7693339838233405e-05, 1.5325070486412483e-05, 9.94082216480857e-06, 1.131527175793998e-05, 9.115562031790243e-06, 30.32786677297207, 3.7384564402172398, 4.231465807159617, 4.309608692386125, 4.419230588163285, 4.372004751552683, 4.367897412046836, 4.863031399597199, 1.8582782655278205, 0.4989174845526915, 1.0843942646398455, 0.1531128181021214, 0.40696253073293254, 0.818875122807061, 0.0511142814680861, 0.11834351619697606, 0.40694961481155234, 0.711378984533064, 0.023515193842725074, 0.030281401622506046, 0.11833668701798057, 0.4069513289177797, 0.6634083656642765, 0.015800865126435668, 0.007570232112869489, 0.030279649287160683, 0.11833607067827753, 2.921567452403827]
+ptq_acc_list:
+[10.0, 12.05, 30.14, 80.45, 87.72, 88.96, 89.09, 89.29, 89.31, 89.38, 89.3, 89.36, 89.34, 89.34, 89.37, 10.0, 10.15, 10.09, 10.48, 10.02, 10.82, 10.04, 10.0, 10.72, 14.7, 12.94, 18.66, 16.28, 13.07, 19.28, 23.28, 20.04, 17.84, 19.17, 28.41, 24.26, 17.14, 19.02, 18.89, 24.28, 25.41, 23.54, 10.0]
+acc_loss_list:
+[0.8880805819809737, 0.8651371012870733, 0.6626748740906547, 0.09960828203693332, 0.018242865137101237, 0.004364857302742033, 0.0029099048684945823, 0.0006715165081140242, 0.00044767767207601615, -0.00033575825405709163, 0.0005595970900950997, -0.00011191941801908356, 0.00011191941801892451, 0.00011191941801892451, -0.00022383883603816713, 0.8880805819809737, 0.8864017907106883, 0.8870733072188024, 0.8827084499160603, 0.8878567431449357, 0.8789031897034136, 0.8876329043088976, 0.8880805819809737, 0.8800223838836038, 0.8354784555120313, 0.85517627308338, 0.7911583659764969, 0.8177951874650251, 0.8537213206491326, 0.7842193620593173, 0.7394515948517068, 0.7757134862898714, 0.800335758254057, 0.7854504756575266, 0.6820369334079462, 0.7284834918858422, 0.8081701175153889, 0.787129266927812, 0.7885842193620592, 0.7282596530498041, 0.7156127588136542, 0.7365416899832121, 0.8880805819809737]
+
+

ykl/ALL-cifar10/ptq_result_sgd_mom95.txt

+AlexNet
+Full_acc: 89.200000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[72.47586903139205, 20.357851490126457, 4.023544864241921, 0.8553805386145032, 0.1899520622672781, 0.04528910654352502, 0.011100158996443895, 0.002750147627405226, 0.0006682028905090966, 0.0001619634768566911, 5.733654785409038e-05, 5.3580024990036894e-05, 2.0562486822339743e-06, 1.342468664395913e-05, 2.570616113086288e-05, 71.84533925852743, 11.468594152888866, 3.1487201197652466, 3.143749871483669, 3.1466604562955096, 3.145095426050407, 3.1421617111409135, 13.983627694914428, 5.072280547738527, 2.111992636667417, 2.8968971556144867, 0.6982631703822858, 0.8812669974501903, 2.1821031558902715, 0.3398179394683658, 0.22720821468439856, 0.8808475097736422, 1.8998833279665155, 0.23134253647984301, 0.05683388879436031, 0.22698963111765585, 0.8808482796073545, 1.7755364120097037, 0.19293769141730546, 0.014322587221196073, 0.05675665580446889, 0.2269928484550443, 0.880848756991021]
+js_flops_wt_log_list:
+[133.76103254420704, 37.572329550425486, 7.425830455317875, 1.578685231269704, 0.35057439562080145, 0.08358530549861494, 0.020486387381241263, 0.005075656093635648, 0.0012332312779141266, 0.0002989188289610865, 0.00010581999148104815, 9.888697524009663e-05, 3.795000330167116e-06, 2.4776521773069227e-05, 4.744313799290413e-05, 132.59733054800603, 21.16636911323429, 5.811259113431139, 5.802086052783091, 5.8074578107894785, 5.804569400283395, 5.799154953506438, 25.808100050174698, 9.361370791228854, 3.8978810407093363, 5.346496149518852, 1.2887103515443308, 1.6264611256335897, 4.027276597724286, 0.6271659666563699, 0.41933412879185017, 1.6256869216743335, 3.5064133629394907, 0.42696440849210654, 0.1048922869119857, 0.41893071226196993, 1.6256883424748512, 3.2769194349002615, 0.35608465501112024, 0.026433681734562498, 0.10474974616549423, 0.418936650160706, 1.625689223531305]
+js_flops_wt_cbrt_list:
+[298.8074779702571, 83.93246389391916, 16.588490892928267, 3.5266096821485826, 0.7831447545216751, 0.1867204062076916, 0.04576434279607774, 0.011338459097812326, 0.0027549034341206305, 0.0006677518833458881, 0.00023639025635388973, 0.0002209026583720119, 8.477614563709042e-06, 5.534803255529001e-05, 0.00010598276748476342, 296.20789532664327, 47.283347416018394, 12.981715575065909, 12.9612239635302, 12.973223881829462, 12.966771489512734, 12.95467620918143, 57.65246529962395, 20.9122757447843, 8.707438788778097, 11.94348608132516, 2.8788376005684615, 3.633335790874815, 8.996494273150313, 1.401019992152298, 0.9367464579771162, 3.631606304131929, 7.832942926445231, 0.9537916660625706, 0.2343178660821458, 0.9358452696895243, 3.631609478044475, 7.320278658365314, 0.7954540697240599, 0.05904994617892657, 0.2339994456860335, 0.9358585343041287, 3.631611446227973]
+js_param_list:
+[20.45373001769639, 5.700772882067947, 1.1512335382606107, 0.2416750284843947, 0.05282092340233431, 0.012649858538273405, 0.003092904958412703, 0.0007565363654742328, 0.00019060084584493709, 4.570834059402884e-05, 2.8692709015644802e-05, 2.3336288244710345e-05, 8.182006996979575e-07, 1.894832657102352e-05, 9.192248824267058e-06, 20.30708351756068, 3.174957531841112, 0.8534434610225798, 0.8526916337761649, 0.8527714110706294, 0.852694038475078, 0.8521871790815859, 3.90941511785069, 1.4328122255005904, 0.577958661354632, 0.8231742646368906, 0.19145508664124247, 0.23685204506682908, 0.6220938169889241, 0.09375707971237274, 0.0611628577031492, 0.23673470411206185, 0.5428423179802724, 0.063915228687228, 0.015442516864355212, 0.06110803706895241, 0.23673495124326852, 0.5080959529013864, 0.0533630359300532, 0.0038643017043627588, 0.015389590095221022, 0.061108798565360625, 0.23673763812869214]
+ptq_acc_list:
+[10.0, 10.01, 26.29, 66.22, 82.64, 88.08, 88.73, 89.1, 89.07, 89.18, 89.17, 89.2, 89.21, 89.22, 89.19, 10.0, 11.31, 67.07, 67.96, 67.98, 68.23, 68.42, 18.79, 62.53, 50.01, 77.49, 81.97, 86.23, 82.91, 86.62, 88.08, 85.83, 84.02, 87.25, 88.72, 88.12, 85.79, 84.35, 87.98, 88.94, 88.65, 88.17, 86.26]
+acc_loss_list:
+[0.8878923766816144, 0.8877802690582959, 0.7052690582959641, 0.2576233183856503, 0.07354260089686102, 0.012556053811659243, 0.005269058295964113, 0.0011210762331839521, 0.0014573991031391217, 0.0002242152466367267, 0.0003363228699551697, 0.0, -0.00011210762331828368, -0.0002242152466367267, 0.000112107623318443, 0.8878923766816144, 0.8732062780269058, 0.24809417040358756, 0.23811659192825121, 0.23789237668161434, 0.2350896860986547, 0.2329596412556054, 0.7893497757847533, 0.29899103139013455, 0.4393497757847534, 0.13127802690582968, 0.08105381165919287, 0.033295964125560525, 0.07051569506726464, 0.02892376681614348, 0.012556053811659243, 0.03778026905829601, 0.05807174887892384, 0.021860986547085234, 0.005381165919282556, 0.01210762331838563, 0.03822869955156947, 0.054372197309417135, 0.013677130044843036, 0.002914798206278084, 0.006165919282511179, 0.011547085201793735, 0.032959641255605354]
+
+
+AlexNet_BN
+Full_acc: 90.340000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[27.634102196940486, 7.012404777478978, 1.29597549108144, 0.282436050917579, 0.0661692774018142, 0.015913561268237546, 0.003950051730942347, 0.000930804933873267, 0.000193001220070606, 5.474206160525341e-05, 2.468341079813552e-05, 3.1409940647373347e-06, 5.556454267522604e-06, 2.712985946714941e-05, 4.206349927864394e-07, 23.994840029573375, 4.178168213490908, 1.184543243998221, 1.197066281760024, 1.1976934437226956, 1.1989435253561826, 1.1954022290911797, 4.627860496950446, 1.6851486437215624, 0.7219299430078794, 0.9531354999576058, 0.2437266980936711, 0.27854797486750116, 0.7139549871749042, 0.1207200565330562, 0.07413060450991123, 0.27822959075987674, 0.6199458789927894, 0.08311994694720047, 0.018361549899065917, 0.07395713336326142, 0.2782296881657688, 0.5803972666606478, 0.06954732001512173, 0.004565916248152199, 0.018351466509887147, 0.0739543569719887, 0.29377269766827296]
+js_flops_wt_log_list:
+[51.03212253315309, 12.949865253664571, 2.393288538541831, 0.5215769651384415, 0.12219534574470232, 0.029387703743436605, 0.0072945928371091154, 0.0017189250839907187, 0.00035641693156876557, 0.00010109261287547825, 4.558305659469075e-05, 5.800499428033851e-06, 1.0261149539407475e-05, 5.010093336008856e-05, 7.767900813505041e-07, 44.311467324897094, 7.7158574110868035, 2.1875056965023667, 2.2106320927570056, 2.211790277882728, 2.2140988138591062, 2.2075590730839623, 8.546307828768958, 3.1119777823828363, 1.3331939300712747, 1.7601631230096528, 0.4500920866933067, 0.5143968232980543, 1.318466513911182, 0.2229347157110161, 0.1368975936270997, 0.5138088607625934, 1.1448591249758802, 0.15349828582556047, 0.03390842436891183, 0.1365772430418442, 0.5138090406428509, 1.0718243791329893, 0.12843360466608708, 0.008431914877900945, 0.033889803291648195, 0.13657211585737408, 0.5425124434099425]
+js_flops_wt_cbrt_list:
+[114.02884523508637, 28.93585662375745, 5.347689157686907, 1.1654388664182933, 0.27303967534014323, 0.0656654231812108, 0.016299419980725056, 0.0038408561636006657, 0.0007963966441453836, 0.00022588662465488412, 0.0001018531671380399, 1.2960939477596364e-05, 2.2928049524160727e-05, 0.00011194814741517927, 1.735700409265222e-06, 99.01186152795677, 17.240715590718928, 4.887877206269246, 4.939552036326575, 4.9421399457824045, 4.947298259379866, 4.932685520349816, 19.096317463672033, 6.953566015049198, 2.9789582869435276, 3.933000596081083, 1.005709313026237, 1.1493951817341692, 2.9460505775504697, 0.49813699555264723, 0.30589114741639556, 1.1480814074752888, 2.558133142375981, 0.3429846028231117, 0.07576675792813391, 0.3051753393039006, 1.1480818094091498, 2.3949404841616864, 0.28697876753880414, 0.01884071186776449, 0.07572515001860543, 0.3051638828546014, 1.212218194677521]
+js_param_list:
+[9.837910329101113, 2.6469255947662456, 0.47554368881281606, 0.10316395532461183, 0.02412694913200428, 0.0058055407041362125, 0.0014614350801527393, 0.0003520262240125475, 8.174534383732279e-05, 2.07342293053987e-05, 6.970436788161117e-06, 1.431238129607415e-05, 6.855751393802123e-06, 8.145102926500502e-06, 2.5903578062655976e-06, 8.8415731001622, 1.5166569904919542, 0.4001755197178754, 0.4033283566324972, 0.40400019518461333, 0.4040595304811504, 0.4030512053954862, 1.6467065412174244, 0.5899300877354661, 0.2623824386434859, 0.33339547346463205, 0.08800422337440013, 0.09900602479190491, 0.2503143578360415, 0.0439282908275743, 0.025642943905155792, 0.09897485903913568, 0.21788912397188961, 0.030420821833556436, 0.0064197899689837365, 0.025596578301106503, 0.09897391789801542, 0.2037844236244302, 0.025543743557473964, 0.0015952880146722559, 0.0064260535966354235, 0.025593228429370528, 0.10337833374226023]
+ptq_acc_list:
+[10.0, 12.35, 31.03, 76.05, 87.93, 90.01, 90.04, 90.23, 90.31, 90.32, 90.3, 90.31, 90.32, 90.35, 90.33, 10.0, 19.57, 69.21, 66.52, 70.22, 67.19, 66.8, 14.3, 56.25, 68.94, 76.64, 84.16, 86.28, 81.22, 87.76, 88.88, 86.16, 82.95, 88.53, 89.99, 89.03, 86.35, 85.02, 89.13, 90.26, 89.96, 88.93, 86.39]
+acc_loss_list:
+[0.889307062209431, 0.8632942218286473, 0.6565198140358645, 0.1581802081027231, 0.02667699800752708, 0.0036528669470887567, 0.0033207881337170373, 0.0012176223156962521, 0.00033207881337171945, 0.00022138587558125116, 0.00044277175116234506, 0.00033207881337171945, 0.00022138587558125116, -0.00011069293779046828, 0.00011069293779062558, 0.889307062209431, 0.7833739207438566, 0.23389417755147232, 0.2636705778171353, 0.2227141908346248, 0.2562541509851672, 0.2605711755589994, 0.8417090989594864, 0.3773522249280496, 0.23688288687181763, 0.1516493247730795, 0.06840823555457169, 0.04494133274297102, 0.10095195926499893, 0.028558777949966772, 0.016161168917423155, 0.0462696479964579, 0.08180208102723047, 0.020035421740093005, 0.003874252822670008, 0.014500774850564559, 0.04416648217843711, 0.05888864290458277, 0.013393845472658933, 0.0008855435023245328, 0.004206331636041728, 0.015607704228470186, 0.04372371042727477]
+
+
+VGG_16
+Full_acc: 92.220000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[12.798857085257719, 3.0714151829653287, 0.5629288525115439, 0.1231144435114923, 0.028789664388994038, 0.0070208288291271105, 0.0017530491709936637, 0.0004289738531499819, 0.0001227577550943897, 6.62499372127492e-05, 3.430705713636773e-05, 3.7814641775733015e-05, 1.3139797419474906e-05, 2.758565502552535e-05, 6.182392511462575e-06, 12.567047884520019, 2.285881997041617, 1.5625290271367738, 1.527472954132658, 1.5272817882302363, 1.5235116078468798, 1.4684720013464767, 2.105950193422988, 0.753767041376289, 0.3231378360543848, 0.4332605055372091, 0.10790372658719181, 0.1292807432982466, 0.3250229428114355, 0.05294004341059009, 0.03405525104476679, 0.12925275638227038, 0.2828611905465128, 0.036058140087808735, 0.008562651105425175, 0.03402574571150178, 0.12924994104162785, 0.26445774647542025, 0.030134576217021394, 0.0021395641067249323, 0.008539756090885224, 0.03402563762507021, 1.0013752978135566]
+js_flops_wt_log_list:
+[32.29040876134581, 7.748914693952702, 1.420220776751261, 0.31060708615855714, 0.07263383167965574, 0.017712943525018687, 0.004422791342464933, 0.0010822639063684432, 0.0003097071921514628, 0.00016714285805075779, 8.655373608383823e-05, 9.540306856842251e-05, 3.3150571718222914e-05, 6.959622025545498e-05, 1.559764125721465e-05, 31.7055742095886, 5.767082449080179, 3.942125507896353, 3.8536820696010428, 3.853199774573502, 3.8436879357529237, 3.7048277719021407, 5.313130080575022, 1.9016890113497606, 0.8152487947124167, 1.093078759357215, 0.27223176375539326, 0.3261641268640622, 0.8200047559157071, 0.1335631478798502, 0.08591845110704474, 0.3260935182971549, 0.7136343037995494, 0.09097156682453639, 0.021602827692594785, 0.08584401170472776, 0.32608641543636474, 0.667204007116525, 0.07602692782766993, 0.0053979350747520765, 0.02154506555233747, 0.08584373901201515, 2.5263832133229136]
+js_flops_wt_cbrt_list:
+[88.74468146194975, 21.296570485470475, 3.9032345911120796, 0.8536506033321793, 0.1996216989203351, 0.048681004396668205, 0.012155287712850895, 0.002974417770251537, 0.000851177398127521, 0.00045936363971058535, 0.0002378781821832147, 0.00026219906329375117, 9.110869265108053e-05, 0.00019127334197514292, 4.286745687116499e-05, 87.13736343797463, 15.849842555139295, 10.834259642450803, 10.591187936021598, 10.589862430392166, 10.563720763605337, 10.182087285386764, 14.602231890324388, 5.226467921147829, 2.240572274377629, 3.004140549255044, 0.7481825744868714, 0.8964064765112555, 2.25364322263198, 0.3670755332103942, 0.23613220977017452, 0.89621242090716, 1.961302176106976, 0.25001983652812326, 0.05937168762483044, 0.2359276257682914, 0.8961928999056468, 1.833696424201744, 0.20894704500222652, 0.01483530395361875, 0.059212938233468376, 0.2359268763188776, 6.943333395814672]
+js_param_list:
+[6.976912401689799, 1.4373783899125656, 0.26901570154830307, 0.058284519951059446, 0.01350861479057626, 0.003299461623223444, 0.0008161555473985124, 0.0001918752196669562, 5.283003149195231e-05, 3.071924683685978e-05, 1.9919553638868325e-05, 1.5286239890305976e-05, 8.089437375349729e-06, 2.210039636837724e-05, 3.863953449166235e-06, 7.863356858366679, 1.177156230138372, 0.907183870553692, 0.9091138071023372, 0.9019609161880194, 0.9005215813559695, 0.8955795034267319, 1.3012998109929585, 0.5154783507405167, 0.15940116402140417, 0.3100955085464897, 0.05297007990265879, 0.08126256545472933, 0.23905359891828473, 0.023623585677493113, 0.023088125370315005, 0.08116838184789234, 0.2104565064633043, 0.015027275095623054, 0.005744859118843579, 0.02308290500464654, 0.08116690751771381, 0.19752701623392252, 0.012293088225753035, 0.0013929968691242039, 0.005785845039526407, 0.023082421464858952, 0.5408517444658735]
+ptq_acc_list:
+[10.0, 10.74, 43.06, 88.77, 91.48, 91.97, 92.19, 92.16, 92.18, 92.22, 92.22, 92.21, 92.22, 92.22, 92.23, 10.0, 15.82, 74.68, 77.59, 77.18, 77.78, 79.16, 13.54, 60.57, 83.52, 83.39, 90.13, 89.92, 86.46, 91.24, 91.71, 89.78, 87.37, 91.54, 92.16, 91.7, 89.81, 87.79, 91.73, 92.15, 92.14, 91.68, 81.94]
+acc_loss_list:
+[0.891563652136196, 0.8835393623942747, 0.5330730860984602, 0.037410540013012396, 0.008024289741921436, 0.0027109086965950986, 0.00032530904359142417, 0.0006506180871828483, 0.0004337453914551295, 0.0, 0.0, 0.00010843634786385943, 0.0, 0.0, -0.00010843634786385943, 0.891563652136196, 0.8284536976794622, 0.19019735415311204, 0.15864237692474512, 0.16308826718716105, 0.15658208631533288, 0.14161787031012799, 0.8531771849924096, 0.3432010409889395, 0.09433962264150947, 0.09574929516373887, 0.022663196703535064, 0.02494036000867488, 0.06245933636955113, 0.01062676209065283, 0.005530253741054057, 0.02645846887876814, 0.052591628713944856, 0.007373671654738588, 0.0006506180871828483, 0.005638690088917762, 0.026133159835176716, 0.04803730210366507, 0.005313381045326338, 0.0007590544350465537, 0.0008674907829104131, 0.005855562784645327, 0.11147256560399046]
+
+
+VGG_19
+Full_acc: 92.450000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[11.40557120539798, 3.3002202330858013, 0.6498871824026272, 0.14036847320115792, 0.0326895689757745, 0.00800360597451915, 0.0019877673714792953, 0.0005031058002731434, 0.00012025253263492614, 4.269202175303555e-05, 1.97552809156436e-05, 1.7502205085866837e-05, 1.2696171827811907e-05, 2.8382741230306863e-05, 2.5105578682709327e-05, 12.081888080531417, 2.4726143499136675, 1.2896726027616525, 1.2568814321939659, 1.3015941785412792, 1.3078544423842748, 1.2924702210575814, 2.055755214652762, 0.729067529771689, 0.3486599880208672, 0.40650076809198243, 0.11969617306991892, 0.11929036397102841, 0.30439390144049583, 0.06127747735837676, 0.031338798432556186, 0.1192135599704649, 0.26470724717521893, 0.04270268822485882, 0.00786699590193287, 0.03128283812768163, 0.11921447783902754, 0.2475859835100893, 0.035986676373552795, 0.0019927131347548086, 0.0078611640620946, 0.03128191218727892, 0.7066002373360015]
+js_flops_wt_log_list:
+[29.900749300216972, 8.651829535578171, 1.703738757533563, 0.36798880575623, 0.08569869838822136, 0.020982185936339114, 0.005211114180202056, 0.0013189379238045258, 0.00031525302558554517, 0.00011192104424833873, 5.179027787178754e-05, 4.588363327439557e-05, 3.3284177009584055e-05, 7.44079550859044e-05, 6.58165733841797e-05, 31.673775917358277, 6.482184930632569, 3.3809948208717455, 3.2950297646068725, 3.4122483234128067, 3.4286601783170165, 3.388329033406816, 5.389350536591658, 1.9113172885470648, 0.914044084692103, 1.0656789860126858, 0.3137944780411499, 0.3127306123292836, 0.797996485364946, 0.16064451795061255, 0.08215752972182669, 0.31252926360901817, 0.6939542871812362, 0.11194900738093086, 0.02062405012194399, 0.08201082465842521, 0.3125316698855025, 0.6490693267233447, 0.09434236734100482, 0.005224079951502432, 0.020608761419798095, 0.082008397220873, 1.85241722414203]
+js_flops_wt_cbrt_list:
+[85.30626546050125, 24.683504071106373, 4.860734066100746, 1.0498650196069166, 0.24449674624949927, 0.05986177488247601, 0.014867209016667458, 0.0037629046524655088, 0.0008994108481298681, 0.00031930859709913725, 0.0001477566715149768, 0.00013090512753034835, 9.495911995734791e-05, 0.00021228447171005025, 0.0001877734241519711, 90.36467646391023, 18.493549539662883, 9.645913512534353, 9.400656852360791, 9.735079157099618, 9.781901865022444, 9.666837880521879, 15.375713930046786, 5.452951642080028, 2.607750279046673, 3.0403617502594256, 0.8952496399019512, 0.8922144513881258, 2.2766686992889213, 0.458315735016372, 0.23439385982137617, 0.8916400073429738, 1.9798382992129864, 0.31938837538121506, 0.058840020258638485, 0.23397531308339078, 0.8916468724037281, 1.851782366113771, 0.2691569683340839, 0.014904200114025743, 0.058796401884038264, 0.23396838765051486, 5.28491088566527]
+js_param_list:
+[6.721109781310786, 1.3023992172094805, 0.24303548088969865, 0.05276293139034465, 0.01243203356773205, 0.00302520512910436, 0.000772872810379482, 0.000172558197085106, 4.710829546218242e-05, 3.4815616769520837e-05, 1.2503731268019178e-05, 1.8285521905184843e-05, 1.1954934042543774e-05, 1.9825276074358968e-05, 2.2090830441526038e-05, 7.533720781550805, 1.1453230886662025, 0.7599914343151031, 0.7589304603734501, 0.7596466840087656, 0.7467759930394964, 0.7696472556168243, 1.277029215790578, 0.5111524834910087, 0.14680767858513416, 0.3088056456114875, 0.04746106623377593, 0.07841530430564249, 0.23859669916671356, 0.020912020222213498, 0.02198602076344491, 0.07817597539824613, 0.20965685372001935, 0.013251418223898604, 0.005385363357572388, 0.021767416709616415, 0.07817577454572809, 0.1967891057241319, 0.010628991196431216, 0.0013589275186255917, 0.005398274808939512, 0.021764705382284595, 0.41974352355577454]
+ptq_acc_list:
+[10.0, 12.48, 51.32, 89.34, 91.66, 92.1, 92.34, 92.43, 92.43, 92.44, 92.47, 92.46, 92.47, 92.46, 92.46, 10.0, 17.92, 76.51, 78.42, 72.81, 75.07, 75.81, 10.05, 25.62, 81.19, 75.97, 90.43, 89.25, 83.61, 91.35, 91.77, 89.26, 85.93, 91.59, 92.38, 91.6, 89.29, 86.69, 91.68, 92.22, 92.42, 91.61, 86.75]
+acc_loss_list:
+[0.8918334234721471, 0.8650081124932395, 0.444889129259059, 0.033639805300162244, 0.008545159545700447, 0.003785830178474943, 0.0011898323418063756, 0.00021633315305566274, 0.00021633315305566274, 0.00010816657652790823, -0.00021633315305566274, -0.00010816657652775451, -0.00021633315305566274, -0.00010816657652775451, -0.00010816657652775451, 0.8918334234721471, 0.8061654948620876, 0.17241752298539748, 0.15175770686857762, 0.2124391563007031, 0.18799351000540843, 0.1799891833423472, 0.8912925905895078, 0.7228772309356408, 0.12179556517036241, 0.1782585181179016, 0.02184964845862624, 0.034613304488912956, 0.09561925365062199, 0.01189832341806391, 0.00735532720389407, 0.03450513791238505, 0.07052460789616004, 0.009302325581395342, 0.0007571660356950502, 0.009194159004867588, 0.034180638182801475, 0.06230394808004332, 0.00832882639264463, 0.0024878312601406597, 0.000324499729583571, 0.009085992428339679, 0.06165494862087618]
+
+
+Inception_BN
+Full_acc: 93.550000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[18.441306121864027, 3.6045267543878516, 0.6025846587846972, 0.13107412977701777, 0.030652094169276486, 0.00742527321994123, 0.0018014076348463986, 0.00046744856262639076, 0.0001271676401872304, 3.2373373750880685e-05, 3.978018151309869e-05, 2.654106994720084e-05, 2.175310462963817e-05, 3.775584490641178e-05, 3.689584788817301e-05, 19.353428608103517, 2.8743176963418686, 2.659766159812214, 2.618483383880818, 2.767666081681822, 2.610112173993742, 2.6674708064858295, 3.3472906298831644, 1.3155811855702504, 0.36693714118039, 0.7878571891784008, 0.11431068730340314, 0.20018055893308564, 0.6054517386109057, 0.05085230042102291, 0.05095218255874415, 0.20011799070424624, 0.5307921271914122, 0.03236978664185223, 0.012872096943486898, 0.05096094424594298, 0.20011920959504145, 0.4973209836613034, 0.026189476122251962, 0.003246621877997434, 0.012862233682811334, 0.05096154199141503, 1.8865822761335538]
+js_flops_wt_log_list:
+[58.54474438956486, 11.443121007074, 1.9129970832053724, 0.4161148550526045, 0.0973097570360041, 0.02357266452882896, 0.005718843818684624, 0.001483986895146522, 0.00040371310688014475, 0.0001027742221049137, 0.00012628826521640386, 8.425868241741298e-05, 6.905855484453612e-05, 0.00011986169930975078, 0.00011713150735502345, 61.44041660800682, 9.124960765508318, 8.443834126130815, 8.312775645314764, 8.786378916817892, 8.28619996028293, 8.46829370438075, 10.626485620330202, 4.176513513954177, 1.1648980281268821, 2.5011730433369253, 0.36289674521691445, 0.6355037748981919, 1.9220990662460882, 0.1614383986739667, 0.16175548978363818, 0.6353051424943857, 1.6850807206964575, 0.10276283427913929, 0.04086443879485729, 0.16178330509856695, 0.6353090120494798, 1.5788214606718434, 0.08314249409134984, 0.010306897283787211, 0.04083312636658986, 0.16178520273291236, 5.989243733402017]
+js_flops_wt_cbrt_list:
+[210.86732111079033, 41.216001488570825, 6.890261020597664, 1.4987686029591305, 0.35049171361283615, 0.08490436968281481, 0.020598215748841398, 0.005345045817619843, 0.0014540998040518224, 0.00037017370424067197, 0.0004548669304406201, 0.0003034841611649709, 0.00024873611818938135, 0.00043171963086987647, 0.0004218859853460231, 221.2969958815052, 32.866417847180344, 30.41312590303907, 29.941077539916375, 31.646908766484536, 29.845356846833194, 30.501224771602626, 38.274632145815055, 15.043027780773894, 4.195746844906218, 9.008761841600727, 1.3070868325006368, 2.2889668400783405, 6.923044676937851, 0.5814711979315236, 0.5826133013524482, 2.28825140296587, 6.0693485151168645, 0.3701326874023654, 0.14718613647074558, 0.58271348696272, 2.288265340386301, 5.686622350057392, 0.29946385764112415, 0.03712353417642865, 0.1470733549062333, 0.5827203218900057, 21.572146137291533]
+js_param_list:
+[17.762675345532376, 3.177229035661919, 0.4999724640691748, 0.1087740307465647, 0.025464538388128413, 0.00616548688825202, 0.0015028108891923398, 0.00038311217811924777, 9.946751366617692e-05, 3.3694740971126795e-05, 2.760401045905604e-05, 2.2713949422416117e-05, 3.2168893819736915e-05, 3.8431876663570844e-05, 2.0427033353135584e-05, 19.80179660211298, 2.9592069602502704, 3.411535315233787, 3.447339609758915, 3.500569967625175, 3.489347311848807, 3.447534423108732, 3.3404348712835015, 1.33055644648669, 0.32487803921057024, 0.8024403172350606, 0.097707746460744, 0.20176217333635624, 0.6174177269206456, 0.04089733817708296, 0.051065661479267874, 0.20172758360647625, 0.541260171830535, 0.02468904815660497, 0.01296737154592153, 0.05107325397118935, 0.20173000734322236, 0.5068886998241199, 0.019491195443066656, 0.0032847893393351483, 0.012960688255798176, 0.05107351280821566, 2.5741448942992275]
+ptq_acc_list:
+[10.0, 12.02, 60.06, 86.43, 92.84, 93.37, 93.51, 93.47, 93.58, 93.54, 93.54, 93.53, 93.52, 93.52, 93.54, 10.0, 12.46, 18.6, 13.53, 17.46, 21.27, 22.16, 11.01, 13.13, 72.51, 38.07, 89.54, 68.47, 36.69, 92.4, 92.18, 84.79, 48.98, 92.77, 93.21, 92.05, 80.52, 77.02, 92.81, 93.37, 93.16, 92.1, 28.8]
+acc_loss_list:
+[0.8931052912880812, 0.8715125601282737, 0.35799037947621587, 0.07610903260288605, 0.007589524318546165, 0.0019241047568144588, 0.00042757883484759, 0.0008551576696953319, -0.00032068412613576845, 0.00010689470871182154, 0.00010689470871182154, 0.000213789417423795, 0.00032068412613576845, 0.00032068412613576845, 0.00010689470871182154, 0.8931052912880812, 0.8668091929449493, 0.801175841795831, 0.8553714591127739, 0.8133618385889899, 0.7726349545697488, 0.763121325494388, 0.8823089257081774, 0.8596472474612508, 0.224906467129877, 0.5930518439337252, 0.04286477819347933, 0.26809192944949223, 0.6078033137359701, 0.012292891501870566, 0.014644575093532767, 0.09363976483164074, 0.47642971672902196, 0.008337787279529676, 0.0036344200962052744, 0.016034206306787816, 0.13928380545163016, 0.17669695350080172, 0.007910208444681934, 0.0019241047568144588, 0.004168893639764838, 0.015499732763228252, 0.692143238909674]
+
+
+MobileNetV2
+Full_acc: 90.360000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[15.211960055412751, 2.340851849567847, 0.41455538577289175, 0.08992781665823964, 0.02101804107199718, 0.005028379926905157, 0.0012546541304640111, 0.00030702210727150716, 9.097884354069217e-05, 1.784341654890209e-05, 1.3981351360675022e-05, 8.318276308207472e-06, 7.541133543433443e-06, 1.1327164939819336e-05, 1.573743689031218e-05, 14.186085935950047, 1.469534034232936, 0.7474639986305238, 0.7473738581786574, 0.7457946312855404, 0.7455695455041967, 0.746570309601244, 2.677340563797658, 1.0572261149515019, 0.2775947958274768, 0.6308651141365945, 0.08527324704617666, 0.17713950377315277, 0.48217016957392533, 0.03534643573135067, 0.046646341666224715, 0.17710393443091055, 0.4210165699398403, 0.020992245105333273, 0.01196307621897752, 0.046639109106894165, 0.17710381690906096, 0.39387305829159386, 0.016395826989814902, 0.0029940810696585563, 0.011968600285551058, 0.04663862580231397, 0.22893320521545044]
+js_flops_wt_log_list:
+[26.95155407907077, 4.147367925297174, 0.7344820692224057, 0.1593282131328568, 0.03723838798712204, 0.008908954075374153, 0.0022229139785111536, 0.0005439616523745711, 0.0001611903536957948, 3.161379627091536e-05, 2.4771242228028465e-05, 1.4737776916889478e-05, 1.3360886287695836e-05, 2.0068728640229907e-05, 2.788255950384847e-05, 25.13397753349076, 2.603624112237347, 1.324307736009397, 1.324148031063135, 1.321350060331032, 1.3209512680921478, 1.3227243563453193, 4.74353658111315, 1.8731239568817128, 0.49182427015056013, 1.1177254724779278, 0.1510815516798064, 0.3138441659122769, 0.8542775127760331, 0.06262449879211497, 0.08264493171349942, 0.31378114648233446, 0.7459295719673519, 0.03719268438363955, 0.021195394577248623, 0.08263211753878312, 0.3137809382648219, 0.6978384765780173, 0.02904905194185342, 0.005304716655320924, 0.021205181756445174, 0.08263126125139593, 0.40560885240188094]
+js_flops_wt_cbrt_list:
+[59.26022782385234, 9.119101904157137, 1.6149560291387437, 0.35032585435778063, 0.08187859406680797, 0.01958872748599262, 0.00488767320848827, 0.001196045740167116, 0.0003544202703486357, 6.95114190408084e-05, 5.446622683078356e-05, 3.240495947468178e-05, 2.9377495747166886e-05, 4.412648813741795e-05, 6.130729321443078e-05, 55.26379779008282, 5.724766653759511, 2.911846118949766, 2.911494964746476, 2.905342874869118, 2.904466023060821, 2.9083646336391373, 10.429938758367683, 4.118565931354752, 1.0814077070626187, 2.457619547623826, 0.3321933550198873, 0.690070662292201, 1.8783584754845082, 0.13769677454915988, 0.18171707158185332, 0.6899320971554835, 1.6401264374457913, 0.08177810242366199, 0.04660376569674617, 0.18168889617828163, 0.6899316393333781, 1.5343852523286161, 0.06387214002913282, 0.01166384382188251, 0.04662528543795438, 0.18168701340055762, 0.8918399632981566]
+js_param_list:
+[13.134231321324735, 1.808311018893426, 0.32459652868811695, 0.0703797420690634, 0.016432307198264138, 0.003957055883056518, 0.0009738632854790072, 0.00023182815258816623, 6.73117360597706e-05, 1.0036738561112203e-05, 9.889758622075638e-06, 5.345097155425636e-06, 4.449658177008708e-06, 9.723265117908844e-06, 1.2464751082451292e-05, 12.487267215418651, 1.1944292480200664, 0.6705370797990136, 0.6702815873603426, 0.6674932025353318, 0.667400040880111, 0.6689245812864402, 2.295846612450366, 0.9041078863217845, 0.22743110036068323, 0.5395656343667699, 0.07017025457690586, 0.1528297940719215, 0.41277599823892136, 0.027589838071319258, 0.04181703432378976, 0.1528020465707898, 0.36084944232187216, 0.015680818674952603, 0.010591959490255695, 0.04181302495044444, 0.1528015971646472, 0.33750161512968324, 0.012029214580045841, 0.002646845238343008, 0.010602870869597537, 0.0418124388176175, 0.21857125704275876]
+ptq_acc_list:
+[10.0, 10.94, 60.47, 85.5, 88.91, 89.78, 90.09, 90.24, 90.23, 90.36, 90.34, 90.36, 90.34, 90.35, 90.35, 10.0, 16.22, 16.35, 17.94, 15.62, 11.49, 15.45, 10.04, 12.46, 66.46, 25.76, 84.7, 71.15, 35.39, 87.69, 88.33, 73.91, 41.82, 88.72, 89.56, 88.17, 71.97, 44.9, 88.92, 89.83, 89.37, 88.12, 71.02]
+acc_loss_list:
+[0.8893315626383356, 0.8789287295263392, 0.33078795927401505, 0.05378486055776892, 0.016046923417441378, 0.00641876936697652, 0.0029880478087648964, 0.0013280212483400239, 0.0014386896857015877, 0.0, 0.00022133687472328486, 0.0, 0.00022133687472328486, 0.00011066843736172108, 0.00011066843736172108, 0.8893315626383356, 0.8204957945993803, 0.8190571049136786, 0.801460823373174, 0.82713590084108, 0.8728419654714475, 0.8290172642762283, 0.8888888888888888, 0.8621071270473661, 0.2644975652943781, 0.7149181053563524, 0.06263833554670205, 0.21259406817175736, 0.6083444001770695, 0.029548472775564428, 0.022465692784417898, 0.18204957945993805, 0.5371845949535192, 0.018149623727312977, 0.008853474988933125, 0.02423638778220449, 0.20351925630810094, 0.5030987162461266, 0.015936254980079657, 0.005865427180168229, 0.010956175298804724, 0.02478972996901278, 0.2140327578574591]
+
+
+ResNet_18
+Full_acc: 89.400000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[19.367526891268323, 3.911784973244561, 0.6956367860863596, 0.14937003377155955, 0.03494359194843906, 0.00849377965842345, 0.002086256098114944, 0.0005057734014148778, 0.00013636415791253495, 4.503875891850218e-05, 2.4424910027127207e-05, 1.3793582579141094e-05, 1.9756839182652803e-05, 9.476420885077941e-06, 2.8321615160289965e-05, 18.061365957713566, 2.621237076481687, 1.4394407819371993, 1.438955261011841, 1.4844383002186519, 1.4509707676077088, 1.3997460623903029, 3.227296406027789, 1.2204744351112982, 0.41343127179822947, 0.7154506848826242, 0.1341263041039015, 0.20219426620801162, 0.5436664837620968, 0.061656702554823604, 0.052932618016427554, 0.2021067634053575, 0.47460648952058027, 0.04076351320729487, 0.013204438437852583, 0.052897639635215224, 0.20210613384362117, 0.4441137741601547, 0.03359468612362406, 0.0033332691576835037, 0.013195326867748122, 0.05289622457353107, 0.6183185029539857]
+js_flops_wt_log_list:
+[30.05955470759487, 6.071323151775524, 1.0796697041069154, 0.23183118459258537, 0.05423453493835261, 0.013182851674869472, 0.0032379936616281496, 0.0007849904282993997, 0.0002116452910038281, 6.990283505341415e-05, 3.7908914406594745e-05, 2.1408461311513707e-05, 3.066379056005608e-05, 1.4707969356460283e-05, 4.395683274884215e-05, 28.03231518137464, 4.068315988119295, 2.2340977852206727, 2.2333442278408677, 2.303936612350281, 2.2519930093753233, 2.172489217409131, 5.0089561470211885, 1.8942489796148925, 0.6416699458954802, 1.1104220545841248, 0.2081720086706798, 0.3138175380245311, 0.8438027479233965, 0.09569487286330819, 0.08215457430439263, 0.31368172846566406, 0.736617525635415, 0.06326739919906225, 0.02049407453176925, 0.08210028577444568, 0.3136807513483707, 0.6892910161277365, 0.052140952796241306, 0.005173432166277187, 0.020479932832544318, 0.08209808951447153, 0.9596671258343779]
+js_flops_wt_cbrt_list:
+[63.74224600709266, 12.874434691184995, 2.289474097505499, 0.49160543275388596, 0.11500606385387598, 0.027954658101513107, 0.006866268996884369, 0.0016645972796540118, 0.00044880060056246083, 0.0001482311947702322, 8.038706399587655e-05, 4.539732610234062e-05, 6.502354743449393e-05, 3.11887188650668e-05, 9.321186804079275e-05, 59.44341983176734, 8.626994014777187, 4.737475721603303, 4.735877779107107, 4.885571185526377, 4.775423116084486, 4.606832785479045, 10.621651520389738, 4.01681547287823, 1.3606816183572186, 2.354685439073355, 0.4414353944213436, 0.6654601142526484, 1.7893106821706959, 0.20292403487032573, 0.17421139923258597, 0.6651721257446472, 1.5620209942872227, 0.13416054107270695, 0.04345833972589313, 0.17409627867059146, 0.6651700537362127, 1.4616636190353114, 0.11056655604224504, 0.010970428173392993, 0.04342835180093141, 0.17409162143115697, 2.0350047968075478]
+js_param_list:
+[22.39341765041649, 4.17178238362394, 0.7487421475161374, 0.1632710178345683, 0.038129659261993146, 0.00928447619442386, 0.0022695326681694324, 0.000554365935911477, 0.00014824012137658473, 7.279415069969987e-05, 2.8333372077377736e-05, 1.0851624200810761e-05, 3.921200700031057e-05, 1.8621368479834868e-05, 4.788302303764979e-05, 22.746121343966685, 2.820303221788956, 1.9163028585870345, 1.9813316155723297, 1.9658387860978548, 1.9550202676193438, 1.8770652302732342, 4.067384202055996, 1.582053249918024, 0.45933514187373287, 0.9410378508907615, 0.1442421479613022, 0.24730084973085972, 0.7203768268320053, 0.06442314769429515, 0.06362281534826547, 0.24720807564227104, 0.6308311053980703, 0.04117114546909489, 0.01601137575506769, 0.06357909487841379, 0.24720749122064958, 0.590736737087964, 0.03337218081195317, 0.00409287931870699, 0.016004820405844385, 0.06357823281396098, 1.1659715189402546]
+ptq_acc_list:
+[10.0, 10.3, 40.8, 79.38, 87.85, 88.92, 89.23, 89.29, 89.36, 89.37, 89.35, 89.37, 89.38, 89.4, 89.4, 10.0, 13.75, 17.84, 25.44, 20.03, 26.91, 18.28, 10.89, 23.18, 53.62, 44.67, 83.16, 76.47, 66.32, 85.86, 87.42, 75.57, 70.61, 86.78, 88.84, 87.29, 76.8, 72.56, 87.39, 89.28, 88.96, 87.42, 63.92]
+acc_loss_list:
+[0.8881431767337807, 0.8847874720357942, 0.5436241610738256, 0.11208053691275179, 0.01733780760626411, 0.0053691275167785674, 0.001901565995525746, 0.0012304250559284052, 0.00044742729306494685, 0.00033557046979867044, 0.0005592841163312234, 0.00033557046979867044, 0.00022371364653255292, 0.0, 0.0, 0.8881431767337807, 0.8461968680089486, 0.8004474272930648, 0.7154362416107383, 0.7759507829977629, 0.6989932885906041, 0.7955257270693512, 0.8781879194630873, 0.7407158836689037, 0.40022371364653253, 0.5003355704697987, 0.0697986577181209, 0.14463087248322154, 0.2581655480984341, 0.039597315436241676, 0.02214765100671145, 0.15469798657718134, 0.210178970917226, 0.02930648769574949, 0.006263982102908302, 0.02360178970917225, 0.14093959731543632, 0.18836689038031323, 0.022483221476510124, 0.0013422818791946818, 0.0049217002237137795, 0.02214765100671145, 0.28501118568232664]
+
+
+ResNet_50
+Full_acc: 90.440000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[31.70300441854657, 4.206904121637693, 0.6697882859372895, 0.14473255885685415, 0.03393772666388998, 0.008206460856987417, 0.0020180336148332323, 0.0004995867565681138, 0.00013071836195911412, 3.163344782210143e-05, 2.613519845483747e-05, 1.3231709869775508e-05, 1.0169978894905671e-05, 5.2713506417494395e-06, 1.0774058667935128e-05, 28.414910502102497, 2.7249128010173878, 2.043537176429337, 2.1493092545042747, 2.078027330063764, 2.0338892493570895, 2.065719579430443, 5.167264761315, 2.0119926445497054, 0.5133394740048283, 1.1926503516529694, 0.1497560510767373, 0.3710968843065892, 0.9065932052081545, 0.057142375262565506, 0.0961019404201235, 0.371079051172069, 0.7905144035322644, 0.031002546199930356, 0.024900701528985093, 0.0961008369323092, 0.37107954399584436, 0.7383992438665229, 0.023144760970550005, 0.006264585903969609, 0.024887740234683008, 0.0961014224842513, 0.8452372961742068]
+js_flops_wt_log_list:
+[61.131071195027225, 8.111930086349759, 1.2915140424127922, 0.2790794286531674, 0.06544015694846705, 0.015824044191036363, 0.003891257590405501, 0.000963324269859791, 0.0002520566627037139, 6.0996949230016106e-05, 5.039499273779177e-05, 2.5513941436036104e-05, 1.9610182545118697e-05, 1.0164440793069503e-05, 2.0774994659611428e-05, 54.79082972616225, 5.254291872154107, 3.940434634293057, 4.144388819514138, 4.006940004242162, 3.921831094106406, 3.9832076800030114, 9.963738005307455, 3.879609136536148, 0.9898428400763429, 2.2997187457419663, 0.28876593837593967, 0.7155646750478833, 1.7481312824751012, 0.11018433976513951, 0.18530781765169266, 0.715530288445226, 1.524303237795177, 0.05978041809395016, 0.048014583661478, 0.18530568985991466, 0.7155312387288301, 1.423812587325378, 0.044628704964469246, 0.012079638946739517, 0.0479895911467529, 0.18530681894588358, 1.62982222905316]
+js_flops_wt_cbrt_list:
+[139.2668730706418, 18.480342575536135, 2.9422864461155585, 0.6357899282162349, 0.14908369595536222, 0.036049837026192265, 0.008864939977892252, 0.0021946148855861293, 0.0005742275174498038, 0.0001389613206516829, 0.00011480827866763896, 5.812505447791452e-05, 4.467529768438478e-05, 2.3156307555062364e-05, 4.732893580537686e-05, 124.82273547219145, 11.970161571369877, 8.976973564008027, 9.441615538541608, 9.128483994533113, 8.934591567108722, 9.074417764007872, 22.699072811391538, 8.838402838680068, 2.255028653579514, 5.239146515847003, 0.6578574283219519, 1.6301768123087481, 3.982537401488925, 0.2510184781727236, 0.4221624096453791, 1.630098473838082, 3.472619428865133, 0.13618985789140398, 0.10938530599258854, 0.4221575621774439, 1.630100638744135, 3.2436848070732913, 0.10167170422656442, 0.027519451418865398, 0.10932836883595289, 0.42216013442538985, 3.7130094576147537]
+js_param_list:
+[31.62515886014933, 3.8913876899162423, 0.6643400032269674, 0.14439967002554566, 0.03372391144542041, 0.008177262746415984, 0.002021563445901279, 0.0004984448170509274, 0.00014424612620721167, 2.2461140143016147e-05, 3.679047422849989e-05, 2.161166255412883e-05, 1.3192266461397928e-05, 6.280298477735425e-06, 1.3245919202714227e-05, 30.644010262165377, 2.751602547440911, 2.473414646473393, 2.6519859704817934, 2.481059306859417, 2.4823001786824603, 2.5135063393927117, 5.46314924686992, 2.120825574144022, 0.5154612897286349, 1.2567502438469402, 0.1518964242666349, 0.3795924463991365, 0.956191248005165, 0.05702688189395775, 0.10038751625063264, 0.37959111828083786, 0.8335983339020387, 0.03027094260496773, 0.026135758720916676, 0.10038506687525245, 0.37959144312521825, 0.778507167856402, 0.02232240900243175, 0.006584992806575628, 0.026125785498218763, 0.1003849405503075, 1.2923677044760538]
+ptq_acc_list:
+[13.12, 9.69, 22.07, 82.68, 88.19, 90.0, 90.55, 90.42, 90.42, 90.43, 90.45, 90.47, 90.45, 90.46, 90.44, 10.0, 12.99, 11.54, 13.84, 10.8, 16.27, 23.04, 9.95, 10.31, 61.35, 13.18, 84.34, 73.75, 19.21, 88.21, 88.32, 70.71, 26.16, 87.97, 89.72, 88.66, 71.53, 28.26, 88.3, 90.31, 90.07, 88.66, 56.86]
+acc_loss_list:
+[0.8549314462627156, 0.8928571428571429, 0.7559708093763822, 0.08580274214949128, 0.024878372401592217, 0.004865103936311342, -0.0012162759840778354, 0.00022114108801410905, 0.00022114108801410905, 0.00011057054400697595, -0.00011057054400713309, -0.00033171163202124214, -0.00011057054400713309, -0.00022114108801410905, 0.0, 0.8894294559929234, 0.8563688633348077, 0.8724015922158338, 0.846970367094206, 0.8805838124723574, 0.8201017249004865, 0.7452454666076958, 0.8899823087129588, 0.8860017691287041, 0.32164971251658553, 0.8542680229986731, 0.06744803184431661, 0.1845422379478107, 0.787593984962406, 0.024657231313578106, 0.023440955329500274, 0.21815568332596202, 0.7107474568774879, 0.02731092436974789, 0.007961079168509497, 0.019681556833259634, 0.20908889871738165, 0.6875276426360017, 0.02366209641751438, 0.0014374170720919445, 0.004091110128261881, 0.019681556833259634, 0.3712958867757629]
+
+
+ResNet_152
+Full_acc: 90.240000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[29.680447121135096, 3.81406479993351, 0.6171960744182122, 0.1336378966308184, 0.031200011409096277, 0.007563481999101995, 0.0018744178909019672, 0.0004657902983917109, 0.00011949545737244453, 3.691498657193737e-05, 2.0492609181348652e-05, 1.639927750614377e-05, 1.2408399133020831e-05, 1.174884206301053e-05, 1.48188612190811e-05, 29.217641629324227, 3.3802445426072825, 3.2157786169377283, 3.2443460803448616, 3.2318705624939996, 3.3229282799630835, 3.2129099619785486, 4.883278329587889, 1.8812193259574128, 0.5051524151371539, 1.1047619549560517, 0.1544778002867222, 0.3826161136602792, 0.8377422884963033, 0.05525364738016926, 0.10798339251068984, 0.38258575636803377, 0.7292481618968714, 0.028157541411315438, 0.02745934933000796, 0.10797174350595944, 0.3825859282447193, 0.6807550789435542, 0.020307372224276737, 0.006865305829494591, 0.027448869916869244, 0.10797209359964278, 2.0107648290105247]
+js_flops_wt_log_list:
+[70.64680574067997, 9.07841764996171, 1.4690793233463306, 0.3180912499174451, 0.07426374461710356, 0.018002958019227217, 0.004461578226060835, 0.0011086961255021039, 0.00028442874628641844, 8.786679912953184e-05, 4.877747879086013e-05, 3.903433689983218e-05, 2.9535059210045322e-05, 2.796515023900662e-05, 3.527255521353078e-05, 69.54521419314563, 8.045818130129216, 7.654348545647642, 7.72234617484746, 7.692651356486835, 7.909391247600109, 7.64752043727006, 11.623410325325114, 4.477767323858413, 1.2023876997532579, 2.6296067206446163, 0.36769537546260017, 0.9107209923317999, 1.9940338659523398, 0.13151734800353518, 0.25702718435430866, 0.9106487344672427, 1.7357910081344459, 0.06702191345371744, 0.06536004359924727, 0.2569994568416165, 0.9106491435762688, 1.6203654757229027, 0.048336569013832695, 0.016341126038539048, 0.06533510000384704, 0.2570002901511801, 4.786117664788919]
+js_flops_wt_cbrt_list:
+[184.45322730428626, 23.703031110823364, 3.8356500271489793, 0.8305111180807693, 0.19389676890144736, 0.04700430080107511, 0.011648828195972908, 0.0028947179749255676, 0.0007426209810988257, 0.00022941326932502873, 0.00012735414274448598, 0.0001019155691665039, 7.711370574790278e-05, 7.301479747815072e-05, 9.209385443817747e-05, 181.5770588210334, 21.00699536017533, 19.98489920887665, 20.162435645569705, 20.08490482130375, 20.650795550293953, 19.967071557457402, 30.34780588181121, 11.691096651065827, 3.1393392718215622, 6.865695357046327, 0.9600235701793965, 2.3778205461401773, 5.206264856178217, 0.34338140318471966, 0.6710776681554418, 2.3776318865656085, 4.532013160671425, 0.17498891998068358, 0.1706499091110072, 0.6710052738108204, 2.377632954716948, 4.230646216428042, 0.1262029621571959, 0.042665388816850745, 0.17058478335445096, 0.6710074495162431, 12.496175025504746]
+js_param_list:
+[29.513813847192072, 3.5252890515550095, 0.5965626359472408, 0.12966470453571305, 0.03034752851740421, 0.007344950904139808, 0.0018198106917150696, 0.00045488868636628783, 0.00012051347361763302, 3.5039409314897576e-05, 2.21514772117209e-05, 1.4700960139617519e-05, 1.3608497827553243e-05, 1.3488181977359483e-05, 1.6929462794170096e-05, 30.130159718162872, 3.62234400291214, 3.623675677763459, 3.664591242641083, 3.633506801137264, 3.7416005874289677, 3.6281029316986824, 4.944391191382817, 1.898867653599768, 0.5017774612880233, 1.1133238658051539, 0.15318789510734107, 0.38802411729615366, 0.843221972982666, 0.05383642240723261, 0.1100842231225798, 0.3879866119535904, 0.7335276638094052, 0.026782762057868142, 0.028050933293330846, 0.11006535839448428, 0.38798644945512895, 0.6844793402720967, 0.01900371541899494, 0.007025297140432401, 0.02803681093088798, 0.11006559345928292, 2.4507077405691904]
+ptq_acc_list:
+[9.98, 10.71, 26.16, 83.73, 89.48, 90.12, 90.06, 90.28, 90.24, 90.26, 90.25, 90.27, 90.24, 90.26, 90.25, 10.0, 9.88, 11.32, 10.79, 12.89, 11.59, 10.96, 10.0, 10.0, 11.77, 12.51, 21.27, 11.46, 14.52, 29.1, 17.2, 14.44, 14.32, 22.37, 23.72, 17.14, 12.51, 16.21, 21.12, 29.28, 23.62, 16.39, 19.35]
+acc_loss_list:
+[0.8894060283687942, 0.8813164893617021, 0.7101063829787234, 0.07214095744680842, 0.008421985815602736, 0.0013297872340424463, 0.001994680851063748, -0.00044326241134758705, 0.0, -0.00022163120567387227, -0.00011081560283693613, -0.0003324468085106509, 0.0, -0.00022163120567387227, -0.00011081560283693613, 0.8891843971631206, 0.8905141843971631, 0.8745567375886524, 0.880429964539007, 0.8571586879432624, 0.8715647163120567, 0.8785460992907802, 0.8891843971631206, 0.8891843971631206, 0.869570035460993, 0.8613696808510638, 0.7642952127659575, 0.8730053191489362, 0.8390957446808511, 0.6775265957446808, 0.8093971631205673, 0.8399822695035462, 0.8413120567375886, 0.7521054964539007, 0.737145390070922, 0.8100620567375887, 0.8613696808510638, 0.8203679078014185, 0.7659574468085105, 0.675531914893617, 0.7382535460992907, 0.8183732269503546, 0.7855718085106382]
+
+

ykl/ALL/train.py → ykl/ALL-cifar10/train.py

@@ -5,65 +5,97 @@ import torch.nn as nn
 import torch.optim as optim
 from torchvision import datasets, transforms
 from torchvision.transforms.functional import InterpolationMode
+from torch.optim.lr_scheduler import CosineAnnealingLR
 import os
 import os.path as osp
-
+import time
+# import sys
 
 def train(model, device, train_loader, optimizer, epoch):
     model.train()
-    lossLayer = torch.nn.CrossEntropyLoss()
-    for batch_idx, (data, target) in enumerate(train_loader):
-        data, target = data.to(device), target.to(device)
+    total_loss = 0.
+    lossLayer = nn.CrossEntropyLoss()
+    start_time = time.time()
+    for batch_idx, (data, targets) in enumerate(train_loader):
+        data,targets = data.to(device), targets.to(device)
         optimizer.zero_grad()
         output = model(data)
-        loss = lossLayer(output, target)
+        loss = lossLayer(output, targets)
         loss.backward()
+        total_loss += loss.item() * len(data)
         optimizer.step()
 
-        if batch_idx % 50 == 0:
-            print('Train Epoch: {} [{}/{}]\tLoss: {:.6f}'.format(
-                epoch, batch_idx * len(data), len(train_loader.dataset), loss.item()
-            ))
+        pred = output.argmax(dim=1, keepdim=True)
+
+        if batch_idx % 200 == 0 and batch_idx > 0:
+            cur_loss = total_loss / 200
+            elapsed = time.time() - start_time
+            lr = optimizer.param_groups[0]['lr']
+            print('| epoch {:3d} | {:5d}/{:5d} batches | lr {:02.4f} | ms/batch {:5.2f} | '
+                'loss {:5.2f}'.format(
+                epoch, batch_idx, len(train_loader.dataset) // len(data), lr,
+                elapsed * 1000 / 200, cur_loss))
             
-def test(model, device, test_loader):
+            total_loss = 0.
+            correct = 0
+            start_time = time.time()
+
+
+def evaluate(model, device, eval_loader):
     model.eval()
-    test_loss = 0
+    total_loss = 0
     correct = 0
-    lossLayer = torch.nn.CrossEntropyLoss(reduction='sum')
-    for data, target in test_loader:
-        data, target = data.to(device), target.to(device)
+    lossLayer = nn.CrossEntropyLoss()
+    with torch.no_grad():
+        for data, targets in eval_loader:
+            data,targets = data.to(device), targets.to(device)
             output = model(data)
-        test_loss += lossLayer(output, target).item()
+            total_loss += len(data) * lossLayer(output, targets).item()
             pred = output.argmax(dim=1, keepdim=True)
-        correct += pred.eq(target.view_as(pred)).sum().item()
+            correct += pred.eq(targets.view_as(pred)).sum().item()
     
-    test_loss /= len(test_loader.dataset)
+    test_loss = total_loss / len(eval_loader.dataset)
+    test_acc = 100. * correct / len(eval_loader.dataset)
+    return test_loss,test_acc
 
-    print('\nTest set: Average loss: {:.4f}, Accuracy: {:.2f}%\n'.format(
-        test_loss, 100. * correct / len(test_loader.dataset)
-    ))
 
 epochs_cfg_table = {
     'AlexNet'       : [20, 30, 20, 20, 10],
     'AlexNet_BN'    : [15, 20, 20, 20, 10, 10],
     'VGG_16'        : [25, 30, 30, 20, 20, 10, 10],  
     'VGG_19'        : [30, 40, 30, 20, 20, 10, 10],
-    'Inception_BN'  : [20, 30, 30, 20, 20, 10, 10]
+    'Inception_BN'  : [20, 30, 30, 20, 20, 10, 10],
+    'ResNet_18'     : [30, 25, 25, 20, 10, 10],
+    'ResNet_50'     : [30, 40, 35, 25, 15, 10, 10],
+    'ResNet_152'    : [50, 60, 50, 40, 25, 15, 10, 10],
+    'MobileNetV2'   : [25, 35, 30, 20, 10, 10], 
 }
 lr_cfg_table = {
     'AlexNet'       : [0.01, 0.005, 0.001, 0.0005, 0.0001],
     'AlexNet_BN'    : [0.01, 0.005, 0.002, 0.001, 0.0005, 0.0001],
     'VGG_16'        : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],  
     'VGG_19'        : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],
-    'Inception_BN'  : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001]
+    'Inception_BN'  : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'ResNet_18'     : [0.01, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'ResNet_50'     : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'ResNet_152'    : [0.01, 0.008, 0.005, 0.003, 0.002, 0.001, 0.0005, 0.0001],
+    'MobileNetV2'   : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0001],
 }
 if __name__ == "__main__":
+    # sys.stdout = open(sys.stdout.fileno(), mode='w', buffering=1)
+
     batch_size = 32
-    seed = 1
-    momentum = 0.5
+    seed = 1111
+    seed_gpu = 1111
+    lr = 0.05   # origin lr
+    # momentum = 0.5
+    t_epochs = 300 #学习率衰减周期
+    patience = 30 #早停参数
     save_model = True
-    append = True
+
+    append = False
     torch.manual_seed(seed)
+    torch.cuda.manual_seed(seed_gpu)
     
     device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
 
@@ -91,23 +123,49 @@ if __name__ == "__main__":
         if not osp.exists('ckpt'):
             os.makedirs('ckpt')
 
-    model_name_list = ['AlexNet', 'AlexNet_BN', 'VGG_16', 'VGG_19', 'Inception_BN']
+    # model_name_list = ['AlexNet', 'AlexNet_BN', 'VGG_16', 'VGG_19', 'Inception_BN', 
+    #                    'ResNet_18', 'ResNet_50', 'ResNet_152', 'MobileNetV2']
+    # model_name_list = ['ResNet_18', 'ResNet_50', 'ResNet_152', 'MobileNetV2']
+    model_name_list = ['ResNet_152']
     for model_name in model_name_list:
         save_path = 'ckpt/cifar10_'+model_name+'.pt'
         if os.path.exists(save_path) and append:
             continue
+        else:
+            print('>>>>>>>>>>>>>>>>>>>>>>>> Train: '+model_name+' <<<<<<<<<<<<<<<<<<<<<<<<')
             model = Model(model_name).to(device)
 
-        epoch_start = 1
-        epochs_cfg = epochs_cfg_table[model_name]
-        lr_cfg = lr_cfg_table[model_name]
-        for epochs,lr in zip(epochs_cfg,lr_cfg):
-            optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum)
-            epoch_end = epoch_start+epochs
-            for epoch in range(epoch_start,epoch_end):
+            best_val_acc = None
+            optimizer = optim.SGD(model.parameters(), lr=lr)
+            lr_scheduler = CosineAnnealingLR(optimizer, T_max=t_epochs)
+            weak_cnt = 0 # 弱于最佳精度的计数器
+            epoch = 0
+            while weak_cnt < patience:
+                epoch += 1
+                epoch_start_time = time.time()
                 train(model, device, train_loader, optimizer, epoch)
-                test(model, device, test_loader)
-            epoch_start += epochs
-        
+                val_loss, val_acc = evaluate(model, device, test_loader)
+                if not best_val_acc or val_acc > best_val_acc:
+                    best_val_acc = val_acc
+                    weak_cnt = 0
                     if save_model:
                         torch.save(model.state_dict(), save_path)
+                else:
+                    weak_cnt += 1
+                print('-' * 89)
+                print('| end of epoch {:3d} | time: {:5.2f}s | test loss {:5.2f} | '
+                        'test acc {:.2f} | weak_cnt {:d}'.format(epoch, (time.time() - epoch_start_time),
+                                                val_loss, val_acc, weak_cnt))
+                print('-' * 89)
+                lr_scheduler.step()
+
+            print('>>> Early Stop: No improvement after patience(%d) epochs.'%patience)
+        
+        model = Model(model_name).to(device)
+        model.load_state_dict(torch.load(save_path))
+
+        test_loss,test_acc = evaluate(model, device, test_loader)
+        print('=' * 89)
+        print('| Test on {:s} | test loss {:5.2f} | test acc {:.2f}'.format(
+            model_name, test_loss, test_acc))
+        print('=' * 89)
\ No newline at end of file

ykl/ALL-cifar10/train_one.py

+from model import *
+
+import torch
+import torch.nn as nn
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torch.optim.lr_scheduler import CosineAnnealingLR
+import os
+import os.path as osp
+import time
+import sys
+
+def train(model, device, train_loader, optimizer, epoch):
+    model.train()
+    total_loss = 0.
+    lossLayer = nn.CrossEntropyLoss()
+    start_time = time.time()
+    for batch_idx, (data, targets) in enumerate(train_loader):
+        data,targets = data.to(device), targets.to(device)
+        optimizer.zero_grad()
+        output = model(data)
+        loss = lossLayer(output, targets)
+        loss.backward()
+        total_loss += loss.item()
+        optimizer.step()
+
+        if batch_idx % 50 == 0 and batch_idx > 0:
+            cur_loss = total_loss / 50
+            elapsed = time.time() - start_time
+            lr = optimizer.param_groups[0]['lr']
+            print('| epoch {:3d} | {:5d}/{:5d} batches | lr {:02.7f} | ms/batch {:5.2f} | '
+                'loss {:5.2f}'.format(
+                epoch, batch_idx, len(train_loader.dataset) // len(data), lr,
+                elapsed * 1000 / 50, cur_loss))
+            
+            total_loss = 0.
+            start_time = time.time()
+
+
+def evaluate(model, device, eval_loader):
+    model.eval()
+    total_loss = 0
+    correct = 0
+    lossLayer = nn.CrossEntropyLoss()
+    with torch.no_grad():
+        for data, targets in eval_loader:
+            data,targets = data.to(device), targets.to(device)
+            output = model(data)
+            total_loss += len(data) * lossLayer(output, targets).item()
+            pred = output.argmax(dim=1, keepdim=True)
+            correct += pred.eq(targets.view_as(pred)).sum().item()
+    
+    test_loss = total_loss / len(eval_loader.dataset)
+    test_acc = 100. * correct / len(eval_loader.dataset)
+    return test_loss,test_acc
+
+if __name__ == "__main__":
+    # sys.stdout = open(sys.stdout.fileno(), mode='w', buffering=1)
+
+    batch_size = 128
+    optim_type = 'adam' 
+    # optim_type = 'sgd'
+    if optim_type is 'adam':
+        lr = 0.001
+        opt_path = 'adam_lr'+str(lr).split('.')[-1]
+    elif optim_type is 'sgd':
+        lr = 0.01
+        momentum = 0.9
+        weight_decay = 1e-4
+        nesterov = True
+        opt_path = 'sgd_lr'+str(lr).split('.')[-1]+'_wd'+str(weight_decay).split('.')[-1]+'_ntr'
+
+    # lr = 0.001   # origin lr
+    # end_lr_bound = 0.00001 #早停时学习率应当低于该值，从而保证充分收敛
+    # momentum = 0.9
+    # weight_decay = 1e-4
+    # nesterov = True
+    t_epochs = 500 #学习率衰减周期
+    patience = 50 #早停参数
+    
+    print(opt_path)
+    print('t_epoches:%d patience:%d'%(t_epochs,patience))
+    save_model = True
+
+    append = False
+    
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+    train_transfrom = transforms.Compose([
+                        transforms.RandomCrop(32, padding=2),
+                        transforms.RandomHorizontalFlip(),
+                        transforms.ToTensor(),
+                        transforms.Normalize(
+                            (0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                        ])
+    eval_transfrom = transforms.Compose([
+                        transforms.ToTensor(),
+                        transforms.Normalize((0.4914, 0.4822, 0.4465),
+                                            (0.2023, 0.1994, 0.2010))
+                        ])
+
+    alltrainset = datasets.CIFAR10(root='/lustre/datasets/CIFAR10',train=True,download=True,transform=train_transfrom)
+    
+    train_size = (int)(0.8 * len(alltrainset))
+    val_size = (int)(0.2 * len(alltrainset))
+    
+    train_idx, val_idx = torch.utils.data.random_split(range(train_size+val_size),[train_size,val_size])
+
+    trainset = torch.utils.data.Subset(alltrainset,train_idx)
+    valset = torch.utils.data.Subset(alltrainset,val_idx)
+
+    train_loader = torch.utils.data.DataLoader(
+     trainset,
+     batch_size=batch_size, shuffle=True, num_workers=4, pin_memory=True
+    )
+    val_loader = torch.utils.data.DataLoader(
+        valset,
+        batch_size=batch_size, shuffle=False, num_workers=4, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+    datasets.CIFAR10('/lustre/datasets/CIFAR10', train=False, download=False, 
+                     transform=eval_transfrom),
+    batch_size=batch_size, shuffle=False, num_workers=4, pin_memory=True
+    )
+
+    # ckpt_path = 'ckpt_sgd_1_momentum_9_ntb'
+    ckpt_path = 'ckpt_'+opt_path
+    if save_model:
+        if not osp.exists(ckpt_path):
+            os.makedirs(ckpt_path)
+
+    model_name = sys.argv[1]
+    save_path = ckpt_path+'/cifar10_'+model_name+'.pt'
+    if os.path.exists(save_path) and append:
+        pass
+    else:
+        print('>>>>>>>>>>>>>>>>>>>>>>>> Train: '+model_name+' <<<<<<<<<<<<<<<<<<<<<<<<')
+        model = Model(model_name).to(device)
+
+        best_val_acc = None
+        if 'adam' in opt_path:
+            optimizer = optim.Adam(model.parameters(), lr=lr)
+        elif 'sgd' in opt_path:
+            optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum,weight_decay = weight_decay,nesterov=nesterov)
+        else:
+            raise ValueError('Illegal opttype')
+        lr_scheduler = CosineAnnealingLR(optimizer, T_max=t_epochs)
+        weak_cnt = 0 # 弱于最佳精度的计数器
+        epoch = 0
+        while True:
+            epoch += 1
+            epoch_start_time = time.time()
+            train(model, device, train_loader, optimizer, epoch)
+            val_loss, val_acc = evaluate(model, device, val_loader)
+            if not best_val_acc or val_acc > best_val_acc:
+                best_val_acc = val_acc
+                weak_cnt = 0
+                if save_model:
+                    torch.save(model.state_dict(), save_path)
+            else:
+                weak_cnt += 1
+            print('-' * 89)
+            print('| end of epoch {:3d} | time: {:5.2f}s | test loss {:5.2f} | '
+                    'test acc {:.2f} | weak_cnt {:d}'.format(epoch, (time.time() - epoch_start_time),
+                                            val_loss, val_acc, weak_cnt))
+            print('-' * 89)
+            lr_scheduler.step()
+            
+            #这里额外限制早停时，学习率曾低于一个限度，保证充分训练
+            if weak_cnt >= patience:
+                break
+                # if optimizer.param_groups[0]['lr'] < end_lr_bound:
+                #     break
+                # elif epoch > t_epochs: # 已经训练完一个周期，曾达到最小精度
+                #     break
+                # else: #若多个epoch未提升，载入当前最佳精度模型继续训练
+                #     print('>> Turn Back: No improvement after %d epochs, back to BestPoint'%patience)
+                #     weak_cnt = 0
+                #     model.load_state_dict(torch.load(save_path))
+
+        print('>>> Early Stop: No improvement after patience(%d) epochs'%patience)
+        # print('>>> Early Stop: No improvement after patience(%d) epochs And lr under bound(%f) '%(patience,end_lr_bound))
+    
+    model = Model(model_name).to(device)
+    model.load_state_dict(torch.load(save_path))
+
+    test_loss,test_acc = evaluate(model, device, test_loader)
+    print('=' * 89)
+    print('| Test on {:s} | test loss {:5.2f} | test acc {:.2f}'.format(
+        model_name, test_loss, test_acc))
+    print('=' * 89)
\ No newline at end of file

ykl/ALL-cifar10/train_one.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 1-06:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-normal             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+python train_one.py $Model
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL/utils.py → ykl/ALL-cifar10/utils.py

@@ -76,6 +76,7 @@ def fold_ratio(layer, par_ratio, flop_ratio):
             [prefix,suffix] = name.split('conv')
             bn_name = prefix+'bn'+suffix
             relu_name = prefix+'relu'+suffix
+            relus_name = prefix+'relus'+suffix
             if bn_name in layer:
                 bn_idx = layer.index(bn_name)
                 par_ratio[conv_idx]+=par_ratio[bn_idx]
@@ -83,7 +84,11 @@ def fold_ratio(layer, par_ratio, flop_ratio):
                 if relu_name in layer:
                     relu_idx = layer.index(relu_name)
                     par_ratio[conv_idx]+=par_ratio[relu_idx]
-                    flop_ratio[conv_idx]+=flop_ratio[bn_idx]
+                    flop_ratio[conv_idx]+=flop_ratio[relu_idx]
+                elif relus_name in layer:
+                    relus_idx = layer.index(relus_name)
+                    par_ratio[conv_idx]+=par_ratio[relus_idx]
+                    flop_ratio[conv_idx]+=flop_ratio[relus_idx]
     return par_ratio,flop_ratio
 
 def fold_model(model):

ykl/ALL-cifar100/README.md

+# ALL-cifar100量化
+
++ cfg最后输出层类别由cifar10的10改为100
++ 从训练集切分验证集
+
++ 训练器：Adam lr=0.0001
++ 全精度准确度
+
+| AlexNet | AlexNet_BN | VGG_16 | VGG_19 | Inception_BN | ResNet_18 | ResNet_50 | ResNet_152 | MobileNetV2 |
+| ------- | ---------- | ------ | ------ | ------------ | --------- | --------- | ---------- | ----------- |
+| 56.88   | 61.60      | 63.29  | 60.84  | 68.44        | 43.26     | 37.1      | 38.56      | 50.3        |
+
++ 拟合结果：R2=0.7989
+
+  <img src="image/adam-ALL.png" alt="adam-ALL"  />
\ No newline at end of file

ykl/ALL-cifar100/cfg.py

+# conv: 'C',''/'B'/'BRL'/'BRS',qi,in_ch,out_ch,kernel_size,stirde,padding,bias
+# relu: 'RL'
+# relu6: 'RS'
+# inception: 'Inc'
+# maxpool: 'MP',kernel_size,stride,padding
+# adaptiveavgpool: 'AAP',output_size
+# view: 'VW': 
+#   dafault: x = x.view(x.size(0),-1)
+# dropout: 'D'
+# MakeLayer: 'ML','BBLK'/'BTNK'/'IRES', ml_idx, blocks
+# softmax: 'SM'
+# class 100
+ResNet_18_cfg_table = [
+    ['C','BRL',True,3,16,3,1,1,True],
+    ['ML','BBLK',0,2],
+    ['ML','BBLK',1,2],
+    ['ML','BBLK',2,2],
+    ['ML','BBLK',3,2],
+    ['AAP',1],
+    ['VW'],
+    ['FC',128,100,True],
+    ['SM']
+]
+ResNet_50_cfg_table = [
+    ['C','BRL',True,3,16,3,1,1,True],
+    ['ML','BTNK',0,3],
+    ['ML','BTNK',1,4],
+    ['ML','BTNK',2,6],
+    ['ML','BTNK',3,3],
+    ['AAP',1],
+    ['VW'],
+    ['FC',512,100,True],
+    ['SM']
+]
+ResNet_152_cfg_table = [
+    ['C','BRL',True,3,16,3,1,1,True],
+    ['ML','BTNK',0,3],
+    ['ML','BTNK',1,8],
+    ['ML','BTNK',2,36],
+    ['ML','BTNK',3,3],
+    ['AAP',1],
+    ['VW'],
+    ['FC',512,100,True],
+    ['SM']
+]
+MobileNetV2_cfg_table = [
+    ['C','BRS',True,3,32,3,1,1,True],
+    ['ML','IRES',0,1],
+    ['ML','IRES',1,2],
+    ['ML','IRES',2,3],
+    ['ML','IRES',3,3],
+    ['ML','IRES',4,3],
+    ['ML','IRES',5,1],
+    ['C','',False,320,1280,1,1,0,True],
+    ['AAP',1],
+    ['VW'],
+    ['FC',1280,100,True]
+]
+AlexNet_cfg_table = [
+    ['C','',True,3,32,3,1,1,True],
+    ['RL'],
+    ['MP',2,2,0],
+    ['C','',False,32,64,3,1,1,True],
+    ['RL'],
+    ['MP',2,2,0],
+    ['C','',False,64,128,3,1,1,True],
+    ['RL'],
+    ['C','',False,128,256,3,1,1,True],
+    ['RL'],
+    ['C','',False,256,256,3,1,1,True],
+    ['RL'],
+    ['MP',3,2,0],
+    ['VW'],
+    ['D',0.5],
+    ['FC',2304,1024,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',1024,512,True],
+    ['RL'],
+    ['FC',512,100,True]
+]
+AlexNet_BN_cfg_table = [
+    ['C','BRL',True,3,32,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,32,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,64,128,3,1,1,True],
+    ['C','BRL',False,128,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['MP',3,2,0],
+    ['VW'],
+    ['D',0.5],
+    ['FC',2304,1024,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',1024,512,True],
+    ['RL'],
+    ['FC',512,100,True]
+]
+VGG_16_cfg_table = [
+    ['C','BRL',True,3,64,3,1,1,True],
+    ['C','BRL',False,64,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,64,128,3,1,1,True],
+    ['C','BRL',False,128,128,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,128,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,256,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['VW'],
+    ['FC',512,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,100,True]
+]
+VGG_19_cfg_table = [
+    ['C','BRL',True,3,64,3,1,1,True],
+    ['C','BRL',False,64,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,64,128,3,1,1,True],
+    ['C','BRL',False,128,128,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,128,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['C','BRL',False,256,256,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,256,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['C','BRL',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['VW'],
+    ['FC',512,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,4096,True],
+    ['RL'],
+    ['D',0.5],
+    ['FC',4096,100,True]
+]
+Inception_BN_cfg_table = [
+    ['C','',True,3,64,3,1,1,True],
+    ['RL'],
+    ['C','',False,64,64,3,1,1,True],
+    ['RL'],
+    ['Inc',0],
+    ['Inc',1],
+    ['MP',3,2,1],
+    ['Inc',2],
+    ['Inc',3],
+    ['Inc',4],
+    ['Inc',5],
+    ['Inc',6],
+    ['MP',3,2,1],
+    ['Inc',7],
+    ['Inc',8],
+    ['AAP',1],
+    ['C','',False,1024,100,1,1,0,True],
+    ['VW']
+]
+
+model_cfg_table = {
+    'AlexNet'       : AlexNet_cfg_table,
+    'AlexNet_BN'    : AlexNet_BN_cfg_table,
+    'VGG_16'        : VGG_16_cfg_table,
+    'VGG_19'        : VGG_19_cfg_table,
+    'Inception_BN'  : Inception_BN_cfg_table,
+    'ResNet_18'     : ResNet_18_cfg_table,
+    'ResNet_50'     : ResNet_50_cfg_table,
+    'ResNet_152'    : ResNet_152_cfg_table,
+    'MobileNetV2'   : MobileNetV2_cfg_table
+}
+
+#每行对应一个Inc结构(channel)的参数表
+inc_ch_table=[
+    [ 64, 64, 96,128, 16, 32, 32],#3a
+    [256,128,128,192, 32, 96, 64],#3b
+    [480,192, 96,208, 16, 48, 64],#4a
+    [512,160,112,224, 24, 64, 64],#4b
+    [512,128,128,256, 24, 64, 64],#4c
+    [512,112,144,288, 32, 64, 64],#4d
+    [528,256,160,320, 32,128,128],#4e
+    [832,256,160,320, 32,128,128],#5a
+    [832,384,192,384, 48,128,128] #5b
+]
+
+# br0,br1,br2,br3 <- br1x1,br3x3,br5x5,brM
+# 每个子数组对应Inc结构中一个分支的结构，均默认含'BRL'参数，bias为False
+# Conv层第2、3个参数是对应Inc结构(即ch_table中的一行)中的索引
+# 由于每个Inc结构操作一致，只有权重不同，使用索引而非具体值，方便复用
+# 各分支后还有Concat操作，由于只有唯一结构，未特殊说明
+# conv: 'C', ('BRL' default), in_ch_idex, out_ch_idx, kernel_size, stride, padding, (bias: True default)
+# maxpool: 'MP', kernel_size, stride, padding
+# relu: 'RL'
+inc_cfg_table = [
+    [
+        ['C',0,1,1,1,0]
+    ],
+    [
+        ['C',0,2,1,1,0],
+        ['C',2,3,3,1,1]
+    ],
+    [
+        ['C',0,4,1,1,0],
+        ['C',4,5,5,1,2]
+    ],
+    [
+        ['MP',3,1,1],
+        ['RL'],
+        ['C',0,6,1,1,0]
+    ]
+]
+
+# ml_cfg_table = []
+
+#BasicBlock
+#value: downsample,inplanes,planes,planes*expansion,stride,1(dafault stride and group) 
+bblk_ch_table = [
+    [False, 16, 16, 16,1,1], #layer1,first
+    [False, 16, 16, 16,1,1], #       other
+    [True,  16, 32, 32,2,1], #layer2
+    [False, 32, 32, 32,1,1], 
+    [True,  32, 64, 64,2,1], #layer3
+    [False, 64, 64, 64,1,1], 
+    [True,  64,128,128,2,1], #layer4
+    [False,128,128,128,1,1]  
+]
+#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
+#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
+bblk_cfg_table = [
+    [
+        ['C','BRL',1,2,3,4,1,5],
+        ['C','B'  ,2,2,3,5,1,5],
+    ],
+    # downsample, 仅当downsample传入为True时使用
+    [
+        ['C','B'  ,1,3,1,4,0,5]
+    ],
+    # 分支交汇后动作
+    [
+        ['AD',True],
+        ['RL']
+    ]
+]
+
+#BottleNeck
+#value: downsample,inplanes,planes,planes*expansion,stride,1(dafault stride and group) 
+btnk_ch_table = [
+    [True,  16, 16, 64,1,1], #layer1,first
+    [False, 64, 16, 64,1,1], #       other 
+    [True,  64, 32,128,2,1], #layer2
+    [False,128, 32,128,1,1], 
+    [True, 128, 64,256,2,1], #layer3
+    [False,256, 64,256,1,1],  
+    [True, 256,128,512,2,1], #layer4
+    [False,512,128,512,1,1]   
+]
+#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
+#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
+btnk_cfg_table = [
+    [
+        ['C','BRL',1,2,1,5,0,5],
+        ['C','BRL',2,2,3,4,1,5],
+        ['C','B'  ,2,3,1,5,0,5]
+    ],
+    # downsample, 仅当downsample传入为True时使用
+    [
+        ['C','B'  ,1,3,1,4,0,5]
+    ], 
+    # 分支交汇后动作
+    [
+        ['AD',True],
+        ['RL']
+    ]
+]
+
+#InvertedResidual
+#value: identity_flag, in_ch, out_ch, in_ch*expand_ratio, stride, 1(dafault stride and group) 
+ires_ch_table = [
+    [False, 32, 16,  32,1,1], #layer1,first
+    [ True, 16, 16,  16,1,1], #       other
+    [False, 16, 24,  96,2,1], #layer2
+    [ True, 24, 24, 144,1,1],
+    [False, 24, 32, 144,2,1], #layer3
+    [ True, 32, 32, 192,1,1],
+    [False, 32, 96, 192,1,1], #layer4
+    [ True, 96, 96, 576,1,1],
+    [False, 96,160, 576,2,1], #layer5
+    [ True,160,160, 960,1,1],
+    [False,160,320, 960,1,1], #layer6
+    [ True,320,320,1920,1,1]
+]
+
+#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
+#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
+ires_cfg_table = [
+    [
+        ['C','BRS',1,3,1,5,0,5],
+        ['C','BRS',3,3,3,4,1,3],
+        ['C','B'  ,3,2,1,5,0,5]
+    ],
+    # identity_br empty
+    [
+
+    ],
+    # 分支汇合后操作
+    [
+        ['AD',False] #有条件的相加
+    ]
+    
+]
\ No newline at end of file

ykl/ALL-cifar100/extract_ratio.py

+import sys
+import os
+
+
+# 从get_param.py输出重定向文件val.txt中提取参数量和计算量
+def extract_ratio(model_name):
+    fr = open('param_flops/'+model_name+'.txt','r')
+    lines = fr.readlines()
+    
+    #跳过warning
+    for i in range(len(lines)):
+        if 'Model' in lines[i]:
+            head = i+1
+            break 
+    Mac = lines[head].split('Mac,')[0].split(',')[-1]
+    if 'M' in Mac:
+        Mac = Mac.split('M')[0]
+        Mac = float(Mac)
+    elif 'G' in Mac:
+        Mac = Mac.split('G')[0]
+        Mac = float(Mac)
+        Mac *= 1024
+    
+    Param = lines[head].split(',')[0]
+    if 'M' in Param:
+        Param = Param.split('M')[0]
+        Param = float(Param)
+    elif 'k' in Param:
+        Param = Param.split('k')[0]
+        Param = float(Param)
+        Param /= 1024
+    
+    layer = []
+    par_ratio = []
+    flop_ratio = []
+    for line in lines:
+        if '(' in line and ')' in line:
+            layer.append(line.split(')')[0].split('(')[1])
+            r1 = line.split('%')[0].split(',')[-1]
+            r1 = float(r1)
+            par_ratio.append(r1)
+            r2 = line.split('%')[-2].split(',')[-1]
+            r2 = float(r2)
+            flop_ratio.append(r2)
+
+    return Mac, Param, layer, par_ratio, flop_ratio
+
+
+if __name__ == "__main__":
+    Mac, Param, layer, par_ratio, flop_ratio = extract_ratio('Inception_BN')
+    print(Mac)
+    print(Param)
+    print(layer)
+    print(par_ratio)
+    print(flop_ratio)
\ No newline at end of file

ykl/ALL-cifar100/function.py

+from torch.autograd import Function
+
+
+class FakeQuantize(Function):
+
+    @staticmethod
+    def forward(ctx, x, qparam):
+        x = qparam.quantize_tensor(x)
+        x = qparam.dequantize_tensor(x)
+        return x
+
+    @staticmethod
+    def backward(ctx, grad_output):
+        return grad_output, None
\ No newline at end of file

ykl/ALL-cifar100/get_param_flops.py

+from model import *
+
+import sys
+import torch
+from ptflops import get_model_complexity_info
+
+
+if __name__ == "__main__":
+    model_name = sys.argv[1]
+    model = Model(model_name)
+
+    # full_file = 'ckpt/cifar10_'+model_name+'.pt'
+    # model.load_state_dict(torch.load(full_file))
+
+    flops, params = get_model_complexity_info(model, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
+

ykl/ALL-cifar100/get_param_flops.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#SBATCH -J ALL              # The job name
+#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
+#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
+
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 0-01:30:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-debug             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000|T4"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+name_list="AlexNet AlexNet_BN VGG_16 VGG_19 Inception_BN ResNet_18 ResNet_50 ResNet_152 MobileNetV2"
+# name_list="MobileNetV2"
+for name in $name_list; do 
+	if [ -f "param_flops/$name.txt" ];then
+		echo "$name: param_flops exists"
+	# elif [ ! -f "ckpt/cifar10_$name.pt" ];then
+	# 	echo "$name: ckpt not exists"
+	else
+		python get_param_flops.py $name > param_flops/$name.txt
+	fi
+done
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL-cifar100/gol.py

+# -*- coding: utf-8 -*-
+
+# 用于多个module之间共享全局变量
+def _init():  # 初始化
+    global _global_dict
+    _global_dict = {}
+ 
+def set_value(value,is_bias=False):
+    # 定义一个全局变量
+    if is_bias:
+        _global_dict[0] = value
+    else:
+        _global_dict[1] = value
+ 
+ 
+def get_value(is_bias=False): # 给bias独立于各变量外的精度
+    if is_bias:
+        return _global_dict[0]
+    else:
+        return _global_dict[1]  
+

ykl/ALL-cifar100/model.py

+import torch.nn as nn
+
+from cfg import *
+from module import *
+from model_deployment import *
+
+
+class Model(nn.Module):
+    def __init__(self,model_name):
+        super(Model, self).__init__()
+        self.cfg_table = model_cfg_table[model_name]
+        make_layers(self,self.cfg_table)
+        # # 参数初始化
+        # for m in self.modules():
+        #     if isinstance(m, nn.Conv2d):
+        #         nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
+        #     elif isinstance(m, nn.BatchNorm2d):
+        #         nn.init.constant_(m.weight, 1)
+        #         nn.init.constant_(m.bias, 0)
+        #     elif isinstance(m, nn.Linear):
+        #         nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
+
+    def forward(self,x):
+        x = model_forward(self,self.cfg_table,x)
+        return x
+
+    def quantize(self, quant_type, num_bits=8, e_bits=3):
+        model_quantize(self,self.cfg_table,quant_type,num_bits,e_bits)
+    
+    def quantize_forward(self,x):
+        return model_utils(self,self.cfg_table,func='forward',x=x)
+
+    def freeze(self):
+        model_utils(self,self.cfg_table,func='freeze')
+    
+    def quantize_inference(self,x):
+        return model_utils(self,self.cfg_table,func='inference',x=x)
+
+    def fakefreeze(self):
+        model_utils(self,self.cfg_table,func='fakefreeze')
+
+# if __name__ == "__main__":
+#     model = Inception_BN()
+#     model.quantize('INT',8,3)
+#     print(model.named_modules)
+#     print('-------')
+#     print(model.named_parameters)
+#     print(len(model.conv0.named_parameters()))
\ No newline at end of file

ykl/ALL-cifar100/model_deployment.py

+import torch.nn as nn
+import torch.nn.functional as F
+from cfg import *
+from module import *
+
+def make_layers(model,cfg_table):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            make_inc_layers(model,cfg[1])
+        elif cfg[0] == 'ML':
+            make_ml_layers(model,cfg[1],cfg[2],cfg[3])
+        elif cfg[0] == 'C':
+            name = 'conv%d'%i
+            layer = nn.Conv2d(cfg[3],cfg[4],kernel_size=cfg[5],stride=cfg[6],padding=cfg[7],bias=cfg[8])
+            model.add_module(name,layer)
+            if 'B' in cfg[1]:
+                name = 'bn%d'%i
+                layer = nn.BatchNorm2d(cfg[4])
+                model.add_module(name,layer)
+                if 'RL' in cfg[1]:
+                    name = 'relu%d'%i
+                    layer = nn.ReLU(True)
+                    model.add_module(name,layer)
+                elif 'RS' in cfg[1]:
+                    name = 'relus%d'%i
+                    layer = nn.ReLU6(True)
+                    model.add_module(name,layer)
+        elif cfg[0] == 'RL':
+            name = 'relu%d'%i
+            layer = nn.ReLU(True)
+            model.add_module(name,layer)
+        elif cfg[0] == 'RS':
+            name = 'relus%d'%i
+            layer = nn.ReLU6(True)
+            model.add_module(name,layer)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            layer = nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
+            model.add_module(name,layer)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            layer = nn.AdaptiveAvgPool2d(cfg[1])
+            model.add_module(name,layer)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i
+            layer = nn.Linear(cfg[1],cfg[2],bias=cfg[3])
+            model.add_module(name,layer)
+        elif cfg[0] == 'D':
+            name = 'drop%d'%i
+            layer = nn.Dropout(cfg[1])
+            model.add_module(name,layer)
+        
+
+def model_forward(model,cfg_table,x):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            x = inc_forward(model,cfg[1],x)
+        elif cfg[0] == 'ML':
+            x = ml_forward(model,cfg[1],cfg[2],cfg[3],x)
+        elif cfg[0] == 'C':
+            name = 'conv%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+            if 'B' in cfg[1]:
+                name = 'bn%d'%i
+                layer = getattr(model,name)
+                x = layer(x)
+                if 'RL' in cfg[1]:
+                    name = 'relu%d'%i
+                    layer = getattr(model,name)
+                    x = layer(x)
+                elif 'RS' in cfg[1]:
+                    name = 'relus%d'%i
+                    layer = getattr(model,name)
+                    x = layer(x)
+        elif cfg[0] == 'RL':
+            name = 'relu%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'RS':
+            name = 'relus%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'D':
+            name = 'drop%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'VW':
+            if len(cfg) == 1: #default
+                x = x.view(x.size(0),-1)
+        elif cfg[0] == 'SM':
+            x = F.softmax(x,dim=1)
+    return x
+
+
+def model_quantize(model,cfg_table,quant_type,num_bits,e_bits):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            inc_quantize(model,cfg[1],quant_type,num_bits,e_bits)
+        elif cfg[0] == 'ML':
+            ml_quantize(model,cfg[1],cfg[2],cfg[3],quant_type,num_bits,e_bits)
+        elif cfg[0] == 'C':
+            conv_name = 'conv%d'%i
+            conv_layer = getattr(model,conv_name)
+            qname = 'q_'+conv_name
+            if 'B' in cfg[1]:
+                bn_name = 'bn%d'%i
+                bn_layer = getattr(model,bn_name)
+                if 'RL' in cfg[1]:
+                    qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+                elif 'RS' in cfg[1]:
+                    qlayer = QConvBNReLU6(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+                else:
+                    qlayer = QConvBN(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+            else:
+                qlayer = QConv2d(quant_type,conv_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'RL':
+            name = 'relu%d'%i
+            qname = 'q_'+name
+            qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'RS':
+            name = 'relus%d'%i
+            qname = 'q_'+name
+            qlayer = QReLU6(quant_type,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            qname = 'q_'+name
+            qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            qname = 'q_'+name
+            qlayer = QAdaptiveAvgPool2d(quant_type,output_size=cfg[1],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i 
+            layer = getattr(model,name)
+            qname = 'q_'+name
+            qlayer = QLinear(quant_type,layer,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+
+
+# 增加了func='fakefreeze'
+def model_utils(model,cfg_table,func,x=None):
+    last_qo = None
+    # 表示已经经过反量化，用于区别反量化不再最后，而是在softmax前的情形
+    done_flag = False
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            x,last_qo = inc_utils(model,cfg[1],func,x,last_qo)
+        elif cfg[0] == 'ML':
+            x,last_qo = ml_utils(model,cfg[1],cfg[2],cfg[3],func,x,last_qo)
+        elif cfg[0] == 'C':
+            qname = 'q_conv%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                # cfg[2]为True表示起始层，需要量化
+                if cfg[2]:
+                    x = qlayer.qi.quantize_tensor(x)
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
+            last_qo = qlayer.qo
+        elif cfg[0] == 'RL':
+            qname = 'q_relu%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'RS':
+            qname = 'q_relus%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'MP':
+            qname = 'q_pool%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'AAP':
+            qname = 'q_aap%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            last_qo = qlayer.qo
+        elif cfg[0] == 'FC':
+            qname = 'q_fc%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
+            last_qo = qlayer.qo
+        elif cfg[0] == 'D':
+            if func == 'forward':
+                name = 'drop%d'%i
+                layer = getattr(model,name)
+                x = layer(x)
+        elif cfg[0] == 'VW':
+            if func == 'inference' or func == 'forward':
+                if len(cfg) == 1: #default
+                    x = x.view(x.size(0),-1)
+        elif cfg[0] == 'SM':
+            if func == 'inference':
+                done_flag = True
+                x = last_qo.dequantize_tensor(x)
+                x = F.softmax(x,dim=1)
+            elif func == 'forward':
+                x = F.softmax(x,dim=1)
+
+    if func == 'inference' and not done_flag:
+        x = last_qo.dequantize_tensor(x)
+
+    return x
+
+
+def make_inc_layers(model,inc_idx):
+    inc_name = 'inc%d'%inc_idx
+    ch = inc_ch_table[inc_idx]
+    for i in range(4): # branch
+        prefix = inc_name+'_br%d_'%i
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                layer =nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
+                model.add_module(name,layer)
+            elif cfg[0] == 'RL':
+                name=prefix+'relu%d'%j
+                layer=nn.ReLU(True)
+                model.add_module(name,layer)
+            elif cfg[0] == 'C': # 'BRL' default
+                name=prefix+'conv%d'%j
+                layer=nn.Conv2d(ch[cfg[1]],ch[cfg[2]],kernel_size=cfg[3],stride=cfg[4],padding=cfg[5],bias=False)
+                model.add_module(name,layer)
+                name=prefix+'bn%d'%j
+                layer=nn.BatchNorm2d(ch[cfg[2]])
+                model.add_module(name,layer)
+                name=prefix+'relu%d'%j
+                layer=nn.ReLU(True)
+                model.add_module(name,layer)
+
+
+def inc_forward(model,inc_idx,x):
+    inc_name = 'inc%d'%inc_idx
+    outs = []
+    for i in range(4):
+        prefix = inc_name+'_br%d_'%i
+        tmp = x
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+            elif cfg[0] == 'RL':
+                name=prefix+'relu%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+            elif cfg[0] == 'C': # 'BRL' default
+                name=prefix+'conv%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                name=prefix+'bn%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                name=prefix+'relu%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+        outs.append(tmp)
+    
+    out = torch.cat(outs,1)
+    return out
+
+
+def inc_quantize(model,inc_idx,quant_type,num_bits,e_bits):
+    inc_name = 'inc%d'%inc_idx
+    for i in range(4):
+        prefix = inc_name+'_br%d_'%i
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                qname = 'q_'+name
+                qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
+                model.add_module(qname,qlayer)
+            elif cfg[0] == 'RL':
+                name = prefix+'relu%d'%j
+                qname = 'q_'+name
+                qlayer = QReLU(quant_type, num_bits=num_bits, e_bits=e_bits)
+                model.add_module(qname,qlayer)
+            elif cfg[0] == 'C': # 'BRL' default
+                conv_name=prefix+'conv%d'%j
+                conv_layer=getattr(model,conv_name)
+                bn_name=prefix+'bn%d'%j
+                bn_layer=getattr(model,bn_name)
+                qname='q_'+conv_name
+                qlayer=QConvBNReLU(quant_type, conv_layer, bn_layer, num_bits=num_bits, e_bits=e_bits)
+                model.add_module(qname,qlayer)
+    qname = 'q_'+inc_name+'_concat'
+    qlayer = QConcat(quant_type,4,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
+    model.add_module(qname,qlayer)
+    
+
+def inc_utils(model,inc_idx,func,x=None,qo=None):
+    inc_name = 'inc%d'%inc_idx
+    outs=[]
+    qos=[]
+    for i in range(4): 
+        qprefix = 'q_'+inc_name+'_br%d_'%i
+        tmp = x
+        last_qo = qo
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                qname = qprefix+'pool%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+            elif cfg[0] == 'RL':
+                qname = qprefix+'relu%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+            elif cfg[0] == 'C': # 'BRL' default
+                qname = qprefix+'conv%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
+                last_qo = qlayer.qo
+        
+        outs.append(tmp)
+        qos.append(last_qo)
+
+    qname = 'q_'+inc_name+'_concat'
+    qlayer = getattr(model,qname)
+    out = None
+    if func == 'forward':
+        out = qlayer(outs)
+    elif func == 'inference':
+        out = qlayer.quantize_inference(outs)
+    elif func == 'freeze':
+        qlayer.freeze(qos)
+    last_qo = qlayer.qo
+    return out,last_qo
+
+
+
+def make_ml_layers(model,blk_type,ml_idx,blocks):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("Make_ml_layers: Illegal blk_type")
+    
+    #一个makelayer对应两行，分别表示第一个blk和其余的特征
+    make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0)
+    for i in range(1,blocks):
+        make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i)
+
+
+# ma表示主分支，ds表示downsample
+# 当cfgtable含有三个元素时，第3个表示分支合并后需经过的层。
+# BasicBlock和BottleNeck合并后需经过relu层，InvertedResidual合并后无需经过relu层
+def make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    for i in range(2):
+        if i == 0:
+            prefix = blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                prefix = blk_name+'_ds_'
+            else:
+                continue
+        
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                name = prefix+'conv%d'%j
+                layer = nn.Conv2d(ch[cfg[2]],ch[cfg[3]],kernel_size=cfg[4],stride=ch[cfg[5]],padding=cfg[6],groups=ch[cfg[7]])
+                model.add_module(name,layer)
+                if 'B' in cfg[1]:
+                    name = prefix+'bn%d'%j
+                    layer=nn.BatchNorm2d(ch[cfg[3]])
+                    model.add_module(name,layer)
+                    if 'RL' in cfg[1]:
+                        name = prefix+'relu%d'%j
+                        layer = nn.ReLU(True)
+                        model.add_module(name,layer)
+                    elif 'RS' in cfg[1]:
+                        name = prefix+'relus%d'%j
+                        layer = nn.ReLU6(True)
+                        model.add_module(name,layer)
+
+    #分支汇总
+    prefix = blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'RL': #当前没有blk出现汇总处有RS
+            name = prefix+'relu%d'%j
+            layer = nn.ReLU(True)
+            model.add_module(name,layer)
+
+
+def ml_forward(model,blk_type,ml_idx,blocks,x):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("ml_forward: Illegal blk_type")
+    
+    x = blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,x)
+    for i in range(1,blocks):
+        x = blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,x)
+
+    return x
+
+
+def blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,x):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    outs = []
+    for i in range(2):
+        tmp=x
+        if i == 0:
+            prefix = blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                prefix = blk_name+'_ds_'
+            else:
+                outs.append(tmp)
+                continue
+       
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                name = prefix+'conv%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                if 'B' in cfg[1]:
+                    name = prefix+'bn%d'%j
+                    layer = getattr(model,name)
+                    tmp = layer(tmp)
+                    if 'RL' in cfg[1]:
+                        name = prefix+'relu%d'%j
+                        layer = getattr(model,name)
+                        tmp = layer(tmp)
+                    elif 'RS' in cfg[1]:
+                        name = prefix+'relus%d'%j
+                        layer = getattr(model,name)
+                        tmp = layer(tmp)
+        
+        outs.append(tmp)
+    
+    #分支汇总
+    prefix = blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'AD':
+            if cfg[1] or ch[0]: #无条件加或flag为true
+                out = outs[0] + outs[1]
+            else:
+                out = outs[0]
+        elif cfg[0] == 'RL':
+            name = prefix+'relu%d'%j
+            layer = getattr(model,name)
+            out = layer(out)
+
+    return out
+
+
+def ml_quantize(model,blk_type,ml_idx,blocks,quant_type,num_bits,e_bits):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("ml_quantize: Illegal blk_type")
+    
+    blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,quant_type,num_bits,e_bits)
+    for i in range(1,blocks):
+        blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,quant_type,num_bits,e_bits)
+
+
+def blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,quant_type,num_bits,e_bits):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    for i in range(2):
+        if i == 0:
+            prefix = blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                prefix = blk_name+'_ds_'
+            else:
+                continue
+
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                conv_name = prefix+'conv%d'%j
+                conv_layer = getattr(model,conv_name)
+                qname = 'q_'+conv_name
+                if 'B' in cfg[1]:
+                    bn_name = prefix+'bn%d'%j
+                    bn_layer = getattr(model,bn_name)
+                    if 'RL' in cfg[1]:
+                        qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
+                    elif 'RS' in cfg[1]:
+                        qlayer = QConvBNReLU6(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
+                    else:
+                        qlayer = QConvBN(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
+                else:
+                    qlayer = QConv2d(quant_type,conv_layer,num_bits=num_bits,e_bits=e_bits)
+                model.add_module(qname,qlayer)
+
+    #分支汇总
+    prefix = blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'AD':
+            if cfg[1] or ch[0]: #无条件加或flag为true
+                qname = 'q_'+prefix+'add%d'%j
+                qlayer = QElementwiseAdd(quant_type,2,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
+                model.add_module(qname,qlayer)
+        elif cfg[0] == 'RL':
+            qname = 'q_'+prefix+'relu%d'%j
+            qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+
+
+def ml_utils(model,blk_type,ml_idx,blocks,func,x=None,qo=None):
+    ml_name = 'ml%d'%ml_idx
+    if blk_type == 'BBLK':
+        blk_ch_table = bblk_ch_table
+        blk_cfg_table = bblk_cfg_table
+    elif blk_type == 'BTNK':
+        blk_ch_table = btnk_ch_table
+        blk_cfg_table = btnk_cfg_table
+    elif blk_type == 'IRES':
+        blk_ch_table = ires_ch_table
+        blk_cfg_table = ires_cfg_table
+    else:
+        raise ValueError("ml_quantize: Illegal blk_type")
+
+    last_qo = qo
+    x,last_qo = blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,func,x,last_qo)
+    for i in range(1,blocks):
+        x,last_qo = blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,func,x,last_qo)
+    
+    return x, last_qo
+
+def blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,func,x=None,qo=None):
+    blk_name = ml_name+'_blk%d'%blk_idx
+    ch = blk_ch_table[ch_idx]
+    outs = []
+    qos = []
+    for i in range(2):
+        tmp=x
+        last_qo = qo
+        if i == 0:
+            qprefix = 'q_'+blk_name+'_ma_'
+        elif i == 1: 
+            if ch[0]: #downsample/identity_flag
+                qprefix = 'q_'+blk_name+'_ds_'
+            else:
+                outs.append(tmp)
+                qos.append(last_qo)
+                continue
+       
+        for j in range(len(blk_cfg_table[i])):
+            cfg = blk_cfg_table[i][j]
+            if cfg[0] == 'C':
+                qname = qprefix+'conv%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
+                last_qo = qlayer.qo
+        
+        outs.append(tmp)
+        qos.append(last_qo)
+    
+    #分支汇总
+    qprefix = 'q_'+blk_name+'_'
+    for j in range(len(blk_cfg_table[-1])):
+        cfg = blk_cfg_table[-1][j]
+        if cfg[0] == 'AD':
+            if cfg[1] or ch[0]: #无条件加或flag为true
+                qname = qprefix+'add%d'%j
+                qlayer = getattr(model,qname)
+                out = None
+                if func == 'forward':
+                    out = qlayer(outs)
+                elif func == 'inference':
+                    out = qlayer.quantize_inference(outs)
+                elif func == 'freeze':
+                    qlayer.freeze(qos)
+                last_qo = qlayer.qo
+            else:
+                out = outs[0]
+                last_qo = qos[0]
+        elif cfg[0] == 'RL':
+            qname = qprefix+'relu%d'%j
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                out = qlayer(out)
+            elif func == 'inference':
+                out = qlayer.quantize_inference(out)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+
+    return out,last_qo
\ No newline at end of file

ykl/ALL-cifar100/module.py

+import math
+import numpy as np
+import gol
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from torch.autograd import Variable
+
+from function import FakeQuantize
+
+
+def js_div(p_output, q_output, get_softmax=True):
+    """
+    Function that measures JS divergence between target and output logits:
+    """
+    KLDivLoss = nn.KLDivLoss(reduction='sum')
+    if get_softmax:
+        p_output = F.softmax(p_output)
+        q_output = F.softmax(q_output)
+    log_mean_output = ((p_output + q_output)/2).log()
+    return (KLDivLoss(log_mean_output, p_output) + KLDivLoss(log_mean_output, q_output))/2
+
+
+# 获取最近的量化值
+def get_nearest_val(quant_type, x, is_bias=False, block_size=1000000):
+    if quant_type == 'INT':
+        return x.round_()
+
+    plist = gol.get_value(is_bias)
+    shape = x.shape
+    xhard = x.view(-1)
+    xout = torch.zeros_like(xhard)
+
+    plist = plist.type_as(x)
+    n_blocks = (x.numel() + block_size - 1) // block_size
+
+    for i in range(n_blocks):
+        start_idx = i * block_size
+        end_idx = min(start_idx + block_size, xhard.numel())
+        block_size_i = end_idx - start_idx
+        xblock = xhard[start_idx:end_idx]
+        plist_block = plist.unsqueeze(1) 
+        idx = (xblock.unsqueeze(0) - plist_block).abs().min(dim=0)[1]
+        xhard_block = plist[idx].view(xblock.shape)
+        xout[start_idx:end_idx] = (xhard_block - xblock).detach() + xblock
+
+    xout = xout.view(shape)
+    return xout
+
+
+# 采用对称有符号量化时，获取量化范围最大值
+def get_qmax(quant_type,num_bits=None, e_bits=None):
+    if quant_type == 'INT':
+        qmax = 2. ** (num_bits - 1) - 1
+    elif quant_type == 'POT':
+        qmax = 1
+    else: #FLOAT
+        m_bits = num_bits - 1 - e_bits
+        dist_m = 2 ** (-m_bits)
+        e = 2 ** (e_bits - 1)
+        expo = 2 ** e
+        m = 2 ** m_bits -1
+        frac = 1. + m * dist_m
+        qmax = frac * expo
+    return qmax
+
+# 都采用有符号量化，zeropoint都置为0
+def calcScaleZeroPoint(min_val, max_val, qmax):
+    scale = torch.max(max_val.abs(),min_val.abs()) / qmax
+    zero_point = torch.tensor(0.)
+    return scale, zero_point
+
+# 将输入进行量化，输入输出都为tensor
+def quantize_tensor(quant_type, x, scale, zero_point, qmax, is_bias=False):
+    # 量化后范围，直接根据位宽确定
+    qmin = -qmax
+    q_x = zero_point + x / scale
+
+    q_x.clamp_(qmin, qmax)
+    q_x = get_nearest_val(quant_type, q_x, is_bias)
+    return q_x
+
+# bias使用不同精度，需要根据量化类型指定num_bits/e_bits
+def bias_qmax(quant_type):
+    if quant_type == 'INT':
+        return get_qmax(quant_type, 64)
+    elif quant_type == 'POT':
+        return get_qmax(quant_type)
+    else:
+        return get_qmax(quant_type, 16, 7)
+        
+
+# 转化为FP32，不需再做限制
+def dequantize_tensor(q_x, scale, zero_point):
+    return scale * (q_x - zero_point)
+
+
+class QParam(nn.Module):
+
+    def __init__(self,quant_type, num_bits=8, e_bits=3):
+        super(QParam, self).__init__()
+        self.quant_type = quant_type
+        self.num_bits = num_bits
+        self.e_bits = e_bits
+        self.qmax = get_qmax(quant_type, num_bits, e_bits)
+
+        scale = torch.tensor([], requires_grad=False)
+        zero_point = torch.tensor([], requires_grad=False)
+        min = torch.tensor([], requires_grad=False)
+        max = torch.tensor([], requires_grad=False)
+        # 通过注册为register，使得buffer可以被记录到state_dict
+        self.register_buffer('scale', scale)
+        self.register_buffer('zero_point', zero_point)
+        self.register_buffer('min', min)
+        self.register_buffer('max', max)
+
+    # 更新统计范围及量化参数
+    def update(self, tensor):
+        if self.max.nelement() == 0 or self.max.data < tensor.max().data:
+            self.max.data = tensor.max().data
+        self.max.clamp_(min=0)
+
+        if self.min.nelement() == 0 or self.min.data > tensor.min().data:
+            self.min.data = tensor.min().data
+        self.min.clamp_(max=0)
+
+        self.scale, self.zero_point = calcScaleZeroPoint(self.min, self.max, self.qmax)
+
+    def quantize_tensor(self, tensor):
+        return quantize_tensor(self.quant_type, tensor, self.scale, self.zero_point, self.qmax)
+
+    def dequantize_tensor(self, q_x):
+        return dequantize_tensor(q_x, self.scale, self.zero_point)
+
+    # 该方法保证了可以从state_dict里恢复
+    def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys,
+                              error_msgs):
+        key_names = ['scale', 'zero_point', 'min', 'max']
+        for key in key_names:
+            value = getattr(self, key)
+            value.data = state_dict[prefix + key].data
+            state_dict.pop(prefix + key)
+
+    # 该方法返回值将是打印该对象的结果
+    def __str__(self):
+        info = 'scale: %.10f ' % self.scale
+        info += 'zp: %.6f ' % self.zero_point
+        info += 'min: %.6f ' % self.min
+        info += 'max: %.6f' % self.max
+        return info
+
+
+# 作为具体量化层的父类，qi和qo分别为量化输入/输出
+class QModule(nn.Module):
+
+    def __init__(self,quant_type, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QModule, self).__init__()
+        if qi:
+            self.qi = QParam(quant_type,num_bits, e_bits)
+        if qo:
+            self.qo = QParam(quant_type,num_bits, e_bits)
+        self.quant_type = quant_type
+        self.num_bits = num_bits
+        self.e_bits = e_bits
+        self.bias_qmax = bias_qmax(quant_type)
+
+    def freeze(self):
+        pass  # 空语句
+
+    def quantize_inference(self, x):
+        raise NotImplementedError('quantize_inference should be implemented.')
+
+    def fakefreeze(self):
+        pass
+
+"""
+QModule  量化卷积
+
+:quant_type: 量化类型
+:conv_module: 卷积模块
+:qi: 是否量化输入特征图
+:qo: 是否量化输出特征图
+:num_bits: 8位bit数
+"""
+
+
+class QConv2d(QModule):
+
+    def __init__(self, quant_type, conv_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConv2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    # freeze方法可以固定真量化的权重参数，并将该值更新到原全精度层上，便于散度计算
+    def freeze(self, qi=None, qo=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        # 这里因为在池化或者激活的输入，不需要对最大值和最小是进行额外的统计，会共享相同的输出
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        # 根据https://zhuanlan.zhihu.com/p/156835141, 这是式3 的系数
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(self.conv_module.weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+
+        #考虑conv层无bias,此时forward和inference传入none亦可
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                        self.conv_module.bias.data, scale=self.qi.scale * self.qw.scale,
+                                                        zero_point=0.,qmax=self.bias_qmax, is_bias=True)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):  # 前向传播,输入张量,x为浮点型数据
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)  # 对输入张量X完成量化
+
+        # foward前更新qw，保证量化weight时候scale正确
+        self.qw.update(self.conv_module.weight.data)
+        # 注意:此处主要为了统计各层x和weight范围，未对bias进行量化操作
+        tmp_wgt = FakeQuantize.apply(self.conv_module.weight, self.qw)
+        x = F.conv2d(x, tmp_wgt, self.conv_module.bias,
+                     stride=self.conv_module.stride,
+                     padding=self.conv_module.padding, 
+                     dilation=self.conv_module.dilation,
+                     groups=self.conv_module.groups)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+
+    # 利用公式 q_a = M(\sigma(q_w-Z_w)(q_x-Z_x) + q_b)
+    def quantize_inference(self, x):  # 此处input为已经量化的qx
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+
+        x = get_nearest_val(self.quant_type,x)
+        x = x + self.qo.zero_point
+        return x
+
+
+class QLinear(QModule):
+
+    def __init__(self, quant_type, fc_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QLinear, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.fc_module = fc_module
+        self.qw = QParam(quant_type, num_bits, e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def freeze(self, qi=None, qo=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+        
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        self.fc_module.weight.data = self.qw.quantize_tensor(self.fc_module.weight.data)
+        self.fc_module.weight.data = self.fc_module.weight.data - self.qw.zero_point
+        if self.fc_module.bias is not None:
+            self.fc_module.bias.data = quantize_tensor(self.quant_type,
+                                                        self.fc_module.bias.data, scale=self.qi.scale * self.qw.scale,
+                                                        zero_point=0., qmax=self.bias_qmax, is_bias=True)
+
+    def fakefreeze(self):
+        self.fc_module.weight.data = self.qw.dequantize_tensor(self.fc_module.weight.data)
+        if self.fc_module.bias is not None:
+            self.fc_module.bias.data = dequantize_tensor(self.fc_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        self.qw.update(self.fc_module.weight.data)
+        tmp_wgt = FakeQuantize.apply(self.fc_module.weight, self.qw)
+        x = F.linear(x, tmp_wgt, self.fc_module.bias)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = self.fc_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x)
+        x = x + self.qo.zero_point
+
+        return x
+
+
+class QReLU(QModule):
+
+    def __init__(self,quant_type, qi=False, qo=False, num_bits=8, e_bits=3):
+        super(QReLU, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+
+    def freeze(self, qi=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.relu(x)
+
+        return x
+
+    def quantize_inference(self, x):
+        x = x.clone()
+        x[x < self.qi.zero_point] = self.qi.zero_point
+        return x
+
+class QReLU6(QModule):
+
+    def __init__(self,quant_type, qi=False, qo=False, num_bits=8, e_bits=3):
+        super(QReLU6, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+
+    def freeze(self, qi=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.relu6(x)
+
+        return x
+
+    def quantize_inference(self, x):
+        x = x.clone()
+        upper = torch.tensor(6)
+        qupper = self.qi.quantize_tensor(upper)
+        x.clamp_(min=0,max=qupper.item())
+        return x
+
+
+class QMaxPooling2d(QModule):
+
+    def __init__(self, quant_type, kernel_size=3, stride=1, padding=0, qi=False, qo=False, num_bits=8,e_bits=3):
+        super(QMaxPooling2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.kernel_size = kernel_size
+        self.stride = stride
+        self.padding = padding
+        
+    def freeze(self, qi=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+        if qi is not None:
+            self.qi = qi
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.max_pool2d(x, self.kernel_size, self.stride, self.padding)
+
+        return x
+
+    def quantize_inference(self, x):
+        return F.max_pool2d(x, self.kernel_size, self.stride, self.padding)
+
+class QAdaptiveAvgPool2d(QModule):
+
+    def __init__(self, quant_type, output_size, qi=False, qo=True, num_bits=8,e_bits=3):
+        super(QAdaptiveAvgPool2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.output_size = output_size
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+        
+        self.M.data = (self.qi.scale / self.qo.scale).data
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.adaptive_avg_pool2d(x, self.output_size)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+        return x
+
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = F.adaptive_avg_pool2d(x, self.output_size)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x)
+        x = x+self.qo.zero_point
+        return x
+    
+class QConvBN(QModule):
+
+    def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConvBN, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.bn_module = bn_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def fold_bn(self, mean, std):
+        if self.bn_module.affine:
+            gamma_ = self.bn_module.weight / std
+            weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
+            else:
+                bias = self.bn_module.bias - gamma_ * mean
+        else:
+            gamma_ = 1 / std
+            weight = self.conv_module.weight * gamma_
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean
+            else:
+                bias = -gamma_ * mean
+            
+        return weight, bias
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(self.bn_module.running_mean, std)
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+        
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                        bias.data, scale=self.qi.scale * self.qw.scale,
+                                                        zero_point=0., qmax=self.bias_qmax,is_bias=True)
+        else:
+            bias = quantize_tensor(self.quant_type,
+                                    bias, scale=self.qi.scale * self.qw.scale,
+                                    zero_point=0., qmax=self.bias_qmax,is_bias=True)
+            self.conv_module.bias = torch.nn.Parameter(bias)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        if self.training:
+            y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias, 
+                            stride=self.conv_module.stride,
+                            padding=self.conv_module.padding,
+                            dilation=self.conv_module.dilation,
+                            groups=self.conv_module.groups)
+            y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
+            y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
+            # mean = y.mean(1)
+            # var = y.var(1)
+            mean = y.mean(1).detach()
+            var = y.var(1).detach()
+            self.bn_module.running_mean = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_mean + \
+                self.bn_module.momentum * mean
+            self.bn_module.running_var = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_var + \
+                self.bn_module.momentum * var
+        else:
+            mean = Variable(self.bn_module.running_mean)
+            var = Variable(self.bn_module.running_var)
+
+        std = torch.sqrt(var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(mean, std)
+
+        self.qw.update(weight.data)
+
+        x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
+                stride=self.conv_module.stride,
+                padding=self.conv_module.padding, 
+                dilation=self.conv_module.dilation, 
+                groups=self.conv_module.groups)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+    
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x) 
+        x = x + self.qo.zero_point        
+        return x
+
+
+class QConvBNReLU(QModule):
+
+    def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConvBNReLU, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.bn_module = bn_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def fold_bn(self, mean, std):
+        if self.bn_module.affine:
+            gamma_ = self.bn_module.weight / std
+            weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
+            else:
+                bias = self.bn_module.bias - gamma_ * mean
+        else:
+            gamma_ = 1 / std
+            weight = self.conv_module.weight * gamma_
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean
+            else:
+                bias = -gamma_ * mean
+            
+        return weight, bias
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(self.bn_module.running_mean, std)
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                            bias.data, scale=self.qi.scale * self.qw.scale,
+                                                            zero_point=0., qmax=self.bias_qmax,is_bias=True)
+        else:
+            bias = quantize_tensor(self.quant_type,
+                                    bias, scale=self.qi.scale * self.qw.scale,
+                                    zero_point=0., qmax=self.bias_qmax,is_bias=True)
+            self.conv_module.bias = torch.nn.Parameter(bias)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        if self.training:
+            y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias, 
+                            stride=self.conv_module.stride,
+                            padding=self.conv_module.padding,
+                            dilation=self.conv_module.dilation,
+                            groups=self.conv_module.groups)
+            y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
+            y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
+            # mean = y.mean(1)
+            # var = y.var(1)
+            mean = y.mean(1).detach()
+            var = y.var(1).detach()
+            self.bn_module.running_mean = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_mean + \
+                self.bn_module.momentum * mean
+            self.bn_module.running_var = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_var + \
+                self.bn_module.momentum * var
+        else:
+            mean = Variable(self.bn_module.running_mean)
+            var = Variable(self.bn_module.running_var)
+
+        std = torch.sqrt(var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(mean, std)
+
+        self.qw.update(weight.data)
+
+        x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
+                stride=self.conv_module.stride,
+                padding=self.conv_module.padding, 
+                dilation=self.conv_module.dilation, 
+                groups=self.conv_module.groups)
+
+        x = F.relu(x)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+    
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x) 
+        x = x + self.qo.zero_point        
+        x.clamp_(min=0)
+        return x
+
+class QConvBNReLU6(QModule):
+
+    def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConvBNReLU6, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.bn_module = bn_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def fold_bn(self, mean, std):
+        if self.bn_module.affine:
+            gamma_ = self.bn_module.weight / std
+            weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
+            else:
+                bias = self.bn_module.bias - gamma_ * mean
+        else:
+            gamma_ = 1 / std
+            weight = self.conv_module.weight * gamma_
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean
+            else:
+                bias = -gamma_ * mean
+            
+        return weight, bias
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(self.bn_module.running_mean, std)
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                            bias.data, scale=self.qi.scale * self.qw.scale,
+                                                            zero_point=0., qmax=self.bias_qmax,is_bias=True)
+        else:
+            bias = quantize_tensor(self.quant_type,
+                                    bias, scale=self.qi.scale * self.qw.scale,
+                                    zero_point=0., qmax=self.bias_qmax,is_bias=True)
+            self.conv_module.bias = torch.nn.Parameter(bias)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        if self.training:
+            y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias, 
+                            stride=self.conv_module.stride,
+                            padding=self.conv_module.padding,
+                            dilation=self.conv_module.dilation,
+                            groups=self.conv_module.groups)
+            y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
+            y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
+            # mean = y.mean(1)
+            # var = y.var(1)
+            mean = y.mean(1).detach()
+            var = y.var(1).detach()
+            self.bn_module.running_mean = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_mean + \
+                self.bn_module.momentum * mean
+            self.bn_module.running_var = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_var + \
+                self.bn_module.momentum * var
+        else:
+            mean = Variable(self.bn_module.running_mean)
+            var = Variable(self.bn_module.running_var)
+
+        std = torch.sqrt(var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(mean, std)
+
+        self.qw.update(weight.data)
+
+        x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
+                stride=self.conv_module.stride,
+                padding=self.conv_module.padding, 
+                dilation=self.conv_module.dilation, 
+                groups=self.conv_module.groups)
+
+        x = F.relu6(x)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+    
+    def quantize_inference(self, x):
+        upper = torch.tensor(6)
+        qupper = self.qo.quantize_tensor(upper)
+
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x) 
+        x = x + self.qo.zero_point        
+        x.clamp_(min=0,max=qupper.item())
+        return x
+    
+
+
+
+# 作为具体量化层的父类，qi和qo分别为量化输入/输出
+# 用于处理多个层结果或qo以array形式传入
+class QModule_array(nn.Module):
+    def __init__(self,quant_type,len,qi_array=False, qo=True, num_bits=8, e_bits=3):
+        super(QModule_array, self).__init__()
+        if qi_array:
+            for i in range(len):
+                self.add_module('qi%d'%i,QParam(quant_type,num_bits, e_bits))
+        if qo:
+            self.qo = QParam(quant_type,num_bits, e_bits)
+        self.quant_type = quant_type
+        self.num_bits = num_bits
+        self.e_bits = e_bits
+        self.bias_qmax = bias_qmax(quant_type)
+        self.len = len
+
+    def freeze(self):
+        pass  # 空语句
+
+    def quantize_inference(self, x):
+        raise NotImplementedError('quantize_inference should be implemented.')
+    
+
+class QElementwiseAdd(QModule_array):
+    def __init__(self, quant_type, len, qi_array=False, qo=True, num_bits=8, e_bits=3):
+        super(QElementwiseAdd,self).__init__(quant_type,len,qi_array,qo,num_bits,e_bits)
+        for i in range(len):
+            self.register_buffer('M%d'%i,torch.tensor([], requires_grad=False))
+
+    def freeze(self, qi_array=None, qo=None):
+        if qi_array is None:
+            raise ValueError('qi_array should be provided')
+        elif len(qi_array) != self.len:
+            raise ValueError('qi_array len no match')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+        
+        for i in range(self.len):
+            self.add_module('qi%d'%i,qi_array[i])
+        if qo is not None:
+            self.qo = qo
+
+        for i in range(self.len):
+            getattr(self,'M%d'%i).data = (getattr(self,'qi%d'%i).scale / self.qo.scale).data
+        
+    def forward(self,x_array):
+        outs=[]
+        for i in range(self.len):
+            x = x_array[i]
+            if hasattr(self,'qi%d'%i):
+                qi = getattr(self,'qi%d'%i)
+                qi.update(x)
+                x = FakeQuantize.apply(x,qi)
+            outs.append(x)
+        out = outs[0]+outs[1]
+        if hasattr(self,'qo'):
+            self.qo.update(x)
+            out = FakeQuantize.apply(out,self.qo)
+        return out
+
+    def quantize_inference(self, x_array):
+        outs=[]
+        for i in range(self.len):
+            qi = getattr(self,'qi%d'%i)
+            x = x_array[i] - qi.zero_point
+            x = getattr(self,'M%d'%i) * x
+            outs.append(x)
+        out = outs[0]+outs[1]
+        out = get_nearest_val(self.quant_type,out)
+        out = out + self.qo.zero_point
+        return out
+
+
+class QConcat(QModule_array):
+    def __init__(self, quant_type, len, qi_array=False, qo=True, num_bits=8, e_bits=3):
+        super(QConcat,self).__init__(quant_type, len, qi_array, qo, num_bits, e_bits)
+        for i in range(len):
+            self.register_buffer('M%d'%i,torch.tensor([], requires_grad=False))
+    
+    def freeze(self, qi_array=None, qo=None):
+        if qi_array is None:
+            raise ValueError('qi_array should be provided')
+        elif len(qi_array) != self.len:
+            raise ValueError('qi_array len no match')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+        
+        for i in range(self.len):
+            self.add_module('qi%d'%i,qi_array[i])
+        if qo is not None:
+            self.qo = qo
+
+        for i in range(self.len):
+            getattr(self,'M%d'%i).data = (getattr(self,'qi%d'%i).scale / self.qo.scale).data
+        
+    def forward(self,x_array):
+        outs=[]
+        for i in range(self.len):
+            x = x_array[i]
+            if hasattr(self,'qi%d'%i):
+                qi = getattr(self,'qi%d'%i)
+                qi.update(x)
+                x = FakeQuantize.apply(x,qi)
+            outs.append(x)
+        out = torch.cat(outs,1)
+        if hasattr(self,'qo'):
+            self.qo.update(x)
+            out = FakeQuantize.apply(out,self.qo)
+        return out
+
+    def quantize_inference(self, x_array):
+        outs=[]
+        for i in range(self.len):
+            qi = getattr(self,'qi%d'%i)
+            x = x_array[i] - qi.zero_point
+            x = getattr(self,'M%d'%i) * x
+            outs.append(x)
+        out = torch.concat(outs,1)
+        out = get_nearest_val(self.quant_type,out)
+        out = out + self.qo.zero_point
+        return out
+
+
+        
\ No newline at end of file

ykl/ALL-cifar100/param_flops/AlexNet.txt

+Model(
+  3.91 M, 100.000% Params, 70.13 MMac, 100.000% MACs, 
+  (conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.308% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
+  (pool2): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv3): Conv2d(18.5 k, 0.472% Params, 4.73 MMac, 6.752% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu4): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool5): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv6): Conv2d(73.86 k, 1.887% Params, 4.73 MMac, 6.740% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu7): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
+  (conv8): Conv2d(295.17 k, 7.540% Params, 18.89 MMac, 26.937% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (conv10): Conv2d(590.08 k, 15.073% Params, 37.77 MMac, 53.851% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu11): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool12): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (drop14): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc15): Linear(2.36 M, 60.290% Params, 2.36 MMac, 3.366% MACs, in_features=2304, out_features=1024, bias=True)
+  (relu16): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
+  (drop17): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc18): Linear(524.8 k, 13.405% Params, 524.8 KMac, 0.748% MACs, in_features=1024, out_features=512, bias=True)
+  (relu19): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
+  (fc20): Linear(51.3 k, 1.310% Params, 51.3 KMac, 0.073% MACs, in_features=512, out_features=100, bias=True)
+)

ykl/ALL-cifar100/param_flops/AlexNet_BN.txt

+Model(
+  3.92 M, 100.000% Params, 70.31 MMac, 100.000% MACs, 
+  (conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.305% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(64, 0.002% Params, 65.54 KMac, 0.093% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
+  (pool1): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv2): Conv2d(18.5 k, 0.472% Params, 4.73 MMac, 6.735% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn2): BatchNorm2d(128, 0.003% Params, 32.77 KMac, 0.047% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool3): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv4): Conv2d(73.86 k, 1.886% Params, 4.73 MMac, 6.723% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn4): BatchNorm2d(256, 0.007% Params, 16.38 KMac, 0.023% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu4): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
+  (conv5): Conv2d(295.17 k, 7.537% Params, 18.89 MMac, 26.868% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn5): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu5): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (conv6): Conv2d(590.08 k, 15.067% Params, 37.77 MMac, 53.713% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn6): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool7): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (drop9): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc10): Linear(2.36 M, 60.268% Params, 2.36 MMac, 3.357% MACs, in_features=2304, out_features=1024, bias=True)
+  (relu11): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
+  (drop12): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc13): Linear(524.8 k, 13.400% Params, 524.8 KMac, 0.746% MACs, in_features=1024, out_features=512, bias=True)
+  (relu14): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
+  (fc15): Linear(51.3 k, 1.310% Params, 51.3 KMac, 0.073% MACs, in_features=512, out_features=100, bias=True)
+)

ykl/ALL-cifar100/param_flops/Inception_BN.txt

+Model(
+  5.97 M, 100.000% Params, 1.46 GMac, 100.000% MACs, 
+  (conv0): Conv2d(1.79 k, 0.030% Params, 1.84 MMac, 0.125% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (conv2): Conv2d(36.93 k, 0.618% Params, 37.81 MMac, 2.583% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu3): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc0_br0_conv0): Conv2d(4.1 k, 0.069% Params, 4.19 MMac, 0.287% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br0_bn0): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc0_br1_conv0): Conv2d(6.14 k, 0.103% Params, 6.29 MMac, 0.430% MACs, 64, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br1_bn0): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br1_relu0): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
+  (inc0_br1_conv1): Conv2d(110.59 k, 1.852% Params, 113.25 MMac, 7.736% MACs, 96, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc0_br1_bn1): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc0_br2_conv0): Conv2d(1.02 k, 0.017% Params, 1.05 MMac, 0.072% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br2_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.002% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br2_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc0_br2_conv1): Conv2d(12.8 k, 0.214% Params, 13.11 MMac, 0.895% MACs, 16, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc0_br2_bn1): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc0_br3_pool0): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc0_br3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc0_br3_conv2): Conv2d(2.05 k, 0.034% Params, 2.1 MMac, 0.143% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br3_bn2): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc1_br0_conv0): Conv2d(32.77 k, 0.549% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br0_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br0_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc1_br1_conv0): Conv2d(32.77 k, 0.549% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br1_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br1_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc1_br1_conv1): Conv2d(221.18 k, 3.704% Params, 226.49 MMac, 15.473% MACs, 128, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc1_br1_bn1): BatchNorm2d(384, 0.006% Params, 393.22 KMac, 0.027% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br1_relu1): ReLU(0, 0.000% Params, 196.61 KMac, 0.013% MACs, inplace=True)
+  (inc1_br2_conv0): Conv2d(8.19 k, 0.137% Params, 8.39 MMac, 0.573% MACs, 256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br2_bn0): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br2_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc1_br2_conv1): Conv2d(76.8 k, 1.286% Params, 78.64 MMac, 5.372% MACs, 32, 96, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc1_br2_bn1): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br2_relu1): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
+  (inc1_br3_pool0): MaxPool2d(0, 0.000% Params, 262.14 KMac, 0.018% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc1_br3_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.018% MACs, inplace=True)
+  (inc1_br3_conv2): Conv2d(16.38 k, 0.274% Params, 16.78 MMac, 1.146% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br3_bn2): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br3_relu2): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (pool6): MaxPool2d(0, 0.000% Params, 491.52 KMac, 0.034% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
+  (inc2_br0_conv0): Conv2d(92.16 k, 1.544% Params, 23.59 MMac, 1.612% MACs, 480, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br0_bn0): BatchNorm2d(384, 0.006% Params, 98.3 KMac, 0.007% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br0_relu0): ReLU(0, 0.000% Params, 49.15 KMac, 0.003% MACs, inplace=True)
+  (inc2_br1_conv0): Conv2d(46.08 k, 0.772% Params, 11.8 MMac, 0.806% MACs, 480, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br1_bn0): BatchNorm2d(192, 0.003% Params, 49.15 KMac, 0.003% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br1_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
+  (inc2_br1_conv1): Conv2d(179.71 k, 3.010% Params, 46.01 MMac, 3.143% MACs, 96, 208, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc2_br1_bn1): BatchNorm2d(416, 0.007% Params, 106.5 KMac, 0.007% MACs, 208, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br1_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
+  (inc2_br2_conv0): Conv2d(7.68 k, 0.129% Params, 1.97 MMac, 0.134% MACs, 480, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br2_bn0): BatchNorm2d(32, 0.001% Params, 8.19 KMac, 0.001% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br2_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.000% MACs, inplace=True)
+  (inc2_br2_conv1): Conv2d(19.2 k, 0.322% Params, 4.92 MMac, 0.336% MACs, 16, 48, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc2_br2_bn1): BatchNorm2d(96, 0.002% Params, 24.58 KMac, 0.002% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br2_relu1): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
+  (inc2_br3_pool0): MaxPool2d(0, 0.000% Params, 122.88 KMac, 0.008% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc2_br3_relu1): ReLU(0, 0.000% Params, 122.88 KMac, 0.008% MACs, inplace=True)
+  (inc2_br3_conv2): Conv2d(30.72 k, 0.515% Params, 7.86 MMac, 0.537% MACs, 480, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc3_br0_conv0): Conv2d(81.92 k, 1.372% Params, 20.97 MMac, 1.433% MACs, 512, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br0_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br0_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
+  (inc3_br1_conv0): Conv2d(57.34 k, 0.960% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br1_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br1_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
+  (inc3_br1_conv1): Conv2d(225.79 k, 3.782% Params, 57.8 MMac, 3.949% MACs, 112, 224, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc3_br1_bn1): BatchNorm2d(448, 0.008% Params, 114.69 KMac, 0.008% MACs, 224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br1_relu1): ReLU(0, 0.000% Params, 57.34 KMac, 0.004% MACs, inplace=True)
+  (inc3_br2_conv0): Conv2d(12.29 k, 0.206% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
+  (inc3_br2_conv1): Conv2d(38.4 k, 0.643% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc3_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc3_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc3_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc3_br3_conv2): Conv2d(32.77 k, 0.549% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc4_br0_conv0): Conv2d(65.54 k, 1.098% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br0_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br0_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc4_br1_conv0): Conv2d(65.54 k, 1.098% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br1_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br1_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc4_br1_conv1): Conv2d(294.91 k, 4.939% Params, 75.5 MMac, 5.158% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc4_br1_bn1): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc4_br2_conv0): Conv2d(12.29 k, 0.206% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
+  (inc4_br2_conv1): Conv2d(38.4 k, 0.643% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc4_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc4_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc4_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc4_br3_conv2): Conv2d(32.77 k, 0.549% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc5_br0_conv0): Conv2d(57.34 k, 0.960% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br0_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br0_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
+  (inc5_br1_conv0): Conv2d(73.73 k, 1.235% Params, 18.87 MMac, 1.289% MACs, 512, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br1_bn0): BatchNorm2d(288, 0.005% Params, 73.73 KMac, 0.005% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br1_relu0): ReLU(0, 0.000% Params, 36.86 KMac, 0.003% MACs, inplace=True)
+  (inc5_br1_conv1): Conv2d(373.25 k, 6.251% Params, 95.55 MMac, 6.527% MACs, 144, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc5_br1_bn1): BatchNorm2d(576, 0.010% Params, 147.46 KMac, 0.010% MACs, 288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br1_relu1): ReLU(0, 0.000% Params, 73.73 KMac, 0.005% MACs, inplace=True)
+  (inc5_br2_conv0): Conv2d(16.38 k, 0.274% Params, 4.19 MMac, 0.287% MACs, 512, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc5_br2_conv1): Conv2d(51.2 k, 0.858% Params, 13.11 MMac, 0.895% MACs, 32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc5_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc5_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc5_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc5_br3_conv2): Conv2d(32.77 k, 0.549% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc6_br0_conv0): Conv2d(135.17 k, 2.264% Params, 34.6 MMac, 2.364% MACs, 528, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br0_bn0): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc6_br1_conv0): Conv2d(84.48 k, 1.415% Params, 21.63 MMac, 1.477% MACs, 528, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br1_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br1_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
+  (inc6_br1_conv1): Conv2d(460.8 k, 7.718% Params, 117.96 MMac, 8.059% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc6_br1_bn1): BatchNorm2d(640, 0.011% Params, 163.84 KMac, 0.011% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br1_relu1): ReLU(0, 0.000% Params, 81.92 KMac, 0.006% MACs, inplace=True)
+  (inc6_br2_conv0): Conv2d(16.9 k, 0.283% Params, 4.33 MMac, 0.295% MACs, 528, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc6_br2_conv1): Conv2d(102.4 k, 1.715% Params, 26.21 MMac, 1.791% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc6_br2_bn1): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc6_br3_pool0): MaxPool2d(0, 0.000% Params, 135.17 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc6_br3_relu1): ReLU(0, 0.000% Params, 135.17 KMac, 0.009% MACs, inplace=True)
+  (inc6_br3_conv2): Conv2d(67.58 k, 1.132% Params, 17.3 MMac, 1.182% MACs, 528, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br3_bn2): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (pool12): MaxPool2d(0, 0.000% Params, 212.99 KMac, 0.015% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
+  (inc7_br0_conv0): Conv2d(212.99 k, 3.567% Params, 13.63 MMac, 0.931% MACs, 832, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br0_bn0): BatchNorm2d(512, 0.009% Params, 32.77 KMac, 0.002% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br0_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc7_br1_conv0): Conv2d(133.12 k, 2.230% Params, 8.52 MMac, 0.582% MACs, 832, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br1_bn0): BatchNorm2d(320, 0.005% Params, 20.48 KMac, 0.001% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br1_relu0): ReLU(0, 0.000% Params, 10.24 KMac, 0.001% MACs, inplace=True)
+  (inc7_br1_conv1): Conv2d(460.8 k, 7.718% Params, 29.49 MMac, 2.015% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc7_br1_bn1): BatchNorm2d(640, 0.011% Params, 40.96 KMac, 0.003% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br1_relu1): ReLU(0, 0.000% Params, 20.48 KMac, 0.001% MACs, inplace=True)
+  (inc7_br2_conv0): Conv2d(26.62 k, 0.446% Params, 1.7 MMac, 0.116% MACs, 832, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br2_bn0): BatchNorm2d(64, 0.001% Params, 4.1 KMac, 0.000% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br2_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (inc7_br2_conv1): Conv2d(102.4 k, 1.715% Params, 6.55 MMac, 0.448% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc7_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc7_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc7_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
+  (inc7_br3_conv2): Conv2d(106.5 k, 1.784% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc8_br0_conv0): Conv2d(319.49 k, 5.351% Params, 20.45 MMac, 1.397% MACs, 832, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br0_bn0): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br0_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
+  (inc8_br1_conv0): Conv2d(159.74 k, 2.675% Params, 10.22 MMac, 0.698% MACs, 832, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br1_bn0): BatchNorm2d(384, 0.006% Params, 24.58 KMac, 0.002% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br1_relu0): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
+  (inc8_br1_conv1): Conv2d(663.55 k, 11.113% Params, 42.47 MMac, 2.901% MACs, 192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc8_br1_bn1): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br1_relu1): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
+  (inc8_br2_conv0): Conv2d(39.94 k, 0.669% Params, 2.56 MMac, 0.175% MACs, 832, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br2_bn0): BatchNorm2d(96, 0.002% Params, 6.14 KMac, 0.000% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br2_relu0): ReLU(0, 0.000% Params, 3.07 KMac, 0.000% MACs, inplace=True)
+  (inc8_br2_conv1): Conv2d(153.6 k, 2.573% Params, 9.83 MMac, 0.672% MACs, 48, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc8_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc8_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc8_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
+  (inc8_br3_conv2): Conv2d(106.5 k, 1.784% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (aap15): AdaptiveAvgPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, output_size=1)
+  (conv16): Conv2d(102.5 k, 1.717% Params, 102.5 KMac, 0.007% MACs, 1024, 100, kernel_size=(1, 1), stride=(1, 1))
+)

ykl/ALL-cifar100/param_flops/MobileNetV2.txt

+Model(
+  2.14 M, 100.000% Params, 59.23 MMac, 100.000% MACs, 
+  (conv0): Conv2d(896, 0.042% Params, 917.5 KMac, 1.549% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(64, 0.003% Params, 65.54 KMac, 0.111% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relus0): ReLU6(0, 0.000% Params, 32.77 KMac, 0.055% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(1.06 k, 0.049% Params, 1.08 MMac, 1.826% MACs, 32, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(64, 0.003% Params, 65.54 KMac, 0.111% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relus0): ReLU6(0, 0.000% Params, 32.77 KMac, 0.055% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(320, 0.015% Params, 327.68 KMac, 0.553% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32)
+  (ml0_blk0_ma_bn1): BatchNorm2d(64, 0.003% Params, 65.54 KMac, 0.111% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relus1): ReLU6(0, 0.000% Params, 32.77 KMac, 0.055% MACs, inplace=True)
+  (ml0_blk0_ma_conv2): Conv2d(528, 0.025% Params, 540.67 KMac, 0.913% MACs, 32, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn2): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.055% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_conv0): Conv2d(1.63 k, 0.076% Params, 1.67 MMac, 2.821% MACs, 16, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(192, 0.009% Params, 196.61 KMac, 0.332% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relus0): ReLU6(0, 0.000% Params, 98.3 KMac, 0.166% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(960, 0.045% Params, 245.76 KMac, 0.415% MACs, 96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96)
+  (ml1_blk0_ma_bn1): BatchNorm2d(192, 0.009% Params, 49.15 KMac, 0.083% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relus1): ReLU6(0, 0.000% Params, 24.58 KMac, 0.041% MACs, inplace=True)
+  (ml1_blk0_ma_conv2): Conv2d(2.33 k, 0.109% Params, 595.97 KMac, 1.006% MACs, 96, 24, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn2): BatchNorm2d(48, 0.002% Params, 12.29 KMac, 0.021% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_conv0): Conv2d(3.6 k, 0.168% Params, 921.6 KMac, 1.556% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(288, 0.013% Params, 73.73 KMac, 0.124% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(1.44 k, 0.067% Params, 368.64 KMac, 0.622% MACs, 144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144)
+  (ml1_blk1_ma_bn1): BatchNorm2d(288, 0.013% Params, 73.73 KMac, 0.124% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relus1): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml1_blk1_ma_conv2): Conv2d(3.48 k, 0.163% Params, 890.88 KMac, 1.504% MACs, 144, 24, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn2): BatchNorm2d(48, 0.002% Params, 12.29 KMac, 0.021% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_conv0): Conv2d(3.6 k, 0.168% Params, 921.6 KMac, 1.556% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(288, 0.013% Params, 73.73 KMac, 0.124% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(1.44 k, 0.067% Params, 92.16 KMac, 0.156% MACs, 144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144)
+  (ml2_blk0_ma_bn1): BatchNorm2d(288, 0.013% Params, 18.43 KMac, 0.031% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relus1): ReLU6(0, 0.000% Params, 9.22 KMac, 0.016% MACs, inplace=True)
+  (ml2_blk0_ma_conv2): Conv2d(4.64 k, 0.217% Params, 296.96 KMac, 0.501% MACs, 144, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn2): BatchNorm2d(64, 0.003% Params, 4.1 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_conv0): Conv2d(6.34 k, 0.296% Params, 405.5 KMac, 0.685% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(384, 0.018% Params, 24.58 KMac, 0.041% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relus0): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(1.92 k, 0.090% Params, 122.88 KMac, 0.207% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
+  (ml2_blk1_ma_bn1): BatchNorm2d(384, 0.018% Params, 24.58 KMac, 0.041% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relus1): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk1_ma_conv2): Conv2d(6.18 k, 0.288% Params, 395.26 KMac, 0.667% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn2): BatchNorm2d(64, 0.003% Params, 4.1 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_conv0): Conv2d(6.34 k, 0.296% Params, 405.5 KMac, 0.685% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn0): BatchNorm2d(384, 0.018% Params, 24.58 KMac, 0.041% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relus0): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk2_ma_conv1): Conv2d(1.92 k, 0.090% Params, 122.88 KMac, 0.207% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
+  (ml2_blk2_ma_bn1): BatchNorm2d(384, 0.018% Params, 24.58 KMac, 0.041% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relus1): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml2_blk2_ma_conv2): Conv2d(6.18 k, 0.288% Params, 395.26 KMac, 0.667% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn2): BatchNorm2d(64, 0.003% Params, 4.1 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_conv0): Conv2d(6.34 k, 0.296% Params, 405.5 KMac, 0.685% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(384, 0.018% Params, 24.58 KMac, 0.041% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relus0): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(1.92 k, 0.090% Params, 122.88 KMac, 0.207% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
+  (ml3_blk0_ma_bn1): BatchNorm2d(384, 0.018% Params, 24.58 KMac, 0.041% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relus1): ReLU6(0, 0.000% Params, 12.29 KMac, 0.021% MACs, inplace=True)
+  (ml3_blk0_ma_conv2): Conv2d(18.53 k, 0.865% Params, 1.19 MMac, 2.002% MACs, 192, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn2): BatchNorm2d(192, 0.009% Params, 12.29 KMac, 0.021% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_conv0): Conv2d(55.87 k, 2.610% Params, 3.58 MMac, 6.037% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(1.15 k, 0.054% Params, 73.73 KMac, 0.124% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(5.76 k, 0.269% Params, 368.64 KMac, 0.622% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
+  (ml3_blk1_ma_bn1): BatchNorm2d(1.15 k, 0.054% Params, 73.73 KMac, 0.124% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relus1): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk1_ma_conv2): Conv2d(55.39 k, 2.587% Params, 3.55 MMac, 5.985% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn2): BatchNorm2d(192, 0.009% Params, 12.29 KMac, 0.021% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_conv0): Conv2d(55.87 k, 2.610% Params, 3.58 MMac, 6.037% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn0): BatchNorm2d(1.15 k, 0.054% Params, 73.73 KMac, 0.124% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk2_ma_conv1): Conv2d(5.76 k, 0.269% Params, 368.64 KMac, 0.622% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
+  (ml3_blk2_ma_bn1): BatchNorm2d(1.15 k, 0.054% Params, 73.73 KMac, 0.124% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relus1): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml3_blk2_ma_conv2): Conv2d(55.39 k, 2.587% Params, 3.55 MMac, 5.985% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn2): BatchNorm2d(192, 0.009% Params, 12.29 KMac, 0.021% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk0_ma_conv0): Conv2d(55.87 k, 2.610% Params, 3.58 MMac, 6.037% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk0_ma_bn0): BatchNorm2d(1.15 k, 0.054% Params, 73.73 KMac, 0.124% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk0_ma_relus0): ReLU6(0, 0.000% Params, 36.86 KMac, 0.062% MACs, inplace=True)
+  (ml4_blk0_ma_conv1): Conv2d(5.76 k, 0.269% Params, 92.16 KMac, 0.156% MACs, 576, 576, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=576)
+  (ml4_blk0_ma_bn1): BatchNorm2d(1.15 k, 0.054% Params, 18.43 KMac, 0.031% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk0_ma_relus1): ReLU6(0, 0.000% Params, 9.22 KMac, 0.016% MACs, inplace=True)
+  (ml4_blk0_ma_conv2): Conv2d(92.32 k, 4.312% Params, 1.48 MMac, 2.494% MACs, 576, 160, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk0_ma_bn2): BatchNorm2d(320, 0.015% Params, 5.12 KMac, 0.009% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk1_ma_conv0): Conv2d(154.56 k, 7.219% Params, 2.47 MMac, 4.175% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk1_ma_bn0): BatchNorm2d(1.92 k, 0.090% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk1_ma_relus0): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk1_ma_conv1): Conv2d(9.6 k, 0.448% Params, 153.6 KMac, 0.259% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
+  (ml4_blk1_ma_bn1): BatchNorm2d(1.92 k, 0.090% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk1_ma_relus1): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk1_ma_conv2): Conv2d(153.76 k, 7.182% Params, 2.46 MMac, 4.153% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk1_ma_bn2): BatchNorm2d(320, 0.015% Params, 5.12 KMac, 0.009% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk2_ma_conv0): Conv2d(154.56 k, 7.219% Params, 2.47 MMac, 4.175% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk2_ma_bn0): BatchNorm2d(1.92 k, 0.090% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk2_ma_relus0): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk2_ma_conv1): Conv2d(9.6 k, 0.448% Params, 153.6 KMac, 0.259% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
+  (ml4_blk2_ma_bn1): BatchNorm2d(1.92 k, 0.090% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml4_blk2_ma_relus1): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml4_blk2_ma_conv2): Conv2d(153.76 k, 7.182% Params, 2.46 MMac, 4.153% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
+  (ml4_blk2_ma_bn2): BatchNorm2d(320, 0.015% Params, 5.12 KMac, 0.009% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml5_blk0_ma_conv0): Conv2d(154.56 k, 7.219% Params, 2.47 MMac, 4.175% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
+  (ml5_blk0_ma_bn0): BatchNorm2d(1.92 k, 0.090% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml5_blk0_ma_relus0): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml5_blk0_ma_conv1): Conv2d(9.6 k, 0.448% Params, 153.6 KMac, 0.259% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
+  (ml5_blk0_ma_bn1): BatchNorm2d(1.92 k, 0.090% Params, 30.72 KMac, 0.052% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml5_blk0_ma_relus1): ReLU6(0, 0.000% Params, 15.36 KMac, 0.026% MACs, inplace=True)
+  (ml5_blk0_ma_conv2): Conv2d(307.52 k, 14.363% Params, 4.92 MMac, 8.307% MACs, 960, 320, kernel_size=(1, 1), stride=(1, 1))
+  (ml5_blk0_ma_bn2): BatchNorm2d(640, 0.030% Params, 10.24 KMac, 0.017% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (conv7): Conv2d(410.88 k, 19.191% Params, 6.57 MMac, 11.099% MACs, 320, 1280, kernel_size=(1, 1), stride=(1, 1))
+  (aap8): AdaptiveAvgPool2d(0, 0.000% Params, 20.48 KMac, 0.035% MACs, output_size=1)
+  (fc10): Linear(128.1 k, 5.983% Params, 128.1 KMac, 0.216% MACs, in_features=1280, out_features=100, bias=True)
+)

ykl/ALL-cifar100/param_flops/ResNet_152.txt

+Model(
+  3.73 M, 100.000% Params, 240.07 MMac, 100.000% MACs, 
+  (conv0): Conv2d(448, 0.012% Params, 458.75 KMac, 0.191% MACs, 3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(272, 0.007% Params, 278.53 KMac, 0.116% MACs, 16, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(2.32 k, 0.062% Params, 2.38 MMac, 0.990% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn1): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk0_ma_conv2): Conv2d(1.09 k, 0.029% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn2): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ds_conv0): Conv2d(1.09 k, 0.029% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ds_bn0): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.027% MACs, inplace=True)
+  (ml0_blk1_ma_conv0): Conv2d(1.04 k, 0.028% Params, 1.06 MMac, 0.444% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk1_ma_conv1): Conv2d(2.32 k, 0.062% Params, 2.38 MMac, 0.990% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn1): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk1_ma_conv2): Conv2d(1.09 k, 0.029% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn2): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.027% MACs, inplace=True)
+  (ml0_blk2_ma_conv0): Conv2d(1.04 k, 0.028% Params, 1.06 MMac, 0.444% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk2_ma_conv1): Conv2d(2.32 k, 0.062% Params, 2.38 MMac, 0.990% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk2_ma_bn1): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.014% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml0_blk2_ma_conv2): Conv2d(1.09 k, 0.029% Params, 1.11 MMac, 0.464% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn2): BatchNorm2d(128, 0.003% Params, 131.07 KMac, 0.055% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.027% MACs, inplace=True)
+  (ml1_blk0_ma_conv0): Conv2d(2.08 k, 0.056% Params, 2.13 MMac, 0.887% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(64, 0.002% Params, 65.54 KMac, 0.027% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml1_blk0_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk0_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ds_conv0): Conv2d(8.32 k, 0.223% Params, 2.13 MMac, 0.887% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2))
+  (ml1_blk0_ds_bn0): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk1_ma_conv0): Conv2d(4.13 k, 0.111% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk1_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk2_ma_conv0): Conv2d(4.13 k, 0.111% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk2_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk2_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk2_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk3_ma_conv0): Conv2d(4.13 k, 0.111% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk3_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk3_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk3_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk4_ma_conv0): Conv2d(4.13 k, 0.111% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk4_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk4_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk4_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk4_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk4_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk4_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk4_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk4_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk5_ma_conv0): Conv2d(4.13 k, 0.111% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk5_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk5_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk5_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk5_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk5_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk5_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk5_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk5_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk6_ma_conv0): Conv2d(4.13 k, 0.111% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk6_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk6_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk6_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk6_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk6_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk6_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk6_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk6_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml1_blk7_ma_conv0): Conv2d(4.13 k, 0.111% Params, 1.06 MMac, 0.440% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk7_ma_bn0): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk7_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk7_ma_conv1): Conv2d(9.25 k, 0.248% Params, 2.37 MMac, 0.986% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk7_ma_bn1): BatchNorm2d(64, 0.002% Params, 16.38 KMac, 0.007% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk7_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml1_blk7_ma_conv2): Conv2d(4.22 k, 0.113% Params, 1.08 MMac, 0.450% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk7_ma_bn2): BatchNorm2d(256, 0.007% Params, 65.54 KMac, 0.027% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk7_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.014% MACs, inplace=True)
+  (ml2_blk0_ma_conv0): Conv2d(8.26 k, 0.221% Params, 2.11 MMac, 0.880% MACs, 128, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(128, 0.003% Params, 32.77 KMac, 0.014% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml2_blk0_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk0_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ds_conv0): Conv2d(33.02 k, 0.885% Params, 2.11 MMac, 0.880% MACs, 128, 256, kernel_size=(1, 1), stride=(2, 2))
+  (ml2_blk0_ds_bn0): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk1_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk1_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk2_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk2_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk2_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk2_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk3_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk3_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk3_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk3_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk4_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk4_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk4_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk4_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk5_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk5_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk5_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk5_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk6_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk6_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk6_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk6_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk6_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk6_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk6_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk6_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk6_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk7_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk7_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk7_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk7_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk7_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk7_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk7_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk7_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk7_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk8_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk8_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk8_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk8_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk8_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk8_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk8_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk8_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk8_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk9_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk9_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk9_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk9_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk9_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk9_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk9_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk9_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk9_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk10_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk10_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk10_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk10_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk10_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk10_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk10_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk10_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk10_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk11_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk11_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk11_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk11_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk11_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk11_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk11_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk11_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk11_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk12_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk12_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk12_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk12_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk12_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk12_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk12_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk12_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk12_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk13_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk13_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk13_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk13_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk13_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk13_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk13_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk13_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk13_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk14_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk14_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk14_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk14_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk14_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk14_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk14_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk14_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk14_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk15_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk15_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk15_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk15_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk15_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk15_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk15_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk15_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk15_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk16_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk16_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk16_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk16_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk16_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk16_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk16_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk16_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk16_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk17_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk17_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk17_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk17_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk17_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk17_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk17_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk17_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk17_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk18_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk18_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk18_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk18_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk18_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk18_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk18_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk18_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk18_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk19_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk19_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk19_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk19_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk19_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk19_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk19_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk19_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk19_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk20_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk20_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk20_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk20_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk20_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk20_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk20_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk20_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk20_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk21_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk21_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk21_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk21_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk21_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk21_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk21_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk21_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk21_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk22_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk22_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk22_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk22_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk22_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk22_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk22_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk22_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk22_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk23_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk23_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk23_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk23_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk23_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk23_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk23_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk23_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk23_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk24_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk24_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk24_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk24_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk24_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk24_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk24_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk24_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk24_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk25_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk25_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk25_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk25_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk25_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk25_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk25_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk25_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk25_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk26_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk26_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk26_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk26_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk26_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk26_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk26_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk26_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk26_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk27_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk27_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk27_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk27_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk27_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk27_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk27_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk27_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk27_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk28_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk28_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk28_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk28_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk28_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk28_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk28_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk28_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk28_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk29_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk29_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk29_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk29_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk29_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk29_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk29_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk29_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk29_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk30_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk30_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk30_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk30_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk30_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk30_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk30_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk30_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk30_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk31_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk31_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk31_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk31_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk31_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk31_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk31_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk31_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk31_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk32_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk32_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk32_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk32_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk32_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk32_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk32_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk32_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk32_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk33_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk33_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk33_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk33_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk33_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk33_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk33_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk33_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk33_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk34_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk34_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk34_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk34_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk34_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk34_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk34_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk34_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk34_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml2_blk35_ma_conv0): Conv2d(16.45 k, 0.441% Params, 1.05 MMac, 0.438% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk35_ma_bn0): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk35_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk35_ma_conv1): Conv2d(36.93 k, 0.989% Params, 2.36 MMac, 0.984% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk35_ma_bn1): BatchNorm2d(128, 0.003% Params, 8.19 KMac, 0.003% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk35_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.002% MACs, inplace=True)
+  (ml2_blk35_ma_conv2): Conv2d(16.64 k, 0.446% Params, 1.06 MMac, 0.444% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk35_ma_bn2): BatchNorm2d(512, 0.014% Params, 32.77 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk35_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.007% MACs, inplace=True)
+  (ml3_blk0_ma_conv0): Conv2d(32.9 k, 0.881% Params, 2.11 MMac, 0.877% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(256, 0.007% Params, 16.38 KMac, 0.007% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(147.58 k, 3.954% Params, 2.36 MMac, 0.984% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml3_blk0_ma_bn1): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk0_ma_conv2): Conv2d(66.05 k, 1.770% Params, 1.06 MMac, 0.440% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.027% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ds_conv0): Conv2d(131.58 k, 3.525% Params, 2.11 MMac, 0.877% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2))
+  (ml3_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.027% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml3_blk1_ma_conv0): Conv2d(65.66 k, 1.759% Params, 1.05 MMac, 0.438% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(147.58 k, 3.954% Params, 2.36 MMac, 0.984% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn1): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk1_ma_conv2): Conv2d(66.05 k, 1.770% Params, 1.06 MMac, 0.440% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.027% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (ml3_blk2_ma_conv0): Conv2d(65.66 k, 1.759% Params, 1.05 MMac, 0.438% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn0): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk2_ma_conv1): Conv2d(147.58 k, 3.954% Params, 2.36 MMac, 0.984% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk2_ma_bn1): BatchNorm2d(256, 0.007% Params, 4.1 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (ml3_blk2_ma_conv2): Conv2d(66.05 k, 1.770% Params, 1.06 MMac, 0.440% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.027% Params, 16.38 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.003% MACs, inplace=True)
+  (aap5): AdaptiveAvgPool2d(0, 0.000% Params, 8.19 KMac, 0.003% MACs, output_size=1)
+  (fc7): Linear(51.3 k, 1.374% Params, 51.3 KMac, 0.021% MACs, in_features=512, out_features=100, bias=True)
+)

ykl/ALL-cifar100/param_flops/ResNet_18.txt

+Model(
+  714.28 k, 100.000% Params, 35.66 MMac, 100.000% MACs, 
+  (conv0): Conv2d(448, 0.063% Params, 458.75 KMac, 1.286% MACs, 3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(32, 0.004% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(2.32 k, 0.325% Params, 2.38 MMac, 6.662% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(32, 0.004% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(2.32 k, 0.325% Params, 2.38 MMac, 6.662% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn1): BatchNorm2d(32, 0.004% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk1_ma_conv0): Conv2d(2.32 k, 0.325% Params, 2.38 MMac, 6.662% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn0): BatchNorm2d(32, 0.004% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml0_blk1_ma_conv1): Conv2d(2.32 k, 0.325% Params, 2.38 MMac, 6.662% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn1): BatchNorm2d(32, 0.004% Params, 32.77 KMac, 0.092% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.046% MACs, inplace=True)
+  (ml1_blk0_ma_conv0): Conv2d(4.64 k, 0.650% Params, 1.19 MMac, 3.331% MACs, 16, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(9.25 k, 1.295% Params, 2.37 MMac, 6.639% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk0_ma_bn1): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ds_conv0): Conv2d(544, 0.076% Params, 139.26 KMac, 0.391% MACs, 16, 32, kernel_size=(1, 1), stride=(2, 2))
+  (ml1_blk0_ds_bn0): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml1_blk1_ma_conv0): Conv2d(9.25 k, 1.295% Params, 2.37 MMac, 6.639% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(9.25 k, 1.295% Params, 2.37 MMac, 6.639% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn1): BatchNorm2d(64, 0.009% Params, 16.38 KMac, 0.046% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.023% MACs, inplace=True)
+  (ml2_blk0_ma_conv0): Conv2d(18.5 k, 2.589% Params, 1.18 MMac, 3.319% MACs, 32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(36.93 k, 5.170% Params, 2.36 MMac, 6.627% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk0_ma_bn1): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ds_conv0): Conv2d(2.11 k, 0.296% Params, 135.17 KMac, 0.379% MACs, 32, 64, kernel_size=(1, 1), stride=(2, 2))
+  (ml2_blk0_ds_bn0): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml2_blk1_ma_conv0): Conv2d(36.93 k, 5.170% Params, 2.36 MMac, 6.627% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(36.93 k, 5.170% Params, 2.36 MMac, 6.627% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn1): BatchNorm2d(128, 0.018% Params, 8.19 KMac, 0.023% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.011% MACs, inplace=True)
+  (ml3_blk0_ma_conv0): Conv2d(73.86 k, 10.340% Params, 1.18 MMac, 3.314% MACs, 64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(147.58 k, 20.662% Params, 2.36 MMac, 6.622% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk0_ma_bn1): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ds_conv0): Conv2d(8.32 k, 1.165% Params, 133.12 KMac, 0.373% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2))
+  (ml3_blk0_ds_bn0): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (ml3_blk1_ma_conv0): Conv2d(147.58 k, 20.662% Params, 2.36 MMac, 6.622% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(147.58 k, 20.662% Params, 2.36 MMac, 6.622% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn1): BatchNorm2d(256, 0.036% Params, 4.1 KMac, 0.011% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.006% MACs, inplace=True)
+  (aap5): AdaptiveAvgPool2d(0, 0.000% Params, 2.05 KMac, 0.006% MACs, output_size=1)
+  (fc7): Linear(12.9 k, 1.806% Params, 12.9 KMac, 0.036% MACs, in_features=128, out_features=100, bias=True)
+)

ykl/ALL-cifar100/param_flops/ResNet_50.txt

+Model(
+  1.54 M, 100.000% Params, 84.81 MMac, 100.000% MACs, 
+  (conv0): Conv2d(448, 0.029% Params, 458.75 KMac, 0.541% MACs, 3, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk0_ma_conv0): Conv2d(272, 0.018% Params, 278.53 KMac, 0.328% MACs, 16, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk0_ma_conv1): Conv2d(2.32 k, 0.151% Params, 2.38 MMac, 2.801% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk0_ma_bn1): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk0_ma_conv2): Conv2d(1.09 k, 0.071% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ma_bn2): BatchNorm2d(128, 0.008% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_ds_conv0): Conv2d(1.09 k, 0.071% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk0_ds_bn0): BatchNorm2d(128, 0.008% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.077% MACs, inplace=True)
+  (ml0_blk1_ma_conv0): Conv2d(1.04 k, 0.068% Params, 1.06 MMac, 1.256% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk1_ma_conv1): Conv2d(2.32 k, 0.151% Params, 2.38 MMac, 2.801% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk1_ma_bn1): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk1_ma_conv2): Conv2d(1.09 k, 0.071% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk1_ma_bn2): BatchNorm2d(128, 0.008% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.077% MACs, inplace=True)
+  (ml0_blk2_ma_conv0): Conv2d(1.04 k, 0.068% Params, 1.06 MMac, 1.256% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn0): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk2_ma_conv1): Conv2d(2.32 k, 0.151% Params, 2.38 MMac, 2.801% MACs, 16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml0_blk2_ma_bn1): BatchNorm2d(32, 0.002% Params, 32.77 KMac, 0.039% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml0_blk2_ma_conv2): Conv2d(1.09 k, 0.071% Params, 1.11 MMac, 1.314% MACs, 16, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml0_blk2_ma_bn2): BatchNorm2d(128, 0.008% Params, 131.07 KMac, 0.155% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml0_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.077% MACs, inplace=True)
+  (ml1_blk0_ma_conv0): Conv2d(2.08 k, 0.135% Params, 2.13 MMac, 2.511% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn0): BatchNorm2d(64, 0.004% Params, 65.54 KMac, 0.077% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk0_ma_conv1): Conv2d(9.25 k, 0.602% Params, 2.37 MMac, 2.791% MACs, 32, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml1_blk0_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk0_ma_conv2): Conv2d(4.22 k, 0.275% Params, 1.08 MMac, 1.275% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk0_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_ds_conv0): Conv2d(8.32 k, 0.541% Params, 2.13 MMac, 2.511% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2))
+  (ml1_blk0_ds_bn0): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk1_ma_conv0): Conv2d(4.13 k, 0.269% Params, 1.06 MMac, 1.246% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn0): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk1_ma_conv1): Conv2d(9.25 k, 0.602% Params, 2.37 MMac, 2.791% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk1_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk1_ma_conv2): Conv2d(4.22 k, 0.275% Params, 1.08 MMac, 1.275% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk1_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk2_ma_conv0): Conv2d(4.13 k, 0.269% Params, 1.06 MMac, 1.246% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn0): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk2_ma_conv1): Conv2d(9.25 k, 0.602% Params, 2.37 MMac, 2.791% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk2_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk2_ma_conv2): Conv2d(4.22 k, 0.275% Params, 1.08 MMac, 1.275% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk2_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml1_blk3_ma_conv0): Conv2d(4.13 k, 0.269% Params, 1.06 MMac, 1.246% MACs, 128, 32, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn0): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk3_ma_conv1): Conv2d(9.25 k, 0.602% Params, 2.37 MMac, 2.791% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml1_blk3_ma_bn1): BatchNorm2d(64, 0.004% Params, 16.38 KMac, 0.019% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml1_blk3_ma_conv2): Conv2d(4.22 k, 0.275% Params, 1.08 MMac, 1.275% MACs, 32, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml1_blk3_ma_bn2): BatchNorm2d(256, 0.017% Params, 65.54 KMac, 0.077% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml1_blk3_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.039% MACs, inplace=True)
+  (ml2_blk0_ma_conv0): Conv2d(8.26 k, 0.537% Params, 2.11 MMac, 2.492% MACs, 128, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn0): BatchNorm2d(128, 0.008% Params, 32.77 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk0_ma_conv1): Conv2d(36.93 k, 2.403% Params, 2.36 MMac, 2.787% MACs, 64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml2_blk0_ma_bn1): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk0_ma_conv2): Conv2d(16.64 k, 1.083% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk0_ma_bn2): BatchNorm2d(512, 0.033% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_ds_conv0): Conv2d(33.02 k, 2.149% Params, 2.11 MMac, 2.492% MACs, 128, 256, kernel_size=(1, 1), stride=(2, 2))
+  (ml2_blk0_ds_bn0): BatchNorm2d(512, 0.033% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk1_ma_conv0): Conv2d(16.45 k, 1.070% Params, 1.05 MMac, 1.241% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn0): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk1_ma_conv1): Conv2d(36.93 k, 2.403% Params, 2.36 MMac, 2.787% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk1_ma_bn1): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk1_ma_conv2): Conv2d(16.64 k, 1.083% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk1_ma_bn2): BatchNorm2d(512, 0.033% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk2_ma_conv0): Conv2d(16.45 k, 1.070% Params, 1.05 MMac, 1.241% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn0): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk2_ma_conv1): Conv2d(36.93 k, 2.403% Params, 2.36 MMac, 2.787% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk2_ma_bn1): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk2_ma_conv2): Conv2d(16.64 k, 1.083% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk2_ma_bn2): BatchNorm2d(512, 0.033% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk3_ma_conv0): Conv2d(16.45 k, 1.070% Params, 1.05 MMac, 1.241% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn0): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk3_ma_conv1): Conv2d(36.93 k, 2.403% Params, 2.36 MMac, 2.787% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk3_ma_bn1): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk3_ma_conv2): Conv2d(16.64 k, 1.083% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk3_ma_bn2): BatchNorm2d(512, 0.033% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk3_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk4_ma_conv0): Conv2d(16.45 k, 1.070% Params, 1.05 MMac, 1.241% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn0): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk4_ma_conv1): Conv2d(36.93 k, 2.403% Params, 2.36 MMac, 2.787% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk4_ma_bn1): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk4_ma_conv2): Conv2d(16.64 k, 1.083% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk4_ma_bn2): BatchNorm2d(512, 0.033% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk4_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml2_blk5_ma_conv0): Conv2d(16.45 k, 1.070% Params, 1.05 MMac, 1.241% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn0): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk5_ma_conv1): Conv2d(36.93 k, 2.403% Params, 2.36 MMac, 2.787% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml2_blk5_ma_bn1): BatchNorm2d(128, 0.008% Params, 8.19 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 4.1 KMac, 0.005% MACs, inplace=True)
+  (ml2_blk5_ma_conv2): Conv2d(16.64 k, 1.083% Params, 1.06 MMac, 1.256% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1))
+  (ml2_blk5_ma_bn2): BatchNorm2d(512, 0.033% Params, 32.77 KMac, 0.039% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml2_blk5_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.019% MACs, inplace=True)
+  (ml3_blk0_ma_conv0): Conv2d(32.9 k, 2.140% Params, 2.11 MMac, 2.482% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn0): BatchNorm2d(256, 0.017% Params, 16.38 KMac, 0.019% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml3_blk0_ma_conv1): Conv2d(147.58 k, 9.602% Params, 2.36 MMac, 2.784% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
+  (ml3_blk0_ma_bn1): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk0_ma_conv2): Conv2d(66.05 k, 4.297% Params, 1.06 MMac, 1.246% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.067% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_ds_conv0): Conv2d(131.58 k, 8.561% Params, 2.11 MMac, 2.482% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2))
+  (ml3_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.067% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml3_blk1_ma_conv0): Conv2d(65.66 k, 4.272% Params, 1.05 MMac, 1.239% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn0): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk1_ma_conv1): Conv2d(147.58 k, 9.602% Params, 2.36 MMac, 2.784% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk1_ma_bn1): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk1_ma_conv2): Conv2d(66.05 k, 4.297% Params, 1.06 MMac, 1.246% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.067% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (ml3_blk2_ma_conv0): Conv2d(65.66 k, 4.272% Params, 1.05 MMac, 1.239% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn0): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk2_ma_conv1): Conv2d(147.58 k, 9.602% Params, 2.36 MMac, 2.784% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (ml3_blk2_ma_bn1): BatchNorm2d(256, 0.017% Params, 4.1 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 2.05 KMac, 0.002% MACs, inplace=True)
+  (ml3_blk2_ma_conv2): Conv2d(66.05 k, 4.297% Params, 1.06 MMac, 1.246% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1))
+  (ml3_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.067% Params, 16.38 KMac, 0.019% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (ml3_blk2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.010% MACs, inplace=True)
+  (aap5): AdaptiveAvgPool2d(0, 0.000% Params, 8.19 KMac, 0.010% MACs, output_size=1)
+  (fc7): Linear(51.3 k, 3.338% Params, 51.3 KMac, 0.060% MACs, in_features=512, out_features=100, bias=True)
+)

ykl/ALL-cifar100/param_flops/VGG_16.txt

+Model(
+  34.02 M, 100.000% Params, 333.73 MMac, 100.000% MACs, 
+  (conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.550% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
+  (conv1): Conv2d(36.93 k, 0.109% Params, 37.81 MMac, 11.331% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
+  (pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.020% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv3): Conv2d(73.86 k, 0.217% Params, 18.91 MMac, 5.665% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
+  (conv4): Conv2d(147.58 k, 0.434% Params, 37.78 MMac, 11.321% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
+  (pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.010% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv6): Conv2d(295.17 k, 0.868% Params, 18.89 MMac, 5.661% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn6): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
+  (conv7): Conv2d(590.08 k, 1.735% Params, 37.77 MMac, 11.316% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn7): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
+  (conv8): Conv2d(590.08 k, 1.735% Params, 37.77 MMac, 11.316% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn8): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
+  (pool9): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.005% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv10): Conv2d(1.18 M, 3.469% Params, 18.88 MMac, 5.658% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn10): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu10): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv11): Conv2d(2.36 M, 6.937% Params, 37.76 MMac, 11.314% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv12): Conv2d(2.36 M, 6.937% Params, 37.76 MMac, 11.314% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (pool13): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv14): Conv2d(2.36 M, 6.937% Params, 9.44 MMac, 2.828% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn14): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu14): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (conv15): Conv2d(2.36 M, 6.937% Params, 9.44 MMac, 2.828% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn15): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu15): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (conv16): Conv2d(2.36 M, 6.937% Params, 9.44 MMac, 2.828% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (pool17): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.001% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (fc19): Linear(2.1 M, 6.177% Params, 2.1 MMac, 0.630% MACs, in_features=512, out_features=4096, bias=True)
+  (relu20): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop21): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc22): Linear(16.78 M, 49.334% Params, 16.78 MMac, 5.028% MACs, in_features=4096, out_features=4096, bias=True)
+  (relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc25): Linear(409.7 k, 1.204% Params, 409.7 KMac, 0.123% MACs, in_features=4096, out_features=100, bias=True)
+)

ykl/ALL-cifar100/param_flops/VGG_19.txt

+Model(
+  39.33 M, 100.000% Params, 418.77 MMac, 100.000% MACs, 
+  (conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.438% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
+  (conv1): Conv2d(36.93 k, 0.094% Params, 37.81 MMac, 9.030% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
+  (pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.016% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv3): Conv2d(73.86 k, 0.188% Params, 18.91 MMac, 4.515% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
+  (conv4): Conv2d(147.58 k, 0.375% Params, 37.78 MMac, 9.022% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
+  (pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.008% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv6): Conv2d(295.17 k, 0.751% Params, 18.89 MMac, 4.511% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn6): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (conv7): Conv2d(590.08 k, 1.500% Params, 37.77 MMac, 9.018% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn7): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (conv8): Conv2d(590.08 k, 1.500% Params, 37.77 MMac, 9.018% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn8): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (conv9): Conv2d(590.08 k, 1.500% Params, 37.77 MMac, 9.018% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn9): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (pool10): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.004% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv11): Conv2d(1.18 M, 3.001% Params, 18.88 MMac, 4.509% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv12): Conv2d(2.36 M, 6.000% Params, 37.76 MMac, 9.016% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv13): Conv2d(2.36 M, 6.000% Params, 37.76 MMac, 9.016% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn13): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu13): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv14): Conv2d(2.36 M, 6.000% Params, 37.76 MMac, 9.016% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn14): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu14): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (pool15): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv16): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (conv17): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn17): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu17): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (conv18): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn18): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu18): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (conv19): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn19): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu19): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (pool20): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.000% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (fc22): Linear(2.1 M, 5.343% Params, 2.1 MMac, 0.502% MACs, in_features=512, out_features=4096, bias=True)
+  (relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc25): Linear(16.78 M, 42.671% Params, 16.78 MMac, 4.007% MACs, in_features=4096, out_features=4096, bias=True)
+  (relu26): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop27): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc28): Linear(409.7 k, 1.042% Params, 409.7 KMac, 0.098% MACs, in_features=4096, out_features=100, bias=True)
+)

ykl/ALL/ptq.py → ykl/ALL-cifar100/ptq.py

@@ -23,7 +23,7 @@ def direct_quantize(model, test_loader,device):
     for i, (data, target) in enumerate(test_loader, 1):
         data = data.to(device)
         output = model.quantize_forward(data).cpu()
-        if i % 500 == 0:
+        if i % 100 == 0:
             break
     print('direct quantization finish')
 
@@ -52,28 +52,30 @@ def quantize_inference(model, test_loader, device):
 
 
 if __name__ == "__main__":
-    batch_size = 32
+    batch_size = 128
 
     device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
     print(device)
     train_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=True, download=True,
+    datasets.CIFAR100('/lustre/datasets/CIFAR100', train=True, download=False,
                      transform=transforms.Compose([
-                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                        transforms.RandomCrop(32, padding=2),
                         transforms.RandomHorizontalFlip(),
                         transforms.ToTensor(),
-                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                        transforms.Normalize(
+                            (0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
                         ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+     batch_size=batch_size, shuffle=True, num_workers=4, pin_memory=True
     )
 
     test_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
-            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+    datasets.CIFAR100('/lustre/datasets/CIFAR100', train=False, download=False, 
+                     transform=transforms.Compose([
                             transforms.ToTensor(),
-            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                            transforms.Normalize((0.4914, 0.4822, 0.4465),
+                                                (0.2023, 0.1994, 0.2010))
                             ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    batch_size=batch_size, shuffle=False, num_workers=4, pin_memory=True
     )
 
     load_ptq = False
@@ -82,7 +84,13 @@ if __name__ == "__main__":
     append = True
     gol._init()
 
-    excel_path = 'ptq_result.xlsx'
+    # ckpt_path = 'ckpt_sgd_5_momentum_95'
+    # excel_path = 'ptq_result_sgd_mom95.xlsx'
+    # optim_type = 'sgd_5_momentum_95_ntb'
+    opt_path = 'adam_lr0001'
+    ckpt_path = 'ckpt_'+opt_path
+    
+    excel_path = 'ptq_result_'+opt_path+'.xlsx'
     if os.path.exists(excel_path) and append:
         workbook = openpyxl.load_workbook(excel_path)
     else:
@@ -90,29 +98,34 @@ if __name__ == "__main__":
     if 'Sheet' in workbook.sheetnames:
         workbook.remove(workbook['Sheet'])
 
-    txt_path = 'ptq_result.txt'
+    # txt_path = 'ptq_result_sgd_mom95.txt'
+    txt_path = 'ptq_result_'+opt_path+'.txt'
     if os.path.exists(txt_path) and append:
         ft = open(txt_path,'a')
     else:
         ft = open(txt_path,'w')
-    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
+    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN',
+                       'MobileNetV2','ResNet_18','ResNet_50','ResNet_152']
+    # model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN',
+    #                    'MobileNetV2','ResNet_18','ResNet_50']
+    # model_name_list = ['MobileNetV2']
     for model_name in model_name_list:
         if model_name in workbook.sheetnames:
             continue
 
         model = Model(model_name)
-        full_file = 'ckpt/cifar10_'+model_name+'.pt'
+        full_file = ckpt_path+'/cifar100_'+model_name+'.pt'
         model.load_state_dict(torch.load(full_file))
         model.to(device)
 
-        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
+        ptq_file_prefix = ckpt_path+'/cifar100_'+model_name+'_ptq_'
 
         model.eval()
         full_acc = full_inference(model, test_loader, device)
         # 传入后可变
         fold_model(model)
     
-        layer, par_ratio, flop_ratio = extract_ratio(model_name)
+        Mac,Param,layer, par_ratio, flop_ratio = extract_ratio(model_name)
         par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
 
         full_names = []
@@ -123,23 +136,14 @@ if __name__ == "__main__":
                 full_names.append(name)
                 full_params.append(param.data.cpu())
     
-        #统计每个参数对应层的参数个数
-        full_par_num=[]
-        for name in full_names:
-            prefix = name.rsplit('.',1)[0]
-            cnt = 0
-            for str in full_names:
-                sprefix = str.rsplit('.',1)[0]
-                if prefix == sprefix:
-                    cnt += 1
-            full_par_num.append(cnt)
-    
         quant_type_list = ['INT','POT','FLOAT']
         # quant_type_list = ['INT']
         # quant_type_list = ['POT']
         # quant_type_list = ['INT','POT']
         title_list = []
         js_flops_list = []
+        js_flops_wt_log_list = []
+        js_flops_wt_cbrt_list = []
         js_param_list = []
         ptq_acc_list = []
         acc_loss_list = []
@@ -205,33 +209,45 @@ if __name__ == "__main__":
                         ptq_param = param.data.cpu()
 
                         js = js_div(ptq_param,full_params[name_idx])
-                        js /= full_par_num[name_idx]
 
                         js = js.item()
                         if js < 0.:
                             js = 0.
                         js_flops = js_flops + js * flop_ratio[layer_idx]
                         js_param = js_param + js * par_ratio[layer_idx]
+                    js_flops_wt_log = js_flops * torch.log10(torch.tensor(Mac)).item()
+                    js_flops_wt_cbrt = js_flops * torch.pow(torch.tensor(Mac),1/3).item()
                     js_flops_list.append(js_flops)
+                    js_flops_wt_log_list.append(js_flops_wt_log)
+                    js_flops_wt_cbrt_list.append(js_flops_wt_cbrt)
                     js_param_list.append(js_param)
 
-                    print(title + ': js_flops: %f js_param: %f acc_loss: %f' % (js_flops, js_param, acc_loss))
+                    print(title + ': js_flops: %f js_flops_wt_log: %f js_flops_wt_cbrt: %f js_param: %f acc_loss: %f' % (js_flops,js_flops_wt_log, js_flops_wt_cbrt, js_param, acc_loss))
         
         # 写入xlsx
         worksheet = workbook.create_sheet(model_name)
         worksheet.cell(row=1,column=1,value='FP32-acc')
         worksheet.cell(row=1,column=2,value=full_acc)
+        worksheet.cell(row=1,column=3,value='Mac')
+        worksheet.cell(row=1,column=4,value=Mac)
+        worksheet.cell(row=1,column=5,value='Param')
+        worksheet.cell(row=1,column=6,value=Param)
+
         worksheet.cell(row=3,column=1,value='title')
         worksheet.cell(row=3,column=2,value='js_flops')
-        worksheet.cell(row=3,column=3,value='js_param')
-        worksheet.cell(row=3,column=4,value='ptq_acc')
-        worksheet.cell(row=3,column=5,value='acc_loss')
+        worksheet.cell(row=3,column=3,value='js_flops_wt_log')
+        worksheet.cell(row=3,column=4,value='js_flops_wt_cbrt')
+        worksheet.cell(row=3,column=5,value='js_param')
+        worksheet.cell(row=3,column=6,value='ptq_acc')
+        worksheet.cell(row=3,column=7,value='acc_loss')
         for i in range(len(title_list)):
             worksheet.cell(row=i+4, column=1, value=title_list[i])
             worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
-            worksheet.cell(row=i+4, column=3, value=js_param_list[i])
-            worksheet.cell(row=i+4, column=4, value=ptq_acc_list[i])
-            worksheet.cell(row=i+4, column=5, value=acc_loss_list[i])
+            worksheet.cell(row=i+4, column=3, value=js_flops_wt_log_list[i])
+            worksheet.cell(row=i+4, column=4, value=js_flops_wt_cbrt_list[i])
+            worksheet.cell(row=i+4, column=5, value=js_param_list[i])
+            worksheet.cell(row=i+4, column=6, value=ptq_acc_list[i])
+            worksheet.cell(row=i+4, column=7, value=acc_loss_list[i])
 
         if 'Sheet' in workbook.sheetnames:
             workbook.remove(workbook['Sheet'])
@@ -243,6 +259,10 @@ if __name__ == "__main__":
         print(title_list,file=ft)
         print('js_flops_list:',file=ft)
         print(js_flops_list, file=ft)
+        print('js_flops_wt_log_list:',file=ft)
+        print(js_flops_wt_log_list, file=ft)
+        print('js_flops_wt_cbrt_list:',file=ft)
+        print(js_flops_wt_cbrt_list, file=ft)
         print('js_param_list:',file=ft)
         print(js_param_list, file=ft)
         print('ptq_acc_list:',file=ft)
@@ -250,6 +270,7 @@ if __name__ == "__main__":
         print('acc_loss_list:',file=ft)
         print(acc_loss_list, file=ft)
         print("\n",file=ft)
+        ft.flush()
 
     ft.close()
 

ykl/ALL/ptq_nodiv.slurm → ykl/ALL-cifar100/ptq.slurm

@@ -5,7 +5,7 @@
 # (TODO)
 # Please modify job name
 
-#SBATCH -J ALL-nodiv            # The job name
+#SBATCH -J PTQ            # The job name
 #SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
 #SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
 
@@ -36,7 +36,8 @@
 ###
 
 # set constraint for RTX8000 to meet my cuda
-#SBATCH --constraint="Ampere|RTX8000|T4"
+### #SBATCH --constraint="Ampere|RTX8000|T4"
+#SBATCH --constraint="Ampere|RTX8000"
 
 #- Log information
 
@@ -82,7 +83,7 @@ echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
 
 #- Job step
 # [EDIT HERE(TODO)]
-python ptq_nodiv.py
+python ptq.py
 
 #- End
 echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL-cifar100/ptq_result_adam_lr0001.txt

+AlexNet_BN
+Full_acc: 61.600000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[30.954255140265232, 7.030371194622713, 1.2766689998565226, 0.2757054729770005, 0.06451857175083886, 0.015585305989074583, 0.0038385109756588084, 0.0009950807589378492, 0.00025233382343069265, 8.335952043027284e-05, 4.7757709274679696e-05, 6.513127580280553e-06, 2.5334270287130868e-05, 5.838298274428465e-05, 2.8918684686232932e-05, 28.06200547785621, 4.513798009748275, 1.7107755508915887, 1.696105464302549, 1.683014471954045, 1.6757721772460925, 1.6458029650269048, 5.219215206849903, 1.953296714437447, 0.7214552016271162, 1.127399234195052, 0.23986693504483542, 0.31747890361149234, 0.852028550485009, 0.11518958928234876, 0.08394291108809784, 0.31737116729413617, 0.7507391195379884, 0.07750244862722218, 0.02080548195662084, 0.08396576977211927, 0.31737483455527477, 0.7028746165055548, 0.06444410228676635, 0.005224109463362765, 0.02069640091183959, 0.08396411549703782, 0.6095935891910232]
+js_flops_wt_log_list:
+[57.17303700760195, 12.985215462816626, 2.358029409672615, 0.5092327093087785, 0.11916690205186196, 0.028786325888006052, 0.007089795217845674, 0.001837931127675612, 0.00046606487411504454, 0.00015396645549698413, 8.820930328922014e-05, 1.2029857688231946e-05, 4.679282916430444e-05, 0.00010783436454625576, 5.341330367295668e-05, 51.83100256888476, 8.337061883310653, 3.159831610859667, 3.1327357108078235, 3.108556424741473, 3.0951797829362855, 3.039826136998688, 9.63997947357203, 3.6077723348636375, 1.3325400580645537, 2.0823256074778063, 0.44303831870854443, 0.586388947918881, 1.5737112596531846, 0.21275713536273982, 0.1550439879886748, 0.5861899571038426, 1.3866279537305686, 0.14314834401490867, 0.03842807990296501, 0.15508620836779813, 0.5861967305976916, 1.298221400549966, 0.11902935568203137, 0.00964901924879733, 0.0382266053534452, 0.15508315289351765, 1.1259297526781527]
+js_flops_wt_cbrt_list:
+[127.75935038177916, 29.01687191947318, 5.269272393573333, 1.137935704303978, 0.2662913637267626, 0.06432616645571654, 0.015842916150341474, 0.004107056388959455, 0.0010414724959387158, 0.0003440547391642728, 0.00019711325260474887, 2.688202138412569e-05, 0.00010456365044526526, 0.00024096757990011566, 0.00011935781858315996, 115.82199519953035, 18.63006875361259, 7.060986350358903, 7.000437623724479, 6.946406387284803, 6.9165148307615185, 6.792821106999562, 21.54157938248916, 8.06195846769949, 2.977705245103374, 4.6531823534113625, 0.9900171611473642, 1.310349685415696, 3.5166284448917593, 0.4754288879112317, 0.34646260235221843, 1.3099050188632115, 3.0985705126400402, 0.3198804960656622, 0.08587171124334571, 0.3465569483672631, 1.309920154969657, 2.9010164837654817, 0.2659840014494256, 0.021561779740399623, 0.0854214945168547, 0.3465501205786828, 2.5160121152660033]
+js_param_list:
+[16.731479038484657, 4.656304031293315, 0.8706428224280062, 0.1886976773523543, 0.04412411803201098, 0.010671571029085497, 0.0026153783759564103, 0.0006713288941462956, 0.00014957330824896432, 5.290273061274497e-05, 2.049375903972184e-05, 1.0072775420606918e-05, 7.688045246927898e-06, 1.6384201149413303e-05, 1.5270479061967546e-05, 15.979923054948255, 2.8109546070051508, 0.7815154856447334, 0.7752273475131825, 0.773336670470333, 0.7735214074768411, 0.765129995435047, 3.0218284756335874, 1.0780299834242775, 0.4795985502521087, 0.6096152108202557, 0.16222198695304593, 0.17722546998411262, 0.45771631314033673, 0.08168191462113031, 0.04587221230256364, 0.17712998097245, 0.3986340598582839, 0.056789196450664124, 0.011615201470204397, 0.0458520675046164, 0.17713069024889352, 0.3729067953998436, 0.04774167281168079, 0.0029558263237104256, 0.011534944288256953, 0.04584962927416049, 0.2590319710287575]
+ptq_acc_list:
+[1.0, 0.99, 9.16, 37.17, 56.28, 60.51, 61.12, 61.35, 61.56, 61.73, 61.63, 61.58, 61.61, 61.61, 61.6, 1.0, 1.85, 13.68, 11.25, 13.22, 9.9, 9.83, 2.22, 9.86, 25.87, 21.26, 47.85, 45.98, 29.56, 53.17, 57.76, 45.59, 35.14, 57.07, 60.37, 57.89, 45.49, 36.72, 57.6, 61.44, 60.37, 58.01, 43.74]
+acc_loss_list:
+[0.9837662337662337, 0.9839285714285714, 0.8512987012987012, 0.39659090909090905, 0.08636363636363636, 0.01769480519480525, 0.007792207792207856, 0.004058441558441558, 0.0006493506493506355, -0.0021103896103895365, -0.00048701298701300544, 0.0003246753246753754, -0.00016233766233763003, -0.00016233766233763003, 0.0, 0.9837662337662337, 0.9699675324675324, 0.7779220779220779, 0.8173701298701299, 0.7853896103896104, 0.8392857142857143, 0.8404220779220779, 0.963961038961039, 0.839935064935065, 0.5800324675324676, 0.6548701298701299, 0.2232142857142857, 0.2535714285714286, 0.5201298701298702, 0.13685064935064933, 0.06233766233766239, 0.25990259740259736, 0.42954545454545456, 0.07353896103896106, 0.019967532467532533, 0.06022727272727274, 0.261525974025974, 0.4038961038961039, 0.06493506493506493, 0.0025974025974026572, 0.019967532467532533, 0.05827922077922083, 0.28993506493506493]
+
+
+VGG_16
+Full_acc: 63.290000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[16.42101660644123, 4.463932477307743, 0.8282606299780116, 0.1791754541527371, 0.042021239718482935, 0.010193030906997514, 0.0023648854705972653, 0.0005985344200239009, 0.00016669397776832847, 5.06575770050397e-05, 2.2607080808961125e-05, 1.639855746564467e-05, 1.4810840757795204e-05, 1.313501123970745e-05, 2.407361345529002e-05, 16.251339425711258, 3.044407251235511, 1.498938339962735, 1.4200699262372136, 1.5197331022553802, 1.4741567021338744, 1.5446690426380056, 2.7547288528300524, 0.9683869668057979, 0.4576982294995759, 0.5384629955250636, 0.15567193144042218, 0.16607241825978808, 0.40134474210030135, 0.07892531679189357, 0.0432792045417152, 0.16593509166381434, 0.34721635812394636, 0.05493566383077612, 0.011307189681095477, 0.043225216519882516, 0.16593891515269915, 0.32421387695148235, 0.04616882660025991, 0.002792132631874573, 0.011259087522936794, 0.043224878627175826, 0.8894719013976583]
+js_flops_wt_log_list:
+[41.43671612446248, 11.264266232360654, 2.0900289807882517, 0.45213049886852513, 0.10603619880207424, 0.02572104628257017, 0.005967540881334133, 0.0015103389423235666, 0.000420634800024152, 0.00012782909171942203, 5.704660145805195e-05, 4.138004283413638e-05, 3.737360595595808e-05, 3.314482562656249e-05, 6.074724304496506e-05, 41.008553524106226, 7.682242948784175, 3.78241396191911, 3.583397777412654, 3.834887367351148, 3.7198800934976544, 3.897810602111538, 6.95126983976195, 2.4436231205333034, 1.1549535610970445, 1.3587549920842121, 0.39282182012490646, 0.4190663596816859, 1.012751435860093, 0.19915977343669036, 0.10921054132447254, 0.4187198303948529, 0.8761641261423541, 0.13862439590809858, 0.02853250929648413, 0.10907430820392842, 0.41872947856873405, 0.8181197732654556, 0.11650220004552485, 0.007045654360001822, 0.028411128536670906, 0.1090734555670605, 2.244489215390195]
+js_flops_wt_cbrt_list:
+[113.90211996715433, 30.96345279598221, 5.745115780640566, 1.2428258592753079, 0.2914745415765391, 0.07070255496497892, 0.01640370234294081, 0.004151651566279505, 0.0011562498174517793, 0.00035137930565212847, 0.00015681090227991044, 0.00011374633523039414, 0.00010273335696811116, 9.110919633369247e-05, 0.000166983303990556, 112.72517757324597, 21.117111581452367, 10.397179341188412, 9.850119452200799, 10.541419345696795, 10.22528492364923, 10.714384061647543, 19.107797269320848, 6.717082815961371, 3.174760728563265, 3.734974404088432, 1.0797969112028285, 1.15193845553033, 2.7838725249789698, 0.5474545892700481, 0.30020024130904277, 1.1509859085118255, 2.408418444816564, 0.3810536656831507, 0.07843076384456708, 0.2998257608315573, 1.1510124296158526, 2.2488649023750757, 0.3202437067204954, 0.01936724343091128, 0.07809710984977603, 0.2998234170853365, 6.1696993340027735]
+js_param_list:
+[20.271194529297738, 5.93027933694188, 1.1318814554174204, 0.24568611637642826, 0.05756450266756366, 0.013878914621940313, 0.0033348875153812346, 0.0008568417282392247, 0.00019412855112987382, 3.899460808457422e-05, 6.123464184651261e-06, 1.580848006667668e-05, 1.8556590947365059e-06, 5.469557139151149e-06, 4.0998876509133865e-05, 21.410733078392443, 3.8634348092078628, 1.4541121459209971, 1.4421389614681546, 1.4213795606912738, 1.4564450181560846, 1.4804955996177944, 3.9016241349323155, 1.3920826560714143, 0.6223750421202993, 0.787901156410768, 0.21140724957664925, 0.2237914482111064, 0.5920410312690766, 0.10723405710502064, 0.05774293502685638, 0.22359087834159397, 0.5159200627882348, 0.07492795409809361, 0.014570543995158319, 0.057658728748871825, 0.2235930175192116, 0.48287569520908163, 0.06290535988771455, 0.0036722136313423162, 0.014510795463934651, 0.05765763115455179, 0.7364045539295738]
+ptq_acc_list:
+[1.0, 1.03, 8.91, 47.1, 60.23, 62.92, 63.21, 63.26, 63.24, 63.3, 63.33, 63.31, 63.29, 63.31, 63.31, 1.0, 1.03, 22.44, 27.23, 22.75, 24.82, 17.93, 1.12, 5.02, 41.43, 29.51, 58.18, 57.96, 42.84, 61.48, 62.45, 57.48, 48.3, 61.35, 63.12, 62.24, 57.82, 49.41, 62.11, 63.12, 63.05, 62.31, 42.94]
+acc_loss_list:
+[0.9841997155948807, 0.9837257070627271, 0.859219465950387, 0.2558066045188813, 0.04834887027966507, 0.0058461052298940976, 0.0012640227524095165, 0.00047400853215359673, 0.0007900142202559197, -0.00015800284405116148, -0.0006320113762047582, -0.00031600568810243523, 0.0, -0.00031600568810243523, -0.00031600568810243523, 0.9841997155948807, 0.9837257070627271, 0.645441617949123, 0.5697582556486017, 0.6405435297835361, 0.6078369410649391, 0.7167009006162111, 0.9823036814662665, 0.9206825722863011, 0.34539421709590773, 0.5337336072049297, 0.08073945331015958, 0.0842155158792858, 0.3231158160846895, 0.028598514773265957, 0.013272238900300147, 0.09179965239374313, 0.23684626323273822, 0.03065255174593139, 0.0026860483488703067, 0.016590298625375213, 0.08642755569600251, 0.21930794754305583, 0.01864433559804076, 0.0026860483488703067, 0.0037920682572286615, 0.015484278717016858, 0.3215357876441776]
+
+
+VGG_19
+Full_acc: 60.840000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[14.015103864594977, 3.8435144527962457, 0.7076177452584133, 0.15381967561171, 0.03602518650714564, 0.008724838120785536, 0.0021896544260230875, 0.0005529540029432198, 0.0001629760751164309, 7.131949008052141e-05, 2.9227706692427575e-05, 4.224468367340833e-05, 4.549431386534403e-05, 2.3138856170440975e-05, 3.33318153549819e-05, 15.39734483339301, 2.714644205355139, 1.4196819338218691, 1.540437303413022, 1.4157837024298614, 1.4685023190873225, 1.535824519309249, 2.5227128354643056, 0.9000526079719748, 0.3973690280265586, 0.5114831995065607, 0.1325113586016059, 0.15449947546200804, 0.3835608915821099, 0.06609197184321991, 0.03961150494349369, 0.15443438377230523, 0.33322336767462624, 0.045447834097109396, 0.010056585171781585, 0.03957312688799412, 0.15443565138102808, 0.3115842624046735, 0.038112580157972815, 0.0025701294365993665, 0.010064025655704877, 0.03957409556427708, 0.844387471733455]
+js_flops_wt_log_list:
+[36.74726120988221, 10.077601345335646, 1.8553565048809766, 0.4033114455328517, 0.09445716218037192, 0.02287631319296751, 0.005741220609553466, 0.0014498319369959012, 0.0004273193021704335, 0.000186997967097853, 7.663433555463681e-05, 0.0001107645323697674, 0.00011928498363807712, 6.066951768663627e-05, 8.739520857517953e-05, 40.371463386773456, 7.117731032881023, 3.722371475882786, 4.038989115804229, 3.7121504080546304, 3.85037733777645, 4.026894508157807, 6.614491652574597, 2.359916031113744, 1.0418919196522805, 1.3410964997708963, 0.3474415569698888, 0.4050938641942251, 1.0056873219805018, 0.17329154151574813, 0.1038604018306481, 0.404923195368135, 0.8737035594940974, 0.1191631148170684, 0.02636812154638754, 0.10375977550323766, 0.4049265190073531, 0.8169663521650905, 0.09993025752635584, 0.006738816826646022, 0.026387630314135442, 0.1037623153488743, 2.2139633987677203]
+js_flops_wt_cbrt_list:
+[104.85476350118113, 28.755462881685077, 5.294080732126931, 1.1508102875240702, 0.2695243965216137, 0.06527535197595877, 0.01638201894243332, 0.0041369555135521855, 0.0012193143895175794, 0.0005335806525351083, 0.00021866868076934857, 0.00031605590358466085, 0.00034036818899715025, 0.0001731146136967676, 0.0002493737934358624, 115.19607465272355, 20.309758592758797, 10.6214277722082, 11.524865651963363, 10.592262941563282, 10.986680145683465, 11.490354856256383, 18.873813587085262, 6.733792646798202, 2.972938043054819, 3.8266894371415896, 0.9913909523347744, 1.1558962471587473, 2.8696317175888852, 0.4944706899000793, 0.29635563338694737, 1.1554092601342043, 2.493028788666761, 0.3400204480775739, 0.0752388901291326, 0.29606850587280825, 1.1554187438179593, 2.3311346430807323, 0.2851413459003671, 0.019228563473071236, 0.07529455651517215, 0.29607575307722, 6.317330895822822]
+js_param_list:
+[15.48721233615551, 5.038649289056825, 0.9805414226093438, 0.21248447040121501, 0.0495578065878288, 0.012001135256917515, 0.003004822777167226, 0.0007228562814786265, 0.00021507286158543073, 6.094618761402669e-05, 1.3936222034342194e-05, 3.5241469722179125e-05, 2.8869367953322092e-05, 1.6632959955752478e-05, 3.3417124642363266e-05, 16.37163630644863, 3.2369734271078485, 1.03654177030809, 1.0745363407793256, 1.0523130489969477, 1.0448524450751904, 1.1006536161908438, 2.9967337832454364, 1.0261928088209593, 0.5314980935211822, 0.5656530882303566, 0.18388125278642936, 0.1706968179771684, 0.4199915941156042, 0.09560615249284497, 0.04429630850628619, 0.1705218979415742, 0.36461441659812127, 0.06754368352095583, 0.011241337203897597, 0.04419386157704079, 0.17052542977686416, 0.34078178281609445, 0.05721551424670466, 0.002847753216944273, 0.011174315993409518, 0.04419458953738452, 0.5084200788650286]
+ptq_acc_list:
+[1.0, 0.97, 5.33, 33.99, 52.84, 59.66, 60.93, 60.81, 60.89, 60.85, 60.85, 60.84, 60.83, 60.85, 60.85, 1.0, 0.99, 23.69, 25.0, 24.98, 24.18, 24.35, 0.96, 1.23, 32.37, 27.31, 52.21, 53.02, 40.75, 58.14, 59.57, 52.88, 44.72, 58.94, 60.39, 59.45, 53.2, 46.95, 59.88, 60.69, 60.63, 59.81, 42.89]
+acc_loss_list:
+[0.9835634451019066, 0.9840565417488495, 0.9123931623931624, 0.4413214990138067, 0.13149243918474687, 0.019395134779750276, -0.0014792899408283416, 0.0004930966469428195, -0.0008218277449046212, -0.0001643655489809009, -0.0001643655489809009, 0.0, 0.00016436554898101768, -0.0001643655489809009, -0.0001643655489809009, 0.9835634451019066, 0.9837278106508875, 0.6106180144641684, 0.589086127547666, 0.5894148586456278, 0.6025641025641026, 0.5997698882314267, 0.9842209072978304, 0.9797830374753452, 0.467948717948718, 0.5511176857330703, 0.14184746877054571, 0.12853385930309008, 0.33021038790269563, 0.04437869822485212, 0.020874424720578617, 0.13083497698882315, 0.264957264957265, 0.031229454306377476, 0.007396449704142058, 0.022846811308349778, 0.12557527942143326, 0.22830374753451677, 0.01577909270216964, 0.002465483234714097, 0.0034516765285996192, 0.01692965154503618, 0.29503616042077585]
+
+
+Inception_BN
+Full_acc: 68.440000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[21.71831660555812, 4.697612319470905, 0.7047081291631722, 0.15340213253769422, 0.03591944913709296, 0.00869243794117787, 0.0021348319103553373, 0.000565929566061959, 0.00010927400352406203, 6.0877663578580146e-05, 4.8938011472047214e-05, 2.3124638859600032e-05, 3.095444712318846e-05, 1.7562018791028706e-05, 1.96634308605024e-05, 20.337903723937306, 2.9287138069294056, 2.3827509775633873, 2.2791241597204044, 2.313675344931919, 2.517667888963041, 2.420379552390045, 3.596532276314623, 1.3790708377263001, 0.4310201285871601, 0.81372898334754, 0.13694590672105744, 0.22301281454319888, 0.6207682134481936, 0.062281192511334835, 0.05752468396335935, 0.22295542729537426, 0.5429266349561096, 0.040273516009724265, 0.01454000506810312, 0.057493410271407586, 0.22295641293271715, 0.5081720800586023, 0.032879550163518, 0.00365648681431155, 0.014561366155643511, 0.05749414078959679, 1.6236628786529401]
+js_flops_wt_log_list:
+[68.9481149459668, 14.913288173153338, 2.2372036459061917, 0.48699851186710197, 0.1140317803114764, 0.02759547257800063, 0.006777350133473437, 0.0017966298899139896, 0.0003469070228792437, 0.00019326544604215078, 0.00015536119587363585, 7.34126998483684e-05, 9.826962269222351e-05, 5.5753312389653275e-05, 6.242456613119945e-05, 64.56578330565321, 9.297649531102756, 7.564407098044206, 7.235427929083466, 7.345115946472126, 7.992721450592023, 7.683864759037016, 11.417741314755448, 4.378071116886187, 1.3683392644827506, 2.583307007521594, 0.4347557082562601, 0.7079882594405575, 1.9707235562020946, 0.19772116311926508, 0.18262089986850474, 0.7078060748527139, 1.7236035699286567, 0.1278544309647863, 0.04615946801760657, 0.18252161675431886, 0.7078092039090789, 1.6132699243939008, 0.10438115647811108, 0.01160807615757133, 0.04622728205429525, 0.1825239358959445, 5.154565928107548]
+js_flops_wt_cbrt_list:
+[248.33833413894462, 53.714900608338795, 8.057992984859936, 1.754078400257928, 0.41072134290607865, 0.0993937788606428, 0.024410759356396312, 0.006471127952883577, 0.0012494948154920021, 0.0006961063250872148, 0.000559582239533728, 0.0002644189416820687, 0.00035394897184606736, 0.00020081310029179553, 0.00022484172010384004, 232.553987604558, 33.4884107820973, 27.24559270328526, 26.060670695625593, 26.455746609371033, 28.78830163568875, 27.67585706300471, 41.124588539626714, 15.769000918464473, 4.9285044811574314, 9.304593161395807, 1.565909502099771, 2.5500425222239356, 7.098181079774878, 0.7121549923668377, 0.6577666421749271, 2.549386327096826, 6.208100679873033, 0.46050796926007986, 0.16625785057670905, 0.657409042794518, 2.549397597378048, 5.810699333178481, 0.3759615840924008, 0.04181013937492564, 0.16650210417110686, 0.657417395912923, 18.565791346126876]
+js_param_list:
+[18.32723971827104, 6.621253743239008, 0.861933701911443, 0.18789982007951292, 0.04395796559434963, 0.010623812268730234, 0.002611874104488947, 0.0006858536781494174, 0.0001485722244224492, 6.123368155953467e-05, 5.664373689038539e-05, 2.1275306823323014e-05, 1.9210450771633464e-05, 2.9407106742084076e-05, 2.2788188711075516e-05, 18.978629559677433, 3.4103876644527706, 2.9848876761403993, 2.7472829539231682, 2.8294276855419622, 3.0776664326209304, 3.0348223591240493, 3.1576499562573233, 1.1330692970868683, 0.4869886890415182, 0.6418872885961755, 0.16301100077808972, 0.1897346798912318, 0.4819072568356359, 0.08109993364229118, 0.04895814031554814, 0.1896555118674395, 0.4194379033173077, 0.05582817152546735, 0.012417857808916272, 0.0489046170295227, 0.18965622798305004, 0.39219254222988986, 0.0468403253289113, 0.0031186607016981892, 0.012428292248452783, 0.04890472579998137, 2.1307484643587236]
+ptq_acc_list:
+[1.0, 0.99, 18.15, 54.29, 65.94, 67.72, 68.38, 68.36, 68.52, 68.4, 68.41, 68.43, 68.43, 68.45, 68.43, 1.0, 1.06, 2.67, 1.78, 2.71, 3.03, 3.66, 1.07, 1.75, 30.72, 2.8, 57.67, 50.06, 6.13, 64.38, 63.32, 46.54, 11.57, 65.84, 67.87, 66.54, 29.29, 18.75, 66.35, 68.27, 67.69, 65.69, 5.27]
+acc_loss_list:
+[0.9853886616014027, 0.9855347749853888, 0.7348042080654588, 0.20675043834015194, 0.03652834599649328, 0.010520163646990047, 0.000876680303915872, 0.0011689070718877601, -0.0011689070718877601, 0.0005844535359437762, 0.000438340151957936, 0.00014611338398584024, 0.00014611338398584024, -0.00014611338398604786, 0.00014611338398584024, 0.9853886616014027, 0.9845119812974868, 0.9609877264757452, 0.9739918176504968, 0.9604032729398013, 0.9557276446522501, 0.9465225014611339, 0.984365867913501, 0.9744301578024547, 0.5511396843950906, 0.9590882524839276, 0.15736411455289298, 0.26855639976621853, 0.9104324956165984, 0.05932203389830512, 0.0748100526008182, 0.3199883109292811, 0.8309468147282291, 0.03798947983635293, 0.008328462887200369, 0.027761542957334767, 0.5720338983050848, 0.7260374050263004, 0.030537697253068433, 0.002483927527761568, 0.010958503798947984, 0.04018118059614261, 0.9229982466393922]
+
+
+MobileNetV2
+Full_acc: 50.300000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[24.943058840686717, 3.893947362340972, 0.6866993578702529, 0.14941589537450753, 0.034537665473809966, 0.008377922613199295, 0.0020675590896694503, 0.0005277259420932586, 0.00013953845297348135, 3.3769229097913906e-05, 9.89793747312717e-06, 9.66509468936261e-06, 1.1227369808491925e-05, 6.138225473374198e-06, 7.894201357233607e-06, 23.738344515592697, 2.285147128881083, 1.1085230231764136, 1.1081885728167844, 1.1079220932485532, 1.107702181324594, 1.1081500840432619, 4.549060714027057, 1.7909816975415174, 0.4559357214826594, 1.0733191392627879, 0.1367259627503318, 0.28954548605424946, 0.8246443175920913, 0.056461880183734235, 0.07344399062115757, 0.2894901687664786, 0.7221736506676362, 0.03366192186997838, 0.018724227613793155, 0.07343372898773493, 0.28949074317336, 0.6769354706149453, 0.026331287861963485, 0.004721447912070984, 0.018710270282897467, 0.07343436432320832, 0.31530800794071917]
+js_flops_wt_log_list:
+[44.21261345213266, 6.902184316436191, 1.2172032893504303, 0.264845914368036, 0.0612194543935329, 0.014850217705659056, 0.003664834830597785, 0.00093541627093357, 0.00024733773521641063, 5.985736882627956e-05, 1.7544507522827225e-05, 1.7131783964764304e-05, 1.9900981856214543e-05, 1.088026099243471e-05, 1.399280157858383e-05, 42.07720700034872, 4.050518716881178, 1.9649033520517059, 1.9643105248221484, 1.9638381786588992, 1.963448375589852, 1.9642423018637347, 8.063400090749399, 3.174589852792386, 0.8081651068412848, 1.9025029976513859, 0.24235247884235359, 0.5132314658555307, 1.4617164912317975, 0.10008104055273692, 0.130182540499748, 0.5131334136528296, 1.2800829547898636, 0.05966716228346395, 0.033189475395375895, 0.1301643513259662, 0.5131344318130149, 1.1998963914368128, 0.04667330736673084, 0.008368963598412701, 0.03316473544348563, 0.13016547748462529, 0.5588966117782364]
+js_flops_wt_cbrt_list:
+[97.22925657385598, 15.178796217269259, 2.6767874975525965, 0.5824304276540145, 0.13462949990469666, 0.032657549842521144, 0.008059445896153148, 0.0020571013904986104, 0.0005439276767429551, 0.00013163409753524834, 3.8582641698974954e-05, 3.767500920255127e-05, 4.376483360488162e-05, 2.3927101454187458e-05, 3.077198085890482e-05, 92.53322153818104, 8.90761465632946, 4.321076662122754, 4.3197729583534885, 4.318734208035145, 4.31787698065919, 4.319622927242231, 17.732459926394817, 6.981333769108412, 1.7772596187325975, 4.183850210335258, 0.5329644530555695, 1.128662387928496, 3.214503660094255, 0.2200911552223829, 0.2862882477053805, 1.1284467584501712, 2.8150680163214767, 0.13121580874140748, 0.07298794997184281, 0.2862482473591804, 1.1284489975161336, 2.6387273901227903, 0.10264064082127715, 0.018404433609162253, 0.07293354361713665, 0.286250723927758, 1.229085951312921]
+js_param_list:
+[26.089298367664295, 4.370053256116123, 0.7869862183623308, 0.17056755863537207, 0.03990258208152068, 0.009672586532405027, 0.0023807657690726406, 0.0006086038460787329, 0.00016859429652491142, 4.0041267543034166e-05, 1.2522874249115467e-05, 9.133500232666013e-06, 9.649836320315574e-06, 3.5987794909059226e-06, 1.0537985858346685e-05, 24.731258628120905, 2.604903153666661, 1.1432817285729853, 1.143141719791697, 1.1425625712807268, 1.1424401731436173, 1.1427752659915693, 4.727417561722829, 1.8532421603793925, 0.49805644984465886, 1.1075984611633638, 0.15327424975276238, 0.28947395726853087, 0.8491768020826799, 0.06628769060108393, 0.07409990300052388, 0.28939540644091033, 0.7436098724468654, 0.041090835950582626, 0.01883602993099688, 0.07408649516268899, 0.2893964389155294, 0.6962273651556354, 0.032906093987645824, 0.004726829206045366, 0.018810946611709636, 0.07408736223387083, 0.3258388955068092]
+ptq_acc_list:
+[1.0, 1.11, 5.68, 33.68, 44.84, 48.81, 50.22, 49.97, 50.27, 50.19, 50.29, 50.28, 50.25, 50.28, 50.28, 1.0, 1.43, 1.59, 2.04, 1.42, 1.49, 1.95, 1.05, 1.22, 11.38, 3.04, 33.19, 16.76, 5.75, 42.03, 45.26, 17.89, 7.87, 44.72, 48.36, 44.23, 18.69, 9.08, 45.43, 50.08, 49.12, 44.75, 19.96]
+acc_loss_list:
+[0.9801192842942346, 0.9779324055666004, 0.8870775347912525, 0.33041749502982104, 0.10854870775347901, 0.029622266401590356, 0.0015904572564611988, 0.006560636182902551, 0.0005964214711728436, 0.0021868787276341835, 0.00019880715705761454, 0.00039761431411522907, 0.000994035785288214, 0.00039761431411522907, 0.00039761431411522907, 0.9801192842942346, 0.9715705765407555, 0.9683896620278329, 0.9594433399602386, 0.9717693836978131, 0.9703777335984095, 0.9612326043737573, 0.9791252485089463, 0.9757455268389662, 0.7737574552683896, 0.9395626242544731, 0.34015904572564615, 0.6667992047713717, 0.885685884691849, 0.16441351888667985, 0.10019880715705765, 0.6443339960238569, 0.8435387673956263, 0.11093439363817095, 0.038568588469184845, 0.12067594433399603, 0.6284294234592445, 0.8194831013916501, 0.09681908548707749, 0.004373757455268367, 0.023459244532803177, 0.11033797216699796, 0.6031809145129224]
+
+
+ResNet_18
+Full_acc: 43.260000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[46.311099294601945, 9.548303298968824, 1.7068367445029802, 0.3690169578949064, 0.08744178551322478, 0.020969894114682308, 0.005193754056287901, 0.001289666218046486, 0.00032020181224332234, 9.251734120127963e-05, 3.093387880740029e-05, 2.2315740581704047e-05, 1.7777078448398154e-05, 1.4924990035261266e-05, 2.5591690802796662e-05, 41.84219818922616, 6.424586394489597, 2.9036512979876132, 3.0275134763395477, 2.8653664753646004, 2.9401542897087785, 3.0035196956581642, 7.661224630509365, 2.8805547522073174, 1.0140070022483827, 1.6784576855410833, 0.32870288355107435, 0.47857924130777696, 1.274373233158023, 0.15485956930700528, 0.12357650602487279, 0.4783970006581733, 1.1129624975475279, 0.10272283076757197, 0.031025385803962198, 0.12349215653738986, 0.47839373946143815, 1.0409729168710533, 0.0847966975499837, 0.007782104196933858, 0.030982875843929922, 0.12349244712610134, 1.3100030176859263]
+js_flops_wt_log_list:
+[71.88322523002209, 14.820698429940787, 2.6493201846816574, 0.5727812447144618, 0.13572550983031995, 0.032549078831111335, 0.008061648250725096, 0.002001795868278863, 0.0004970112853967791, 0.00014360369277672418, 4.801499017341939e-05, 3.463807663482422e-05, 2.7593249858072923e-05, 2.3166291377271925e-05, 3.972294551446642e-05, 64.946680220695, 9.972123267189673, 4.506993429694263, 4.699249994525819, 4.447568441529013, 4.56365269314146, 4.662007295321729, 11.891610090043557, 4.471143401781969, 1.5739227709486943, 2.605270738258521, 0.510206489847105, 0.7428417791272324, 1.9780583821487325, 0.24037013737692053, 0.19181314956953593, 0.7425589085873173, 1.7275196465289284, 0.1594444634828936, 0.04815702562807657, 0.1916822238670774, 0.7425538466185235, 1.615778761065105, 0.13161985359000175, 0.01207923710022796, 0.048091042460368105, 0.1916826749134597, 2.0333622696644182]
+js_flops_wt_cbrt_list:
+[152.43295444381062, 31.428234353250442, 5.618052079972356, 1.2146190867540996, 0.28781458247907515, 0.06902239339949089, 0.017095238236911624, 0.004244935552331316, 0.0010539440652880636, 0.00030452077086066834, 0.00010181884280100494, 7.345224620628446e-05, 5.8513242625285314e-05, 4.9125595392301354e-05, 8.42350343157908e-05, 137.72356924273325, 21.14652211042868, 9.557366436310195, 9.965058719102375, 9.431351966524907, 9.677516010793385, 9.886083204954595, 25.21691609299277, 9.481344170279737, 3.337603415461339, 5.524642425099377, 1.0819253361953858, 1.5752432742732378, 4.194598702109221, 0.5097201757876295, 0.4067519925058309, 1.5746434292887315, 3.663315366519636, 0.3381121334029251, 0.10212003802311528, 0.40647435622038985, 1.5746326950617118, 3.4263617066231378, 0.27910827709781644, 0.025614791110486035, 0.10198011651618263, 0.40647531269295245, 4.311874115660411]
+js_param_list:
+[97.81722707993141, 22.73239077300788, 4.1429612857486955, 0.9051066244569851, 0.2123234938429031, 0.05120082275701424, 0.0125970394999174, 0.0031343074684991737, 0.0007879633449540001, 0.00022757400273937378, 6.47977291347397e-05, 4.595541245012092e-05, 2.6343920545673693e-05, 2.5207733679211142e-05, 1.710209183983391e-05, 96.76805130718965, 15.417926002759907, 6.8064063417673175, 7.183530213004662, 6.713597808537161, 6.874996418338524, 7.066415702786158, 17.7478938276552, 6.646934943267997, 2.405351970591189, 3.8718121712536933, 0.7873832018324953, 1.0715453586195163, 2.941115688672916, 0.37673368505084254, 0.27741269676365804, 1.0711173153398603, 2.5703957050923085, 0.25275853273544635, 0.07004410235166722, 0.2771727906181428, 1.0711062086309968, 2.4056015001106976, 0.20973276395637827, 0.017605165510262966, 0.06992661593008742, 0.27717556472297894, 3.3668324246675914]
+ptq_acc_list:
+[1.0, 1.42, 9.05, 31.18, 40.7, 42.74, 43.14, 43.14, 43.25, 43.25, 43.28, 43.3, 43.27, 43.27, 43.29, 1.0, 1.5, 3.79, 1.6, 2.49, 2.56, 2.13, 1.14, 3.64, 15.84, 13.98, 33.29, 27.58, 21.1, 39.24, 40.47, 28.76, 24.81, 40.48, 42.96, 40.82, 27.59, 26.0, 40.9, 43.15, 42.8, 40.33, 16.98]
+acc_loss_list:
+[0.9768839574664817, 0.967175219602404, 0.7907998150716596, 0.27924179380490055, 0.05917706888580664, 0.012020342117429405, 0.0027739251040221325, 0.0027739251040221325, 0.00023116042533513663, 0.00023116042533513663, -0.0004623208506704375, -0.0009246417013407108, -0.0002311604253353009, -0.0002311604253353009, -0.0006934812760055742, 0.9768839574664817, 0.9653259361997226, 0.9123901987979658, 0.9630143319463708, 0.9424410540915394, 0.9408229311141932, 0.950762829403606, 0.9736477115117892, 0.9158576051779935, 0.6338418862690707, 0.6768377253814146, 0.23046694405917706, 0.36245954692556637, 0.5122515025427646, 0.09292649098474333, 0.06449375866851594, 0.33518261673601474, 0.4264909847434119, 0.0642625982431808, 0.006934812760055413, 0.05640314378178451, 0.36222838650023115, 0.39898289412852517, 0.054553860379103085, 0.002542764678686996, 0.01063337956541842, 0.06773000462320851, 0.6074895977808599]
+
+
+ResNet_50
+Full_acc: 37.100000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[38.37237809889991, 5.246261241387925, 0.7754120557277933, 0.1668912034928133, 0.03906485368457993, 0.009485100609602, 0.002337019188278503, 0.0005853575140575716, 0.00014612157339234067, 3.7508880691788716e-05, 2.1247760613417826e-05, 1.6771571663420586e-05, 1.273911537180461e-05, 1.1578714388590811e-05, 1.0678311601581226e-05, 33.472594844739106, 3.1253180157294405, 2.310301727722747, 2.3562521237083858, 2.450588535171482, 2.3449475271695834, 2.40319084852709, 6.129155830656942, 2.389971497133795, 0.6015240414213472, 1.4174562107697137, 0.1776237283207883, 0.4437904127539243, 1.0797579519238398, 0.06561295493894756, 0.11793938875573966, 0.4437573132314104, 0.9408953669480978, 0.035047808297243986, 0.029808419005175665, 0.11792546678810481, 0.4437592433449091, 0.8786793832680858, 0.025876138724446832, 0.007555920273236011, 0.02979701265824952, 0.11792465578571457, 0.9838703174427896]
+js_flops_wt_log_list:
+[73.99909773976927, 10.11713679483832, 1.4953410589384712, 0.3218408420594206, 0.07533450021117902, 0.018291513892426853, 0.0045068176615849385, 0.001128830946672456, 0.00028178771103213636, 7.233388874849514e-05, 4.0975180384470415e-05, 3.2343087196010536e-05, 2.4566708925016877e-05, 2.232893712070371e-05, 2.0592558059980522e-05, 64.55012538278302, 6.0270101768974, 4.4552944547112805, 4.5439073583723095, 4.725830128816315, 4.522107042998907, 4.634426201814515, 11.819752224010912, 4.608933383170657, 1.1600072379990285, 2.7334891888183006, 0.34253794745364136, 0.8558262937133854, 2.082255991895274, 0.12653110664677447, 0.2274398613870332, 0.8557624630382228, 1.814466855357199, 0.0675878410524368, 0.05748395644770392, 0.22741301361020047, 0.8557661851598248, 1.694486627771432, 0.04990076230519226, 0.01457119184461297, 0.057461959912100465, 0.2274114496350674, 1.8973417700633484]
+js_flops_wt_cbrt_list:
+[168.59101358531396, 23.049718157663378, 3.4068126839698896, 0.7332450723488758, 0.17163344062937108, 0.04167327658478704, 0.010267813808795344, 0.002571798295909496, 0.000641992635989423, 0.00016479719338689003, 9.335312731990723e-05, 7.368676131740369e-05, 5.596995753501045e-05, 5.0871676229088e-05, 4.691571034900252e-05, 147.0635642560528, 13.731245186061955, 10.15042927391165, 10.352314698210842, 10.766786566114712, 10.302647372830652, 10.558542395988434, 26.928760871936024, 10.50046249683306, 2.6428267640276024, 6.227666647882724, 0.780398971291869, 1.949815966912249, 4.7439720069593925, 0.2882738867258745, 0.5181727606434839, 1.9496705424604357, 4.133872109324737, 0.15398434544627654, 0.13096481955082095, 0.5181115937637922, 1.949679022513203, 3.860522990258247, 0.11368814421626247, 0.0331973237142475, 0.13091470517988912, 0.518108030583146, 4.322684761071398]
+js_param_list:
+[49.945718323491185, 7.44807442429426, 1.2068698334049077, 0.26343882101115146, 0.06137353593833893, 0.014881871606304302, 0.003682648051475471, 0.0009193966025286882, 0.00022807705200574002, 6.18077100265939e-05, 3.442866724336601e-05, 2.5122784867850573e-05, 2.0130562501819766e-05, 1.3678309485367636e-05, 1.0095147817679504e-05, 45.007906204119216, 4.589554523953724, 3.0832926106129595, 3.150104947891661, 3.303810391657527, 3.1297490888092074, 3.2184388272932445, 8.444122701642135, 3.320489217723546, 0.8392835371246071, 1.984797846403652, 0.25266529455515846, 0.5507232081921996, 1.5199836485663667, 0.10228204035022223, 0.1430896593264097, 0.5506574324215415, 1.3291535361589786, 0.05952715689294671, 0.03659003048680164, 0.14305481919196644, 0.550659740459459, 1.243222111597989, 0.04618834437089072, 0.009243521839648905, 0.03658159618440049, 0.14305265911851456, 1.414802210943476]
+ptq_acc_list:
+[1.0, 1.04, 3.5, 23.35, 34.27, 36.68, 36.76, 36.92, 37.08, 37.14, 37.12, 37.11, 37.11, 37.1, 37.11, 1.0, 1.01, 1.77, 2.31, 1.75, 2.12, 1.69, 1.0, 1.54, 5.97, 1.66, 26.26, 20.46, 3.58, 33.43, 34.02, 18.35, 4.11, 34.56, 36.38, 33.36, 18.47, 4.98, 35.06, 36.86, 36.49, 33.71, 8.05]
+acc_loss_list:
+[0.9730458221024259, 0.9719676549865229, 0.9056603773584906, 0.37061994609164417, 0.07628032345013472, 0.011320754716981178, 0.009164420485175294, 0.004851752021563334, 0.0005390835579515668, -0.0010781671159029419, -0.0005390835579513752, -0.0002695417789756876, -0.0002695417789756876, 0.0, -0.0002695417789756876, 0.9730458221024259, 0.9727762803234502, 0.9522911051212937, 0.9377358490566037, 0.9528301886792453, 0.942857142857143, 0.9544474393530998, 0.9730458221024259, 0.9584905660377359, 0.8390835579514825, 0.955256064690027, 0.2921832884097035, 0.4485175202156334, 0.9035040431266846, 0.09892183288409707, 0.08301886792452826, 0.5053908355795148, 0.8892183288409704, 0.06846361185983825, 0.019407008086253338, 0.10080862533692728, 0.502156334231806, 0.865768194070081, 0.05498652291105119, 0.006469002695417843, 0.0164420485175202, 0.0913746630727763, 0.7830188679245282]
+
+
+ResNet_152
+Full_acc: 38.560000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[33.66854346221386, 4.154697929171488, 0.6787129152558027, 0.14747151575352796, 0.03448815229468351, 0.008341607924202353, 0.0020673235097247796, 0.000512707024238025, 0.00012720980204321603, 3.9677057618262354e-05, 2.0056850957330397e-05, 1.8724513245024113e-05, 1.640912415376573e-05, 1.3640162378495634e-05, 1.3084535140359858e-05, 32.37820625905684, 3.5554461953275474, 3.4296427937882705, 3.579874842605743, 3.567857679796559, 3.6048307928777223, 3.398603467915731, 5.664133830023334, 2.1992671598090263, 0.5476609775673071, 1.301018702611541, 0.16208696107237489, 0.41433650181429, 0.9889468949881792, 0.05892148576366311, 0.11162578635099828, 0.41430854036032155, 0.8615209273455552, 0.030365965428374823, 0.0286719451513968, 0.11160863344587821, 0.4143088747267852, 0.8044598801299715, 0.022023678061550856, 0.00720633651260789, 0.028664658465049106, 0.11160903338209736, 2.139758314664933]
+js_flops_wt_log_list:
+[80.14251184444474, 9.889585166415209, 1.6155661117598328, 0.3510320460182508, 0.08209345785545138, 0.01985584593575831, 0.004920928612500273, 0.0012204159888544257, 0.00030280231986927627, 9.44448061346346e-05, 4.774208356278029e-05, 4.457066953920834e-05, 3.9059261008009835e-05, 3.2468196202356765e-05, 3.1145615599386294e-05, 77.07107322686993, 8.463163520605345, 8.163708909233511, 8.521311956875948, 8.492707048147048, 8.580715552475684, 8.089824823805957, 13.482552729690216, 5.234999090528037, 1.303618210591343, 3.0968641961274055, 0.38582174521899265, 0.9862609008169458, 2.3540278282031006, 0.14025304884383472, 0.26570709584809876, 0.9861943431068279, 2.050710961153298, 0.07228126000587932, 0.0682489192464756, 0.26566626613702754, 0.9861951390120116, 1.9148863847957367, 0.05242379676706152, 0.017153516307139856, 0.06823157447040644, 0.26566721812038896, 5.09334792785858]
+js_flops_wt_cbrt_list:
+[209.25233064489606, 25.82173552533868, 4.218247799031159, 0.9165456892958245, 0.21434625633264953, 0.05184367127209551, 0.012848558866004821, 0.003186509683147471, 0.0007906177345715357, 0.0002465956624785721, 0.00012465471851875388, 0.00011637414731379309, 0.00010198384366910652, 8.477455436343253e-05, 8.132129257679828e-05, 201.2327955146802, 22.097344474963432, 21.31546761697896, 22.249170210555512, 22.174482710979444, 22.404273170792877, 21.122556055842832, 35.20298424394774, 13.668597794178515, 3.4037509251873566, 8.08592138948926, 1.007381694714773, 2.5751300697993544, 6.146373480413772, 0.3662011169251766, 0.6937619971659957, 2.57495628742538, 5.354412261662942, 0.18872657931582398, 0.17819812590900544, 0.6936553906720815, 2.5749583655361215, 4.999773899231263, 0.13687868526075386, 0.04478788078156616, 0.17815283864843104, 0.6936578763033967, 13.298746198003496]
+js_param_list:
+[35.64931536330689, 4.658213924601981, 0.8023829211088676, 0.17474871555875185, 0.0408733507736506, 0.009907835396837386, 0.002445121924268682, 0.000602245475093518, 0.00014976819273500744, 4.356943795178302e-05, 2.1737175879709135e-05, 1.9698749128066407e-05, 1.9920456092062944e-05, 1.5797760223147713e-05, 1.5120632188557826e-05, 35.4227787977079, 4.133444357825877, 3.8702658650770454, 4.041708078037307, 4.02899935226451, 4.0655213543433195, 3.8387725234234376, 6.259461479874493, 2.4405641709634938, 0.6076009961373882, 1.4491747943412618, 0.1810519820147383, 0.43515448218732355, 1.1046365092390036, 0.06890011038225868, 0.11574239945601895, 0.4351194006313603, 0.9638271859206384, 0.03748131594762573, 0.029742992361991686, 0.1157150903462242, 0.43511924572879146, 0.9006129278534172, 0.028047717722324936, 0.007477854161178018, 0.02973171027232506, 0.11571523498310131, 2.5197745411619548]
+ptq_acc_list:
+[1.0, 1.19, 3.77, 31.35, 37.12, 38.47, 38.39, 38.55, 38.46, 38.54, 38.51, 38.52, 38.54, 38.56, 38.53, 1.0, 0.99, 1.01, 1.03, 1.0, 0.97, 0.91, 1.0, 0.99, 1.32, 1.0, 2.77, 1.9, 1.13, 7.99, 3.02, 1.66, 1.06, 6.68, 3.68, 2.76, 1.84, 1.28, 11.22, 5.78, 5.13, 2.94, 1.15]
+acc_loss_list:
+[0.9740663900414938, 0.9691390041493777, 0.9022302904564314, 0.18698132780082988, 0.03734439834024909, 0.0023340248962656486, 0.004408713692946102, 0.0002593360995851949, 0.002593360995850659, 0.0005186721991702056, 0.0012966804979254217, 0.0010373443983402268, 0.0005186721991702056, 0.0, 0.0007780082987552162, 0.9740663900414938, 0.9743257261410788, 0.9738070539419088, 0.9732883817427386, 0.9740663900414938, 0.9748443983402489, 0.9764004149377594, 0.9740663900414938, 0.9743257261410788, 0.9657676348547718, 0.9740663900414938, 0.9281639004149377, 0.9507261410788382, 0.9706950207468878, 0.7927904564315352, 0.9216804979253111, 0.9569502074688797, 0.9725103734439834, 0.8267634854771785, 0.9045643153526971, 0.9284232365145229, 0.9522821576763485, 0.966804979253112, 0.7090248962655602, 0.850103734439834, 0.866960580912863, 0.9237551867219918, 0.9701763485477178]
+
+
+AlexNet
+Full_acc: 56.880000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[33.87359446965535, 16.740572236264075, 3.8246736674371498, 0.833928087283217, 0.1946716118354284, 0.047067667751298656, 0.011615297169904466, 0.002902892826769317, 0.000730158297205726, 0.00020891966597511133, 7.95563599103879e-05, 5.2209678902559896e-05, 7.887115795261579e-06, 7.152630183898668e-06, 1.6723434333818066e-05, 33.8311148521711, 10.488390582380545, 1.3429597361471546, 1.3382069314135554, 1.338223249744882, 1.3381893231696378, 1.338210073262586, 7.258449801385262, 2.0160824353992757, 1.8050176598979768, 0.8980064073289054, 0.647597640019581, 0.3764314204520171, 0.5963570555891465, 0.35556753508190725, 0.09788623073706697, 0.37576247481030245, 0.4969683201711997, 0.26011238737184705, 0.024912719409893935, 0.0975442185037343, 0.3757619024857827, 0.4587752260822144, 0.2235077581418984, 0.006317632878853032, 0.024808516446777278, 0.09754686903029722, 0.3757628998916201]
+js_flops_wt_log_list:
+[62.52739923899464, 30.901487134582304, 7.059979937379987, 1.5393510864895057, 0.3593450823365269, 0.08688239021607877, 0.02144072203712518, 0.0053584611131251335, 0.0013478020290390354, 0.0003856456206620018, 0.00014685339291565844, 9.63740485174821e-05, 1.4558857596771575e-05, 1.3203067761768185e-05, 3.0869852214074516e-05, 62.44898594853805, 19.36056080220566, 2.4789745788328323, 2.470201358165361, 2.4702314802363645, 2.4701688550397085, 2.4702071577166556, 13.398400603573009, 3.7214943766851367, 3.3318890900397844, 1.6576335056656009, 1.1954029921450808, 0.6948562170982561, 1.100817798867428, 0.6563434902784869, 0.18068857247434625, 0.6936214077470918, 0.9173557471827785, 0.48014246338035554, 0.04598648525673686, 0.18005725076807239, 0.6936203512910438, 0.8468549668651244, 0.4125738364988264, 0.011661743002069475, 0.045794136603495524, 0.18006214338532153, 0.6936221924063426]
+js_flops_wt_cbrt_list:
+[139.68911739076165, 69.03535916139256, 15.772323465423446, 3.438981906225315, 0.8027936232952317, 0.19409929972770973, 0.047899570013122235, 0.011971051292316686, 0.003011052404951551, 0.000861550248875403, 0.0003280773083783373, 0.00021530410573011345, 3.252516485033172e-05, 2.9496267315427067e-05, 6.896468527797117e-05, 139.5139384537053, 43.252393087919664, 5.538144480113536, 5.518544697192303, 5.518611991299587, 5.518472083698219, 5.518557653659573, 29.93264996689242, 8.31399148502611, 7.4435951582215365, 3.7032303307343666, 2.6705858700562013, 1.5523410994424647, 2.4592781501126844, 1.4663018769057627, 0.40366667283032076, 1.5495824779339848, 2.049415395761236, 1.0726605895611911, 0.10273594641068477, 0.402256270782381, 1.549580117762049, 1.8919133742827652, 0.9217091352026001, 0.026052876131477637, 0.10230623057523344, 0.40226720111681974, 1.5495842308992371]
+js_param_list:
+[16.908999565606642, 8.938215392372381, 2.2406724487195606, 0.49450757096648745, 0.11290482808020397, 0.02724221263697541, 0.006689312570274382, 0.001667825756809197, 0.0004132059555793042, 9.423551553131659e-05, 4.1809971545706765e-05, 1.7976497508185753e-05, 5.959596778938092e-06, 3.0228186088649967e-06, 7.170491602584759e-06, 16.900737113597582, 5.7971680827348155, 0.6602696296814521, 0.6573461876334223, 0.6574078853478714, 0.6573612661856714, 0.6573367159300887, 3.9010552175892337, 1.0336175916249588, 0.9979067410050385, 0.4347818470896162, 0.360209614136694, 0.18502362163867106, 0.2783401308316823, 0.1995933769954112, 0.04802116166788542, 0.18465816480058367, 0.22852847589063074, 0.14686012481332805, 0.012197579452241356, 0.04785543629494371, 0.18465781703645243, 0.21003939159009058, 0.12643302822656555, 0.003118306121547932, 0.012115905553535242, 0.04785527191586849, 0.18465819751280046]
+ptq_acc_list:
+[1.0, 0.98, 11.4, 38.64, 51.7, 55.32, 56.63, 56.79, 56.73, 56.87, 56.83, 56.93, 56.87, 56.89, 56.89, 1.0, 0.96, 25.06, 25.67, 25.69, 25.05, 25.18, 4.83, 23.01, 20.31, 43.33, 39.41, 48.06, 47.11, 49.65, 51.15, 48.14, 49.37, 51.37, 55.57, 51.37, 48.05, 48.35, 52.41, 56.65, 55.61, 51.32, 48.18]
+acc_loss_list:
+[0.9824191279887482, 0.9827707454289734, 0.79957805907173, 0.3206751054852321, 0.0910689170182841, 0.027426160337552782, 0.004395218002812939, 0.001582278481012718, 0.0026371308016878633, 0.00017580872011260752, 0.0008790436005626628, -0.0008790436005625379, 0.00017580872011260752, -0.0001758087201124826, -0.0001758087201124826, 0.9824191279887482, 0.9831223628691983, 0.559423347398031, 0.5486990154711674, 0.5483473980309423, 0.5595991561181435, 0.5573136427566807, 0.9150843881856541, 0.5954641350210971, 0.6429324894514769, 0.23822081575246137, 0.3071378340365683, 0.1550632911392405, 0.17176511954992973, 0.12710970464135027, 0.10073839662447263, 0.15365682137834039, 0.1320323488045008, 0.09687060478199727, 0.023030942334739842, 0.09687060478199727, 0.1552390998593531, 0.1499648382559775, 0.07858649789029545, 0.004043600562587974, 0.022327707454289785, 0.09774964838255981, 0.15295358649789034]
+
+

ykl/ALL-cifar100/train.py

+from model import *
+
+import torch
+import torch.nn as nn
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+from torch.optim.lr_scheduler import CosineAnnealingLR
+import os
+import os.path as osp
+import time
+# import sys
+
+def train(model, device, train_loader, optimizer, epoch):
+    model.train()
+    total_loss = 0.
+    lossLayer = nn.CrossEntropyLoss()
+    start_time = time.time()
+    for batch_idx, (data, targets) in enumerate(train_loader):
+        data,targets = data.to(device), targets.to(device)
+        optimizer.zero_grad()
+        output = model(data)
+        loss = lossLayer(output, targets)
+        loss.backward()
+        total_loss += loss.item() * len(data)
+        optimizer.step()
+
+        pred = output.argmax(dim=1, keepdim=True)
+
+        if batch_idx % 200 == 0 and batch_idx > 0:
+            cur_loss = total_loss / 200
+            elapsed = time.time() - start_time
+            lr = optimizer.param_groups[0]['lr']
+            print('| epoch {:3d} | {:5d}/{:5d} batches | lr {:02.4f} | ms/batch {:5.2f} | '
+                'loss {:5.2f}'.format(
+                epoch, batch_idx, len(train_loader.dataset) // len(data), lr,
+                elapsed * 1000 / 200, cur_loss))
+            
+            total_loss = 0.
+            correct = 0
+            start_time = time.time()
+
+
+def evaluate(model, device, eval_loader):
+    model.eval()
+    total_loss = 0
+    correct = 0
+    lossLayer = nn.CrossEntropyLoss()
+    with torch.no_grad():
+        for data, targets in eval_loader:
+            data,targets = data.to(device), targets.to(device)
+            output = model(data)
+            total_loss += len(data) * lossLayer(output, targets).item()
+            pred = output.argmax(dim=1, keepdim=True)
+            correct += pred.eq(targets.view_as(pred)).sum().item()
+    
+    test_loss = total_loss / len(eval_loader.dataset)
+    test_acc = 100. * correct / len(eval_loader.dataset)
+    return test_loss,test_acc
+
+
+epochs_cfg_table = {
+    'AlexNet'       : [20, 30, 20, 20, 10],
+    'AlexNet_BN'    : [15, 20, 20, 20, 10, 10],
+    'VGG_16'        : [25, 30, 30, 20, 20, 10, 10],  
+    'VGG_19'        : [30, 40, 30, 20, 20, 10, 10],
+    'Inception_BN'  : [20, 30, 30, 20, 20, 10, 10],
+    'ResNet_18'     : [30, 25, 25, 20, 10, 10],
+    'ResNet_50'     : [30, 40, 35, 25, 15, 10, 10],
+    'ResNet_152'    : [50, 60, 50, 40, 25, 15, 10, 10],
+    'MobileNetV2'   : [25, 35, 30, 20, 10, 10], 
+}
+lr_cfg_table = {
+    'AlexNet'       : [0.01, 0.005, 0.001, 0.0005, 0.0001],
+    'AlexNet_BN'    : [0.01, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'VGG_16'        : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],  
+    'VGG_19'        : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'Inception_BN'  : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'ResNet_18'     : [0.01, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'ResNet_50'     : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'ResNet_152'    : [0.01, 0.008, 0.005, 0.003, 0.002, 0.001, 0.0005, 0.0001],
+    'MobileNetV2'   : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0001],
+}
+if __name__ == "__main__":
+    # sys.stdout = open(sys.stdout.fileno(), mode='w', buffering=1)
+
+    batch_size = 32
+    seed = 1111
+    seed_gpu = 1111
+    lr = 0.05   # origin lr
+    # momentum = 0.5
+    t_epochs = 300 #学习率衰减周期
+    patience = 30 #早停参数
+    save_model = True
+
+    append = False
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed(seed_gpu)
+    
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+    train_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=True, download=True,
+                       transform=transforms.Compose([
+                           transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                           transforms.RandomHorizontalFlip(),
+                           transforms.ToTensor(),
+                           transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                       ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
+            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+            transforms.ToTensor(),
+            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+        ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    if save_model:
+        if not osp.exists('ckpt'):
+            os.makedirs('ckpt')
+
+    # model_name_list = ['AlexNet', 'AlexNet_BN', 'VGG_16', 'VGG_19', 'Inception_BN', 
+    #                    'ResNet_18', 'ResNet_50', 'ResNet_152', 'MobileNetV2']
+    # model_name_list = ['ResNet_18', 'ResNet_50', 'ResNet_152', 'MobileNetV2']
+    model_name_list = ['ResNet_152']
+    for model_name in model_name_list:
+        save_path = 'ckpt/cifar10_'+model_name+'.pt'
+        if os.path.exists(save_path) and append:
+            continue
+        else:
+            print('>>>>>>>>>>>>>>>>>>>>>>>> Train: '+model_name+' <<<<<<<<<<<<<<<<<<<<<<<<')
+            model = Model(model_name).to(device)
+
+            best_val_acc = None
+            optimizer = optim.SGD(model.parameters(), lr=lr)
+            lr_scheduler = CosineAnnealingLR(optimizer, T_max=t_epochs)
+            weak_cnt = 0 # 弱于最佳精度的计数器
+            epoch = 0
+            while weak_cnt < patience:
+                epoch += 1
+                epoch_start_time = time.time()
+                train(model, device, train_loader, optimizer, epoch)
+                val_loss, val_acc = evaluate(model, device, test_loader)
+                if not best_val_acc or val_acc > best_val_acc:
+                    best_val_acc = val_acc
+                    weak_cnt = 0
+                    if save_model:
+                        torch.save(model.state_dict(), save_path)
+                else:
+                    weak_cnt += 1
+                print('-' * 89)
+                print('| end of epoch {:3d} | time: {:5.2f}s | test loss {:5.2f} | '
+                        'test acc {:.2f} | weak_cnt {:d}'.format(epoch, (time.time() - epoch_start_time),
+                                                val_loss, val_acc, weak_cnt))
+                print('-' * 89)
+                lr_scheduler.step()
+
+            print('>>> Early Stop: No improvement after patience(%d) epochs.'%patience)
+        
+        model = Model(model_name).to(device)
+        model.load_state_dict(torch.load(save_path))
+
+        test_loss,test_acc = evaluate(model, device, test_loader)
+        print('=' * 89)
+        print('| Test on {:s} | test loss {:5.2f} | test acc {:.2f}'.format(
+            model_name, test_loss, test_acc))
+        print('=' * 89)
\ No newline at end of file

ykl/ALL/ptq.slurm → ykl/ALL-cifar100/train.slurm

@@ -82,7 +82,7 @@ echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
 
 #- Job step
 # [EDIT HERE(TODO)]
-python ptq.py
+python train.py
 
 #- End
 echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL-cifar100/train_one.py

+from model import *
+
+import torch
+import torch.nn as nn
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torch.optim.lr_scheduler import CosineAnnealingLR
+import os
+import os.path as osp
+import time
+import sys
+
+def train(model, device, train_loader, optimizer, epoch):
+    model.train()
+    total_loss = 0.
+    lossLayer = nn.CrossEntropyLoss()
+    start_time = time.time()
+    for batch_idx, (data, targets) in enumerate(train_loader):
+        data,targets = data.to(device), targets.to(device)
+        optimizer.zero_grad()
+        output = model(data)
+        loss = lossLayer(output, targets)
+        loss.backward()
+        total_loss += loss.item()
+        optimizer.step()
+
+        if batch_idx % 50 == 0 and batch_idx > 0:
+            cur_loss = total_loss / 50
+            elapsed = time.time() - start_time
+            lr = optimizer.param_groups[0]['lr']
+            print('| epoch {:3d} | {:5d}/{:5d} batches | lr {:02.7f} | ms/batch {:5.2f} | '
+                'loss {:5.2f}'.format(
+                epoch, batch_idx, len(train_loader.dataset) // len(data), lr,
+                elapsed * 1000 / 50, cur_loss))
+            
+            total_loss = 0.
+            start_time = time.time()
+
+
+def evaluate(model, device, eval_loader):
+    model.eval()
+    total_loss = 0
+    correct = 0
+    lossLayer = nn.CrossEntropyLoss()
+    with torch.no_grad():
+        for data, targets in eval_loader:
+            data,targets = data.to(device), targets.to(device)
+            output = model(data)
+            total_loss += len(data) * lossLayer(output, targets).item()
+            pred = output.argmax(dim=1, keepdim=True)
+            correct += pred.eq(targets.view_as(pred)).sum().item()
+    
+    test_loss = total_loss / len(eval_loader.dataset)
+    test_acc = 100. * correct / len(eval_loader.dataset)
+    return test_loss,test_acc
+
+if __name__ == "__main__":
+    # sys.stdout = open(sys.stdout.fileno(), mode='w', buffering=1)
+
+    batch_size = 128
+    optim_type = 'sgd' 
+    # optim_type = 'sgd'
+    if optim_type is 'adam':
+        lr = 0.0001
+        opt_path = 'adam_lr'+str(lr).split('.')[-1]
+    elif optim_type is 'sgd':
+        lr = 0.01
+        momentum = 0.9
+        weight_decay = 1e-4
+        nesterov = True
+        opt_path = 'sgd_lr'+str(lr).split('.')[-1]+'_wd'+str(weight_decay).split('.')[-1]+'_ntr'
+
+    # lr = 0.001   # origin lr
+    # end_lr_bound = 0.00001 #早停时学习率应当低于该值，从而保证充分收敛
+    # momentum = 0.9
+    # weight_decay = 1e-4
+    # nesterov = True
+    t_epochs = 800 #学习率衰减周期
+    patience = 50 #早停参数
+    
+    print(opt_path)
+    print('t_epoches:%d patience:%d'%(t_epochs,patience))
+    save_model = True
+
+    append = False
+    
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+    train_transfrom = transforms.Compose([
+                        transforms.RandomCrop(32, padding=2),
+                        transforms.RandomHorizontalFlip(),
+                        transforms.ToTensor(),
+                        transforms.Normalize(
+                            (0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                        ])
+    eval_transfrom = transforms.Compose([
+                        transforms.ToTensor(),
+                        transforms.Normalize((0.4914, 0.4822, 0.4465),
+                                            (0.2023, 0.1994, 0.2010))
+                        ])
+
+    alltrainset = datasets.CIFAR100(root='/lustre/datasets/CIFAR100',train=True,download=True,transform=train_transfrom)
+    
+    train_size = (int)(0.8 * len(alltrainset))
+    val_size = (int)(0.2 * len(alltrainset))
+    
+    train_idx, val_idx = torch.utils.data.random_split(range(train_size+val_size),[train_size,val_size])
+
+    trainset = torch.utils.data.Subset(alltrainset,train_idx)
+    valset = torch.utils.data.Subset(alltrainset,val_idx)
+
+    train_loader = torch.utils.data.DataLoader(
+     trainset,
+     batch_size=batch_size, shuffle=True, num_workers=4, pin_memory=True
+    )
+    val_loader = torch.utils.data.DataLoader(
+        valset,
+        batch_size=batch_size, shuffle=False, num_workers=4, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+    datasets.CIFAR100('/lustre/datasets/CIFAR100', train=False, download=False, 
+                     transform=eval_transfrom),
+    batch_size=batch_size, shuffle=False, num_workers=4, pin_memory=True
+    )
+
+    # ckpt_path = 'ckpt_sgd_1_momentum_9_ntb'
+    ckpt_path = 'ckpt_'+opt_path
+    if save_model:
+        if not osp.exists(ckpt_path):
+            os.makedirs(ckpt_path)
+
+    model_name = sys.argv[1]
+    save_path = ckpt_path+'/cifar100_'+model_name+'.pt'
+    if os.path.exists(save_path) and append:
+        pass
+    else:
+        print('>>>>>>>>>>>>>>>>>>>>>>>> Train: '+model_name+' <<<<<<<<<<<<<<<<<<<<<<<<')
+        model = Model(model_name).to(device)
+
+        best_val_acc = None
+        if 'adam' in opt_path:
+            optimizer = optim.Adam(model.parameters(), lr=lr)
+        elif 'sgd' in opt_path:
+            optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum,weight_decay = weight_decay,nesterov=nesterov)
+        else:
+            raise ValueError('Illegal opttype')
+        lr_scheduler = CosineAnnealingLR(optimizer, T_max=t_epochs)
+        weak_cnt = 0 # 弱于最佳精度的计数器
+        epoch = 0
+        while True:
+            epoch += 1
+            epoch_start_time = time.time()
+            train(model, device, train_loader, optimizer, epoch)
+            val_loss, val_acc = evaluate(model, device, val_loader)
+            if not best_val_acc or val_acc > best_val_acc:
+                best_val_acc = val_acc
+                weak_cnt = 0
+                if save_model:
+                    torch.save(model.state_dict(), save_path)
+            else:
+                weak_cnt += 1
+            print('-' * 89)
+            print('| end of epoch {:3d} | time: {:5.2f}s | test loss {:5.2f} | '
+                    'test acc {:.2f} | weak_cnt {:d}'.format(epoch, (time.time() - epoch_start_time),
+                                            val_loss, val_acc, weak_cnt))
+            print('-' * 89)
+            lr_scheduler.step()
+            
+            #这里额外限制早停时，学习率曾低于一个限度，保证充分训练
+            if weak_cnt >= patience:
+                break
+                # if optimizer.param_groups[0]['lr'] < end_lr_bound:
+                #     break
+                # elif epoch > t_epochs: # 已经训练完一个周期，曾达到最小精度
+                #     break
+                # else: #若多个epoch未提升，载入当前最佳精度模型继续训练
+                #     print('>> Turn Back: No improvement after %d epochs, back to BestPoint'%patience)
+                #     weak_cnt = 0
+                #     model.load_state_dict(torch.load(save_path))
+
+        print('>>> Early Stop: No improvement after patience(%d) epochs'%patience)
+        # print('>>> Early Stop: No improvement after patience(%d) epochs And lr under bound(%f) '%(patience,end_lr_bound))
+    
+    model = Model(model_name).to(device)
+    model.load_state_dict(torch.load(save_path))
+
+    test_loss,test_acc = evaluate(model, device, test_loader)
+    print('=' * 89)
+    print('| Test on {:s} | test loss {:5.2f} | test acc {:.2f}'.format(
+        model_name, test_loss, test_acc))
+    print('=' * 89)
\ No newline at end of file

ykl/ALL/ptq_weightratio.slurm → ykl/ALL-cifar100/train_one.slurm

@@ -5,22 +5,17 @@
 # (TODO)
 # Please modify job name
 
-#SBATCH -J ALL-weightratio            # The job name
-#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
-#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
-
-
 #- Resources
 
 # (TODO)
 # Please modify your requirements
 
 #SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
-#SBATCH -t 3-00:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH -t 1-06:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
 #SBATCH --nodes=1                    # Request N nodes
 #SBATCH --gres=gpu:1                 # Request M GPU per node
 #SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
-#SBATCH --qos=gpu-long             # Request QOS Type
+#SBATCH --qos=gpu-normal             # Request QOS Type
 
 ###
 ### The system will alloc 8 or 16 cores per gpu by default.
@@ -36,7 +31,7 @@
 ###
 
 # set constraint for RTX8000 to meet my cuda
-#SBATCH --constraint="Ampere|RTX8000|T4"
+#SBATCH --constraint="Ampere"
 
 #- Log information
 
@@ -82,7 +77,7 @@ echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
 
 #- Job step
 # [EDIT HERE(TODO)]
-python ptq_weightratio.py
+python train_one.py $Model
 
 #- End
 echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL-cifar100/utils.py

+import torch
+import torch.nn as nn
+
+def ebit_list(quant_type, num_bits):
+    if quant_type == 'FLOAT':
+        e_bit_list = list(range(1,num_bits-1))
+    else:
+        e_bit_list = [0]
+    return e_bit_list
+
+
+def numbit_list(quant_type):
+    if quant_type == 'INT':
+        num_bit_list = list(range(2,17))
+        # num_bit_list = [4,5]
+    elif quant_type == 'POT':
+        num_bit_list = list(range(2,9))
+        # num_bit_list = [5]
+    else:
+        num_bit_list = list(range(2,9))
+        # num_bit_list = [8]
+    
+    return num_bit_list     
+
+def build_bias_list(quant_type):
+    if quant_type == 'POT':
+        return build_pot_list(8)
+    else:
+        return build_float_list(16,7)
+    
+def build_list(quant_type, num_bits, e_bits):
+    if quant_type == 'POT':
+        return build_pot_list(num_bits)
+    else:
+        return build_float_list(num_bits,e_bits)
+
+def build_pot_list(num_bits):
+    plist = [0.]
+    for i in range(-2 ** (num_bits-1) + 2, 1): 
+        # i最高到0，即pot量化最大值为1
+        plist.append(2. ** i)
+        plist.append(-2. ** i)
+    plist = torch.Tensor(list(set(plist)))
+    # plist = plist.mul(1.0 / torch.max(plist))
+    return plist
+
+def build_float_list(num_bits,e_bits):
+    m_bits = num_bits - 1 - e_bits
+    plist = [0.]
+    # 相邻尾数的差值
+    dist_m = 2 ** (-m_bits)
+    e = -2 ** (e_bits - 1) + 1
+    for m in range(1, 2 ** m_bits):
+        frac = m * dist_m   # 尾数部分
+        expo = 2 ** e       # 指数部分
+        flt = frac * expo
+        plist.append(flt)
+        plist.append(-flt)
+
+    for e in range(-2 ** (e_bits - 1) + 2, 2 ** (e_bits - 1) + 1):
+        expo = 2 ** e
+        for m in range(0, 2 ** m_bits):
+            frac = 1. + m * dist_m
+            flt = frac * expo
+            plist.append(flt)
+            plist.append(-flt)
+    plist = torch.Tensor(list(set(plist)))
+    return plist
+
+#此处不必cfg，直接取同前缀同后缀即可。将relu一起考虑进去
+def fold_ratio(layer, par_ratio, flop_ratio):
+    idx = -1
+    for name in layer:
+        if 'conv' in name:
+            conv_idx = layer.index(name)
+            [prefix,suffix] = name.split('conv')
+            bn_name = prefix+'bn'+suffix
+            relu_name = prefix+'relu'+suffix
+            relus_name = prefix+'relus'+suffix
+            if bn_name in layer:
+                bn_idx = layer.index(bn_name)
+                par_ratio[conv_idx]+=par_ratio[bn_idx]
+                flop_ratio[conv_idx]+=flop_ratio[bn_idx]
+                if relu_name in layer:
+                    relu_idx = layer.index(relu_name)
+                    par_ratio[conv_idx]+=par_ratio[relu_idx]
+                    flop_ratio[conv_idx]+=flop_ratio[relu_idx]
+                elif relus_name in layer:
+                    relus_idx = layer.index(relus_name)
+                    par_ratio[conv_idx]+=par_ratio[relus_idx]
+                    flop_ratio[conv_idx]+=flop_ratio[relus_idx]
+    return par_ratio,flop_ratio
+
+def fold_model(model):
+    for name, module in model.named_modules():
+        if 'conv' in name:
+            [prefix,suffix] = name.split('conv')
+            bn_name = prefix+'bn'+suffix
+            if hasattr(model,bn_name):
+                bn_layer = getattr(model,bn_name)
+                fold_bn(module,bn_layer)
+
+def fold_bn(conv, bn):
+    # 获取 BN 层的参数
+    mean = bn.running_mean
+    var = bn.running_var
+    eps = bn.eps
+    std = torch.sqrt(var + eps)
+
+    if bn.affine:
+        gamma_ = bn.weight / std
+        weight = conv.weight * gamma_.view(conv.out_channels, 1, 1, 1)
+        if conv.bias is not None:
+            bias = gamma_ * conv.bias - gamma_ * mean + bn.bias
+        else:
+            bias = bn.bias - gamma_ * mean
+    else:
+        gamma_ = 1 / std
+        weight = conv.weight * gamma_
+        if conv.bias is not None:
+            bias = gamma_ * conv.bias - gamma_ * mean
+        else:
+            bias = -gamma_ * mean
+
+    # 设置新的 weight 和 bias
+    conv.weight.data = weight.data
+    if conv.bias is not None:
+        conv.bias.data = bias.data
+    else:
+        conv.bias = torch.nn.Parameter(bias)

ykl/ALL/cfg.py

-# conv: 'C',''/'B'/'BR',qi,in_ch,out_ch,kernel_size,stirde,padding,bias
-# relu: 'R'
-# inception: 'Inc'
-# maxpool: 'MP',kernel_size,stride,padding
-# adaptiveavgpool: 'AAP',output_size
-# flatten: 'FT'
-# dropout: 'D'
-
-# class 10
-AlexNet_cfg_table = [
-    ['C','',True,3,32,3,1,1,True],
-    ['R'],
-    ['MP',2,2,0],
-    ['C','',False,32,64,3,1,1,True],
-    ['R'],
-    ['MP',2,2,0],
-    ['C','',False,64,128,3,1,1,True],
-    ['R'],
-    ['C','',False,128,256,3,1,1,True],
-    ['R'],
-    ['C','',False,256,256,3,1,1,True],
-    ['R'],
-    ['MP',3,2,0],
-    ['FT'],
-    ['D',0.5],
-    ['FC',2304,1024,True],
-    ['R'],
-    ['D',0.5],
-    ['FC',1024,512,True],
-    ['R'],
-    ['FC',512,10,True]
-]
-AlexNet_BN_cfg_table = [
-    ['C','BR',True,3,32,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,32,64,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,64,128,3,1,1,True],
-    ['C','BR',False,128,256,3,1,1,True],
-    ['C','BR',False,256,256,3,1,1,True],
-    ['MP',3,2,0],
-    ['FT'],
-    ['D',0.5],
-    ['FC',2304,1024,True],
-    ['R'],
-    ['D',0.5],
-    ['FC',1024,512,True],
-    ['R'],
-    ['FC',512,10,True]
-]
-VGG_16_cfg_table = [
-    ['C','BR',True,3,64,3,1,1,True],
-    ['C','BR',False,64,64,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,64,128,3,1,1,True],
-    ['C','BR',False,128,128,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,128,256,3,1,1,True],
-    ['C','BR',False,256,256,3,1,1,True],
-    ['C','BR',False,256,256,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,256,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['MP',2,2,0],
-    ['FT'],
-    ['FC',512,4096,True],
-    ['R'],
-    ['D',0.5],
-    ['FC',4096,4096,True],
-    ['R'],
-    ['D',0.5],
-    ['FC',4096,10,True]
-]
-VGG_19_cfg_table = [
-    ['C','BR',True,3,64,3,1,1,True],
-    ['C','BR',False,64,64,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,64,128,3,1,1,True],
-    ['C','BR',False,128,128,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,128,256,3,1,1,True],
-    ['C','BR',False,256,256,3,1,1,True],
-    ['C','BR',False,256,256,3,1,1,True],
-    ['C','BR',False,256,256,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,256,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['MP',2,2,0],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['C','BR',False,512,512,3,1,1,True],
-    ['MP',2,2,0],
-    ['FT'],
-    ['FC',512,4096,True],
-    ['R'],
-    ['D',0.5],
-    ['FC',4096,4096,True],
-    ['R'],
-    ['D',0.5],
-    ['FC',4096,10,True]
-]
-Inception_BN_cfg_table = [
-    ['C','',True,3,64,3,1,1,True],
-    ['R'],
-    ['C','',False,64,64,3,1,1,True],
-    ['R'],
-    ['Inc',0],
-    ['Inc',1],
-    ['MP',3,2,1],
-    ['Inc',2],
-    ['Inc',3],
-    ['Inc',4],
-    ['Inc',5],
-    ['Inc',6],
-    ['MP',3,2,1],
-    ['Inc',7],
-    ['Inc',8],
-    ['AAP',1],
-    ['C','',False,1024,10,1,1,0,True],
-    ['FT']
-]
-
-model_cfg_table = {
-    'AlexNet'       : AlexNet_cfg_table,
-    'AlexNet_BN'    : AlexNet_BN_cfg_table,
-    'VGG_16'        : VGG_16_cfg_table,
-    'VGG_19'        : VGG_19_cfg_table,
-    'Inception_BN'  : Inception_BN_cfg_table
-}
-
-
-inc_ch_table=[
-    [64,  64, 96,128, 16, 32, 32],#3a
-    [256,128,128,192, 32, 96, 64],#3b
-    [480,192, 96,208, 16, 48, 64],#4a
-    [512,160,112,224, 24, 64, 64],#4b
-    [512,128,128,256, 24, 64, 64],#4c
-    [512,112,144,288, 32, 64, 64],#4d
-    [528,256,160,320, 32,128,128],#4e
-    [832,256,160,320, 32,128,128],#5a
-    [832,384,192,384, 48,128,128] #5b
-]
-
-# br0,br1,br2,br3 <- br1x1,br3x3,br5x5,brM
-# 这里的第2,3个参数是channel中的索引
-# 对于cfg拓展，可以认为'C'有'BR'参数，且bias为false。这里的一个项根据量化后可融合结构指定
-inc_cfg_table = [
-    [['C',0,1,1,1,0]],
-    [['C',0,2,1,1,0],
-     ['C',2,3,3,1,1]],
-    [['C',0,4,1,1,0],
-     ['C',4,5,5,1,2]],
-    [['MP',3,1,1],
-     ['R'],
-     ['C',0,6,1,1,0]]
-]
\ No newline at end of file

ykl/ALL/model.py

-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-
-from cfg import *
-from module import *
-import module
-
-def make_layers(model,cfg_table):
-    for i in range(len(cfg_table)):
-        cfg = cfg_table[i]
-        if cfg[0] == 'Inc':
-            make_inc_layers(model,cfg[1])
-        elif cfg[0] == 'C':
-            name = 'conv%d'%i
-            layer = nn.Conv2d(cfg[3],cfg[4],kernel_size=cfg[5],stride=cfg[6],padding=cfg[7],bias=cfg[8])
-            model.add_module(name,layer)
-            if 'B' in cfg[1]:
-                name = 'bn%d'%i
-                layer = nn.BatchNorm2d(cfg[4])
-                model.add_module(name,layer)
-                if 'R' in cfg[1]:
-                    name = 'relu%d'%i
-                    layer = nn.ReLU(True)
-                    model.add_module(name,layer)
-        elif cfg[0] == 'R':
-            name = 'relu%d'%i
-            layer = nn.ReLU(True)
-            model.add_module(name,layer)
-        elif cfg[0] == 'MP':
-            name = 'pool%d'%i
-            layer = nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
-            model.add_module(name,layer)
-        elif cfg[0] == 'AAP':
-            name = 'aap%d'%i
-            layer = nn.AdaptiveAvgPool2d(cfg[1])
-            model.add_module(name,layer)
-        elif cfg[0] == 'FC':
-            name = 'fc%d'%i
-            layer = nn.Linear(cfg[1],cfg[2],bias=cfg[3])
-            model.add_module(name,layer)
-        elif cfg[0] == 'D':
-            name = 'drop%d'%i
-            layer = nn.Dropout(cfg[1])
-            model.add_module(name,layer)
-        
-
-def model_forward(model,cfg_table,x):
-    for i in range(len(cfg_table)):
-        cfg = cfg_table[i]
-        if cfg[0] == 'Inc':
-            x = inc_forward(model,cfg[1],x)
-        elif cfg[0] == 'C':
-            name = 'conv%d'%i
-            layer = getattr(model,name)
-            x = layer(x)
-            if 'B' in cfg[1]:
-                name = 'bn%d'%i
-                layer = getattr(model,name)
-                x = layer(x)
-                if 'R' in cfg[1]:
-                    name = 'relu%d'%i
-                    layer = getattr(model,name)
-                    x = layer(x)
-        elif cfg[0] == 'R':
-            name = 'relu%d'%i
-            layer = getattr(model,name)
-            x = layer(x)
-        elif cfg[0] == 'MP':
-            name = 'pool%d'%i
-            layer = getattr(model,name)
-            x = layer(x)
-        elif cfg[0] == 'AAP':
-            name = 'aap%d'%i
-            layer = getattr(model,name)
-            x = layer(x)
-        elif cfg[0] == 'FC':
-            name = 'fc%d'%i
-            layer = getattr(model,name)
-            x = layer(x)
-        elif cfg[0] == 'D':
-            name = 'drop%d'%i
-            layer = getattr(model,name)
-            x = layer(x)
-        elif cfg[0] == 'FT':
-            x = torch.flatten(x, start_dim=1)
-
-    return x
-
-def model_quantize(model,cfg_table,quant_type,num_bits,e_bits):
-    for i in range(len(cfg_table)):
-        cfg = cfg_table[i]
-        if cfg[0] == 'Inc':
-            inc_quantize(model,cfg[1],quant_type,num_bits,e_bits)
-        elif cfg[0] == 'C':
-            conv_name = 'conv%d'%i
-            conv_layer = getattr(model,conv_name)
-            qname = 'q_'+conv_name
-            if 'B' in cfg[1]:
-                bn_name = 'bn%d'%i
-                bn_layer = getattr(model,bn_name)
-                if 'R' in cfg[1]:
-                    qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
-                else:
-                    qlayer = QConvBN(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
-            else:
-                qlayer = QConv2d(quant_type,conv_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
-            model.add_module(qname,qlayer)
-        elif cfg[0] == 'R':
-            name = 'relu%d'%i
-            qname = 'q_'+name
-            qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
-            model.add_module(qname,qlayer)
-        elif cfg[0] == 'MP':
-            name = 'pool%d'%i
-            qname = 'q_'+name
-            qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
-            model.add_module(qname,qlayer)
-        elif cfg[0] == 'AAP':
-            name = 'aap%d'%i
-            qname = 'q_'+name
-            qlayer = QAdaptiveAvgPool2d(quant_type,output_size=cfg[1],num_bits=num_bits,e_bits=e_bits)
-            model.add_module(qname,qlayer)
-        elif cfg[0] == 'FC':
-            name = 'fc%d'%i 
-            layer = getattr(model,name)
-            qname = 'q_'+name
-            qlayer = QLinear(quant_type,layer,num_bits=num_bits,e_bits=e_bits)
-            model.add_module(qname,qlayer)
-
-# 增加了func='fakefreeze'
-def model_utils(model,cfg_table,func,x=None):
-    last_qo = None
-    for i in range(len(cfg_table)):
-        cfg = cfg_table[i]
-        if cfg[0] == 'Inc':
-            x,last_qo = inc_utils(model,cfg[1],func,x,last_qo)
-        elif cfg[0] == 'C':
-            qname = 'q_conv%d'%i
-            qlayer = getattr(model,qname)
-            if func == 'forward':
-                x = qlayer(x)
-            elif func == 'inference':
-                # cfg[2]为True表示起始层，需要量化
-                if cfg[2]:
-                    x = qlayer.qi.quantize_tensor(x)
-                x = qlayer.quantize_inference(x)
-            elif func == 'freeze':
-                qlayer.freeze(last_qo)
-            elif func == 'fakefreeze':
-                qlayer.fakefreeze()
-            last_qo = qlayer.qo
-        elif cfg[0] == 'R':
-            qname = 'q_relu%d'%i
-            qlayer = getattr(model,qname)
-            if func == 'forward':
-                x = qlayer(x)
-            elif func == 'inference':
-                x = qlayer.quantize_inference(x)
-            elif func == 'freeze':
-                qlayer.freeze(last_qo)
-        elif cfg[0] == 'MP':
-            qname = 'q_pool%d'%i
-            qlayer = getattr(model,qname)
-            if func == 'forward':
-                x = qlayer(x)
-            elif func == 'inference':
-                x = qlayer.quantize_inference(x)
-            elif func == 'freeze':
-                qlayer.freeze(last_qo)
-        elif cfg[0] == 'AAP':
-            qname = 'q_aap%d'%i
-            qlayer = getattr(model,qname)
-            if func == 'forward':
-                x = qlayer(x)
-            elif func == 'inference':
-                x = qlayer.quantize_inference(x)
-            elif func == 'freeze':
-                qlayer.freeze(last_qo)
-            last_qo = qlayer.qo
-        elif cfg[0] == 'FC':
-            qname = 'q_fc%d'%i
-            qlayer = getattr(model,qname)
-            if func == 'forward':
-                x = qlayer(x)
-            elif func == 'inference':
-                x = qlayer.quantize_inference(x)
-            elif func == 'freeze':
-                qlayer.freeze(last_qo)
-            elif func == 'fakefreeze':
-                qlayer.fakefreeze()
-            last_qo = qlayer.qo
-        elif cfg[0] == 'D':
-            if func == 'forward':
-                name = 'drop%d'%i
-                layer = getattr(model,name)
-                x = layer(x)
-        elif cfg[0] == 'FT':
-            if func == 'inference' or func == 'forward':
-                x = torch.flatten(x,start_dim=1)
-    
-    if func == 'inference':
-        x = last_qo.dequantize_tensor(x)
-
-    return x
-
-def make_inc_layers(model,inc_idx):
-    inc_name = 'inc%d'%inc_idx
-    ch = inc_ch_table[inc_idx]
-    for i in range(4): # branch
-        prefix = inc_name+'_br%d_'%i
-        for j in range(len(inc_cfg_table[i])):
-            cfg = inc_cfg_table[i][j]
-            if cfg[0] == 'MP':
-                name = prefix+'pool%d'%j
-                layer =nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
-                model.add_module(name,layer)
-            elif cfg[0] == 'R':
-                name=prefix+'relu%d'%j
-                layer=nn.ReLU(True)
-                model.add_module(name,layer)
-            elif cfg[0] == 'C':
-                name=prefix+'conv%d'%j
-                layer=nn.Conv2d(ch[cfg[1]],ch[cfg[2]],kernel_size=cfg[3],stride=cfg[4],padding=cfg[5],bias=False)
-                model.add_module(name,layer)
-                name=prefix+'bn%d'%j
-                layer=nn.BatchNorm2d(ch[cfg[2]])
-                model.add_module(name,layer)
-                name=prefix+'relu%d'%j
-                layer=nn.ReLU(True)
-                model.add_module(name,layer)
-
-def inc_forward(model,inc_idx,x=None):
-    inc_name = 'inc%d'%inc_idx
-    outs = []
-    for i in range(4):
-        prefix = inc_name+'_br%d_'%i
-        tmp = x
-        for j in range(len(inc_cfg_table[i])):
-            cfg = inc_cfg_table[i][j]
-            if cfg[0] == 'MP':
-                name = prefix+'pool%d'%j
-                layer = getattr(model,name)
-                tmp = layer(tmp)
-            elif cfg[0] == 'R':
-                name=prefix+'relu%d'%j
-                layer = getattr(model,name)
-                tmp = layer(tmp)
-            else:
-                name=prefix+'conv%d'%j
-                layer = getattr(model,name)
-                tmp = layer(tmp)
-                name=prefix+'bn%d'%j
-                layer = getattr(model,name)
-                tmp = layer(tmp)
-                name=prefix+'relu%d'%j
-                layer = getattr(model,name)
-                tmp = layer(tmp)
-        outs.append(tmp)
-    
-    out = torch.cat(outs,1)
-    return out
-
-def inc_quantize(model,inc_idx,quant_type,num_bits,e_bits):
-    inc_name = 'inc%d'%inc_idx
-    for i in range(4):
-        prefix = inc_name+'_br%d_'%i
-        for j in range(len(inc_cfg_table[i])):
-            cfg = inc_cfg_table[i][j]
-            if cfg[0] == 'MP':
-                name = prefix+'pool%d'%j
-                qname = 'q_'+name
-                qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
-                model.add_module(qname,qlayer)
-            elif cfg[0] == 'R':
-                name = prefix+'relu%d'%j
-                qname = 'q_'+name
-                qlayer = QReLU(quant_type, num_bits=num_bits, e_bits=e_bits)
-                model.add_module(qname,qlayer)
-            else:
-                conv_name=prefix+'conv%d'%j
-                conv_layer=getattr(model,conv_name)
-                bn_name=prefix+'bn%d'%j
-                bn_layer=getattr(model,bn_name)
-                qname='q_'+conv_name
-                qlayer=QConvBNReLU(quant_type, conv_layer, bn_layer, qi=False, qo=True, num_bits=num_bits, e_bits=e_bits)
-                model.add_module(qname,qlayer)
-    qname = 'q_'+inc_name+'_concat'
-    qlayer = QConcat(quant_type,4,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
-    model.add_module(qname,qlayer)
-    
-def inc_utils(model,inc_idx,func,x=None,qo=None):
-    inc_name = 'inc%d'%inc_idx
-    outs=[]
-    qos=[]
-    for i in range(4): 
-        qprefix = 'q_'+inc_name+'_br%d_'%i
-        tmp = x
-        last_qo = qo
-        for j in range(len(inc_cfg_table[i])):
-            cfg = inc_cfg_table[i][j]
-            if cfg[0] == 'MP':
-                qname = qprefix+'pool%d'%j
-                qlayer = getattr(model,qname)
-                if func == 'forward':
-                    tmp = qlayer(tmp)
-                elif func == 'inference':
-                    tmp = qlayer.quantize_inference(tmp)
-                elif func == 'freeze':
-                    qlayer.freeze(last_qo)
-            elif cfg[0] == 'R':
-                qname = qprefix+'relu%d'%j
-                qlayer = getattr(model,qname)
-                if func == 'forward':
-                    tmp = qlayer(tmp)
-                elif func == 'inference':
-                    tmp = qlayer.quantize_inference(tmp)
-                elif func == 'freeze':
-                    qlayer.freeze(last_qo)
-            elif cfg[0] == 'C':
-                qname = qprefix+'conv%d'%j
-                qlayer = getattr(model,qname)
-                if func == 'forward':
-                    tmp = qlayer(tmp)
-                elif func == 'inference':
-                    tmp = qlayer.quantize_inference(tmp)
-                elif func == 'freeze':
-                    qlayer.freeze(last_qo)
-                elif func == 'fakefreeze':
-                    qlayer.fakefreeze()
-                last_qo = qlayer.qo
-        
-        outs.append(tmp)
-        qos.append(last_qo)
-
-    qname = 'q_'+inc_name+'_concat'
-    qlayer = getattr(model,qname)
-    out = None
-    if func == 'forward':
-        out = qlayer(outs)
-    elif func == 'inference':
-        out = qlayer.quantize_inference(outs)
-    elif func == 'freeze':
-        qlayer.freeze(qos)
-    last_qo = qlayer.qo
-    return out,last_qo
-
-
-class Model(nn.Module):
-    def __init__(self,model_name,num_classes=10):
-        super(Model, self).__init__()
-        self.cfg_table = model_cfg_table[model_name]
-        make_layers(self,self.cfg_table)
-    
-    def forward(self,x):
-        x = model_forward(self,self.cfg_table,x)
-        return x
-
-    def quantize(self, quant_type, num_bits=8, e_bits=3):
-        model_quantize(self,self.cfg_table,quant_type,num_bits,e_bits)
-    
-    def quantize_forward(self,x):
-        return model_utils(self,self.cfg_table,func='forward',x=x)
-
-    def freeze(self):
-        model_utils(self,self.cfg_table,func='freeze')
-    
-    def quantize_inference(self,x):
-        return model_utils(self,self.cfg_table,func='inference',x=x)
-
-    def fakefreeze(self):
-        model_utils(self,self.cfg_table,func='fakefreeze')
-
-# if __name__ == "__main__":
-#     model = Inception_BN()
-#     model.quantize('INT',8,3)
-#     print(model.named_modules)
-#     print('-------')
-#     print(model.named_parameters)
-#     print(len(model.conv0.named_parameters()))
\ No newline at end of file

ykl/ALL/ptq_L2.py

-from torch.serialization import load
-from model import *
-from extract_ratio import *
-from utils import *
-
-import gol
-import openpyxl
-import sys
-import argparse
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-import torch.optim as optim
-from torchvision import datasets, transforms
-from torchvision.transforms.functional import InterpolationMode
-import torch.utils.bottleneck as bn
-import os
-import os.path as osp
-from torch.utils.tensorboard import SummaryWriter
-
-
-def direct_quantize(model, test_loader,device):
-    for i, (data, target) in enumerate(test_loader, 1):
-        data = data.to(device)
-        output = model.quantize_forward(data).cpu()
-        if i % 500 == 0:
-            break
-    print('direct quantization finish')
-
-
-def full_inference(model, test_loader, device):
-    correct = 0
-    for i, (data, target) in enumerate(test_loader, 1):
-        data = data.to(device)
-        output = model(data).cpu()
-        pred = output.argmax(dim=1, keepdim=True)
-        # print(pred)
-        correct += pred.eq(target.view_as(pred)).sum().item()
-    print('\nTest set: Full Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
-    return 100. * correct / len(test_loader.dataset)
-
-
-def quantize_inference(model, test_loader, device):
-    correct = 0
-    for i, (data, target) in enumerate(test_loader, 1):
-        data = data.to(device)
-        output = model.quantize_inference(data).cpu()
-        pred = output.argmax(dim=1, keepdim=True)
-        correct += pred.eq(target.view_as(pred)).sum().item()
-    print('Test set: Quant Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
-    return 100. * correct / len(test_loader.dataset)
-
-
-if __name__ == "__main__":
-    batch_size = 32
-
-    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
-    print(device)
-    train_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=True, download=True,
-                         transform=transforms.Compose([
-                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
-                             transforms.RandomHorizontalFlip(),
-                             transforms.ToTensor(),
-                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
-                         ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
-    )
-
-    test_loader = torch.utils.data.DataLoader(
-        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
-            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
-            transforms.ToTensor(),
-            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
-        ])),
-        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
-    )
-
-    load_ptq = False
-    store_ptq = False
-
-    append = True
-    gol._init()
-
-    excel_path = 'ptq_result_L2.xlsx'
-    if os.path.exists(excel_path) and append:
-        workbook = openpyxl.load_workbook(excel_path)
-    else:
-        workbook = openpyxl.Workbook()
-    if 'Sheet' in workbook.sheetnames:
-        workbook.remove(workbook['Sheet'])
-
-    txt_path = 'ptq_result_L2.txt'
-    if os.path.exists(txt_path) and append:
-        ft = open(txt_path,'a')
-    else:
-        ft = open(txt_path,'w')
-        
-    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
-    for model_name in model_name_list:
-        if model_name in workbook.sheetnames:
-            continue
-
-        model = Model(model_name)
-        full_file = 'ckpt/cifar10_'+model_name+'.pt'
-        model.load_state_dict(torch.load(full_file))
-        model.to(device)
-
-        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
-
-        model.eval()
-        full_acc = full_inference(model, test_loader, device)
-        # 传入后可变
-        fold_model(model)
-    
-        layer, par_ratio, flop_ratio = extract_ratio(model_name)
-        par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
-
-        full_names = []
-        full_params = []
-        full_params_norm = []
-        for name, param in model.named_parameters():
-            if 'conv' in name or 'fc' in name:
-                full_names.append(name)
-                full_params.append(param.data.cpu())
-                param_norm = F.normalize(param.data.cpu(),p=2,dim=-1)
-                full_params_norm.append(param_norm)
-    
-        #统计每个参数对应层的参数个数
-        full_par_num=[]
-        for name in full_names:
-            prefix = name.rsplit('.',1)[0]
-            cnt = 0
-            for str in full_names:
-                sprefix = str.rsplit('.',1)[0]
-                if prefix == sprefix:
-                    cnt += 1
-            full_par_num.append(cnt)
-    
-        quant_type_list = ['INT','POT','FLOAT']
-        # quant_type_list = ['INT']
-        # quant_type_list = ['POT']
-        # quant_type_list = ['INT','POT']
-        title_list = []
-        js_flops_list = []
-        js_param_list = []
-        ptq_acc_list = []
-        acc_loss_list = []
-
-        for quant_type in quant_type_list:
-            num_bit_list = numbit_list(quant_type)
-            # 对一个量化类别，只需设置一次bias量化表
-            # int由于位宽大，使用量化表开销过大，直接_round即可
-            if quant_type != 'INT':
-                bias_list = build_bias_list(quant_type)
-                gol.set_value(bias_list, is_bias=True)
-
-            for num_bits in num_bit_list:
-                e_bit_list = ebit_list(quant_type,num_bits)
-                for e_bits in e_bit_list:
-                    model_ptq = Model(model_name)
-                    if quant_type == 'FLOAT':
-                        title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
-                    else:
-                        title = '%s_%d' % (quant_type, num_bits)
-                    print('\n'+model_name+': PTQ: '+title)
-                    title_list.append(title)
-
-                    # 设置量化表
-                    if quant_type != 'INT':
-                        plist = build_list(quant_type, num_bits, e_bits)
-                        gol.set_value(plist)
-                    # 判断是否需要载入
-                    if load_ptq is True and osp.exists(ptq_file_prefix + title + '.pt'):
-                        model_ptq.quantize(quant_type,num_bits,e_bits)
-                        model_ptq.load_state_dict(torch.load(ptq_file_prefix + title + '.pt'))
-                        model_ptq.to(device)
-                        print('Successfully load ptq model: ' + title)
-                    else:
-                        model_ptq.load_state_dict(torch.load(full_file))
-                        model_ptq.to(device)
-                        model_ptq.quantize(quant_type,num_bits,e_bits)
-                        model_ptq.eval()
-                        direct_quantize(model_ptq, train_loader, device)
-                        if store_ptq:
-                            torch.save(model_ptq.state_dict(), ptq_file_prefix + title + '.pt')
-
-                    model_ptq.freeze()
-                    ptq_acc = quantize_inference(model_ptq, test_loader, device)
-                    ptq_acc_list.append(ptq_acc)
-                    acc_loss = (full_acc - ptq_acc) / full_acc
-                    acc_loss_list.append(acc_loss)
-
-                    # 获取计算量/参数量下的js-div
-                    js_flops = 0.
-                    js_param = 0.
-                    for name, param in model_ptq.named_parameters():
-                        if 'conv' not in name and 'fc' not in name:
-                            continue
-                            
-                        prefix = name.rsplit('.',1)[0]
-                        layer_idx = layer.index(prefix)
-                        name_idx = full_names.index(name)
-
-                        layer_idx = layer.index(prefix)
-                        ptq_param = param.data.cpu()
-                        ptq_param_norm = F.normalize(ptq_param,p=2,dim=-1)
-                        js = js_div(ptq_param_norm,full_params_norm[name_idx])
-                        js /= full_par_num[name_idx]
-
-                        js = js.item()
-                        if js < 0.:
-                            js = 0.
-                        js_flops = js_flops + js * flop_ratio[layer_idx]
-                        js_param = js_param + js * par_ratio[layer_idx]
-                    js_flops_list.append(js_flops)
-                    js_param_list.append(js_param)
-
-                    print(title + ': js_flops: %f js_param: %f acc_loss: %f' % (js_flops, js_param, acc_loss))
-        
-        # 写入xlsx
-        worksheet = workbook.create_sheet(model_name)
-        worksheet.cell(row=1,column=1,value='FP32-acc')
-        worksheet.cell(row=1,column=2,value=full_acc)
-        worksheet.cell(row=3,column=1,value='title')
-        worksheet.cell(row=3,column=2,value='js_flops')
-        worksheet.cell(row=3,column=3,value='js_param')
-        worksheet.cell(row=3,column=4,value='ptq_acc')
-        worksheet.cell(row=3,column=5,value='acc_loss')
-        for i in range(len(title_list)):
-            worksheet.cell(row=i+4, column=1, value=title_list[i])
-            worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
-            worksheet.cell(row=i+4, column=3, value=js_param_list[i])
-            worksheet.cell(row=i+4, column=4, value=ptq_acc_list[i])
-            worksheet.cell(row=i+4, column=5, value=acc_loss_list[i])
-
-        if 'Sheet' in workbook.sheetnames:
-            workbook.remove(workbook['Sheet'])
-        workbook.save(excel_path)
-        
-        print(model_name,file=ft)
-        print('Full_acc: %f'%full_acc,file=ft)
-        print('title_list:',file=ft)
-        print(title_list,file=ft)
-        print('js_flops_list:',file=ft)
-        print(js_flops_list, file=ft)
-        print('js_param_list:',file=ft)
-        print(js_param_list, file=ft)
-        print('ptq_acc_list:',file=ft)
-        print(ptq_acc_list, file=ft)
-        print('acc_loss_list:',file=ft)
-        print(acc_loss_list, file=ft)
-        print("\n",file=ft)
-
-    ft.close()
-

ykl/ALL/ptq_result.txt

-AlexNet
-Full_acc: 86.060000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[10.147603472887434, 3.4735777398315735, 0.657128777104439, 0.1402944938181503, 0.03300085677953831, 0.007995885722661755, 0.0019068787727882519, 0.0004704087742350774, 0.00011290004340042615, 3.4677666427763485e-05, 4.835759235449544e-06, 2.3190564678341904e-05, 3.196400098729291e-06, 3.3872935928732365e-06, 1.3684112457913457e-06, 10.090646800237572, 2.005292868116476, 0.43355013904831324, 0.43289340966483714, 0.43298123987911274, 0.43326893616506545, 0.4331247501839425, 1.987108513502595, 0.6841263244261785, 0.35485575695990856, 0.3752167238172064, 0.121914769596053, 0.11863332237249592, 0.2772744841621785, 0.06223004378240952, 0.031475393945171834, 0.11856597328382808, 0.24017891433004604, 0.043757322622821594, 0.007918827552432828, 0.03150159800820788, 0.11856755637997582, 0.2234448545845582, 0.036928952491247675, 0.0019824746774589172, 0.007832565623933951, 0.03150104707959592, 0.11856715338828598]
-js_param_list:
-[3.2355718958631003, 0.9536900908692879, 0.17661893352285823, 0.03780415278800973, 0.008858951155161035, 0.00214154849155571, 0.000509927082053201, 0.00012164070394154658, 3.33694225388701e-05, 1.2162544774288353e-05, 7.1603672381082e-07, 1.1702497226528164e-05, 5.0466627657161924e-06, 9.523126802622739e-07, 2.9889347233081897e-06, 3.2234406567916567, 0.5354475822504783, 0.1312188550424844, 0.13103709245073697, 0.13105757329236303, 0.1311427863025942, 0.13111267005364427, 0.6156447703609608, 0.22233144522128817, 0.09576509216086894, 0.1266601765790008, 0.03225347850497606, 0.036223997687183665, 0.09544844878311845, 0.016100888160090848, 0.009473680968640114, 0.03622018735765729, 0.08326961787407997, 0.01109786654496731, 0.0023947080163857853, 0.009485698022477014, 0.036221162514389005, 0.0778945488972154, 0.009306042998032716, 0.0005849357445567093, 0.0023701389889276645, 0.00948535279272403, 0.03622079684057701]
-ptq_acc_list:
-[10.0, 10.21, 52.06, 77.31, 83.98, 85.6, 85.9, 86.09, 86.04, 86.09, 86.06, 86.07, 86.06, 86.06, 86.08, 10.0, 13.18, 74.37, 74.43, 74.18, 74.4, 74.31, 21.82, 68.32, 51.82, 79.4, 78.8, 82.72, 81.08, 83.47, 85.4, 82.88, 82.16, 84.38, 86.07, 85.5, 82.82, 82.46, 84.98, 86.08, 86.0, 85.34, 82.66]
-acc_loss_list:
-[0.883801998605624, 0.881361840576342, 0.3950732047408784, 0.10167325122007902, 0.02416918429003019, 0.005345108064141389, 0.0018591680223099766, -0.0003485940041831413, 0.0002323960027887058, -0.0003485940041831413, 0.0, -0.00011619800139427033, 0.0, 0.0, -0.0002323960027887058, 0.883801998605624, 0.8468510341622123, 0.13583546363002552, 0.13513827562165925, 0.13804322565651866, 0.1354868696258424, 0.13653265163839182, 0.7464559609574716, 0.20613525447362316, 0.3978619567743435, 0.07738786892865439, 0.08435974901231705, 0.038810132465721625, 0.057866604694399304, 0.03009528236114343, 0.007669068092028777, 0.03695096444341165, 0.04531722054380671, 0.01952126423425525, -0.00011619800139427033, 0.006507088078085083, 0.03764815245177793, 0.04183128050197547, 0.012549384150592589, -0.0002323960027887058, 0.0006971880083662826, 0.00836625610039506, 0.03950732047408791]
-
-
-AlexNet_BN
-Full_acc: 87.310000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[14.261531958147645, 2.5216079781917964, 0.46116271187287083, 0.10116751776414573, 0.02366857999726347, 0.005727456404133103, 0.0013995787291137347, 0.00032755864702621547, 9.083468712851506e-05, 3.9279267814657185e-05, 1.9521001999016182e-05, 1.4907377340701574e-05, 1.472309998261742e-06, 2.927860141339056e-06, 2.140239534597049e-05, 11.00797021621244, 1.7347626455421232, 0.684466797606898, 0.683273225078019, 0.7042803192576432, 0.7042800060567285, 0.7062439671835439, 2.1415266825120525, 0.8162350530415978, 0.27655595945712447, 0.48138285659911956, 0.08826544681744933, 0.13263285956507462, 0.36659157678143833, 0.04149385722275749, 0.0331260620201096, 0.1325675098211588, 0.3212629515020388, 0.027438579008588024, 0.008646288753879492, 0.033061484833924536, 0.13256908968462155, 0.30083965669229856, 0.02259296380787109, 0.002152848971541399, 0.008646190645337604, 0.03306166334274963, 0.28528407163580505]
-js_param_list:
-[4.525975088124574, 0.7756408789073711, 0.13962123018543932, 0.030527044743903838, 0.007106442539618428, 0.0017230344973903195, 0.00042786997111233664, 0.00010198592832773555, 2.910676937132983e-05, 9.735338251863014e-06, 6.254242391436015e-06, 5.110885341159888e-06, 5.134026811320913e-07, 5.412170504028867e-07, 8.626456577724872e-06, 3.615088068764246, 0.5070590858920699, 0.20256617721543332, 0.20223268982982928, 0.20818900100468749, 0.20818330171155638, 0.20904725377374708, 0.6815435977802837, 0.26233617206078796, 0.08174246558663437, 0.15525741248561856, 0.026175585917729102, 0.04077145732814419, 0.11869481379609015, 0.012033564115370663, 0.010570620047083423, 0.04074582656946896, 0.10400372156451279, 0.007843797978007105, 0.002655443829455315, 0.010547276042144588, 0.04074616626033381, 0.09741770355331004, 0.0064264679913963955, 0.0006802731478472274, 0.0026576990691152174, 0.010548301767420753, 0.08400151890829698]
-ptq_acc_list:
-[10.0, 15.54, 38.44, 76.15, 85.69, 86.96, 87.23, 87.34, 87.25, 87.34, 87.31, 87.3, 87.31, 87.3, 87.32, 10.0, 19.54, 56.21, 54.07, 52.04, 51.95, 54.64, 20.28, 62.25, 70.81, 72.49, 82.61, 83.32, 76.46, 84.89, 86.82, 83.55, 78.17, 85.69, 86.95, 86.57, 83.64, 78.4, 85.96, 87.19, 87.0, 86.62, 79.87]
-acc_loss_list:
-[0.8854655824075135, 0.822013515061276, 0.5597296987744818, 0.12782041003321493, 0.01855457564998287, 0.004008704615737126, 0.0009162753407398728, -0.0003436032527774726, 0.0006872065055549452, -0.0003436032527774726, 0.0, 0.00011453441759254514, 0.0, 0.00011453441759254514, -0.00011453441759238237, 0.8854655824075135, 0.7761997480242814, 0.3562020387126332, 0.3807124040774253, 0.4039628908487001, 0.4049937006070324, 0.37418394227465357, 0.7677242011224373, 0.2870232504867713, 0.1889817890276028, 0.16974000687206514, 0.05383117626846871, 0.04569923261940223, 0.124269843087848, 0.02771732905738176, 0.005612186462031945, 0.043064941014774996, 0.1046844576795327, 0.01855457564998287, 0.004123239033329509, 0.008475546901844109, 0.04203413125644258, 0.10205016607490547, 0.01546214637498578, 0.0013744130111098905, 0.0035505669453671086, 0.007902874813881544, 0.08521360668880996]
-
-
-VGG_16
-Full_acc: 90.060000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[7.15133747064537, 1.4625925835782612, 0.27528038309802805, 0.05999375723854592, 0.013721293740212743, 0.0033732424131763853, 0.0008308842262242227, 0.00022161440470198947, 6.398155155004749e-05, 2.3345051897377455e-05, 1.3154840854179507e-05, 1.1042119061156764e-05, 8.383129310645444e-06, 1.0039571268441182e-05, 1.0307767134309586e-05, 7.473873573128817, 1.2138168958012272, 0.8954533307613973, 0.8847200729870529, 0.889480495729074, 0.8828377520433295, 0.9202384247243933, 1.2985884732391204, 0.5001098400915878, 0.15797619371473212, 0.2963979378403137, 0.05224890302044636, 0.07668234531548482, 0.2280040599808785, 0.02401062359604003, 0.020395652708399853, 0.07667591820866455, 0.1997822583827158, 0.015853223921508276, 0.0050662161921924565, 0.020394644590516577, 0.07667526110132153, 0.18710542968170488, 0.013228032043279565, 0.0012830579038248617, 0.005075002328472146, 0.020394361774078447, 0.5904796029343199]
-js_param_list:
-[3.823939875624223, 0.5685185211098528, 0.1061155145856618, 0.023117101046099858, 0.005345663248595699, 0.0013503269855487474, 0.00032222047705326686, 8.086141579548302e-05, 3.209479985588981e-05, 9.997605488204276e-06, 8.46697402182539e-06, 3.035080993068207e-06, 4.645341375592637e-06, 5.930448596222854e-06, 5.909594486887659e-06, 4.305893745952836, 0.6018669058635838, 0.5400431259542849, 0.5416126547356545, 0.5406803652791912, 0.5390025287208834, 0.5470329327782539, 0.7519722318836962, 0.3038645326029812, 0.07427020366235762, 0.18383715082417446, 0.02218821705174367, 0.04942817720813972, 0.14148808249856845, 0.00878762265164005, 0.012693733649980042, 0.049462073173177655, 0.12408389114459065, 0.005096161222710954, 0.003090655671655611, 0.012692062625119162, 0.049462032314656366, 0.11622787586640881, 0.003961143226002479, 0.0007856883944763987, 0.003065281986746431, 0.012690010835378107, 0.3441573944792919]
-ptq_acc_list:
-[10.0, 11.51, 76.07, 88.33, 89.57, 89.74, 89.95, 89.97, 90.06, 90.06, 90.07, 90.05, 90.08, 90.08, 90.09, 10.0, 23.06, 77.66, 78.29, 74.35, 77.1, 76.48, 10.56, 56.9, 82.52, 79.68, 87.58, 86.88, 84.19, 89.07, 89.05, 86.94, 85.38, 89.28, 89.74, 89.44, 87.22, 85.69, 89.49, 89.87, 89.77, 89.54, 82.56]
-acc_loss_list:
-[0.8889629136131468, 0.872196313568732, 0.15534088385520772, 0.01920941594492565, 0.005440817232955908, 0.0035531867643793847, 0.001221407950255379, 0.0009993337774817166, 0.0, 0.0, -0.00011103708638675222, 0.00011103708638691001, -0.00022207417277366224, -0.00022207417277366224, -0.00033311125916057226, 0.8889629136131468, 0.7439484787919165, 0.13768598711969804, 0.1306906506773262, 0.17443926271374646, 0.14390406395736186, 0.15078836331334664, 0.882744836775483, 0.3681989784588053, 0.08372196313568739, 0.11525649566955358, 0.02753719742393964, 0.035309793471019396, 0.06517876970908289, 0.010992671552298569, 0.011214745725072231, 0.03464357095269825, 0.05196535642904738, 0.008660892738174563, 0.0035531867643793847, 0.006884299355984949, 0.03153453253386635, 0.048523206751054905, 0.006329113924050715, 0.0021097046413501856, 0.0032200755052188123, 0.005773928492116323, 0.0832778147901399]
-
-
-VGG_19
-Full_acc: 90.140000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[6.52908130056615, 1.3380062847274221, 0.24543462269497923, 0.05393969104397018, 0.012381510740945257, 0.003067060214909937, 0.0007555918574419032, 0.00017501117606359432, 4.4104190648609716e-05, 2.3327871023182897e-05, 1.0675439046810562e-05, 8.983655080683937e-06, 1.1246136322410839e-05, 1.669374629310751e-05, 6.347073531497991e-06, 6.932484220693536, 1.0687040042001579, 0.8323984445929756, 0.8230926166285857, 0.8200582224272311, 0.8417420472174938, 0.8317440393798151, 1.2031666133766876, 0.4699059099648023, 0.1461144158851069, 0.27725188424588565, 0.04629791970679128, 0.07348240968071523, 0.21177775297649584, 0.021228071993016986, 0.0187992093161517, 0.07344999091598375, 0.18586181536894214, 0.013847674161837173, 0.004719387421439763, 0.018766820174169, 0.07344825979729348, 0.17349586010003093, 0.011314319153131549, 0.001195520195639009, 0.004732545275858459, 0.018766792234023475, 0.5598509816229902]
-js_param_list:
-[3.9388183524565687, 0.5906007422253898, 0.11337101319621945, 0.024587731384019467, 0.005748740971112084, 0.0013975448353345961, 0.00034048951357184523, 7.219580933576352e-05, 2.420215829820016e-05, 2.245575096393826e-05, 1.1588766272794481e-05, 4.6564651206769664e-06, 1.1169147267451386e-05, 1.0010399586171422e-05, 3.3232170155925167e-06, 4.392429723980739, 0.5858373864659538, 0.5174166440347002, 0.5125834551246481, 0.5181444012623768, 0.5165617022044846, 0.5127922743171613, 0.785298571239749, 0.32687467701633877, 0.0757253679493511, 0.20164512084223385, 0.022964817771256384, 0.04912063785704141, 0.15688899149660557, 0.009258595397256264, 0.01287157719588226, 0.04911382975037919, 0.13826518313312058, 0.005405974137344003, 0.0031803287470254043, 0.012817747347664115, 0.04911231637448609, 0.12977071412854377, 0.004209915508246866, 0.000807348196237399, 0.0031895961527889497, 0.012816874513219311, 0.29721936760612694]
-ptq_acc_list:
-[10.0, 10.56, 69.87, 88.36, 89.56, 90.07, 90.03, 90.05, 90.11, 90.13, 90.15, 90.14, 90.14, 90.14, 90.14, 10.0, 17.15, 66.08, 64.32, 66.0, 58.73, 61.57, 10.58, 61.44, 80.12, 82.18, 87.95, 86.74, 84.47, 89.05, 89.36, 86.86, 86.0, 89.26, 89.82, 89.5, 87.16, 86.77, 89.53, 90.1, 89.82, 89.28, 79.38]
-acc_loss_list:
-[0.889061459951187, 0.8828489017084535, 0.2248724206789438, 0.01974706012868872, 0.006434435322831133, 0.0007765697803417727, 0.0012203239405369362, 0.0009984468604393544, 0.0003328156201464515, 0.0001109385400488697, -0.0001109385400488697, 0.0, 0.0, 0.0, 0.0, 0.889061459951187, 0.8097404038162859, 0.266918127357444, 0.28644331040603516, 0.2678056356778345, 0.34845795429332155, 0.3169514089194586, 0.882627024628356, 0.3183936099400932, 0.11116041712891053, 0.08830707787885504, 0.024295540270690013, 0.037719103616596465, 0.06290215220767696, 0.01209230086532065, 0.008653206123807423, 0.036387841136010665, 0.04592855558020857, 0.00976259152429549, 0.0035500332815620967, 0.007100066563124036, 0.03305968493454631, 0.03738628799645002, 0.006767250942977584, 0.0004437541601953212, 0.0035500332815620967, 0.009540714444197908, 0.1193698690925228]
-
-
-Inception_BN
-Full_acc: 92.850000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[12.330741426000127, 1.7774022898716264, 0.28821054098072, 0.06272624750607707, 0.014643773238671408, 0.0035362488362275716, 0.0008576211587906442, 0.00021727906401486407, 5.245980780163249e-05, 1.6362107243046427e-05, 6.144467100252098e-06, 7.116624913113964e-06, 5.504538717570937e-06, 3.849723618814388e-06, 1.0516858692556946e-05, 11.647423497409791, 1.3409273597202798, 1.3517284432634658, 1.2545879744691506, 1.2791368790498459, 1.3251419114745038, 1.3162237365748197, 2.1519596291810594, 0.8640877339820748, 0.19610413380825797, 0.5233173089245113, 0.0577209691448774, 0.13109263975912017, 0.4030272721882487, 0.023144755090275103, 0.03308999561031262, 0.13108261789492087, 0.3533817432909395, 0.013407919874326895, 0.008421085777519849, 0.03309604428632267, 0.13108322365407005, 0.3308494735980438, 0.010379484824287365, 0.0021142344319224275, 0.008399573286408415, 0.033096178092927774, 0.9498731099086688]
-js_param_list:
-[11.743628415486707, 1.8138152564298837, 0.2575987088433499, 0.056091978816504955, 0.0131183142169655, 0.003185085333115615, 0.0007642336413234433, 0.0001867811492769769, 4.84294604077158e-05, 1.6515516075695125e-05, 7.525543685547781e-06, 6.304620082507371e-06, 9.163043695968708e-06, 3.1847959617624517e-06, 1.4797626839592367e-05, 11.975132118833136, 1.3209527384787245, 1.8069678172592176, 1.6552403779967813, 1.6623214155416337, 1.7549714599696822, 1.7484103230481454, 2.1317498641722077, 0.8521769299783177, 0.18539618435504263, 0.5143901131955009, 0.05361950389674753, 0.13206790208872876, 0.3951690287541315, 0.020658682417329204, 0.03345737808203658, 0.13206631898251822, 0.3459066061409915, 0.011401936022077864, 0.008596322544040389, 0.03346616042846384, 0.13206817134822676, 0.32360442639471093, 0.008604766847197439, 0.00217485133267553, 0.008572328166388418, 0.03346495066046156, 1.2860953403025934]
-ptq_acc_list:
-[10.0, 11.24, 60.8, 88.36, 92.18, 92.92, 92.96, 92.89, 92.9, 92.87, 92.85, 92.85, 92.84, 92.85, 92.85, 10.0, 10.21, 20.71, 22.62, 19.12, 26.69, 24.87, 11.59, 15.18, 72.27, 27.03, 90.96, 79.08, 39.19, 91.77, 90.92, 77.58, 44.48, 92.16, 92.7, 91.96, 81.45, 40.77, 92.38, 92.7, 92.71, 91.93, 21.51]
-acc_loss_list:
-[0.8922994076467421, 0.8789445341949381, 0.3451803984921917, 0.04835756596661277, 0.007215939687668148, -0.0007539041464728853, -0.0011847065158858312, -0.00043080236941309916, -0.0005385029617664122, -0.0002154011847066261, 0.0, 0.0, 0.00010770059235316, 0.0, 0.0, 0.8922994076467421, 0.8900376952073236, 0.7769520732364027, 0.7563812600969305, 0.7940764674205707, 0.7125471190091546, 0.7321486268174474, 0.875175013462574, 0.8365105008077544, 0.22164781906300485, 0.7088852988691438, 0.02035541195476576, 0.14830371567043615, 0.5779213785675821, 0.01163166397415184, 0.020786214324178705, 0.16445880452342485, 0.5209477652127087, 0.007431340872374774, 0.0016155088852987773, 0.009585352719439964, 0.12277867528271397, 0.5609046849757673, 0.005061927840603111, 0.0016155088852987773, 0.0015078082929456174, 0.009908454496499597, 0.7683360258481421]
-
-

ykl/ALL/ptq_result_L2.txt

-AlexNet
-Full_acc: 86.070000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[3806.6554297042253, 1742.015153345221, 425.358251800215, 102.70556529578258, 25.13516598845947, 6.284990710624555, 1.5226463858972383, 0.37637188590897513, 0.0937218483840339, 0.02356942857993605, 0.005913245649293143, 0.0014840293248440614, 0.0004124549331586565, 0.00012464459306113474, 4.4241644032453234e-05, 3806.626823601337, 1119.5684059386515, 67.22826637442971, 65.2584135547821, 65.25849993476412, 65.25854846334343, 65.2585426561475, 663.9931633855356, 150.21247755919424, 156.37541828099458, 46.0525243022747, 55.87336577063952, 18.37679391322827, 21.364867094811032, 31.17330007882125, 4.767773367697066, 18.12624943589473, 14.65885675436464, 23.174095613967566, 1.229944644119404, 4.67310792527002, 18.126250551950505, 12.643838288241945, 20.052928560737637, 0.3380601265152883, 1.1750614507345885, 4.673106245697662, 18.12777438545329]
-js_param_list:
-[1064.8615010864344, 482.2225667072016, 116.87681175824119, 28.202862327123864, 6.8951693631537, 1.724479789021284, 0.4170550164496115, 0.10312659284974346, 0.02567170871100962, 0.006452665587276742, 0.0016207288256828358, 0.000411124521601818, 0.00011448728661560505, 3.4554701236429785e-05, 1.256408327153913e-05, 1064.8533560327176, 308.84553439789335, 18.799006637521117, 18.2646496648573, 18.264688604564057, 18.264684586733694, 18.264687473646386, 183.50218430540403, 41.5957977020034, 43.26997180631621, 12.847152788616123, 15.472483192451259, 5.139796185842727, 6.030818858640326, 8.635462553039364, 1.3329236199853485, 5.072315166766056, 4.170099277689426, 6.420378956568966, 0.3434601272612782, 1.3073140127584826, 5.0723155455971, 3.6093530667552316, 5.556115140184075, 0.09416124202888529, 0.3287037001727009, 1.3073135613517266, 5.075342519571067]
-ptq_acc_list:
-[10.0, 10.06, 51.65, 77.22, 83.96, 85.47, 86.07, 86.0, 86.06, 86.04, 86.07, 86.06, 86.09, 86.06, 86.07, 10.0, 12.28, 74.5, 74.64, 74.57, 74.5, 74.25, 20.54, 68.16, 52.38, 79.55, 79.14, 82.92, 81.3, 83.3, 85.45, 82.91, 82.06, 84.44, 85.91, 85.51, 82.82, 82.62, 85.01, 86.06, 85.98, 85.48, 82.88]
-acc_loss_list:
-[0.8838154990124317, 0.8831183920065063, 0.39990705239920993, 0.10282328337399785, 0.024514929708376897, 0.00697107005925403, 0.0, 0.0008132915069128986, 0.0001161845009874626, 0.0003485535029625529, 0.0, 0.0001161845009874626, -0.00023236900197525542, 0.0001161845009874626, 0.0, 0.8838154990124317, 0.8573254327872661, 0.1344254676426164, 0.13279888462879044, 0.1336121761357035, 0.1344254676426164, 0.1373300801673056, 0.7613570349715348, 0.20808644126873474, 0.3914255838271174, 0.07575229464389446, 0.08051585918438472, 0.03659811781108391, 0.05542000697107002, 0.03218310677355636, 0.007203439061229121, 0.03671430231207153, 0.04658998489601477, 0.018938073660973574, 0.0018589520158010526, 0.0065063320553036845, 0.03775996282095969, 0.04008365284071092, 0.012315557104682098, 0.0001161845009874626, 0.001045660508887989, 0.0068548855582664025, 0.037062855815034254]
-
-
-AlexNet_BN
-Full_acc: 87.310000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[3632.5462719527304, 1153.6671007438044, 258.53006140593516, 61.17532279693054, 14.891223739277061, 3.708468609799996, 0.9327535718553865, 0.2290387991705687, 0.0558845133577796, 0.014370449214296995, 0.003571630291049879, 0.0008625562856728075, 0.00023265116615085678, 2.582518881799345e-05, 5.500940913937269e-06, 3632.5290002264333, 707.8994232280825, 66.09579658259983, 65.1197204024375, 65.11971100025693, 65.11958369494646, 65.11959010588745, 502.53342155216836, 125.74848822462886, 106.7977490011123, 50.340123700280365, 37.91919519151391, 18.217442279952905, 30.826013119263497, 20.928424709525203, 4.712420598057065, 18.085518803241957, 24.785147412778223, 15.410974606040625, 1.2015424694844759, 4.66620720799309, 18.085515618418324, 22.628848887626468, 13.29073707534331, 0.3150020990484127, 1.1736005029204484, 4.666208204984376, 18.096500236447497]
-js_param_list:
-[1015.8729315394377, 316.80440575739027, 70.45411939495952, 16.637757426403944, 4.0480464268365335, 1.008581365757596, 0.2542549327402086, 0.06247395741987976, 0.01521756945701585, 0.0039055598906891386, 0.0009750390687994653, 0.0002388051307656589, 6.268083549538516e-05, 1.1420448279403141e-05, 9.514402586825453e-07, 1015.8657756982868, 193.51164061931982, 18.487584223270957, 18.224827841004352, 18.224824861932166, 18.22477461172962, 18.224794934502942, 138.7951197584608, 34.86550339450208, 29.34660124544504, 14.111192248397439, 10.428013148625217, 5.099791885771236, 8.714878912611212, 5.751938045273537, 1.319394969858532, 5.063761997837521, 7.033866884554059, 4.234439261320504, 0.3362723593071352, 1.3068606576090431, 5.063761194055209, 6.430008897204431, 3.651427775331254, 0.08801919927931902, 0.32871521234415546, 1.30685917092583, 5.069624667109006]
-ptq_acc_list:
-[10.0, 13.33, 38.66, 79.33, 85.48, 87.0, 87.49, 87.36, 87.24, 87.29, 87.25, 87.28, 87.31, 87.31, 87.32, 10.0, 14.81, 49.61, 54.45, 54.28, 56.42, 50.9, 20.48, 61.29, 70.85, 72.36, 81.95, 83.32, 76.64, 84.76, 86.75, 83.52, 77.85, 85.42, 86.87, 86.71, 83.64, 78.32, 85.69, 87.16, 87.07, 86.73, 79.48]
-acc_loss_list:
-[0.8854655824075135, 0.8473256213492154, 0.5572099415874471, 0.09139846523880431, 0.020959798419425017, 0.0035505669453671086, -0.002061619516664673, -0.0005726720879624001, 0.0008017409231474904, 0.0002290688351849275, 0.0006872065055549452, 0.0003436032527774726, 0.0, 0.0, -0.00011453441759238237, 0.8854655824075135, 0.8303745275455274, 0.4317947543236743, 0.37636009620891075, 0.3783071813079831, 0.3537968159431909, 0.4170198144542435, 0.7654335127705876, 0.29801855457565, 0.18852365135723292, 0.1712289543007674, 0.06139044782957278, 0.04569923261940223, 0.12220822357118316, 0.029206276486084034, 0.006413927385179272, 0.04340854426755247, 0.10834955904249235, 0.02164700492497996, 0.0050395143740693815, 0.00687206505554929, 0.04203413125644258, 0.1029664414156455, 0.01855457564998287, 0.0017180162638873633, 0.002748826022219781, 0.0066429962203642, 0.08968044897491693]
-
-
-VGG_16
-Full_acc: 90.060000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[4803.919675134635, 1287.1382930872844, 279.9264367424686, 65.25738246026505, 15.72176171355718, 3.863929102877906, 0.9633695234465877, 0.24037554092498606, 0.06003798427713084, 0.014859649305759608, 0.0036989349617955183, 0.0009297590950600473, 0.00023777455542695435, 6.177451735386848e-05, 1.687348277800094e-05, 4803.9126735699, 780.6742574825653, 90.85871885068245, 89.98441565230418, 89.98439740063434, 89.98447368346076, 89.98460526006676, 609.734857829675, 163.97601192015594, 121.10715205180763, 73.45937603752851, 42.80441037433894, 24.85925755673281, 48.867139323033896, 23.336643195538816, 6.477995998407114, 24.744689884301835, 40.82625887892399, 17.053899721476558, 1.6444280482850528, 6.436495589986837, 24.7446915417418, 37.74966086128042, 14.645385768954204, 0.42431402914077926, 1.619551796382776, 6.436509160682666, 24.763244303170193]
-js_param_list:
-[3468.943478398751, 780.3214366978973, 166.04031924610246, 38.151388298670724, 9.14175442167032, 2.2421934202373506, 0.5572632738247393, 0.1404745912003612, 0.03522789057832243, 0.008541134041069957, 0.0021640374431170254, 0.0005385031232900044, 0.0001413481065553399, 3.903370734086286e-05, 1.5885526939603433e-05, 3468.9449224664572, 471.25549248508725, 68.06340397833709, 67.6246439579611, 67.62469670868062, 67.62460844256277, 67.62469563891236, 410.2968176053128, 118.54872617685369, 75.29372245845758, 58.36820486947816, 26.416316296085885, 18.41059254585881, 41.195915783732055, 14.112047898448875, 4.859251633385958, 18.353329865236514, 35.23899741349211, 10.192427088266959, 1.2270888971031126, 4.8384185581718, 18.353331704123484, 32.851415181489216, 8.718847369799665, 0.3129867228456511, 1.2147891895642164, 4.838426537801942, 18.35858860501769]
-ptq_acc_list:
-[10.0, 12.22, 75.73, 87.93, 89.36, 89.98, 89.9, 89.98, 90.04, 90.04, 90.07, 90.07, 90.05, 90.04, 90.06, 10.0, 21.34, 73.34, 75.79, 76.82, 76.91, 76.59, 11.03, 54.96, 82.07, 79.84, 87.91, 87.24, 84.55, 89.4, 89.31, 87.16, 85.59, 89.38, 89.84, 89.41, 87.09, 85.57, 89.41, 89.89, 89.83, 89.35, 82.0]
-acc_loss_list:
-[0.8889629136131468, 0.8643126804352654, 0.15911614479236064, 0.02365089940039968, 0.0077725960470797556, 0.0008882966910948067, 0.0017765933821896134, 0.0008882966910948067, 0.00022207417277366224, 0.00022207417277366224, -0.00011103708638675222, -0.00011103708638675222, 0.00011103708638691001, 0.00022207417277366224, 0.0, 0.8889629136131468, 0.7630468576504552, 0.18565400843881855, 0.15844992227403948, 0.14701310237619375, 0.14601376859871204, 0.14956695536309125, 0.8775260937153009, 0.38974017321785476, 0.08871863202309581, 0.11347990228736396, 0.023872973573173504, 0.031312458361092685, 0.061181434599156176, 0.007328447701532273, 0.008327781479013991, 0.032200755052187495, 0.04963357761492337, 0.007550521874306094, 0.002442815900510758, 0.007217410615145522, 0.03297801465689539, 0.04985565178769719, 0.007217410615145522, 0.0018876304685765235, 0.002553852986897668, 0.007883633133466666, 0.0894958916278037]
-
-
-VGG_19
-Full_acc: 90.140000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[5144.206275389696, 1314.5031915215789, 284.75228418572925, 66.08828138577456, 15.924547195347134, 3.945508489581075, 0.9744824621662432, 0.240145204857737, 0.060970177786898, 0.015168279581215922, 0.0038762319348639397, 0.0009231650198397274, 0.00023264343031430685, 5.850058697982251e-05, 2.2746126922737273e-05, 5144.199599261901, 796.5924672633316, 98.79804986382494, 97.90326891150325, 97.90339725838194, 97.90342733252247, 97.9033895034332, 646.5794756970722, 179.30711630045755, 124.80812987710833, 83.28759482337827, 43.904681597578175, 27.154587219689308, 56.692876023912184, 23.776916070113252, 7.026013234767483, 27.04269180301487, 47.72767446760083, 17.299138622395738, 1.7806228432719848, 6.983728619409324, 27.04269089121566, 44.240986013374624, 14.849377566405725, 0.45780853948607875, 1.7553917796752694, 6.983728425473142, 27.059538268599393]
-js_param_list:
-[4036.5180986336604, 829.4606793443687, 174.56442102431618, 40.01640309397311, 9.582595067338994, 2.350277999553833, 0.5824392818690054, 0.14468797282940982, 0.03658160405467158, 0.00915174920249928, 0.002339405326979436, 0.0005618663309452209, 0.00013767900536115093, 4.960963871634183e-05, 1.221981967301472e-05, 4036.5197593916123, 499.33260286171014, 81.26284288453259, 80.81754200216422, 80.81764201292202, 80.81764351085528, 80.81763076484798, 474.4480804814884, 145.68195465957422, 81.48766551558924, 75.4339569457211, 28.164690222709147, 22.24136690419591, 54.58554288393205, 14.858657158099888, 5.747360506023278, 22.184649628754382, 47.06499217070044, 10.642578971997688, 1.4449041773182094, 5.72664701483485, 22.18464837310006, 43.9639668331577, 9.077944964333957, 0.3670860046163598, 1.4324142167409286, 5.726646439297058, 22.190228135948722]
-ptq_acc_list:
-[10.0, 10.7, 71.16, 88.29, 89.71, 90.02, 89.98, 90.13, 90.21, 90.12, 90.13, 90.14, 90.14, 90.13, 90.14, 10.0, 19.03, 68.1, 66.57, 66.96, 61.69, 65.38, 11.04, 62.01, 78.65, 81.66, 87.93, 86.94, 84.67, 89.09, 89.4, 87.17, 85.78, 89.24, 89.94, 89.56, 87.18, 86.51, 89.38, 90.05, 89.81, 89.53, 78.62]
-acc_loss_list:
-[0.889061459951187, 0.8812957621477701, 0.21056134901264703, 0.020523629909030335, 0.004770357222099033, 0.001331262480585806, 0.0017750166407809694, 0.0001109385400488697, -0.0007765697803416151, 0.00022187708009758177, 0.0001109385400488697, 0.0, 0.0, 0.0001109385400488697, 0.0, 0.889061459951187, 0.7888839582871089, 0.24450854226758384, 0.26148213889505223, 0.2571555358331485, 0.3156201464388729, 0.27468382516086093, 0.8775238517861105, 0.31207011315731087, 0.12746838251608603, 0.09407588196139344, 0.024517417350787593, 0.03550033281562018, 0.060683381406700675, 0.011648546705125329, 0.008209451963612103, 0.032948746394497436, 0.04836920346128244, 0.00998446860439323, 0.0022187708009762906, 0.006434435322831133, 0.032837807854448564, 0.040270690037719055, 0.00843132904370984, 0.0009984468604393544, 0.0036609718216108087, 0.006767250942977584, 0.12780119813623247]
-
-
-Inception_BN
-Full_acc: 92.850000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[2511.2094588241143, 514.6224944443054, 121.72162693805204, 34.91936699749214, 11.830727046720341, 4.61389921181382, 1.9815037610235493, 0.9090898451724668, 0.445693145985502, 0.22169278207806847, 0.10797906282345335, 0.05346285125971105, 0.02532186120284057, 0.012302718294603427, 0.005881463105781766, 2511.197114436539, 317.43921605650036, 53.10258916165484, 49.56627028116844, 49.55384989557891, 49.55381038799659, 49.55343630864873, 314.49013802009296, 104.54996263349719, 53.443062402939, 54.960874258820255, 19.04872998828755, 14.524522578886899, 38.56171912282987, 9.638177596411147, 3.80774697251023, 13.940455390214797, 32.05389710796874, 6.417397878853529, 1.0179720739751408, 3.5521641738404086, 13.938914251013315, 29.187816906242272, 5.153629153112564, 0.2840957063401789, 0.8982905850077583, 3.5514526481610735, 13.972900328027036]
-js_param_list:
-[2744.038379989026, 516.638349887526, 129.91342844973485, 40.73366137533873, 15.109998796040331, 6.340951315617309, 2.873664540067067, 1.3703147138534952, 0.6729131813965019, 0.33525752212052057, 0.16545026015840456, 0.07993866540433915, 0.038260218032434255, 0.018713906292927, 0.009382849493313599, 2744.0333295413316, 324.6756368084103, 60.080916610025525, 54.766434780856635, 54.747717700492004, 54.747529170172136, 54.74718325245878, 349.4332999050808, 124.12611787269181, 56.2261930410738, 67.01936850231559, 20.017832250515283, 16.607070029075818, 47.101684942696515, 9.810136391111984, 4.36989863546681, 15.72518888008714, 38.93910436603853, 6.231147896753903, 1.1879888395822027, 3.9838019674079073, 15.722349515453137, 35.26415154400586, 4.7847528600807925, 0.339524681842086, 1.0097705140107518, 3.9826169398840015, 15.75149388072123]
-ptq_acc_list:
-[10.0, 10.91, 58.96, 88.94, 91.98, 92.73, 92.86, 92.93, 92.88, 92.89, 92.86, 92.85, 92.85, 92.85, 92.85, 10.0, 11.58, 23.08, 12.96, 23.49, 14.39, 25.34, 11.25, 15.72, 69.72, 26.39, 89.61, 75.18, 44.08, 91.88, 91.65, 80.29, 54.62, 92.51, 92.64, 91.66, 83.23, 47.87, 92.44, 92.81, 92.66, 91.2, 29.07]
-acc_loss_list:
-[0.8922994076467421, 0.8824986537425956, 0.3649973074851911, 0.042110931610123824, 0.009369951534733337, 0.0012924071082389913, -0.00010770059235331305, -0.0008616047388261983, -0.0003231017770597861, -0.00043080236941309916, -0.00010770059235331305, 0.0, 0.0, 0.0, 0.0, 0.8922994076467421, 0.8752827140549273, 0.7514270328486806, 0.8604200323101776, 0.7470113085621971, 0.8450188476036617, 0.7270866989768443, 0.8788368336025848, 0.8306946688206786, 0.2491114701130856, 0.7157781367797522, 0.03489499192245552, 0.19030694668820666, 0.525255788906839, 0.010446957458266008, 0.01292407108239083, 0.13527194399569187, 0.4117393645665051, 0.0036618201400106538, 0.0022617124394183497, 0.01281637049003767, 0.10360796984383404, 0.4844372644049542, 0.004415724286483539, 0.0004308023694129461, 0.0020463112547118765, 0.01777059773828747, 0.6869143780290792]
-
-

ykl/ALL/ptq_result_nodiv.txt

-AlexNet
-Full_acc: 86.060000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[20.29526915572396, 6.947154405565474, 1.3097954986483278, 0.2806041501204682, 0.06600583186782864, 0.015990427906057616, 0.003813768549117867, 0.0009404779007737146, 0.00022623305712286078, 6.948210405950218e-05, 9.299055436569503e-06, 4.588463785228213e-05, 6.323176802503827e-06, 6.744560112952103e-06, 2.7236538125521296e-06, 20.181323343778978, 4.010774698034702, 0.8670328491284963, 0.8666524823918315, 0.8660510033487936, 0.8660729611264287, 0.8664076192754275, 3.9742171794465233, 1.3682528628687918, 0.7097116431336872, 0.7504338769105835, 0.24382982572517414, 0.2372659926125888, 0.5545490309437322, 0.12446012187624539, 0.06295130822442739, 0.23713186361064312, 0.4803567676666072, 0.08751503503961693, 0.01583786418054705, 0.06300327659273015, 0.2371345332262401, 0.44688978958546705, 0.07385774436728046, 0.003964355381077269, 0.015665392460334736, 0.06300194864679118, 0.23713405078669267]
-js_param_list:
-[6.472271200739325, 1.9073607385855067, 0.3519049896730976, 0.07560821810352604, 0.017718723069791623, 0.004282963800527071, 0.001020443650571906, 0.00024306038592283128, 6.666702871534726e-05, 2.4975186637540503e-05, 1.3491321849468107e-06, 2.3169196998707343e-05, 1.0058927602988542e-05, 1.8907509311902882e-06, 5.974142388311065e-06, 6.447438061039491, 1.0709440642338761, 0.2624273939040286, 0.2623439973305667, 0.2621623043957625, 0.2621589243725357, 0.2621859683158946, 1.2312889431290226, 0.4446631646350267, 0.1915297162811837, 0.25331989546362266, 0.06450702860498402, 0.07244799510226177, 0.19089616432803086, 0.032201968185343655, 0.018948033485787396, 0.07244055043973999, 0.1665385503147391, 0.02219587159786096, 0.004789412520292082, 0.018971298231359603, 0.07244168608599733, 0.15578948141977655, 0.01861220259767605, 0.0011699346276903527, 0.004740313570757627, 0.018970282687255417, 0.07244162541940319]
-ptq_acc_list:
-[10.0, 10.17, 54.88, 77.5, 84.15, 85.84, 85.94, 86.08, 86.11, 86.09, 86.06, 86.06, 86.07, 86.07, 86.08, 10.0, 12.87, 74.42, 73.74, 74.54, 74.54, 74.52, 20.48, 67.91, 51.8, 79.33, 78.92, 82.86, 81.23, 83.41, 85.49, 82.7, 82.14, 84.26, 86.06, 85.62, 83.09, 82.39, 84.76, 85.96, 85.99, 85.53, 82.7]
-acc_loss_list:
-[0.883801998605624, 0.8818266325819196, 0.3623053683476644, 0.09946548919358589, 0.022193818266325778, 0.002556356030676259, 0.0013943760167325651, -0.0002323960027887058, -0.000580990006971847, -0.0003485940041831413, 0.0, 0.0, -0.00011619800139427033, -0.00011619800139427033, -0.0002323960027887058, 0.883801998605624, 0.850453172205438, 0.13525447362305368, 0.14315593771787133, 0.13386009760632112, 0.13386009760632112, 0.13409249360911, 0.7620264931443179, 0.21089937253079252, 0.3980943527771323, 0.07820125493841511, 0.08296537299558449, 0.03718336044620036, 0.056123634673483594, 0.03079247036950971, 0.006623286079479518, 0.039042528468510335, 0.045549616546595416, 0.02091564025098765, 0.0, 0.005112712061352518, 0.034510806414129666, 0.04264466651173602, 0.015105740181268848, 0.0011619800139438593, 0.000813386009760718, 0.006158494073901942, 0.039042528468510335]
-
-
-AlexNet_BN
-Full_acc: 87.310000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[28.523062559045258, 5.0572556337657435, 0.9222994515743476, 0.20228252172057867, 0.04733840794454623, 0.011456054210043627, 0.0027986331414157524, 0.0006547017959301428, 0.00018235688076410206, 7.889945211402782e-05, 3.896137579417227e-05, 3.0078232088860448e-05, 3.1099101543519137e-06, 5.861285826331164e-06, 4.279552306759359e-05, 22.01594004242986, 3.427301854013008, 1.3063413452820585, 1.4045641196426313, 1.356906249917974, 1.351080683822659, 1.3664221708008975, 4.283054483794392, 1.6324676164185732, 0.5531126050153183, 0.9627616968657938, 0.17653089935503774, 0.2652652914052897, 0.7331839328597248, 0.08298583534512317, 0.06625189064409207, 0.26513542399370454, 0.642524352709163, 0.05487889254662182, 0.017293229228320214, 0.0661235968993205, 0.26514059830510045, 0.6016803287731487, 0.04518600073910273, 0.004305085389680972, 0.017293366508607133, 0.06612324530096758, 0.5719832788177283]
-js_param_list:
-[9.051924388498541, 1.5532132254895608, 0.279235384076741, 0.06104835425311557, 0.014213577461106068, 0.0034475495538411212, 0.0008562487643766872, 0.00020370005038261856, 5.8245655176918376e-05, 1.9824314206317698e-05, 1.2478366906568228e-05, 1.027524534715929e-05, 1.080271489503737e-06, 1.080119362859344e-06, 1.7222549393826723e-05, 7.2301684811811935, 1.0026643585816455, 0.3883847860336896, 0.4152343837972867, 0.40173012752731296, 0.40068117689267724, 0.40442175933627295, 1.36308850130799, 0.5246718536237667, 0.1634848500226413, 0.3105133429143706, 0.05235117714725387, 0.08154295575886782, 0.23739024316614557, 0.024066363948542654, 0.021141573431830138, 0.08149152594104668, 0.20800683060508368, 0.01568816634762468, 0.005310852279338676, 0.02109452384982742, 0.08149265449474423, 0.19483586797504632, 0.012852646679949089, 0.0013603533217732317, 0.005315723494355893, 0.021096808338497135, 0.16840667478229432]
-ptq_acc_list:
-[10.0, 13.16, 37.45, 80.39, 85.4, 87.04, 87.38, 87.26, 87.23, 87.27, 87.25, 87.32, 87.3, 87.31, 87.31, 10.0, 14.77, 55.59, 51.7, 53.81, 55.49, 52.13, 17.32, 62.24, 69.42, 72.16, 81.77, 83.47, 76.18, 84.66, 86.6, 83.71, 78.13, 85.64, 86.99, 86.76, 83.56, 78.52, 85.64, 87.18, 87.02, 86.69, 79.88]
-acc_loss_list:
-[0.8854655824075135, 0.8492727064482878, 0.5710686061161379, 0.07925781697400071, 0.02187607376016489, 0.003092429274997091, -0.0008017409231473277, 0.0005726720879624001, 0.0009162753407398728, 0.00045813767037001775, 0.0006872065055549452, -0.00011453441759238237, 0.00011453441759254514, 0.0, 0.0, 0.8854655824075135, 0.8308326652158974, 0.3633031726033673, 0.40785706104684455, 0.3836902989348299, 0.3644485167792922, 0.40293208109036766, 0.8016263887298134, 0.28713778490436376, 0.20490207307295843, 0.17351964265261718, 0.06345206734623761, 0.04398121635551487, 0.12747680678043746, 0.030351620662009, 0.008131943649066635, 0.04123239033329525, 0.10514259534990272, 0.01912724773794527, 0.0036651013629596537, 0.006299392967586727, 0.04295040659718245, 0.10067575306379574, 0.01912724773794527, 0.001488947428702273, 0.003321498110182181, 0.007101133890734217, 0.08509907227121757]
-
-
-VGG_16
-Full_acc: 90.060000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[14.262825155716163, 2.924289265562907, 0.5505950024563626, 0.11999170611395046, 0.02744225430159203, 0.006746828169761675, 0.0016618247316436267, 0.00044322956430803987, 0.00012768623230197918, 4.650674238497832e-05, 2.652061807080142e-05, 2.204594599459253e-05, 1.6918118686923485e-05, 2.011766220617495e-05, 2.067195538459942e-05, 14.917494972939796, 2.4171221602368393, 1.7598931061155216, 1.8031076352626845, 1.7405455628994735, 1.793194596897975, 1.8030161795108837, 2.5971766763412623, 1.0002184887481798, 0.31595235279138284, 0.5927951847220075, 0.10449790573630993, 0.15336463195724515, 0.456008781548831, 0.04802145042877176, 0.040792344467324326, 0.1533515894473453, 0.39956516741739173, 0.03170617607986788, 0.010133194077074878, 0.04078978612397213, 0.15334956958540794, 0.37421014652121226, 0.02645526524379531, 0.0025665850063961667, 0.010150208235922479, 0.040788831687284116, 1.2360329844220124]
-js_param_list:
-[7.646327343346346, 1.1369253976412526, 0.21223178228730832, 0.0462335545707114, 0.010691317072738505, 0.002701012583115348, 0.0006444077532065897, 0.00016180418230428466, 6.402129794029498e-05, 1.968498314365874e-05, 1.6924953659230635e-05, 6.065384498061266e-06, 9.299935263935805e-06, 1.1852805417376545e-05, 1.1821849182492628e-05, 8.602750549829853, 1.198344465680804, 1.052990177854398, 1.0729257331977795, 1.0820562338446593, 1.0805086622820688, 1.0756339188081119, 1.503943954991838, 0.6077287851397646, 0.14854021027605877, 0.36767343901811383, 0.04437681481233167, 0.09885630898936866, 0.2829765056435155, 0.01757503223494139, 0.025387881883922547, 0.09892526136629265, 0.2481673048125344, 0.010192293341661128, 0.0061815967921599645, 0.02538376152549722, 0.09892383354598411, 0.23245635974338605, 0.007921735530031428, 0.0015711629407955173, 0.006130501581707569, 0.025380028273212177, 0.6825620173760718]
-ptq_acc_list:
-[10.0, 10.79, 77.47, 88.08, 89.5, 89.93, 89.99, 90.01, 90.01, 90.03, 90.06, 90.07, 90.06, 90.09, 90.09, 10.0, 22.78, 77.33, 75.72, 77.74, 76.09, 77.69, 10.92, 55.56, 81.89, 80.03, 87.53, 86.94, 84.0, 89.09, 89.59, 87.0, 85.36, 89.28, 89.73, 89.35, 87.03, 85.68, 89.53, 89.9, 89.73, 89.45, 81.45]
-acc_loss_list:
-[0.8889629136131468, 0.8801909837885855, 0.13979569176104822, 0.021985343104596978, 0.006218076837663805, 0.0014434821230290412, 0.0007772596047080544, 0.0005551854319342345, 0.0005551854319342345, 0.00033311125916057226, 0.0, -0.00011103708638675222, 0.0, -0.00033311125916057226, -0.00033311125916057226, 0.8889629136131468, 0.7470575172107484, 0.14135021097046419, 0.15922718187874754, 0.13679769042860324, 0.1551188096824339, 0.13735287586053746, 0.8787475016655563, 0.38307794803464357, 0.09071729957805909, 0.11137019764601377, 0.028092382855873874, 0.03464357095269825, 0.06728847435043307, 0.010770597379524748, 0.005218743060182088, 0.03397734843437711, 0.05218743060182104, 0.008660892738174563, 0.0036642238507661367, 0.007883633133466666, 0.033644237175216536, 0.04863424383744165, 0.005884965578503233, 0.0017765933821896134, 0.0036642238507661367, 0.00677326226959804, 0.0956029313790806]
-
-
-VGG_19
-Full_acc: 90.140000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[13.048974717343004, 2.6749312021528913, 0.4909782092613753, 0.10788846263262551, 0.024765729210361388, 0.006134660841831837, 0.001511714235343558, 0.00035217524779276296, 9.027838452787228e-05, 4.826876430404425e-05, 2.3137291736999405e-05, 1.935483645987856e-05, 2.0907217644717642e-05, 3.387203643630215e-05, 1.4449608674907743e-05, 13.81147885113297, 2.1226757078950094, 1.6530962572212622, 1.6875658424517532, 1.6481537280948702, 1.6834940394264948, 1.6566304454586653, 2.406336585421847, 0.939816429984288, 0.2922284769134882, 0.5545077160998108, 0.09259667141740417, 0.146967465597935, 0.42356516614972445, 0.04246028388371049, 0.03759871228427055, 0.14690249068281197, 0.3717209019427504, 0.027698966422784608, 0.009442294391095476, 0.037535971730376476, 0.14689964408599956, 0.3469994616021775, 0.02262714849993608, 0.0023915509517363483, 0.009467019535196476, 0.03753423478901108, 1.103711294382473]
-js_param_list:
-[7.87702175577721, 1.1811605933279172, 0.22674393351088745, 0.04917663913573108, 0.01150250664366638, 0.0027928304104144464, 0.0006812588208013448, 0.0001419666987856605, 4.487908372373006e-05, 4.4161934990693646e-05, 2.0421922284708402e-05, 1.0125851299536251e-05, 2.2663438017443415e-05, 1.7626631301269656e-05, 7.0688205745594876e-06, 8.778726407792607, 1.1730341880807205, 1.0298268062193983, 1.0443298310858409, 1.0362971008666828, 1.0415261153131727, 1.0389181506047174, 1.5705945195303064, 0.6537306062275565, 0.15144308109334406, 0.4032963612294232, 0.045927748342518185, 0.09826016473130708, 0.31379083728006496, 0.018516901788017837, 0.02573785465867853, 0.0982350784265641, 0.2765367058530107, 0.010813489695802557, 0.006358928752244081, 0.02563535210744919, 0.09823167663379691, 0.25956214400602157, 0.008427520460296526, 0.001612199066969595, 0.0063761376736868125, 0.02563587243935141, 0.5868812546140629]
-ptq_acc_list:
-[10.0, 10.82, 67.51, 88.12, 89.54, 90.05, 90.1, 90.12, 90.16, 90.13, 90.16, 90.18, 90.16, 90.17, 90.17, 10.0, 17.7, 67.89, 58.46, 68.4, 59.88, 61.53, 10.75, 59.68, 78.68, 81.82, 88.0, 86.99, 84.53, 89.13, 89.58, 87.11, 85.86, 89.36, 89.81, 89.53, 86.96, 86.77, 89.47, 90.12, 89.88, 89.48, 79.32]
-acc_loss_list:
-[0.889061459951187, 0.8799644996671843, 0.2510539161304637, 0.022409585089860174, 0.006656312402928714, 0.0009984468604393544, 0.0004437541601953212, 0.00022187708009758177, -0.00022187708009758177, 0.0001109385400488697, -0.00022187708009758177, -0.0004437541601953212, -0.00022187708009758177, -0.0003328156201464515, -0.0003328156201464515, 0.889061459951187, 0.803638784113601, 0.24683825160860884, 0.35145329487463944, 0.2411803860661193, 0.335700022187708, 0.3173951630796539, 0.8807410694475261, 0.33791879298868427, 0.12713556689593958, 0.09230086532061246, 0.023740847570445978, 0.034945640115376146, 0.062236520967384065, 0.011204792544930165, 0.006212558242733551, 0.03361437763479034, 0.047481695140891955, 0.008653206123807423, 0.0036609718216108087, 0.006767250942977584, 0.035278455735522594, 0.03738628799645002, 0.007432882183270487, 0.00022187708009758177, 0.0028844020412691937, 0.0073219436432216175, 0.1200355003328157]
-
-
-Inception_BN
-Full_acc: 92.850000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[24.501583051204396, 3.5853632714637826, 0.5764639203863853, 0.1254596836956013, 0.029294242308528968, 0.007074992528733897, 0.0017265789356264636, 0.00044525835147108414, 0.00010715587044753592, 2.763500892255966e-05, 1.3372514338448216e-05, 1.651875332239356e-05, 1.071087798827436e-05, 1.1416507626689443e-05, 1.8697256957320165e-05, 23.2714172433988, 2.7164069977501053, 2.8011785806535356, 2.6953667618375734, 2.8191085641023816, 2.773406742070304, 2.6137332471433825, 4.303955467683129, 1.728203312661921, 0.39222141068340766, 1.0466501003638464, 0.11544128460953085, 0.26217790777428135, 0.8060525439567119, 0.04629405404231517, 0.06617748628761817, 0.26216117578349835, 0.7067635183298785, 0.02682603064554569, 0.016838761834673067, 0.06618181366908193, 0.26216038591779744, 0.6617165182206839, 0.020762169553687215, 0.004244293114838931, 0.01680512193717317, 0.06618269148789903, 1.8439418092752666]
-js_param_list:
-[23.398090191439067, 3.662370609719416, 0.5152455334173763, 0.11217440188269354, 0.02624451610255668, 0.0063719985735158444, 0.001544632694210822, 0.00038196865652848907, 0.00010483607892653654, 2.9065382911872993e-05, 2.1424251686981593e-05, 1.6622652370079758e-05, 2.136174368302968e-05, 1.0926635195267737e-05, 1.957231372594437e-05, 23.937170004690962, 2.759034655091616, 3.7002130819772976, 3.532238910924999, 3.6297386953671507, 3.620141221546079, 3.4493545951768345, 4.26354002827198, 1.704408792013372, 0.37079297092866725, 1.0288180768202424, 0.10723518968423576, 0.2641344164185636, 0.7903383537153296, 0.041324052299147755, 0.06690974766269926, 0.2641313817787517, 0.6918113077016262, 0.022814864784890004, 0.017182124843102154, 0.06691562325891631, 0.264131453339321, 0.6472324961904546, 0.017213687801549447, 0.0043651611752884645, 0.017148119773601003, 0.06691453707221243, 2.5415331853471583]
-ptq_acc_list:
-[10.0, 10.71, 69.45, 89.97, 92.08, 92.86, 92.87, 92.9, 92.9, 92.88, 92.84, 92.86, 92.85, 92.85, 92.84, 10.0, 10.36, 18.61, 23.06, 16.98, 14.41, 19.57, 11.41, 13.97, 79.08, 38.49, 90.11, 84.51, 40.49, 91.79, 91.6, 79.59, 48.48, 92.36, 92.53, 91.33, 80.13, 51.38, 92.61, 92.73, 92.63, 90.82, 29.59]
-acc_loss_list:
-[0.8922994076467421, 0.8846526655896606, 0.2520193861066235, 0.03101777059773824, 0.008292945611200819, -0.00010770059235331305, -0.0002154011847066261, -0.0005385029617664122, -0.0005385029617664122, -0.0003231017770597861, 0.00010770059235316, -0.00010770059235331305, 0.0, 0.0, 0.00010770059235316, 0.8922994076467421, 0.8884221863220247, 0.799569197630587, 0.7516424340333872, 0.8171243941841679, 0.8448034464189553, 0.7892299407646742, 0.8771136241249328, 0.8495422724824987, 0.14830371567043615, 0.5854604200323101, 0.029509962304792622, 0.08982229402261702, 0.5639203015616585, 0.011416262789445214, 0.013462574044157244, 0.14281098546041995, 0.4778675282714055, 0.005277329025309584, 0.003446418955304181, 0.016370490037695164, 0.1369951534733441, 0.4466343564889606, 0.0025848142164781357, 0.0012924071082389913, 0.0023694130317716625, 0.021863220247711375, 0.6813139472267097]
-
-

ykl/ALL/ptq_result_weightratio.txt

-AlexNet
-Full_acc: 86.060000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[15.390794522744232, 4.717097541596072, 0.861433466201176, 0.18508192637631454, 0.043397405458327654, 0.010496797691919834, 0.0024850090520978957, 0.000624175093904678, 0.00014904738718305474, 5.291579966091702e-05, 4.8962531373354375e-06, 4.193895417504035e-05, 1.592819528135614e-06, 2.967417045749432e-06, 1.0413106493855072e-06, 15.321349499009886, 2.669359363004649, 0.665240323544293, 0.6648249490573711, 0.6647401888858064, 0.6646321902074231, 0.6648930262034427, 2.970237193983619, 1.0721955650797796, 0.4770905332198318, 0.6054695109850212, 0.16107606526699011, 0.181228082062255, 0.45318591305589634, 0.07980226898451764, 0.04805752515345307, 0.18115329705371844, 0.39433974657405857, 0.055063869704067014, 0.012139171526851388, 0.048132818635071795, 0.1811559349720972, 0.3671160558876084, 0.04605680247234207, 0.003032838870538176, 0.011984056400302881, 0.04813150319087321, 0.18115602343582896]
-js_flops_weighted_list:
-[28.40515877286409, 8.70584714896398, 1.589856478399669, 0.3415861018050526, 0.08009399323423358, 0.019372827348516423, 0.004586317917020654, 0.0011519738385324405, 0.0002750809707295301, 9.766108492581001e-05, 9.036495650216035e-06, 7.740228422577126e-05, 2.9396982414617645e-06, 5.476647239052785e-06, 1.921836737145626e-06, 28.276991450362036, 4.926553753669294, 1.227763581955602, 1.2269969692746403, 1.2268405364064507, 1.2266412146883807, 1.227122612050269, 5.481852087849688, 1.978837753739401, 0.8805154487062422, 1.1174509260221974, 0.29728102741256435, 0.3344734696751785, 0.8363972240000284, 0.14728259269462676, 0.08869468240885497, 0.33433544690850636, 0.7277910893641049, 0.1016255502132089, 0.02240398272367832, 0.08883364361042133, 0.3343403154349667, 0.6775472078549858, 0.08500216053589742, 0.005597389369524905, 0.02211770316928806, 0.0888312158344373, 0.3343404787031035]
-js_param_list:
-[5.335234710880941, 1.5364356992134998, 0.2810079498291621, 0.06031047856052035, 0.014110041947520574, 0.0034040126288667374, 0.0008046306976175277, 0.00019470572779706148, 5.339504844640815e-05, 2.3103931722405794e-05, 1.118676972143684e-06, 2.186891678345004e-05, 7.638901720498192e-06, 6.725816434148269e-07, 4.19329140017799e-06, 5.313908139577299, 0.8537195505193228, 0.21603135769650977, 0.21582717195610415, 0.21581634718014503, 0.21575105452221033, 0.21583294142343779, 1.0080567176520165, 0.36710793002522313, 0.15308999403878698, 0.21012211564543096, 0.05127532080061556, 0.059764869329634644, 0.15863713336428933, 0.02542085053727128, 0.015573244047684945, 0.05975132050282547, 0.1384792977857137, 0.01742679923812338, 0.003952068904423522, 0.015598955194465434, 0.059751507791787915, 0.12954601789882095, 0.014575061413335639, 0.0009603470887152325, 0.0039045494639076486, 0.015598733274334387, 0.059751990616446676]
-ptq_acc_list:
-[10.0, 10.31, 56.17, 77.08, 84.22, 85.71, 85.98, 86.09, 86.08, 86.02, 86.06, 86.05, 86.08, 86.08, 86.08, 10.0, 16.15, 73.31, 74.31, 74.16, 74.21, 74.55, 20.62, 67.09, 53.42, 79.19, 79.39, 82.32, 81.78, 83.36, 85.47, 82.97, 82.16, 84.49, 86.01, 85.78, 82.91, 82.59, 84.78, 86.05, 86.04, 85.75, 83.02]
-acc_loss_list:
-[0.883801998605624, 0.8801998605623983, 0.3473158261677899, 0.1043458052521497, 0.021380432256565225, 0.00406693004880326, 0.0009295840111549883, -0.0003485940041831413, -0.0002323960027887058, 0.0004647920055775767, 0.0, 0.00011619800139443546, -0.0002323960027887058, -0.0002323960027887058, -0.0002323960027887058, 0.883801998605624, 0.8123402277480827, 0.14815245177782943, 0.13653265163839182, 0.1382756216593075, 0.13769463165233567, 0.13374389960492686, 0.7603997211247966, 0.22042760864513128, 0.3792702765512433, 0.07982802695793638, 0.07750406693004883, 0.04345805252149673, 0.049732744596792944, 0.03137346037648156, 0.0068556820822682244, 0.03590518243086223, 0.04531722054380671, 0.01824308621891712, 0.000580990006971847, 0.0032535440390425415, 0.03660237043922851, 0.040320706483848466, 0.014873344178480142, 0.00011619800139443546, 0.0002323960027887058, 0.003602138043225683, 0.03532419242389038]
-
-
-AlexNet_BN
-Full_acc: 87.310000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[11.591798533785282, 1.700183197789268, 0.3038094135267426, 0.06701544888451295, 0.01593624543282047, 0.003825953228443132, 0.0009532233854295787, 0.00019551855882143858, 6.55307570327729e-05, 3.47072260035146e-05, 3.505022686533898e-05, 2.3009420528141446e-05, 8.312853958041267e-07, 3.116905667386982e-06, 3.833276897941617e-05, 7.000022231570889, 1.5005345800833543, 0.745846478904552, 0.7646294650365152, 0.7972771082666167, 0.8348834173962452, 0.8141783554712251, 1.377012987292044, 0.5252011904953136, 0.18856212240816972, 0.31178739055080124, 0.05899102144452863, 0.09192247716102836, 0.23655059730355277, 0.027952782309607387, 0.021727784158718434, 0.09180928235111235, 0.20792645939896084, 0.018443659406444046, 0.006097508898828769, 0.02164334660618966, 0.09181375890446229, 0.19470294027215662, 0.01519870107475935, 0.001524288569272028, 0.006088090455227164, 0.021643227963696866, 0.33145092225344597]
-js_flops_weighted_list:
-[21.406669156099774, 3.1397422163400726, 0.5610473286712618, 0.12375797754223267, 0.029429594775694786, 0.007065419117601705, 0.0017603254218295182, 0.00036106572162803395, 0.000121016184957042, 6.409411811443945e-05, 6.472754061121002e-05, 4.249168507232945e-05, 1.5351415391158844e-06, 5.75601518763951e-06, 7.078943797961337e-05, 12.9269983048652, 2.7710494811923083, 1.377360792513532, 1.41204748662046, 1.4723381563828981, 1.541786035016542, 1.50354982782284, 2.5429411455607536, 0.9698933338562424, 0.34821921361029456, 0.5757803240896372, 0.10893920176087167, 0.16975399036310762, 0.4368399226743099, 0.051620631703546804, 0.04012487670702703, 0.1695449525818697, 0.3839794931028098, 0.034060056667721286, 0.011260319552066803, 0.039968945188197914, 0.1695532194695526, 0.35955951218252213, 0.028067565577641885, 0.0028149161672995126, 0.011242926435238788, 0.03996872609013615, 0.6120931289035025]
-js_param_list:
-[2.734479088940684, 0.3132424172357634, 0.054306647194035426, 0.011796604135106659, 0.0027847107051644014, 0.0006822415151577091, 0.0001651089941727384, 3.228843341085658e-05, 1.2759462925018916e-05, 8.564187390072141e-06, 9.326278173153569e-06, 5.965686148541275e-06, 3.5330878817685375e-08, 8.826677324214048e-07, 1.0788902521246867e-05, 1.4564256642500042, 0.31047654192531055, 0.17060282426356335, 0.1771587081095755, 0.18545945737715597, 0.19559617351750125, 0.189959581337747, 0.265386519272884, 0.10192080314098644, 0.03236375259538735, 0.06019249236191452, 0.010293599322833998, 0.016283465321462246, 0.04594274741460937, 0.004729445591036409, 0.004158369800705201, 0.016253072643470482, 0.0402070007169081, 0.0030832450417326886, 0.001062630465564137, 0.004136040389603472, 0.016254232375065597, 0.03763443684725491, 0.0025390305847557497, 0.00027616659750303706, 0.0010603967983225746, 0.004136140717399854, 0.08413230006865291]
-ptq_acc_list:
-[10.0, 15.29, 37.89, 80.02, 85.56, 87.09, 87.42, 87.42, 87.25, 87.29, 87.28, 87.32, 87.29, 87.32, 87.3, 10.0, 16.25, 50.55, 54.82, 52.0, 55.79, 53.02, 18.65, 61.89, 69.85, 72.73, 82.03, 83.5, 76.15, 84.4, 86.66, 83.66, 78.13, 85.68, 86.93, 86.54, 83.27, 78.21, 85.91, 87.23, 87.0, 87.04, 81.06]
-acc_loss_list:
-[0.8854655824075135, 0.8248768755010881, 0.5660290917420685, 0.08349559042492276, 0.020043523078685146, 0.0025197571870346908, -0.0012598785935173454, -0.0012598785935173454, 0.0006872065055549452, 0.0002290688351849275, 0.0003436032527774726, -0.00011453441759238237, 0.0002290688351849275, -0.00011453441759238237, 0.00011453441759254514, 0.8854655824075135, 0.8138815714122094, 0.42102851906998057, 0.3721223227579888, 0.40442102851906997, 0.3610124842515176, 0.3927385179246363, 0.7863933111900125, 0.2911464895201008, 0.1999770931164816, 0.16699118084984535, 0.060474172488832904, 0.0436376131027374, 0.12782041003321493, 0.033329515519413544, 0.00744473714351169, 0.04180506242125765, 0.10514259534990272, 0.018669110067575254, 0.004352307868514436, 0.008819150154621418, 0.046271904707364635, 0.10422632000916285, 0.01603481846294818, 0.0009162753407398728, 0.0035505669453671086, 0.003092429274997091, 0.07158401099530409]
-
-
-VGG_16
-Full_acc: 90.060000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[5.920568558509058, 1.2366942369029248, 0.24333976529364282, 0.05313804637395972, 0.011805343923029208, 0.0028956135593713197, 0.0006858387721491292, 0.00017475428648558512, 5.5037716377499444e-05, 1.8363316636701207e-05, 3.0147309189941645e-06, 3.5938150094587322e-06, 7.588472673062187e-07, 2.761265478140623e-06, 1.8205751891677835e-06, 6.655998976309741, 1.2806109655501847, 1.148895211956822, 1.110608810491263, 1.1338888989171876, 1.1246610427359278, 1.1214109082880812, 1.0629437809132753, 0.403389070458992, 0.13440872344714155, 0.23736481179302008, 0.046645787173690494, 0.06196931592022412, 0.1834877783166125, 0.021485302765101157, 0.017332589659448675, 0.06194557184806153, 0.16064264766655828, 0.014416940182581594, 0.004258526972833182, 0.01732019862118995, 0.06194491639496624, 0.15031629789717732, 0.012258144588710359, 0.001051457151644622, 0.004272414206520595, 0.017319316052076844, 0.8666441153492498]
-js_flops_weighted_list:
-[14.937082083214763, 3.1200725311944497, 0.6139251682302425, 0.13406269222045603, 0.02978386103568301, 0.00730538241220721, 0.0017303118668762724, 0.0004408899408617193, 0.00013885539523427596, 4.632905863884517e-05, 7.605905201638485e-06, 9.066884245605717e-06, 1.914505981707713e-06, 6.966433830287458e-06, 4.593153642344031e-06, 16.792509380208696, 3.2308706368405034, 2.898563189737484, 2.8019699122996187, 2.8607035606453954, 2.8374224781160025, 2.829222670183841, 2.6817151677984326, 1.0177157138494521, 0.33910157698362114, 0.5988508776447673, 0.11768328412449634, 0.15634322099196396, 0.4629237849197624, 0.05420555945151566, 0.04372862367851055, 0.15628331675280557, 0.40528749739950476, 0.03637269238045072, 0.01074389500234279, 0.04369736216135425, 0.15628166310137173, 0.3792350106153517, 0.03092623792799905, 0.002652735396252887, 0.01077893129107887, 0.04369513551586299, 2.1864681001458894]
-js_param_list:
-[1.770296244959528, 0.29806992793707837, 0.055665464831966924, 0.012101417041225202, 0.0028447136298671597, 0.0006880367651531245, 0.00017020795157846893, 4.522436604353743e-05, 2.085685768523233e-05, 1.3865664827935832e-05, 1.9972193056226206e-06, 5.692071212596929e-07, 1.8434384391604333e-06, 2.8346000195430685e-06, 2.562304531827042e-06, 2.6600226813451946, 0.6233435306363425, 0.7163984416773069, 0.6869824985889844, 0.7096845057337227, 0.7146866621934539, 0.7032853391142566, 0.3446638133393568, 0.140197618797341, 0.03384653692463511, 0.08580447942745584, 0.010451821497441623, 0.01977311058211095, 0.06660686501570992, 0.0044870860230231315, 0.005106004645241131, 0.019767490033555546, 0.05864023434517889, 0.002784702570814231, 0.0012759569718724024, 0.005101844752063444, 0.01976665091010429, 0.05504161767683782, 0.0022191752052733086, 0.00031513865524340187, 0.0012912743585447657, 0.005102061834463343, 0.5556901116653865]
-ptq_acc_list:
-[10.0, 13.22, 76.72, 88.11, 89.56, 89.98, 89.99, 90.0, 90.0, 90.04, 90.07, 90.06, 90.06, 90.07, 90.09, 10.0, 20.94, 77.81, 75.14, 75.06, 76.45, 77.19, 11.53, 54.29, 81.34, 80.0, 87.6, 86.8, 84.21, 88.85, 89.41, 87.05, 85.21, 89.39, 89.84, 89.38, 86.9, 85.74, 89.45, 89.86, 89.78, 89.4, 81.93]
-acc_loss_list:
-[0.8889629136131468, 0.8532089717965801, 0.14812347324006223, 0.021652231845436406, 0.00555185431934266, 0.0008882966910948067, 0.0007772596047080544, 0.0006662225183211445, 0.0006662225183211445, 0.00022207417277366224, -0.00011103708638675222, 0.0, 0.0, -0.00011103708638675222, -0.00033311125916057226, 0.8889629136131468, 0.7674883411059295, 0.13602043082389517, 0.165667332889185, 0.1665556295802798, 0.15112147457250721, 0.14290473017988012, 0.8719742393959582, 0.39717965800577393, 0.09682433932933598, 0.11170330890517435, 0.027315123251165977, 0.0361980901621142, 0.06495669553630921, 0.013435487452809326, 0.007217410615145522, 0.03342216300244287, 0.0538529868976239, 0.007439484787919184, 0.002442815900510758, 0.007550521874306094, 0.03508771929824558, 0.04796802131912067, 0.00677326226959804, 0.002220741727737096, 0.0031090384188319025, 0.007328447701532273, 0.0902731512325116]
-
-
-VGG_19
-Full_acc: 90.140000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[5.42539688208821, 1.2431814495773714, 0.22893087958060832, 0.050875650693093466, 0.011424559252241116, 0.002888330566046987, 0.0006958012938553044, 0.0001620122109056303, 4.628456898334069e-05, 4.606778419700862e-06, 6.266021337973151e-06, 3.404626571397867e-07, 4.28695999660748e-06, 3.0337623828983415e-07, 1.0465506947944175e-06, 6.098423281023068, 1.10155611081281, 0.9965644293881576, 0.9874767728771963, 0.9815533180848163, 0.9920825228870539, 1.006248223283427, 0.9964726594305288, 0.38478060783526064, 0.13371221116731546, 0.22201018739741246, 0.04292414200270462, 0.062727570196778, 0.16807050915013197, 0.020087426776406736, 0.016173171638683738, 0.06268975506483192, 0.14755320645575126, 0.013336430846658188, 0.004058001192906326, 0.01615557232713148, 0.06268919379337706, 0.13696270689396448, 0.01092882410397449, 0.001026471090943476, 0.004062249384958819, 0.016155447628954833, 0.7151647488167303]
-js_flops_weighted_list:
-[14.223174719098854, 3.2591140056975556, 0.6001632635617812, 0.1333751768731794, 0.0299505282037136, 0.0075720230575257045, 0.0018241068049697996, 0.0004247298460221305, 0.00012133923577492387, 1.2077091460700653e-05, 1.6426948704495657e-05, 8.925540311742125e-07, 1.1238658179430174e-05, 7.95328588230114e-07, 2.743628476950432e-06, 15.987575051583281, 2.887830211778206, 2.612584905045913, 2.5887607813639586, 2.5732319022268664, 2.6008351767540856, 2.6379718574679347, 2.612344321698798, 1.0087375970282038, 0.3505388053339556, 0.5820200352896387, 0.11252956873763163, 0.16444606910857248, 0.4406122296163039, 0.052661028659708, 0.042399450395679474, 0.16434693327752012, 0.386824241874752, 0.03496267465472304, 0.010638421710229424, 0.04235331218891159, 0.1643454618530067, 0.35906027752273645, 0.02865091311905997, 0.002690987956806161, 0.010649558734693455, 0.042352985281223145, 1.8748698752238893]
-js_param_list:
-[1.4984730901682581, 0.2683235732843003, 0.04958604921625643, 0.010778911066837874, 0.002515988105136827, 0.0006044757002238476, 0.00015185584462366542, 3.125311773258917e-05, 9.291434369348796e-06, 2.403748311446711e-07, 2.2523717069928906e-06, 3.05053520044847e-08, 5.350493385841079e-06, 6.700927991514511e-07, 1.7557961942484982e-07, 2.1026192189821815, 0.45659539551001177, 0.5084291166478593, 0.5101433777932717, 0.508929468342814, 0.5094047352848247, 0.5180348597519879, 0.2931382886776165, 0.11863554595409948, 0.029916657741284044, 0.0723272689470671, 0.00924226226738377, 0.01679713209566103, 0.05600879079504824, 0.004060626606009455, 0.00428490928954925, 0.01680431404310862, 0.049292483042490044, 0.0025609913043603437, 0.0010654431936954005, 0.004274791409264434, 0.01680445136301286, 0.046263886113338384, 0.00205286507975312, 0.0002642645241999461, 0.0010691611662199975, 0.00427478016792876, 0.3638833699602575]
-ptq_acc_list:
-[10.0, 10.19, 65.93, 87.82, 89.82, 90.02, 90.07, 90.04, 90.18, 90.19, 90.17, 90.15, 90.17, 90.16, 90.16, 10.0, 16.77, 59.02, 62.25, 64.35, 61.93, 61.73, 10.55, 59.85, 79.42, 81.95, 87.93, 87.04, 84.63, 89.18, 89.48, 87.21, 86.08, 89.5, 89.82, 89.43, 86.75, 86.71, 89.59, 90.05, 89.86, 89.58, 78.65]
-acc_loss_list:
-[0.889061459951187, 0.8869536276902596, 0.2685822054581761, 0.025737741291324688, 0.0035500332815620967, 0.001331262480585806, 0.0007765697803417727, 0.0011093854004880666, -0.0004437541601953212, -0.0005546927002440333, -0.0003328156201464515, -0.0001109385400488697, -0.0003328156201464515, -0.00022187708009758177, -0.00022187708009758177, 0.889061459951187, 0.8139560683381407, 0.3452407366319059, 0.30940758819613934, 0.2861104947858887, 0.31295762147770134, 0.31517639227867766, 0.8829598402485024, 0.3360328378078544, 0.11892611493232748, 0.09085866429997778, 0.024517417350787593, 0.034390947415131955, 0.061127135566895995, 0.010650099844685975, 0.0073219436432216175, 0.032504992234302275, 0.045041047259818084, 0.007100066563124036, 0.0035500332815620967, 0.007876636343465651, 0.0376081650765476, 0.03805191923674292, 0.006101619702684681, 0.0009984468604393544, 0.0031062791213667756, 0.006212558242733551, 0.12746838251608603]
-
-
-Inception_BN
-Full_acc: 92.850000
-title_list:
-['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
-js_flops_list:
-[7.715147804903006, 1.180638829948032, 0.17330341588504639, 0.03757641893210694, 0.00878982153678008, 0.0021136845718360358, 0.0005197591695297085, 0.00012868349861507722, 3.2617573467141134e-05, 1.95771371272391e-06, 2.329043574392525e-06, 1.9973666121921525e-06, 2.181171911720007e-06, 5.134343259917914e-06, 4.130203057900563e-06, 7.258357623221218, 1.135985132290838, 1.4849991181524151, 1.5395437466854156, 1.449550683059265, 1.3787184235186551, 1.3649998008469453, 1.2069771223431678, 0.48469862476839726, 0.11355577484142695, 0.2937877086812499, 0.03402380401823649, 0.07264058418816968, 0.2264499587330589, 0.013962832913529065, 0.01846691453157432, 0.07264100003421901, 0.19874527130821865, 0.008312257117463705, 0.004663653123973545, 0.018472925707157563, 0.07264083738327363, 0.1861251700828908, 0.006510681819443857, 0.0011782965721076142, 0.004656104987596719, 0.018474123863005152, 1.1891994561788195]
-js_flops_weighted_list:
-[24.492915695935718, 3.7481183848335724, 0.5501781770652018, 0.11929208413536399, 0.02790463168390087, 0.00671021467571623, 0.0016500549105994422, 0.0004085254311050457, 0.00010354947142152921, 6.215061348798991e-06, 7.393904739388733e-06, 6.34094553771341e-06, 6.924463549246638e-06, 1.6299757282582123e-05, 1.3111960763728517e-05, 23.042765459856884, 3.606358397872026, 4.714356630514774, 4.887516889022836, 4.601820156263742, 4.376952324129888, 4.333400460049073, 3.8317333189245306, 1.5387498534822412, 0.36050016024452497, 0.9326739763377017, 0.1080137651989134, 0.23060863506641904, 0.7188999989179939, 0.044327146812426284, 0.05862604219953586, 0.23060995523325012, 0.6309472349997668, 0.026388530455838718, 0.01480547951757709, 0.05864512559502185, 0.23060943887296853, 0.5908827951208971, 0.020669154364788552, 0.0037406825293873503, 0.014781516805181413, 0.05864892932385605, 3.77529540099415]
-js_param_list:
-[5.547292775694633, 1.0830498393821684, 0.11597159954861848, 0.025184683478010235, 0.005889836213039606, 0.001418800737950504, 0.0003470743045252829, 8.729454098248694e-05, 2.0691080441622126e-05, 1.3027382812223066e-06, 5.085435185989181e-06, 2.6182420731614744e-06, 3.434516785198397e-06, 6.510479031259873e-06, 4.896755871723168e-06, 6.431020460639781, 1.2042858915688233, 2.1661643185195403, 2.2217083989716477, 2.12019065583259, 2.0296487615468775, 1.996163205688216, 0.8993658060076932, 0.3600575347311997, 0.08029751063059903, 0.21764768767983095, 0.0235711444064559, 0.05512862455451699, 0.1674148989599356, 0.009286143104694698, 0.014028389319905675, 0.05513198123092901, 0.14665927416585386, 0.005281656984956287, 0.003588795191323551, 0.014033740564019261, 0.05512987087623732, 0.13726783858604585, 0.004046517151685216, 0.0009180459749539567, 0.003582893701133105, 0.01403383522314854, 1.7935260885146647]
-ptq_acc_list:
-[10.0, 10.36, 67.33, 89.38, 92.15, 92.8, 92.81, 92.9, 92.89, 92.84, 92.85, 92.85, 92.85, 92.86, 92.84, 10.0, 12.36, 17.68, 19.75, 24.11, 23.29, 22.15, 10.89, 12.39, 81.29, 33.15, 89.88, 75.48, 28.39, 91.96, 91.1, 85.62, 63.83, 92.39, 92.58, 91.25, 82.3, 57.86, 92.35, 92.82, 92.72, 91.78, 25.49]
-acc_loss_list:
-[0.8922994076467421, 0.8884221863220247, 0.27485191168551426, 0.0373721055465805, 0.007539041464727934, 0.0005385029617662591, 0.0004308023694129461, -0.0005385029617664122, -0.00043080236941309916, 0.00010770059235316, 0.0, 0.0, 0.0, -0.00010770059235331305, 0.00010770059235316, 0.8922994076467421, 0.8668820678513732, 0.8095853527194399, 0.7872913301023156, 0.7403338718362951, 0.7491653204092623, 0.7614431879375336, 0.882714054927302, 0.8665589660743134, 0.12450188476036606, 0.6429725363489499, 0.0319870759289176, 0.18707592891760896, 0.6942380183091007, 0.009585352719439964, 0.01884760366182014, 0.07786752827140539, 0.31254711900915455, 0.004954227248249799, 0.0029079159935379217, 0.017232094776521212, 0.1136241249326871, 0.3768443726440495, 0.005385029617662898, 0.0003231017770597861, 0.0014001077005923043, 0.011523963381798527, 0.7254711900915456]
-
-

ykl/README.md

 # 改动说明
 
+## update2: 2023/05/26
+
++ 添加了cifar100数据集支持，详见ALL-cifar100。原先ALL文件夹重命名为ALL-cifar10
++ 在ALL-cifar10中罗列了多种训练策略的结果，并进行简要分析。针对cifar100，同样进行了多种尝试，展示了全精度准确度和拟合效果均较好的训练策略和结果。
++ 对poly2添加新的分段拟合matlab脚本，分界点等可自行指定。
+
+## update1: 2023/05/26
+
++ 添加针对BasicBlock、BottleNeck、InvertedResidual的Makelayer方法和Relu6的框架。支持了当前所有不含LSTM的网络
++ 新的全精度模型训练方案，详见ALL/README.md
++ 更改了flop_ratio将relu层参数量加到conv层处的笔误。
++ conv层新增groups参数的cfg
++ ALL/README.md部分更换为集合所有当前模型，采用fakefreeze方法，且weight：bias由0.5:0.5变为1：1的拟合效果，并包含对计算量的log和cuberoot加权。之前的模型拟合结果和尝试分析移动至`fit_bkp.md`
+
 ## update: 2023/04/26
 
 + 将新框架应用于所有模型（AlexNet、AlexNet_BN、VGG_16、VGG_19、Inception_BN），并采用不同方式对单个模型拟合，以及对所有模型一同拟合。详见ALL