feat: ALL: new frame and ptq for all model

ykl/ALL/README.md

+# 模型整合说明
+
++ 该文件夹下实现了基于cifar10数据集的AlexNet、AlexNet_BN、VGG_16、VGG_19、Inception_BN的整合。
+
+### 部署说明
+
+#### cfg_table
+
+在该通用框架下，当前所有模型部署只需要提供一个cfg_table，如包含特殊结构（如Inception_BN的inception多分支结构），额外针对特殊结构提供cfg_table即可。详见`cfg.py`
+
+cfg_table书写规则说明如下：
+
++ 每项根据进行量化的合并单位给出，例如Conv后接BN或BN+ReLu时，将被统一合并为一个量化层，则在cfg_table中表现为一项，''/'B'/'BR'参数可选。
++ 针对不涉及量化，但位置需特别指定的操作，如flatten、drop，同样在cfg_table中单独指定。
+
+根据cfg_table，也相应简化了fold_ratio和fold_model方法，合并与量化层进行对比的全精度层的参数值、参数量、计算量。
+
+#### 训练方案
+
++ 为每个模型提供了梯度学习率的训练方案，以获得更好的全精度精确度。根据传入的模型名字会自动从epochs_cfg_table和lr_cfg_table中提取，详见`train.py`
+
+
+
+### PTQ部分概述
+
+#### matlab脚本
+
++ `flops\param`、`flops\param_all`、`flops_all_weight`分别是对单个模型、全体模型、计算量加权后全体模型的拟合。
+  针对单个模型，脚本使用不同颜色标记不同量化类别的点；针对全体模型，脚本使用不同颜色标记不同模型的点。
++ 脚本内添加了拟合条件限制，以保证拟合曲线一定是单调不降的。如果想允许拟合曲线在某个位置出现略微的转折下降，可以改动tolerance为一个较小的负数，如-1e-5。
++ 拟合条件限制基于采样实现，对于fakefreeze，采样步长设为0.1，对于L2方法，采样步长设为10
++ 脚本同时在图像上输出拟合效果评估，如SSE、R方、RMSE指标。
++ 支持选择拟合模型，当前可选择poly为2/3/4分别表示rational模型分子分母的多项式次数
++ 由于每次拟合结果会具有细微差别，后续可以更改脚本，使得脚本在每次拟合完成后再进行三次符合约束的拟合，效果均不好于当前拟合才终止，否则取效果更好的拟合结果，并重复该过程。该步骤目前由人工完成
+
+#### 拟合方式及结果简述
+
++ 分别使用了fakefreeze方法和取L2范数的方法将量化层分布rescale到和全精度层一致
+
+  + L2范数在针对平面结构的单个模型的拟合上能取得更好的效果
+  + fakefreeze在对非平面结构模型（Inception_BN）和所有模型整合上取得较好效果
+  + 为了对多模型的普适性，综合考虑下选择fakefreeze方法
+
++ 在对fakefreeze方法的flops模型整合分布观察可以发现，计算量较大的模型在较小js散度时就达到了较大的精度损失，因此同时将模型计算量考虑对散度加权。
+
+  具体而言，分别将LOG10(Mac)和Mac^(1/3)作为散度系数，拟合效果均取得了一定提升。
+
+  PS:对param使用相同方法拟合效果反而变差，需要后续实验。对L2方法采用了乘sqrt(Mac)方案，拟合效果也有一定提高。
+
+#### 基于fakefreeze方法的其他尝试
+
++ 根据weight和bias参数比
+
+  已初步试验，模型整合加权前后flops拟合效果R方分别为0.78和0.81。可能是由于weight和bias的参数比过于接近1，导致bias的js散度损失被忽略。
+
++ 将weight和bias从0.5:0.5变为1:1，即有bias的层将增大影响。
+
+  经过试验，取得了比原方案更好的拟合效果，将单个模型拟合结果和模型整合拟合结果罗列于下。
+
+#### 拟合中发现的问题
+
+在VGG_16 VGG_19 Inception_BN的fakefreeze方式中，都观察到POT量化点扎堆（acc_loss具有略小差距，js_div相近，在图上表现为连续的一竖列点），影响了量化效果。
+
+观察这些模型的权重参数分布，可以发现出现问题的模型存在着无尖端的权重分布。而有尖无尖的分布在面对不同量化方式的分布如下：
+
+![diff1](image/diff1.png)
+
+![diff2](image/diff2.png)
+
+根据不同模型权重分布的特点，可以推测出现问题的模型POT量化散度较大且集中的重要原因是量化后分布与原分布趋势不同。基于此，我们可能需要在相似度之外额外的考虑模型参数分布与量化方式的适配性。这需要进行实验的验证，例如，直接度量全精度模型-量化模型用于衡量分布趋势的系数；度量全精度权重的尖锐程度和量化表的尖锐程度等。并将所得值作用于原先所求js散度上。
+
++ 方案一：度量全精度模型、量化模型分布趋势相似度
+
+  使用pearson相关系数或余弦相似度，并作用于js散度。例如，若POT量化的余弦相似度较小（趋势差异较大），考虑将js散度乘余弦相似度，从而矫正因趋势过大的散度。
+
++ 方案二：考虑尖锐程度
+
+  考虑到无尖端分布遇到有极大尖端的POT量化点列表会产生不同趋势的问题，从分布和量化点的角度入手。例如，衡量在均值范围内的比例，差异较大可能说明尖锐程度差异大，从而矫正js散度。或者可以考虑对原分布做bins切割，若某个bins有量化点则统计该bins内元素，考虑所有和量化点在同一bins的点数以衡量分布与量化方式的适配度。
+
+#### 后续增强拟合效果的方案
+
++ 针对POT量化点扎堆，可以考虑使用更关注趋势的Pearson相关系数、余弦相似度等对js散度进行修正，或者考虑将量化范围切分多个bins评估量化点覆盖率的方式修正。
+
++ 对weight和bias采取更合理的加权方式
+
+  + 根据对精度的影响（不易衡量，不易确定基准）
+
++ 在模型整合上，尝试更有效的加权方式
+
++ 考虑到js散度达到一定值后acc_loss不会再上升（因为最差效果是随机分类，准确度也有10%），采取分段拟合的方式。
+
+  
+
+## ptq拟合结果图示
+
++ 数据拟合
+
+  + L2：使用L2范数将量化层参数rescale
+
+  + fakefreeze：使用dequantize_tensor将量化层参数rescale
+
+  + fakefreeze-nodiv：weight和bias不再是0.5:0.5而是1:1
+
+  + fakefreeze-weightratio：weight和bias按照参数比加权，该参数比通常接近于1
+
+    
+
+### L2
+
++ 所有模型拟合
+
+  ![L2_param](image/L2_param.png)
+
+  ![L2_flops](image/L2_flops.png)
+
+  ![L2_flops_weighted](image/L2_flops_weighted.png)
+
++ 单个模型拟合
+
+  ![L2_AlexNet](image/L2_AlexNet.png)
+
+  ![L2_AlexNet_BN](image/L2_AlexNet_BN.png)
+
+  ![L2_VGG_16](image/L2_VGG_16.png)
+
+  ![L2_VGG_19](image/L2_VGG_19.png)
+
+  ![L2_Inception_BN](image/L2_Inception_BN.png)
+
+
+
+### fakefreeze
+
++ 所有模型拟合
+
+  ![fakefreeze_param](image/fakefreeze_param.png)
+
+  ![fakefreeze_flops](image/fakefreeze_flops.png)
+
+  ![fakefreeze_flops_weighted_log](image/fakefreeze_flops_weighted_log.png)
+
+  ![fakefreeze_flops_weighted_cuberoot](image/fakefreeze_flops_weighted_cuberoot.png)
+
++ 单个模型拟合
+
+  ![fakefreeze_AlexNet](image/fakefreeze_AlexNet.png)
+
+  ![fakefreeze_AlexNet_BN](image/fakefreeze_AlexNet_BN.png)
+
+  ![fakefreeze_VGG_16](image/fakefreeze_VGG_16.png)
+
+  ![fakefreeze_VGG_19](image/fakefreeze_VGG_19.png)
+
+  ![fakefreeze_Inception_BN](image/fakefreeze_Inception_BN.png)
+
+  
+
+#### fakefreeze_nodiv
+
++ 所有模型拟合
+
+  ![fakefreeze_nodiv_param](image/fakefreeze_nodiv_param.png)
+
+  ![fakefreeze_nodiv_flops](image/fakefreeze_nodiv_flops.png)
+
+  ![fakefreeze_nodiv_flops_weighted_log](image/fakefreeze_nodiv_flops_weighted_log.png)
+
+  ![fakefreeze_nodiv_flops_weighted_cuderoot](image/fakefreeze_nodiv_flops_weighted_cuderoot.png)
+
++ 单个模型拟合
+
+  ![fakefreeze_nodiv_AlexNet](image/fakefreeze_nodiv_AlexNet.png)
+
+  ![fakefreeze_nodiv_AlexNet_BN](image/fakefreeze_nodiv_AlexNet_BN.png)
+
+  ![fakefreeze_nodiv_VGG_16](image/fakefreeze_nodiv_VGG_16.png)
+
+  ![fakefreeze_nodiv_VGG_19](image/fakefreeze_nodiv_VGG_19.png)
+
+  ![fakefreeze_nodiv_Inception_BN](image/fakefreeze_nodiv_Inception_BN.png)
+

ykl/ALL/cfg.py

+# conv: 'C',''/'B'/'BR',qi,in_ch,out_ch,kernel_size,stirde,padding,bias
+# relu: 'R'
+# inception: 'Inc'
+# maxpool: 'MP',kernel_size,stride,padding
+# adaptiveavgpool: 'AAP',output_size
+# flatten: 'FT'
+# dropout: 'D'
+
+# class 10
+AlexNet_cfg_table = [
+    ['C','',True,3,32,3,1,1,True],
+    ['R'],
+    ['MP',2,2,0],
+    ['C','',False,32,64,3,1,1,True],
+    ['R'],
+    ['MP',2,2,0],
+    ['C','',False,64,128,3,1,1,True],
+    ['R'],
+    ['C','',False,128,256,3,1,1,True],
+    ['R'],
+    ['C','',False,256,256,3,1,1,True],
+    ['R'],
+    ['MP',3,2,0],
+    ['FT'],
+    ['D',0.5],
+    ['FC',2304,1024,True],
+    ['R'],
+    ['D',0.5],
+    ['FC',1024,512,True],
+    ['R'],
+    ['FC',512,10,True]
+]
+AlexNet_BN_cfg_table = [
+    ['C','BR',True,3,32,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,32,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,64,128,3,1,1,True],
+    ['C','BR',False,128,256,3,1,1,True],
+    ['C','BR',False,256,256,3,1,1,True],
+    ['MP',3,2,0],
+    ['FT'],
+    ['D',0.5],
+    ['FC',2304,1024,True],
+    ['R'],
+    ['D',0.5],
+    ['FC',1024,512,True],
+    ['R'],
+    ['FC',512,10,True]
+]
+VGG_16_cfg_table = [
+    ['C','BR',True,3,64,3,1,1,True],
+    ['C','BR',False,64,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,64,128,3,1,1,True],
+    ['C','BR',False,128,128,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,128,256,3,1,1,True],
+    ['C','BR',False,256,256,3,1,1,True],
+    ['C','BR',False,256,256,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,256,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['FT'],
+    ['FC',512,4096,True],
+    ['R'],
+    ['D',0.5],
+    ['FC',4096,4096,True],
+    ['R'],
+    ['D',0.5],
+    ['FC',4096,10,True]
+]
+VGG_19_cfg_table = [
+    ['C','BR',True,3,64,3,1,1,True],
+    ['C','BR',False,64,64,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,64,128,3,1,1,True],
+    ['C','BR',False,128,128,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,128,256,3,1,1,True],
+    ['C','BR',False,256,256,3,1,1,True],
+    ['C','BR',False,256,256,3,1,1,True],
+    ['C','BR',False,256,256,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,256,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['C','BR',False,512,512,3,1,1,True],
+    ['MP',2,2,0],
+    ['FT'],
+    ['FC',512,4096,True],
+    ['R'],
+    ['D',0.5],
+    ['FC',4096,4096,True],
+    ['R'],
+    ['D',0.5],
+    ['FC',4096,10,True]
+]
+Inception_BN_cfg_table = [
+    ['C','',True,3,64,3,1,1,True],
+    ['R'],
+    ['C','',False,64,64,3,1,1,True],
+    ['R'],
+    ['Inc',0],
+    ['Inc',1],
+    ['MP',3,2,1],
+    ['Inc',2],
+    ['Inc',3],
+    ['Inc',4],
+    ['Inc',5],
+    ['Inc',6],
+    ['MP',3,2,1],
+    ['Inc',7],
+    ['Inc',8],
+    ['AAP',1],
+    ['C','',False,1024,10,1,1,0,True],
+    ['FT']
+]
+
+model_cfg_table = {
+    'AlexNet'       : AlexNet_cfg_table,
+    'AlexNet_BN'    : AlexNet_BN_cfg_table,
+    'VGG_16'        : VGG_16_cfg_table,
+    'VGG_19'        : VGG_19_cfg_table,
+    'Inception_BN'  : Inception_BN_cfg_table
+}
+
+
+inc_ch_table=[
+    [64,  64, 96,128, 16, 32, 32],#3a
+    [256,128,128,192, 32, 96, 64],#3b
+    [480,192, 96,208, 16, 48, 64],#4a
+    [512,160,112,224, 24, 64, 64],#4b
+    [512,128,128,256, 24, 64, 64],#4c
+    [512,112,144,288, 32, 64, 64],#4d
+    [528,256,160,320, 32,128,128],#4e
+    [832,256,160,320, 32,128,128],#5a
+    [832,384,192,384, 48,128,128] #5b
+]
+
+# br0,br1,br2,br3 <- br1x1,br3x3,br5x5,brM
+# 这里的第2,3个参数是channel中的索引
+# 对于cfg拓展，可以认为'C'有'BR'参数，且bias为false。这里的一个项根据量化后可融合结构指定
+inc_cfg_table = [
+    [['C',0,1,1,1,0]],
+    [['C',0,2,1,1,0],
+     ['C',2,3,3,1,1]],
+    [['C',0,4,1,1,0],
+     ['C',4,5,5,1,2]],
+    [['MP',3,1,1],
+     ['R'],
+     ['C',0,6,1,1,0]]
+]
\ No newline at end of file

ykl/ALL/extract_ratio.py

+import sys
+import os
+
+
+# 从get_param.py输出重定向文件val.txt中提取参数量和计算量
+def extract_ratio(model_name):
+    fr = open('param_flops/'+model_name+'.txt','r')
+    lines = fr.readlines()
+
+    Mac = lines[1].split('Mac,')[0].split(',')[-1]
+    if 'M' in Mac:
+        Mac = Mac.split('M')[0]
+        Mac = float(Mac)
+    elif 'G' in Mac:
+        Mac = Mac.split('G')[0]
+        Mac = float(Mac)
+        Mac *= 1024
+    
+    Param = lines[1].split('M,')[0]
+    Param = float(Param)
+    
+    layer = []
+    par_ratio = []
+    flop_ratio = []
+    weight_ratio = []
+    for line in lines:
+        if '(' in line and ')' in line:
+            layer.append(line.split(')')[0].split('(')[1])
+            r1 = line.split('%')[0].split(',')[-1]
+            r1 = float(r1)
+            par_ratio.append(r1)
+            r2 = line.split('%')[-2].split(',')[-1]
+            r2 = float(r2)
+            flop_ratio.append(r2)
+            if 'conv' in line:
+                #无论是否bias=false都计算，fold之后直接使用conv的近似计算
+                inch = line.split(',')[4]
+                # outch = line.split(',')[5]
+                klsz = line.split(',')[6].split('(')[-1]
+                inch = float(inch)
+                # outch = float(outch)
+                klsz = float(klsz)
+                wr = inch * klsz * klsz
+                wr = wr / (1+wr)
+                weight_ratio.append(wr)
+            elif 'fc' in line:
+                inch = line.split(',')[4].split('=')[-1]
+                inch = float(inch)
+                wr = inch / (1+inch)
+                weight_ratio.append(wr)
+            else:
+                weight_ratio.append(0)
+
+
+    return Mac, Param, layer, par_ratio, flop_ratio, weight_ratio
+
+
+if __name__ == "__main__":
+    Mac, Param, layer, par_ratio, flop_ratio, weight_ratio = extract_ratio('Inception_BN')
+    print(Mac)
+    print(Param)
+    print(layer)
+    print(par_ratio)
+    print(flop_ratio)
+    print(weight_ratio)
\ No newline at end of file

ykl/ALL/function.py

+from torch.autograd import Function
+
+
+class FakeQuantize(Function):
+
+    @staticmethod
+    def forward(ctx, x, qparam):
+        x = qparam.quantize_tensor(x)
+        x = qparam.dequantize_tensor(x)
+        return x
+
+    @staticmethod
+    def backward(ctx, grad_output):
+        return grad_output, None
\ No newline at end of file

ykl/ALL/get_param_flops.py

+from model import *
+
+import sys
+import torch
+from ptflops import get_model_complexity_info
+
+
+if __name__ == "__main__":
+    model_name = sys.argv[1]
+    model = Model(model_name)
+
+    full_file = 'ckpt/cifar10_'+model_name+'.pt'
+    model.load_state_dict(torch.load(full_file))
+
+    flops, params = get_model_complexity_info(model, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
+

ykl/ALL/get_param_flops.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#SBATCH -J ALL              # The job name
+#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
+#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
+
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 0-01:30:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-debug             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000|T4"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+name_list="AlexNet AlexNet_BN VGG_16 VGG_19 Inception_BN"
+for name in $name_list; do 
+	if [ -f "param_flops/$name.txt" ];then
+		echo "$name: param_flops exists"
+	elif [ ! -f "ckpt/cifar10_$name.pt" ];then
+		echo "$name: ckpt not exists"
+	else
+		python get_param_flops.py $name > param_flops/$name.txt
+	fi
+done
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL/gol.py

+# -*- coding: utf-8 -*-
+
+# 用于多个module之间共享全局变量
+def _init():  # 初始化
+    global _global_dict
+    _global_dict = {}
+ 
+def set_value(value,is_bias=False):
+    # 定义一个全局变量
+    if is_bias:
+        _global_dict[0] = value
+    else:
+        _global_dict[1] = value
+ 
+ 
+def get_value(is_bias=False): # 给bias独立于各变量外的精度
+    if is_bias:
+        return _global_dict[0]
+    else:
+        return _global_dict[1]  
+

ykl/ALL/model.py

+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+from cfg import *
+from module import *
+import module
+
+def make_layers(model,cfg_table):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            make_inc_layers(model,cfg[1])
+        elif cfg[0] == 'C':
+            name = 'conv%d'%i
+            layer = nn.Conv2d(cfg[3],cfg[4],kernel_size=cfg[5],stride=cfg[6],padding=cfg[7],bias=cfg[8])
+            model.add_module(name,layer)
+            if 'B' in cfg[1]:
+                name = 'bn%d'%i
+                layer = nn.BatchNorm2d(cfg[4])
+                model.add_module(name,layer)
+                if 'R' in cfg[1]:
+                    name = 'relu%d'%i
+                    layer = nn.ReLU(True)
+                    model.add_module(name,layer)
+        elif cfg[0] == 'R':
+            name = 'relu%d'%i
+            layer = nn.ReLU(True)
+            model.add_module(name,layer)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            layer = nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
+            model.add_module(name,layer)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            layer = nn.AdaptiveAvgPool2d(cfg[1])
+            model.add_module(name,layer)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i
+            layer = nn.Linear(cfg[1],cfg[2],bias=cfg[3])
+            model.add_module(name,layer)
+        elif cfg[0] == 'D':
+            name = 'drop%d'%i
+            layer = nn.Dropout(cfg[1])
+            model.add_module(name,layer)
+        
+
+def model_forward(model,cfg_table,x):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            x = inc_forward(model,cfg[1],x)
+        elif cfg[0] == 'C':
+            name = 'conv%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+            if 'B' in cfg[1]:
+                name = 'bn%d'%i
+                layer = getattr(model,name)
+                x = layer(x)
+                if 'R' in cfg[1]:
+                    name = 'relu%d'%i
+                    layer = getattr(model,name)
+                    x = layer(x)
+        elif cfg[0] == 'R':
+            name = 'relu%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'D':
+            name = 'drop%d'%i
+            layer = getattr(model,name)
+            x = layer(x)
+        elif cfg[0] == 'FT':
+            x = torch.flatten(x, start_dim=1)
+
+    return x
+
+def model_quantize(model,cfg_table,quant_type,num_bits,e_bits):
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            inc_quantize(model,cfg[1],quant_type,num_bits,e_bits)
+        elif cfg[0] == 'C':
+            conv_name = 'conv%d'%i
+            conv_layer = getattr(model,conv_name)
+            qname = 'q_'+conv_name
+            if 'B' in cfg[1]:
+                bn_name = 'bn%d'%i
+                bn_layer = getattr(model,bn_name)
+                if 'R' in cfg[1]:
+                    qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+                else:
+                    qlayer = QConvBN(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+            else:
+                qlayer = QConv2d(quant_type,conv_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'R':
+            name = 'relu%d'%i
+            qname = 'q_'+name
+            qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'MP':
+            name = 'pool%d'%i
+            qname = 'q_'+name
+            qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'AAP':
+            name = 'aap%d'%i
+            qname = 'q_'+name
+            qlayer = QAdaptiveAvgPool2d(quant_type,output_size=cfg[1],num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+        elif cfg[0] == 'FC':
+            name = 'fc%d'%i 
+            layer = getattr(model,name)
+            qname = 'q_'+name
+            qlayer = QLinear(quant_type,layer,num_bits=num_bits,e_bits=e_bits)
+            model.add_module(qname,qlayer)
+
+# 增加了func='fakefreeze'
+def model_utils(model,cfg_table,func,x=None):
+    last_qo = None
+    for i in range(len(cfg_table)):
+        cfg = cfg_table[i]
+        if cfg[0] == 'Inc':
+            x,last_qo = inc_utils(model,cfg[1],func,x,last_qo)
+        elif cfg[0] == 'C':
+            qname = 'q_conv%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                # cfg[2]为True表示起始层，需要量化
+                if cfg[2]:
+                    x = qlayer.qi.quantize_tensor(x)
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
+            last_qo = qlayer.qo
+        elif cfg[0] == 'R':
+            qname = 'q_relu%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'MP':
+            qname = 'q_pool%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+        elif cfg[0] == 'AAP':
+            qname = 'q_aap%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            last_qo = qlayer.qo
+        elif cfg[0] == 'FC':
+            qname = 'q_fc%d'%i
+            qlayer = getattr(model,qname)
+            if func == 'forward':
+                x = qlayer(x)
+            elif func == 'inference':
+                x = qlayer.quantize_inference(x)
+            elif func == 'freeze':
+                qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
+            last_qo = qlayer.qo
+        elif cfg[0] == 'D':
+            if func == 'forward':
+                name = 'drop%d'%i
+                layer = getattr(model,name)
+                x = layer(x)
+        elif cfg[0] == 'FT':
+            if func == 'inference' or func == 'forward':
+                x = torch.flatten(x,start_dim=1)
+    
+    if func == 'inference':
+        x = last_qo.dequantize_tensor(x)
+
+    return x
+
+def make_inc_layers(model,inc_idx):
+    inc_name = 'inc%d'%inc_idx
+    ch = inc_ch_table[inc_idx]
+    for i in range(4): # branch
+        prefix = inc_name+'_br%d_'%i
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                layer =nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
+                model.add_module(name,layer)
+            elif cfg[0] == 'R':
+                name=prefix+'relu%d'%j
+                layer=nn.ReLU(True)
+                model.add_module(name,layer)
+            elif cfg[0] == 'C':
+                name=prefix+'conv%d'%j
+                layer=nn.Conv2d(ch[cfg[1]],ch[cfg[2]],kernel_size=cfg[3],stride=cfg[4],padding=cfg[5],bias=False)
+                model.add_module(name,layer)
+                name=prefix+'bn%d'%j
+                layer=nn.BatchNorm2d(ch[cfg[2]])
+                model.add_module(name,layer)
+                name=prefix+'relu%d'%j
+                layer=nn.ReLU(True)
+                model.add_module(name,layer)
+
+def inc_forward(model,inc_idx,x=None):
+    inc_name = 'inc%d'%inc_idx
+    outs = []
+    for i in range(4):
+        prefix = inc_name+'_br%d_'%i
+        tmp = x
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+            elif cfg[0] == 'R':
+                name=prefix+'relu%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+            else:
+                name=prefix+'conv%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                name=prefix+'bn%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+                name=prefix+'relu%d'%j
+                layer = getattr(model,name)
+                tmp = layer(tmp)
+        outs.append(tmp)
+    
+    out = torch.cat(outs,1)
+    return out
+
+def inc_quantize(model,inc_idx,quant_type,num_bits,e_bits):
+    inc_name = 'inc%d'%inc_idx
+    for i in range(4):
+        prefix = inc_name+'_br%d_'%i
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                name = prefix+'pool%d'%j
+                qname = 'q_'+name
+                qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
+                model.add_module(qname,qlayer)
+            elif cfg[0] == 'R':
+                name = prefix+'relu%d'%j
+                qname = 'q_'+name
+                qlayer = QReLU(quant_type, num_bits=num_bits, e_bits=e_bits)
+                model.add_module(qname,qlayer)
+            else:
+                conv_name=prefix+'conv%d'%j
+                conv_layer=getattr(model,conv_name)
+                bn_name=prefix+'bn%d'%j
+                bn_layer=getattr(model,bn_name)
+                qname='q_'+conv_name
+                qlayer=QConvBNReLU(quant_type, conv_layer, bn_layer, qi=False, qo=True, num_bits=num_bits, e_bits=e_bits)
+                model.add_module(qname,qlayer)
+    qname = 'q_'+inc_name+'_concat'
+    qlayer = QConcat(quant_type,4,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
+    model.add_module(qname,qlayer)
+    
+def inc_utils(model,inc_idx,func,x=None,qo=None):
+    inc_name = 'inc%d'%inc_idx
+    outs=[]
+    qos=[]
+    for i in range(4): 
+        qprefix = 'q_'+inc_name+'_br%d_'%i
+        tmp = x
+        last_qo = qo
+        for j in range(len(inc_cfg_table[i])):
+            cfg = inc_cfg_table[i][j]
+            if cfg[0] == 'MP':
+                qname = qprefix+'pool%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+            elif cfg[0] == 'R':
+                qname = qprefix+'relu%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+            elif cfg[0] == 'C':
+                qname = qprefix+'conv%d'%j
+                qlayer = getattr(model,qname)
+                if func == 'forward':
+                    tmp = qlayer(tmp)
+                elif func == 'inference':
+                    tmp = qlayer.quantize_inference(tmp)
+                elif func == 'freeze':
+                    qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
+                last_qo = qlayer.qo
+        
+        outs.append(tmp)
+        qos.append(last_qo)
+
+    qname = 'q_'+inc_name+'_concat'
+    qlayer = getattr(model,qname)
+    out = None
+    if func == 'forward':
+        out = qlayer(outs)
+    elif func == 'inference':
+        out = qlayer.quantize_inference(outs)
+    elif func == 'freeze':
+        qlayer.freeze(qos)
+    last_qo = qlayer.qo
+    return out,last_qo
+
+
+class Model(nn.Module):
+    def __init__(self,model_name,num_classes=10):
+        super(Model, self).__init__()
+        self.cfg_table = model_cfg_table[model_name]
+        make_layers(self,self.cfg_table)
+    
+    def forward(self,x):
+        x = model_forward(self,self.cfg_table,x)
+        return x
+
+    def quantize(self, quant_type, num_bits=8, e_bits=3):
+        model_quantize(self,self.cfg_table,quant_type,num_bits,e_bits)
+    
+    def quantize_forward(self,x):
+        return model_utils(self,self.cfg_table,func='forward',x=x)
+
+    def freeze(self):
+        model_utils(self,self.cfg_table,func='freeze')
+    
+    def quantize_inference(self,x):
+        return model_utils(self,self.cfg_table,func='inference',x=x)
+
+    def fakefreeze(self):
+        model_utils(self,self.cfg_table,func='fakefreeze')
+
+# if __name__ == "__main__":
+#     model = Inception_BN()
+#     model.quantize('INT',8,3)
+#     print(model.named_modules)
+#     print('-------')
+#     print(model.named_parameters)
+#     print(len(model.conv0.named_parameters()))
\ No newline at end of file

ykl/ALL/module.py

+import math
+import numpy as np
+import gol
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from torch.autograd import Variable
+
+from function import FakeQuantize
+
+
+def js_div(p_output, q_output, get_softmax=True):
+    """
+    Function that measures JS divergence between target and output logits:
+    """
+    KLDivLoss = nn.KLDivLoss(reduction='sum')
+    if get_softmax:
+        p_output = F.softmax(p_output)
+        q_output = F.softmax(q_output)
+    log_mean_output = ((p_output + q_output)/2).log()
+    return (KLDivLoss(log_mean_output, p_output) + KLDivLoss(log_mean_output, q_output))/2
+
+
+# 获取最近的量化值
+def get_nearest_val(quant_type, x, is_bias=False, block_size=1000000):
+    if quant_type == 'INT':
+        return x.round_()
+
+    plist = gol.get_value(is_bias)
+    shape = x.shape
+    xhard = x.view(-1)
+    xout = torch.zeros_like(xhard)
+
+    plist = plist.type_as(x)
+    n_blocks = (x.numel() + block_size - 1) // block_size
+
+    for i in range(n_blocks):
+        start_idx = i * block_size
+        end_idx = min(start_idx + block_size, xhard.numel())
+        block_size_i = end_idx - start_idx
+        # print(x.numel())
+        # print(block_size_i)
+        # print(start_idx)
+        # print(end_idx)
+        xblock = xhard[start_idx:end_idx]
+        # xblock = xblock.view(shape[start_idx:end_idx])
+        plist_block = plist.unsqueeze(1) #.expand(-1, block_size_i)
+        idx = (xblock.unsqueeze(0) - plist_block).abs().min(dim=0)[1]
+        # print(xblock.shape)
+        xhard_block = plist[idx].view(xblock.shape)
+        xout[start_idx:end_idx] = (xhard_block - xblock).detach() + xblock
+
+    xout = xout.view(shape)
+    return xout
+
+
+# 采用对称有符号量化时，获取量化范围最大值
+def get_qmax(quant_type,num_bits=None, e_bits=None):
+    if quant_type == 'INT':
+        qmax = 2. ** (num_bits - 1) - 1
+    elif quant_type == 'POT':
+        qmax = 1
+    else: #FLOAT
+        m_bits = num_bits - 1 - e_bits
+        dist_m = 2 ** (-m_bits)
+        e = 2 ** (e_bits - 1)
+        expo = 2 ** e
+        m = 2 ** m_bits -1
+        frac = 1. + m * dist_m
+        qmax = frac * expo
+    return qmax
+
+# 都采用有符号量化，zeropoint都置为0
+def calcScaleZeroPoint(min_val, max_val, qmax):
+    scale = torch.max(max_val.abs(),min_val.abs()) / qmax
+    zero_point = torch.tensor(0.)
+    return scale, zero_point
+
+# 将输入进行量化，输入输出都为tensor
+def quantize_tensor(quant_type, x, scale, zero_point, qmax, is_bias=False):
+    # 量化后范围，直接根据位宽确定
+    qmin = -qmax
+    q_x = zero_point + x / scale
+
+    q_x.clamp_(qmin, qmax)
+    q_x = get_nearest_val(quant_type, q_x, is_bias)
+    return q_x
+
+# bias使用不同精度，需要根据量化类型指定num_bits/e_bits
+def bias_qmax(quant_type):
+    if quant_type == 'INT':
+        return get_qmax(quant_type, 64)
+    elif quant_type == 'POT':
+        return get_qmax(quant_type)
+    else:
+        return get_qmax(quant_type, 16, 7)
+        
+
+# 转化为FP32，不需再做限制
+def dequantize_tensor(q_x, scale, zero_point):
+    return scale * (q_x - zero_point)
+
+
+class QParam(nn.Module):
+
+    def __init__(self,quant_type, num_bits=8, e_bits=3):
+        super(QParam, self).__init__()
+        self.quant_type = quant_type
+        self.num_bits = num_bits
+        self.e_bits = e_bits
+        self.qmax = get_qmax(quant_type, num_bits, e_bits)
+
+        scale = torch.tensor([], requires_grad=False)
+        zero_point = torch.tensor([], requires_grad=False)
+        min = torch.tensor([], requires_grad=False)
+        max = torch.tensor([], requires_grad=False)
+        # 通过注册为register，使得buffer可以被记录到state_dict
+        self.register_buffer('scale', scale)
+        self.register_buffer('zero_point', zero_point)
+        self.register_buffer('min', min)
+        self.register_buffer('max', max)
+
+    # 更新统计范围及量化参数
+    def update(self, tensor):
+        if self.max.nelement() == 0 or self.max.data < tensor.max().data:
+            self.max.data = tensor.max().data
+        self.max.clamp_(min=0)
+
+        if self.min.nelement() == 0 or self.min.data > tensor.min().data:
+            self.min.data = tensor.min().data
+        self.min.clamp_(max=0)
+
+        self.scale, self.zero_point = calcScaleZeroPoint(self.min, self.max, self.qmax)
+
+    def quantize_tensor(self, tensor):
+        return quantize_tensor(self.quant_type, tensor, self.scale, self.zero_point, self.qmax)
+
+    def dequantize_tensor(self, q_x):
+        return dequantize_tensor(q_x, self.scale, self.zero_point)
+
+    # 该方法保证了可以从state_dict里恢复
+    def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys,
+                              error_msgs):
+        key_names = ['scale', 'zero_point', 'min', 'max']
+        for key in key_names:
+            value = getattr(self, key)
+            value.data = state_dict[prefix + key].data
+            state_dict.pop(prefix + key)
+
+    # 该方法返回值将是打印该对象的结果
+    def __str__(self):
+        info = 'scale: %.10f ' % self.scale
+        info += 'zp: %.6f ' % self.zero_point
+        info += 'min: %.6f ' % self.min
+        info += 'max: %.6f' % self.max
+        return info
+
+
+# 作为具体量化层的父类，qi和qo分别为量化输入/输出
+class QModule(nn.Module):
+
+    def __init__(self,quant_type, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QModule, self).__init__()
+        if qi:
+            self.qi = QParam(quant_type,num_bits, e_bits)
+        if qo:
+            self.qo = QParam(quant_type,num_bits, e_bits)
+        self.quant_type = quant_type
+        self.num_bits = num_bits
+        self.e_bits = e_bits
+        self.bias_qmax = bias_qmax(quant_type)
+
+    def freeze(self):
+        pass  # 空语句
+
+    def quantize_inference(self, x):
+        raise NotImplementedError('quantize_inference should be implemented.')
+
+    def fakefreeze(self):
+        pass
+
+"""
+QModule  量化卷积
+
+:quant_type: 量化类型
+:conv_module: 卷积模块
+:qi: 是否量化输入特征图
+:qo: 是否量化输出特征图
+:num_bits: 8位bit数
+"""
+
+
+class QConv2d(QModule):
+
+    def __init__(self, quant_type, conv_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConv2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    # freeze方法可以固定真量化的权重参数，并将该值更新到原全精度层上，便于散度计算
+    def freeze(self, qi=None, qo=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        # 这里因为在池化或者激活的输入，不需要对最大值和最小是进行额外的统计，会共享相同的输出
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        # 根据https://zhuanlan.zhihu.com/p/156835141, 这是式3 的系数
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(self.conv_module.weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+
+        #考虑conv层无bias,此时forward和inference传入none亦可
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                        self.conv_module.bias.data, scale=self.qi.scale * self.qw.scale,
+                                                        zero_point=0.,qmax=self.bias_qmax, is_bias=True)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):  # 前向传播,输入张量,x为浮点型数据
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)  # 对输入张量X完成量化
+
+        # foward前更新qw，保证量化weight时候scale正确
+        self.qw.update(self.conv_module.weight.data)
+        # 注意:此处主要为了统计各层x和weight范围，未对bias进行量化操作
+        tmp_wgt = FakeQuantize.apply(self.conv_module.weight, self.qw)
+        x = F.conv2d(x, tmp_wgt, self.conv_module.bias,
+                     stride=self.conv_module.stride,
+                     padding=self.conv_module.padding, dilation=self.conv_module.dilation,
+                     groups=self.conv_module.groups)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+
+    # 利用公式 q_a = M(\sigma(q_w-Z_w)(q_x-Z_x) + q_b)
+    def quantize_inference(self, x):  # 此处input为已经量化的qx
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+
+        x = get_nearest_val(self.quant_type,x)
+        x = x + self.qo.zero_point
+        return x
+
+
+class QLinear(QModule):
+
+    def __init__(self, quant_type, fc_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QLinear, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.fc_module = fc_module
+        self.qw = QParam(quant_type, num_bits, e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def freeze(self, qi=None, qo=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+        
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        self.fc_module.weight.data = self.qw.quantize_tensor(self.fc_module.weight.data)
+        self.fc_module.weight.data = self.fc_module.weight.data - self.qw.zero_point
+        if self.fc_module.bias is not None:
+            self.fc_module.bias.data = quantize_tensor(self.quant_type,
+                                                        self.fc_module.bias.data, scale=self.qi.scale * self.qw.scale,
+                                                        zero_point=0., qmax=self.bias_qmax, is_bias=True)
+
+    def fakefreeze(self):
+        self.fc_module.weight.data = self.qw.dequantize_tensor(self.fc_module.weight.data)
+        if self.fc_module.bias is not None:
+            self.fc_module.bias.data = dequantize_tensor(self.fc_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        self.qw.update(self.fc_module.weight.data)
+        tmp_wgt = FakeQuantize.apply(self.fc_module.weight, self.qw)
+        x = F.linear(x, tmp_wgt, self.fc_module.bias)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = self.fc_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x)
+        x = x + self.qo.zero_point
+
+        return x
+
+
+class QReLU(QModule):
+
+    def __init__(self,quant_type, qi=False, qo=False, num_bits=8, e_bits=3):
+        super(QReLU, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+
+    def freeze(self, qi=None):
+
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.relu(x)
+
+        return x
+
+    def quantize_inference(self, x):
+        x = x.clone()
+        x[x < self.qi.zero_point] = self.qi.zero_point
+        return x
+
+
+class QMaxPooling2d(QModule):
+
+    def __init__(self, quant_type, kernel_size=3, stride=1, padding=0, qi=False, qo=False, num_bits=8,e_bits=3):
+        super(QMaxPooling2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.kernel_size = kernel_size
+        self.stride = stride
+        self.padding = padding
+        
+    def freeze(self, qi=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+        if qi is not None:
+            self.qi = qi
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.max_pool2d(x, self.kernel_size, self.stride, self.padding)
+
+        return x
+
+    def quantize_inference(self, x):
+        return F.max_pool2d(x, self.kernel_size, self.stride, self.padding)
+
+class QAdaptiveAvgPool2d(QModule):
+
+    def __init__(self, quant_type, output_size, qi=False, qo=True, num_bits=8,e_bits=3):
+        super(QAdaptiveAvgPool2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.output_size = output_size
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+        
+        self.M.data = (self.qi.scale / self.qo.scale).data
+
+    def forward(self, x):
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        x = F.adaptive_avg_pool2d(x, self.output_size)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+        return x
+
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = F.adaptive_avg_pool2d(x, self.output_size)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x)
+        x = x+self.qo.zero_point
+        return x
+    
+
+class QConvBNReLU(QModule):
+
+    def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConvBNReLU, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.bn_module = bn_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def fold_bn(self, mean, std):
+        if self.bn_module.affine:
+            gamma_ = self.bn_module.weight / std
+            weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
+            else:
+                bias = self.bn_module.bias - gamma_ * mean
+        else:
+            gamma_ = 1 / std
+            weight = self.conv_module.weight * gamma_
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean
+            else:
+                bias = -gamma_ * mean
+            
+        return weight, bias
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(self.bn_module.running_mean, std)
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                            bias.data, scale=self.qi.scale * self.qw.scale,
+                                                            zero_point=0., qmax=self.bias_qmax,is_bias=True)
+        else:
+            bias = quantize_tensor(self.quant_type,
+                                    bias, scale=self.qi.scale * self.qw.scale,
+                                    zero_point=0., qmax=self.bias_qmax,is_bias=True)
+            self.conv_module.bias = torch.nn.Parameter(bias)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        if self.training:
+            y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias, 
+                            stride=self.conv_module.stride,
+                            padding=self.conv_module.padding,
+                            dilation=self.conv_module.dilation,
+                            groups=self.conv_module.groups)
+            y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
+            y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
+            # mean = y.mean(1)
+            # var = y.var(1)
+            mean = y.mean(1).detach()
+            var = y.var(1).detach()
+            self.bn_module.running_mean = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_mean + \
+                self.bn_module.momentum * mean
+            self.bn_module.running_var = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_var + \
+                self.bn_module.momentum * var
+        else:
+            mean = Variable(self.bn_module.running_mean)
+            var = Variable(self.bn_module.running_var)
+
+        std = torch.sqrt(var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(mean, std)
+
+        self.qw.update(weight.data)
+
+        x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
+                stride=self.conv_module.stride,
+                padding=self.conv_module.padding, dilation=self.conv_module.dilation, 
+                groups=self.conv_module.groups)
+
+        x = F.relu(x)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+    
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x) 
+        x = x + self.qo.zero_point        
+        x.clamp_(min=0)
+        return x
+
+class QConvBN(QModule):
+
+    def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
+        super(QConvBN, self).__init__(quant_type, qi, qo, num_bits, e_bits)
+        self.conv_module = conv_module
+        self.bn_module = bn_module
+        self.qw = QParam(quant_type, num_bits,e_bits)
+        self.register_buffer('M', torch.tensor([], requires_grad=False))  # 将M注册为buffer
+
+    def fold_bn(self, mean, std):
+        if self.bn_module.affine:
+            gamma_ = self.bn_module.weight / std
+            weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
+            else:
+                bias = self.bn_module.bias - gamma_ * mean
+        else:
+            gamma_ = 1 / std
+            weight = self.conv_module.weight * gamma_
+            if self.conv_module.bias is not None:
+                bias = gamma_ * self.conv_module.bias - gamma_ * mean
+            else:
+                bias = -gamma_ * mean
+            
+        return weight, bias
+
+    def freeze(self, qi=None, qo=None):
+        if hasattr(self, 'qi') and qi is not None:
+            raise ValueError('qi has been provided in init function.')
+        if not hasattr(self, 'qi') and qi is None:
+            raise ValueError('qi is not existed, should be provided.')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+
+        if qi is not None:
+            self.qi = qi
+        if qo is not None:
+            self.qo = qo
+
+        self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
+
+        std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(self.bn_module.running_mean, std)
+
+        self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
+        self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
+        
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = quantize_tensor(self.quant_type,
+                                                        bias.data, scale=self.qi.scale * self.qw.scale,
+                                                        zero_point=0., qmax=self.bias_qmax,is_bias=True)
+        else:
+            bias = quantize_tensor(self.quant_type,
+                                    bias, scale=self.qi.scale * self.qw.scale,
+                                    zero_point=0., qmax=self.bias_qmax,is_bias=True)
+            self.conv_module.bias = torch.nn.Parameter(bias)
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
+    def forward(self, x):
+
+        if hasattr(self, 'qi'):
+            self.qi.update(x)
+            x = FakeQuantize.apply(x, self.qi)
+
+        if self.training:
+            y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias, 
+                            stride=self.conv_module.stride,
+                            padding=self.conv_module.padding,
+                            dilation=self.conv_module.dilation,
+                            groups=self.conv_module.groups)
+            y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
+            y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
+            # mean = y.mean(1)
+            # var = y.var(1)
+            mean = y.mean(1).detach()
+            var = y.var(1).detach()
+            self.bn_module.running_mean = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_mean + \
+                self.bn_module.momentum * mean
+            self.bn_module.running_var = \
+                (1 - self.bn_module.momentum) * self.bn_module.running_var + \
+                self.bn_module.momentum * var
+        else:
+            mean = Variable(self.bn_module.running_mean)
+            var = Variable(self.bn_module.running_var)
+
+        std = torch.sqrt(var + self.bn_module.eps)
+
+        weight, bias = self.fold_bn(mean, std)
+
+        self.qw.update(weight.data)
+
+        x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias, 
+                stride=self.conv_module.stride,
+                padding=self.conv_module.padding, dilation=self.conv_module.dilation, 
+                groups=self.conv_module.groups)
+
+        if hasattr(self, 'qo'):
+            self.qo.update(x)
+            x = FakeQuantize.apply(x, self.qo)
+
+        return x
+    
+    def quantize_inference(self, x):
+        x = x - self.qi.zero_point
+        x = self.conv_module(x)
+        x = self.M * x
+        x = get_nearest_val(self.quant_type,x) 
+        x = x + self.qo.zero_point        
+        return x
+
+
+# 作为具体量化层的父类，qi和qo分别为量化输入/输出
+# 用于处理多个层结果或qo以array形式传入
+class QModule_array(nn.Module):
+    def __init__(self,quant_type,len,qi_array=False, qo=True, num_bits=8, e_bits=3):
+        super(QModule_array, self).__init__()
+        if qi_array:
+            for i in range(len):
+                self.add_module('qi%d'%i,QParam(quant_type,num_bits, e_bits))
+        if qo:
+            self.qo = QParam(quant_type,num_bits, e_bits)
+        self.quant_type = quant_type
+        self.num_bits = num_bits
+        self.e_bits = e_bits
+        self.bias_qmax = bias_qmax(quant_type)
+        self.len = len
+
+    def freeze(self):
+        pass  # 空语句
+
+    def quantize_inference(self, x):
+        raise NotImplementedError('quantize_inference should be implemented.')
+    
+class QConcat(QModule_array):
+    def __init__(self, quant_type, len, qi_array=False, qo=True, num_bits=8, e_bits=3):
+        super(QConcat,self).__init__(quant_type, len, qi_array, qo, num_bits, e_bits)
+        for i in range(len):
+            self.register_buffer('M%d'%i,torch.tensor([], requires_grad=False))
+    
+    def freeze(self, qi_array=None, qo=None):
+        if qi_array is None:
+            raise ValueError('qi_array should be provided')
+        elif len(qi_array) != self.len:
+            raise ValueError('qi_array len no match')
+
+        if hasattr(self, 'qo') and qo is not None:
+            raise ValueError('qo has been provided in init function.')
+        if not hasattr(self, 'qo') and qo is None:
+            raise ValueError('qo is not existed, should be provided.')
+        
+        for i in range(self.len):
+            self.add_module('qi%d'%i,qi_array[i])
+        if qo is not None:
+            self.qo = qo
+
+        for i in range(self.len):
+            getattr(self,'M%d'%i).data = (getattr(self,'qi%d'%i).scale / self.qo.scale).data
+        
+    def forward(self,x_array):
+        outs=[]
+        for i in range(self.len):
+            x = x_array[i]
+            if hasattr(self,'qi%d'%i):
+                qi = getattr(self,'qi%d'%i)
+                qi.update(x)
+                x = FakeQuantize.apply(x,qi)
+            outs.append(x)
+        out = torch.cat(outs,1)
+        if hasattr(self,'qo'):
+            self.qo.update(x)
+            out = FakeQuantize.apply(out,self.qo)
+        return out
+
+    def quantize_inference(self, x_array):
+        outs=[]
+        for i in range(self.len):
+            qi = getattr(self,'qi%d'%i)
+            x = x_array[i] - qi.zero_point
+            x = getattr(self,'M%d'%i) * x
+            outs.append(x)
+        out = torch.concat(outs,1)
+        out = get_nearest_val(self.quant_type,out)
+        out = out + self.qo.zero_point
+        return out
+
+
+        
\ No newline at end of file

ykl/ALL/param_flops/AlexNet.txt

+Model(
+  3.87 M, 100.000% Params, 70.08 MMac, 100.000% MACs, 
+  (conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.309% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
+  (pool2): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv3): Conv2d(18.5 k, 0.478% Params, 4.73 MMac, 6.756% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu4): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool5): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv6): Conv2d(73.86 k, 1.909% Params, 4.73 MMac, 6.745% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu7): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
+  (conv8): Conv2d(295.17 k, 7.630% Params, 18.89 MMac, 26.955% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (conv10): Conv2d(590.08 k, 15.252% Params, 37.77 MMac, 53.887% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu11): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool12): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (drop14): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc15): Linear(2.36 M, 61.010% Params, 2.36 MMac, 3.368% MACs, in_features=2304, out_features=1024, bias=True)
+  (relu16): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
+  (drop17): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc18): Linear(524.8 k, 13.565% Params, 524.8 KMac, 0.749% MACs, in_features=1024, out_features=512, bias=True)
+  (relu19): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
+  (fc20): Linear(5.13 k, 0.133% Params, 5.13 KMac, 0.007% MACs, in_features=512, out_features=10, bias=True)
+)

ykl/ALL/param_flops/AlexNet_BN.txt

+Model(
+  3.87 M, 100.000% Params, 70.26 MMac, 100.000% MACs, 
+  (conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.306% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(64, 0.002% Params, 65.54 KMac, 0.093% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
+  (pool1): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv2): Conv2d(18.5 k, 0.478% Params, 4.73 MMac, 6.739% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn2): BatchNorm2d(128, 0.003% Params, 32.77 KMac, 0.047% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool3): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv4): Conv2d(73.86 k, 1.908% Params, 4.73 MMac, 6.727% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn4): BatchNorm2d(256, 0.007% Params, 16.38 KMac, 0.023% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu4): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
+  (conv5): Conv2d(295.17 k, 7.627% Params, 18.89 MMac, 26.886% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn5): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu5): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (conv6): Conv2d(590.08 k, 15.247% Params, 37.77 MMac, 53.748% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn6): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
+  (pool7): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (drop9): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc10): Linear(2.36 M, 60.987% Params, 2.36 MMac, 3.359% MACs, in_features=2304, out_features=1024, bias=True)
+  (relu11): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
+  (drop12): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc13): Linear(524.8 k, 13.560% Params, 524.8 KMac, 0.747% MACs, in_features=1024, out_features=512, bias=True)
+  (relu14): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
+  (fc15): Linear(5.13 k, 0.133% Params, 5.13 KMac, 0.007% MACs, in_features=512, out_features=10, bias=True)
+)

ykl/ALL/param_flops/Inception_BN.txt

+Model(
+  5.88 M, 100.000% Params, 1.46 GMac, 100.000% MACs, 
+  (conv0): Conv2d(1.79 k, 0.030% Params, 1.84 MMac, 0.125% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (conv2): Conv2d(36.93 k, 0.628% Params, 37.81 MMac, 2.583% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (relu3): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc0_br0_conv0): Conv2d(4.1 k, 0.070% Params, 4.19 MMac, 0.287% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br0_bn0): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc0_br1_conv0): Conv2d(6.14 k, 0.105% Params, 6.29 MMac, 0.430% MACs, 64, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br1_bn0): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br1_relu0): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
+  (inc0_br1_conv1): Conv2d(110.59 k, 1.881% Params, 113.25 MMac, 7.737% MACs, 96, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc0_br1_bn1): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc0_br2_conv0): Conv2d(1.02 k, 0.017% Params, 1.05 MMac, 0.072% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br2_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.002% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br2_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc0_br2_conv1): Conv2d(12.8 k, 0.218% Params, 13.11 MMac, 0.895% MACs, 16, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc0_br2_bn1): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc0_br3_pool0): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc0_br3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc0_br3_conv2): Conv2d(2.05 k, 0.035% Params, 2.1 MMac, 0.143% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc0_br3_bn2): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc0_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc1_br0_conv0): Conv2d(32.77 k, 0.557% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br0_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br0_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc1_br1_conv0): Conv2d(32.77 k, 0.557% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br1_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br1_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc1_br1_conv1): Conv2d(221.18 k, 3.763% Params, 226.49 MMac, 15.474% MACs, 128, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc1_br1_bn1): BatchNorm2d(384, 0.007% Params, 393.22 KMac, 0.027% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br1_relu1): ReLU(0, 0.000% Params, 196.61 KMac, 0.013% MACs, inplace=True)
+  (inc1_br2_conv0): Conv2d(8.19 k, 0.139% Params, 8.39 MMac, 0.573% MACs, 256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br2_bn0): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br2_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc1_br2_conv1): Conv2d(76.8 k, 1.306% Params, 78.64 MMac, 5.373% MACs, 32, 96, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc1_br2_bn1): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br2_relu1): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
+  (inc1_br3_pool0): MaxPool2d(0, 0.000% Params, 262.14 KMac, 0.018% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc1_br3_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.018% MACs, inplace=True)
+  (inc1_br3_conv2): Conv2d(16.38 k, 0.279% Params, 16.78 MMac, 1.146% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc1_br3_bn2): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc1_br3_relu2): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (pool6): MaxPool2d(0, 0.000% Params, 491.52 KMac, 0.034% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
+  (inc2_br0_conv0): Conv2d(92.16 k, 1.568% Params, 23.59 MMac, 1.612% MACs, 480, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br0_bn0): BatchNorm2d(384, 0.007% Params, 98.3 KMac, 0.007% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br0_relu0): ReLU(0, 0.000% Params, 49.15 KMac, 0.003% MACs, inplace=True)
+  (inc2_br1_conv0): Conv2d(46.08 k, 0.784% Params, 11.8 MMac, 0.806% MACs, 480, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br1_bn0): BatchNorm2d(192, 0.003% Params, 49.15 KMac, 0.003% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br1_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
+  (inc2_br1_conv1): Conv2d(179.71 k, 3.057% Params, 46.01 MMac, 3.143% MACs, 96, 208, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc2_br1_bn1): BatchNorm2d(416, 0.007% Params, 106.5 KMac, 0.007% MACs, 208, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br1_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
+  (inc2_br2_conv0): Conv2d(7.68 k, 0.131% Params, 1.97 MMac, 0.134% MACs, 480, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br2_bn0): BatchNorm2d(32, 0.001% Params, 8.19 KMac, 0.001% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br2_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.000% MACs, inplace=True)
+  (inc2_br2_conv1): Conv2d(19.2 k, 0.327% Params, 4.92 MMac, 0.336% MACs, 16, 48, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc2_br2_bn1): BatchNorm2d(96, 0.002% Params, 24.58 KMac, 0.002% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br2_relu1): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
+  (inc2_br3_pool0): MaxPool2d(0, 0.000% Params, 122.88 KMac, 0.008% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc2_br3_relu1): ReLU(0, 0.000% Params, 122.88 KMac, 0.008% MACs, inplace=True)
+  (inc2_br3_conv2): Conv2d(30.72 k, 0.523% Params, 7.86 MMac, 0.537% MACs, 480, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc2_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc2_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc3_br0_conv0): Conv2d(81.92 k, 1.394% Params, 20.97 MMac, 1.433% MACs, 512, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br0_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br0_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
+  (inc3_br1_conv0): Conv2d(57.34 k, 0.975% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br1_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br1_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
+  (inc3_br1_conv1): Conv2d(225.79 k, 3.841% Params, 57.8 MMac, 3.949% MACs, 112, 224, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc3_br1_bn1): BatchNorm2d(448, 0.008% Params, 114.69 KMac, 0.008% MACs, 224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br1_relu1): ReLU(0, 0.000% Params, 57.34 KMac, 0.004% MACs, inplace=True)
+  (inc3_br2_conv0): Conv2d(12.29 k, 0.209% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
+  (inc3_br2_conv1): Conv2d(38.4 k, 0.653% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc3_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc3_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc3_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc3_br3_conv2): Conv2d(32.77 k, 0.557% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc3_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc3_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc4_br0_conv0): Conv2d(65.54 k, 1.115% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br0_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br0_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc4_br1_conv0): Conv2d(65.54 k, 1.115% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br1_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br1_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc4_br1_conv1): Conv2d(294.91 k, 5.017% Params, 75.5 MMac, 5.158% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc4_br1_bn1): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc4_br2_conv0): Conv2d(12.29 k, 0.209% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
+  (inc4_br2_conv1): Conv2d(38.4 k, 0.653% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc4_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc4_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc4_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc4_br3_conv2): Conv2d(32.77 k, 0.557% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc4_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc4_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc5_br0_conv0): Conv2d(57.34 k, 0.975% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br0_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br0_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
+  (inc5_br1_conv0): Conv2d(73.73 k, 1.254% Params, 18.87 MMac, 1.289% MACs, 512, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br1_bn0): BatchNorm2d(288, 0.005% Params, 73.73 KMac, 0.005% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br1_relu0): ReLU(0, 0.000% Params, 36.86 KMac, 0.003% MACs, inplace=True)
+  (inc5_br1_conv1): Conv2d(373.25 k, 6.349% Params, 95.55 MMac, 6.528% MACs, 144, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc5_br1_bn1): BatchNorm2d(576, 0.010% Params, 147.46 KMac, 0.010% MACs, 288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br1_relu1): ReLU(0, 0.000% Params, 73.73 KMac, 0.005% MACs, inplace=True)
+  (inc5_br2_conv0): Conv2d(16.38 k, 0.279% Params, 4.19 MMac, 0.287% MACs, 512, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc5_br2_conv1): Conv2d(51.2 k, 0.871% Params, 13.11 MMac, 0.895% MACs, 32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc5_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc5_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc5_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
+  (inc5_br3_conv2): Conv2d(32.77 k, 0.557% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc5_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc5_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc6_br0_conv0): Conv2d(135.17 k, 2.299% Params, 34.6 MMac, 2.364% MACs, 528, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br0_bn0): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
+  (inc6_br1_conv0): Conv2d(84.48 k, 1.437% Params, 21.63 MMac, 1.478% MACs, 528, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br1_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br1_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
+  (inc6_br1_conv1): Conv2d(460.8 k, 7.839% Params, 117.96 MMac, 8.059% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc6_br1_bn1): BatchNorm2d(640, 0.011% Params, 163.84 KMac, 0.011% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br1_relu1): ReLU(0, 0.000% Params, 81.92 KMac, 0.006% MACs, inplace=True)
+  (inc6_br2_conv0): Conv2d(16.9 k, 0.287% Params, 4.33 MMac, 0.296% MACs, 528, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc6_br2_conv1): Conv2d(102.4 k, 1.742% Params, 26.21 MMac, 1.791% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc6_br2_bn1): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (inc6_br3_pool0): MaxPool2d(0, 0.000% Params, 135.17 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc6_br3_relu1): ReLU(0, 0.000% Params, 135.17 KMac, 0.009% MACs, inplace=True)
+  (inc6_br3_conv2): Conv2d(67.58 k, 1.150% Params, 17.3 MMac, 1.182% MACs, 528, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc6_br3_bn2): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc6_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
+  (pool12): MaxPool2d(0, 0.000% Params, 212.99 KMac, 0.015% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
+  (inc7_br0_conv0): Conv2d(212.99 k, 3.623% Params, 13.63 MMac, 0.931% MACs, 832, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br0_bn0): BatchNorm2d(512, 0.009% Params, 32.77 KMac, 0.002% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br0_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
+  (inc7_br1_conv0): Conv2d(133.12 k, 2.265% Params, 8.52 MMac, 0.582% MACs, 832, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br1_bn0): BatchNorm2d(320, 0.005% Params, 20.48 KMac, 0.001% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br1_relu0): ReLU(0, 0.000% Params, 10.24 KMac, 0.001% MACs, inplace=True)
+  (inc7_br1_conv1): Conv2d(460.8 k, 7.839% Params, 29.49 MMac, 2.015% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc7_br1_bn1): BatchNorm2d(640, 0.011% Params, 40.96 KMac, 0.003% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br1_relu1): ReLU(0, 0.000% Params, 20.48 KMac, 0.001% MACs, inplace=True)
+  (inc7_br2_conv0): Conv2d(26.62 k, 0.453% Params, 1.7 MMac, 0.116% MACs, 832, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br2_bn0): BatchNorm2d(64, 0.001% Params, 4.1 KMac, 0.000% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br2_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (inc7_br2_conv1): Conv2d(102.4 k, 1.742% Params, 6.55 MMac, 0.448% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc7_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc7_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc7_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
+  (inc7_br3_conv2): Conv2d(106.5 k, 1.812% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc7_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc7_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc8_br0_conv0): Conv2d(319.49 k, 5.435% Params, 20.45 MMac, 1.397% MACs, 832, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br0_bn0): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br0_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
+  (inc8_br1_conv0): Conv2d(159.74 k, 2.717% Params, 10.22 MMac, 0.698% MACs, 832, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br1_bn0): BatchNorm2d(384, 0.007% Params, 24.58 KMac, 0.002% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br1_relu0): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
+  (inc8_br1_conv1): Conv2d(663.55 k, 11.288% Params, 42.47 MMac, 2.901% MACs, 192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
+  (inc8_br1_bn1): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br1_relu1): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
+  (inc8_br2_conv0): Conv2d(39.94 k, 0.679% Params, 2.56 MMac, 0.175% MACs, 832, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br2_bn0): BatchNorm2d(96, 0.002% Params, 6.14 KMac, 0.000% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br2_relu0): ReLU(0, 0.000% Params, 3.07 KMac, 0.000% MACs, inplace=True)
+  (inc8_br2_conv1): Conv2d(153.6 k, 2.613% Params, 9.83 MMac, 0.672% MACs, 48, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
+  (inc8_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (inc8_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
+  (inc8_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
+  (inc8_br3_conv2): Conv2d(106.5 k, 1.812% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
+  (inc8_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (inc8_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
+  (aap15): AdaptiveAvgPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, output_size=1)
+  (conv16): Conv2d(10.25 k, 0.174% Params, 10.25 KMac, 0.001% MACs, 1024, 10, kernel_size=(1, 1), stride=(1, 1))
+)

ykl/ALL/param_flops/VGG_16.txt

+Model(
+  33.65 M, 100.000% Params, 333.36 MMac, 100.000% MACs, 
+  (conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.550% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
+  (conv1): Conv2d(36.93 k, 0.110% Params, 37.81 MMac, 11.343% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
+  (pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.020% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv3): Conv2d(73.86 k, 0.220% Params, 18.91 MMac, 5.672% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
+  (conv4): Conv2d(147.58 k, 0.439% Params, 37.78 MMac, 11.334% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
+  (pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.010% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv6): Conv2d(295.17 k, 0.877% Params, 18.89 MMac, 5.667% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn6): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
+  (conv7): Conv2d(590.08 k, 1.754% Params, 37.77 MMac, 11.329% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn7): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
+  (conv8): Conv2d(590.08 k, 1.754% Params, 37.77 MMac, 11.329% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn8): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
+  (pool9): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.005% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv10): Conv2d(1.18 M, 3.508% Params, 18.88 MMac, 5.664% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn10): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu10): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv11): Conv2d(2.36 M, 7.013% Params, 37.76 MMac, 11.326% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv12): Conv2d(2.36 M, 7.013% Params, 37.76 MMac, 11.326% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (pool13): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv14): Conv2d(2.36 M, 7.013% Params, 9.44 MMac, 2.832% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn14): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu14): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (conv15): Conv2d(2.36 M, 7.013% Params, 9.44 MMac, 2.832% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn15): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu15): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (conv16): Conv2d(2.36 M, 7.013% Params, 9.44 MMac, 2.832% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
+  (pool17): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.001% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (fc19): Linear(2.1 M, 6.245% Params, 2.1 MMac, 0.630% MACs, in_features=512, out_features=4096, bias=True)
+  (relu20): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop21): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc22): Linear(16.78 M, 49.875% Params, 16.78 MMac, 5.034% MACs, in_features=4096, out_features=4096, bias=True)
+  (relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc25): Linear(40.97 k, 0.122% Params, 40.97 KMac, 0.012% MACs, in_features=4096, out_features=10, bias=True)
+)

ykl/ALL/param_flops/VGG_19.txt

+Model(
+  38.96 M, 100.000% Params, 418.4 MMac, 100.000% MACs, 
+  (conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.439% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
+  (conv1): Conv2d(36.93 k, 0.095% Params, 37.81 MMac, 9.038% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
+  (pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.016% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv3): Conv2d(73.86 k, 0.190% Params, 18.91 MMac, 4.519% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
+  (conv4): Conv2d(147.58 k, 0.379% Params, 37.78 MMac, 9.030% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
+  (pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.008% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv6): Conv2d(295.17 k, 0.758% Params, 18.89 MMac, 4.515% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn6): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (conv7): Conv2d(590.08 k, 1.515% Params, 37.77 MMac, 9.026% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn7): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (conv8): Conv2d(590.08 k, 1.515% Params, 37.77 MMac, 9.026% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn8): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (conv9): Conv2d(590.08 k, 1.515% Params, 37.77 MMac, 9.026% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn9): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
+  (pool10): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.004% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv11): Conv2d(1.18 M, 3.029% Params, 18.88 MMac, 4.513% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv12): Conv2d(2.36 M, 6.057% Params, 37.76 MMac, 9.024% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv13): Conv2d(2.36 M, 6.057% Params, 37.76 MMac, 9.024% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn13): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu13): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (conv14): Conv2d(2.36 M, 6.057% Params, 37.76 MMac, 9.024% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn14): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu14): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
+  (pool15): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (conv16): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (conv17): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn17): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu17): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (conv18): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn18): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu18): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (conv19): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
+  (bn19): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
+  (relu19): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
+  (pool20): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.000% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
+  (fc22): Linear(2.1 M, 5.393% Params, 2.1 MMac, 0.502% MACs, in_features=512, out_features=4096, bias=True)
+  (relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc25): Linear(16.78 M, 43.074% Params, 16.78 MMac, 4.011% MACs, in_features=4096, out_features=4096, bias=True)
+  (relu26): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
+  (drop27): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
+  (fc28): Linear(40.97 k, 0.105% Params, 40.97 KMac, 0.010% MACs, in_features=4096, out_features=10, bias=True)
+)

ykl/ALL/ptq.py

+from torch.serialization import load
+from model import *
+from extract_ratio import *
+from utils import *
+
+import gol
+import openpyxl
+import sys
+import argparse
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+import torch.utils.bottleneck as bn
+import os
+import os.path as osp
+from torch.utils.tensorboard import SummaryWriter
+
+
+def direct_quantize(model, test_loader,device):
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_forward(data).cpu()
+        if i % 500 == 0:
+            break
+    print('direct quantization finish')
+
+
+def full_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        # print(pred)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('\nTest set: Full Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+def quantize_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_inference(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('Test set: Quant Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+if __name__ == "__main__":
+    batch_size = 32
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(device)
+    train_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=True, download=True,
+                         transform=transforms.Compose([
+                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                             transforms.RandomHorizontalFlip(),
+                             transforms.ToTensor(),
+                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                         ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
+            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+            transforms.ToTensor(),
+            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+        ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    load_ptq = False
+    store_ptq = False
+
+    append = True
+    gol._init()
+
+    excel_path = 'ptq_result.xlsx'
+    if os.path.exists(excel_path) and append:
+        workbook = openpyxl.load_workbook(excel_path)
+    else:
+        workbook = openpyxl.Workbook()
+    if 'Sheet' in workbook.sheetnames:
+        workbook.remove(workbook['Sheet'])
+
+    txt_path = 'ptq_result.txt'
+    if os.path.exists(txt_path) and append:
+        ft = open(txt_path,'a')
+    else:
+        ft = open(txt_path,'w')
+    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
+    for model_name in model_name_list:
+        if model_name in workbook.sheetnames:
+            continue
+
+        model = Model(model_name)
+        full_file = 'ckpt/cifar10_'+model_name+'.pt'
+        model.load_state_dict(torch.load(full_file))
+        model.to(device)
+
+        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
+
+        model.eval()
+        full_acc = full_inference(model, test_loader, device)
+        # 传入后可变
+        fold_model(model)
+    
+        layer, par_ratio, flop_ratio = extract_ratio(model_name)
+        par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
+
+        full_names = []
+        full_params = []
+
+        for name, param in model.named_parameters():
+            if 'conv' in name or 'fc' in name:
+                full_names.append(name)
+                full_params.append(param.data.cpu())
+    
+        #统计每个参数对应层的参数个数
+        full_par_num=[]
+        for name in full_names:
+            prefix = name.rsplit('.',1)[0]
+            cnt = 0
+            for str in full_names:
+                sprefix = str.rsplit('.',1)[0]
+                if prefix == sprefix:
+                    cnt += 1
+            full_par_num.append(cnt)
+    
+        quant_type_list = ['INT','POT','FLOAT']
+        # quant_type_list = ['INT']
+        # quant_type_list = ['POT']
+        # quant_type_list = ['INT','POT']
+        title_list = []
+        js_flops_list = []
+        js_param_list = []
+        ptq_acc_list = []
+        acc_loss_list = []
+
+        for quant_type in quant_type_list:
+            num_bit_list = numbit_list(quant_type)
+            # 对一个量化类别，只需设置一次bias量化表
+            # int由于位宽大，使用量化表开销过大，直接_round即可
+            if quant_type != 'INT':
+                bias_list = build_bias_list(quant_type)
+                gol.set_value(bias_list, is_bias=True)
+
+            for num_bits in num_bit_list:
+                e_bit_list = ebit_list(quant_type,num_bits)
+                for e_bits in e_bit_list:
+                    model_ptq = Model(model_name)
+                    if quant_type == 'FLOAT':
+                        title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
+                    else:
+                        title = '%s_%d' % (quant_type, num_bits)
+                    print('\n'+model_name+': PTQ: '+title)
+                    title_list.append(title)
+
+                    # 设置量化表
+                    if quant_type != 'INT':
+                        plist = build_list(quant_type, num_bits, e_bits)
+                        gol.set_value(plist)
+                    # 判断是否需要载入
+                    if load_ptq is True and osp.exists(ptq_file_prefix + title + '.pt'):
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.load_state_dict(torch.load(ptq_file_prefix + title + '.pt'))
+                        model_ptq.to(device)
+                        print('Successfully load ptq model: ' + title)
+                    else:
+                        model_ptq.load_state_dict(torch.load(full_file))
+                        model_ptq.to(device)
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.eval()
+                        direct_quantize(model_ptq, train_loader, device)
+                        if store_ptq:
+                            torch.save(model_ptq.state_dict(), ptq_file_prefix + title + '.pt')
+
+                    model_ptq.freeze()
+                    ptq_acc = quantize_inference(model_ptq, test_loader, device)
+                    ptq_acc_list.append(ptq_acc)
+                    acc_loss = (full_acc - ptq_acc) / full_acc
+                    acc_loss_list.append(acc_loss)
+
+                    #将量化后分布反量化到全精度相同的scale
+                    model_ptq.fakefreeze()
+                    # 获取计算量/参数量下的js-div
+                    js_flops = 0.
+                    js_param = 0.
+                    for name, param in model_ptq.named_parameters():
+                        if 'conv' not in name and 'fc' not in name:
+                            continue
+                            
+                        prefix = name.rsplit('.',1)[0]
+                        layer_idx = layer.index(prefix)
+                        name_idx = full_names.index(name)
+
+                        layer_idx = layer.index(prefix)
+                        ptq_param = param.data.cpu()
+
+                        js = js_div(ptq_param,full_params[name_idx])
+                        js /= full_par_num[name_idx]
+
+                        js = js.item()
+                        if js < 0.:
+                            js = 0.
+                        js_flops = js_flops + js * flop_ratio[layer_idx]
+                        js_param = js_param + js * par_ratio[layer_idx]
+                    js_flops_list.append(js_flops)
+                    js_param_list.append(js_param)
+
+                    print(title + ': js_flops: %f js_param: %f acc_loss: %f' % (js_flops, js_param, acc_loss))
+        
+        # 写入xlsx
+        worksheet = workbook.create_sheet(model_name)
+        worksheet.cell(row=1,column=1,value='FP32-acc')
+        worksheet.cell(row=1,column=2,value=full_acc)
+        worksheet.cell(row=3,column=1,value='title')
+        worksheet.cell(row=3,column=2,value='js_flops')
+        worksheet.cell(row=3,column=3,value='js_param')
+        worksheet.cell(row=3,column=4,value='ptq_acc')
+        worksheet.cell(row=3,column=5,value='acc_loss')
+        for i in range(len(title_list)):
+            worksheet.cell(row=i+4, column=1, value=title_list[i])
+            worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
+            worksheet.cell(row=i+4, column=3, value=js_param_list[i])
+            worksheet.cell(row=i+4, column=4, value=ptq_acc_list[i])
+            worksheet.cell(row=i+4, column=5, value=acc_loss_list[i])
+
+        if 'Sheet' in workbook.sheetnames:
+            workbook.remove(workbook['Sheet'])
+        workbook.save(excel_path)
+        
+        print(model_name,file=ft)
+        print('Full_acc: %f'%full_acc,file=ft)
+        print('title_list:',file=ft)
+        print(title_list,file=ft)
+        print('js_flops_list:',file=ft)
+        print(js_flops_list, file=ft)
+        print('js_param_list:',file=ft)
+        print(js_param_list, file=ft)
+        print('ptq_acc_list:',file=ft)
+        print(ptq_acc_list, file=ft)
+        print('acc_loss_list:',file=ft)
+        print(acc_loss_list, file=ft)
+        print("\n",file=ft)
+
+    ft.close()
+

ykl/ALL/ptq.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#SBATCH -J ALL            # The job name
+#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
+#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
+
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 3-00:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-long             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000|T4"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+python ptq.py
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL/ptq_L2.py

+from torch.serialization import load
+from model import *
+from extract_ratio import *
+from utils import *
+
+import gol
+import openpyxl
+import sys
+import argparse
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+import torch.utils.bottleneck as bn
+import os
+import os.path as osp
+from torch.utils.tensorboard import SummaryWriter
+
+
+def direct_quantize(model, test_loader,device):
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_forward(data).cpu()
+        if i % 500 == 0:
+            break
+    print('direct quantization finish')
+
+
+def full_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        # print(pred)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('\nTest set: Full Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+def quantize_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_inference(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('Test set: Quant Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+if __name__ == "__main__":
+    batch_size = 32
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(device)
+    train_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=True, download=True,
+                         transform=transforms.Compose([
+                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                             transforms.RandomHorizontalFlip(),
+                             transforms.ToTensor(),
+                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                         ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
+            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+            transforms.ToTensor(),
+            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+        ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    load_ptq = False
+    store_ptq = False
+
+    append = True
+    gol._init()
+
+    excel_path = 'ptq_result_L2.xlsx'
+    if os.path.exists(excel_path) and append:
+        workbook = openpyxl.load_workbook(excel_path)
+    else:
+        workbook = openpyxl.Workbook()
+    if 'Sheet' in workbook.sheetnames:
+        workbook.remove(workbook['Sheet'])
+
+    txt_path = 'ptq_result_L2.txt'
+    if os.path.exists(txt_path) and append:
+        ft = open(txt_path,'a')
+    else:
+        ft = open(txt_path,'w')
+        
+    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
+    for model_name in model_name_list:
+        if model_name in workbook.sheetnames:
+            continue
+
+        model = Model(model_name)
+        full_file = 'ckpt/cifar10_'+model_name+'.pt'
+        model.load_state_dict(torch.load(full_file))
+        model.to(device)
+
+        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
+
+        model.eval()
+        full_acc = full_inference(model, test_loader, device)
+        # 传入后可变
+        fold_model(model)
+    
+        layer, par_ratio, flop_ratio = extract_ratio(model_name)
+        par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
+
+        full_names = []
+        full_params = []
+        full_params_norm = []
+        for name, param in model.named_parameters():
+            if 'conv' in name or 'fc' in name:
+                full_names.append(name)
+                full_params.append(param.data.cpu())
+                param_norm = F.normalize(param.data.cpu(),p=2,dim=-1)
+                full_params_norm.append(param_norm)
+    
+        #统计每个参数对应层的参数个数
+        full_par_num=[]
+        for name in full_names:
+            prefix = name.rsplit('.',1)[0]
+            cnt = 0
+            for str in full_names:
+                sprefix = str.rsplit('.',1)[0]
+                if prefix == sprefix:
+                    cnt += 1
+            full_par_num.append(cnt)
+    
+        quant_type_list = ['INT','POT','FLOAT']
+        # quant_type_list = ['INT']
+        # quant_type_list = ['POT']
+        # quant_type_list = ['INT','POT']
+        title_list = []
+        js_flops_list = []
+        js_param_list = []
+        ptq_acc_list = []
+        acc_loss_list = []
+
+        for quant_type in quant_type_list:
+            num_bit_list = numbit_list(quant_type)
+            # 对一个量化类别，只需设置一次bias量化表
+            # int由于位宽大，使用量化表开销过大，直接_round即可
+            if quant_type != 'INT':
+                bias_list = build_bias_list(quant_type)
+                gol.set_value(bias_list, is_bias=True)
+
+            for num_bits in num_bit_list:
+                e_bit_list = ebit_list(quant_type,num_bits)
+                for e_bits in e_bit_list:
+                    model_ptq = Model(model_name)
+                    if quant_type == 'FLOAT':
+                        title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
+                    else:
+                        title = '%s_%d' % (quant_type, num_bits)
+                    print('\n'+model_name+': PTQ: '+title)
+                    title_list.append(title)
+
+                    # 设置量化表
+                    if quant_type != 'INT':
+                        plist = build_list(quant_type, num_bits, e_bits)
+                        gol.set_value(plist)
+                    # 判断是否需要载入
+                    if load_ptq is True and osp.exists(ptq_file_prefix + title + '.pt'):
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.load_state_dict(torch.load(ptq_file_prefix + title + '.pt'))
+                        model_ptq.to(device)
+                        print('Successfully load ptq model: ' + title)
+                    else:
+                        model_ptq.load_state_dict(torch.load(full_file))
+                        model_ptq.to(device)
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.eval()
+                        direct_quantize(model_ptq, train_loader, device)
+                        if store_ptq:
+                            torch.save(model_ptq.state_dict(), ptq_file_prefix + title + '.pt')
+
+                    model_ptq.freeze()
+                    ptq_acc = quantize_inference(model_ptq, test_loader, device)
+                    ptq_acc_list.append(ptq_acc)
+                    acc_loss = (full_acc - ptq_acc) / full_acc
+                    acc_loss_list.append(acc_loss)
+
+                    # 获取计算量/参数量下的js-div
+                    js_flops = 0.
+                    js_param = 0.
+                    for name, param in model_ptq.named_parameters():
+                        if 'conv' not in name and 'fc' not in name:
+                            continue
+                            
+                        prefix = name.rsplit('.',1)[0]
+                        layer_idx = layer.index(prefix)
+                        name_idx = full_names.index(name)
+
+                        layer_idx = layer.index(prefix)
+                        ptq_param = param.data.cpu()
+                        ptq_param_norm = F.normalize(ptq_param,p=2,dim=-1)
+                        js = js_div(ptq_param_norm,full_params_norm[name_idx])
+                        js /= full_par_num[name_idx]
+
+                        js = js.item()
+                        if js < 0.:
+                            js = 0.
+                        js_flops = js_flops + js * flop_ratio[layer_idx]
+                        js_param = js_param + js * par_ratio[layer_idx]
+                    js_flops_list.append(js_flops)
+                    js_param_list.append(js_param)
+
+                    print(title + ': js_flops: %f js_param: %f acc_loss: %f' % (js_flops, js_param, acc_loss))
+        
+        # 写入xlsx
+        worksheet = workbook.create_sheet(model_name)
+        worksheet.cell(row=1,column=1,value='FP32-acc')
+        worksheet.cell(row=1,column=2,value=full_acc)
+        worksheet.cell(row=3,column=1,value='title')
+        worksheet.cell(row=3,column=2,value='js_flops')
+        worksheet.cell(row=3,column=3,value='js_param')
+        worksheet.cell(row=3,column=4,value='ptq_acc')
+        worksheet.cell(row=3,column=5,value='acc_loss')
+        for i in range(len(title_list)):
+            worksheet.cell(row=i+4, column=1, value=title_list[i])
+            worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
+            worksheet.cell(row=i+4, column=3, value=js_param_list[i])
+            worksheet.cell(row=i+4, column=4, value=ptq_acc_list[i])
+            worksheet.cell(row=i+4, column=5, value=acc_loss_list[i])
+
+        if 'Sheet' in workbook.sheetnames:
+            workbook.remove(workbook['Sheet'])
+        workbook.save(excel_path)
+        
+        print(model_name,file=ft)
+        print('Full_acc: %f'%full_acc,file=ft)
+        print('title_list:',file=ft)
+        print(title_list,file=ft)
+        print('js_flops_list:',file=ft)
+        print(js_flops_list, file=ft)
+        print('js_param_list:',file=ft)
+        print(js_param_list, file=ft)
+        print('ptq_acc_list:',file=ft)
+        print(ptq_acc_list, file=ft)
+        print('acc_loss_list:',file=ft)
+        print(acc_loss_list, file=ft)
+        print("\n",file=ft)
+
+    ft.close()
+

ykl/ALL/ptq_L2.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#SBATCH -J ALL-L2            # The job name
+#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
+#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
+
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 3-00:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-long             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000|T4"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+python ptq_L2.py
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL/ptq_nodiv.py

+from torch.serialization import load
+from model import *
+from extract_ratio import *
+from utils import *
+
+import gol
+import openpyxl
+import sys
+import argparse
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+import torch.utils.bottleneck as bn
+import os
+import os.path as osp
+from torch.utils.tensorboard import SummaryWriter
+
+
+def direct_quantize(model, test_loader,device):
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_forward(data).cpu()
+        if i % 500 == 0:
+            break
+    print('direct quantization finish')
+
+
+def full_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        # print(pred)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('\nTest set: Full Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+def quantize_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_inference(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('Test set: Quant Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+if __name__ == "__main__":
+    batch_size = 32
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(device)
+    train_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=True, download=True,
+                         transform=transforms.Compose([
+                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                             transforms.RandomHorizontalFlip(),
+                             transforms.ToTensor(),
+                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                         ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
+            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+            transforms.ToTensor(),
+            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+        ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    load_ptq = False
+    store_ptq = False
+
+    append = True
+    gol._init()
+
+    excel_path = 'ptq_result_nodiv.xlsx'
+    if os.path.exists(excel_path) and append:
+        workbook = openpyxl.load_workbook(excel_path)
+    else:
+        workbook = openpyxl.Workbook()
+    if 'Sheet' in workbook.sheetnames:
+        workbook.remove(workbook['Sheet'])
+
+    txt_path = 'ptq_result_nodiv.txt'
+    if os.path.exists(txt_path) and append:
+        ft = open(txt_path,'a')
+    else:
+        ft = open(txt_path,'w')
+    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
+    for model_name in model_name_list:
+        if model_name in workbook.sheetnames:
+            continue
+
+        model = Model(model_name)
+        full_file = 'ckpt/cifar10_'+model_name+'.pt'
+        model.load_state_dict(torch.load(full_file))
+        model.to(device)
+
+        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
+
+        model.eval()
+        full_acc = full_inference(model, test_loader, device)
+        # 传入后可变
+        fold_model(model)
+    
+        layer, par_ratio, flop_ratio = extract_ratio(model_name)
+        par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
+
+        full_names = []
+        full_params = []
+
+        for name, param in model.named_parameters():
+            if 'conv' in name or 'fc' in name:
+                full_names.append(name)
+                full_params.append(param.data.cpu())
+    
+        #统计每个参数对应层的参数个数
+        full_par_num=[]
+        for name in full_names:
+            prefix = name.rsplit('.',1)[0]
+            cnt = 0
+            for str in full_names:
+                sprefix = str.rsplit('.',1)[0]
+                if prefix == sprefix:
+                    cnt += 1
+            full_par_num.append(cnt)
+    
+        quant_type_list = ['INT','POT','FLOAT']
+        # quant_type_list = ['INT']
+        # quant_type_list = ['POT']
+        # quant_type_list = ['INT','POT']
+        title_list = []
+        js_flops_list = []
+        js_param_list = []
+        ptq_acc_list = []
+        acc_loss_list = []
+
+        for quant_type in quant_type_list:
+            num_bit_list = numbit_list(quant_type)
+            # 对一个量化类别，只需设置一次bias量化表
+            # int由于位宽大，使用量化表开销过大，直接_round即可
+            if quant_type != 'INT':
+                bias_list = build_bias_list(quant_type)
+                gol.set_value(bias_list, is_bias=True)
+
+            for num_bits in num_bit_list:
+                e_bit_list = ebit_list(quant_type,num_bits)
+                for e_bits in e_bit_list:
+                    model_ptq = Model(model_name)
+                    if quant_type == 'FLOAT':
+                        title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
+                    else:
+                        title = '%s_%d' % (quant_type, num_bits)
+                    print('\n'+model_name+': PTQ: '+title)
+                    title_list.append(title)
+
+                    # 设置量化表
+                    if quant_type != 'INT':
+                        plist = build_list(quant_type, num_bits, e_bits)
+                        gol.set_value(plist)
+                    # 判断是否需要载入
+                    if load_ptq is True and osp.exists(ptq_file_prefix + title + '.pt'):
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.load_state_dict(torch.load(ptq_file_prefix + title + '.pt'))
+                        model_ptq.to(device)
+                        print('Successfully load ptq model: ' + title)
+                    else:
+                        model_ptq.load_state_dict(torch.load(full_file))
+                        model_ptq.to(device)
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.eval()
+                        direct_quantize(model_ptq, train_loader, device)
+                        if store_ptq:
+                            torch.save(model_ptq.state_dict(), ptq_file_prefix + title + '.pt')
+
+                    model_ptq.freeze()
+                    ptq_acc = quantize_inference(model_ptq, test_loader, device)
+                    ptq_acc_list.append(ptq_acc)
+                    acc_loss = (full_acc - ptq_acc) / full_acc
+                    acc_loss_list.append(acc_loss)
+
+                    #将量化后分布反量化到全精度相同的scale
+                    model_ptq.fakefreeze()
+                    # 获取计算量/参数量下的js-div
+                    js_flops = 0.
+                    js_param = 0.
+                    for name, param in model_ptq.named_parameters():
+                        if 'conv' not in name and 'fc' not in name:
+                            continue
+                            
+                        prefix = name.rsplit('.',1)[0]
+                        layer_idx = layer.index(prefix)
+                        name_idx = full_names.index(name)
+
+                        layer_idx = layer.index(prefix)
+                        ptq_param = param.data.cpu()
+
+                        js = js_div(ptq_param,full_params[name_idx])
+
+                        js = js.item()
+                        if js < 0.:
+                            js = 0.
+                        js_flops = js_flops + js * flop_ratio[layer_idx]
+                        js_param = js_param + js * par_ratio[layer_idx]
+                    js_flops_list.append(js_flops)
+                    js_param_list.append(js_param)
+
+                    print(title + ': js_flops: %f js_param: %f acc_loss: %f' % (js_flops, js_param, acc_loss))
+        
+        # 写入xlsx
+        worksheet = workbook.create_sheet(model_name)
+        worksheet.cell(row=1,column=1,value='FP32-acc')
+        worksheet.cell(row=1,column=2,value=full_acc)
+        worksheet.cell(row=3,column=1,value='title')
+        worksheet.cell(row=3,column=2,value='js_flops')
+        worksheet.cell(row=3,column=3,value='js_param')
+        worksheet.cell(row=3,column=4,value='ptq_acc')
+        worksheet.cell(row=3,column=5,value='acc_loss')
+        for i in range(len(title_list)):
+            worksheet.cell(row=i+4, column=1, value=title_list[i])
+            worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
+            worksheet.cell(row=i+4, column=3, value=js_param_list[i])
+            worksheet.cell(row=i+4, column=4, value=ptq_acc_list[i])
+            worksheet.cell(row=i+4, column=5, value=acc_loss_list[i])
+
+        if 'Sheet' in workbook.sheetnames:
+            workbook.remove(workbook['Sheet'])
+        workbook.save(excel_path)
+        
+        print(model_name,file=ft)
+        print('Full_acc: %f'%full_acc,file=ft)
+        print('title_list:',file=ft)
+        print(title_list,file=ft)
+        print('js_flops_list:',file=ft)
+        print(js_flops_list, file=ft)
+        print('js_param_list:',file=ft)
+        print(js_param_list, file=ft)
+        print('ptq_acc_list:',file=ft)
+        print(ptq_acc_list, file=ft)
+        print('acc_loss_list:',file=ft)
+        print(acc_loss_list, file=ft)
+        print("\n",file=ft)
+
+    ft.close()
+

ykl/ALL/ptq_nodiv.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#SBATCH -J ALL-nodiv            # The job name
+#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
+#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
+
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 3-00:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-long             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000|T4"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+python ptq_nodiv.py
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL/ptq_result.txt

+AlexNet
+Full_acc: 86.060000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[10.147603472887434, 3.4735777398315735, 0.657128777104439, 0.1402944938181503, 0.03300085677953831, 0.007995885722661755, 0.0019068787727882519, 0.0004704087742350774, 0.00011290004340042615, 3.4677666427763485e-05, 4.835759235449544e-06, 2.3190564678341904e-05, 3.196400098729291e-06, 3.3872935928732365e-06, 1.3684112457913457e-06, 10.090646800237572, 2.005292868116476, 0.43355013904831324, 0.43289340966483714, 0.43298123987911274, 0.43326893616506545, 0.4331247501839425, 1.987108513502595, 0.6841263244261785, 0.35485575695990856, 0.3752167238172064, 0.121914769596053, 0.11863332237249592, 0.2772744841621785, 0.06223004378240952, 0.031475393945171834, 0.11856597328382808, 0.24017891433004604, 0.043757322622821594, 0.007918827552432828, 0.03150159800820788, 0.11856755637997582, 0.2234448545845582, 0.036928952491247675, 0.0019824746774589172, 0.007832565623933951, 0.03150104707959592, 0.11856715338828598]
+js_param_list:
+[3.2355718958631003, 0.9536900908692879, 0.17661893352285823, 0.03780415278800973, 0.008858951155161035, 0.00214154849155571, 0.000509927082053201, 0.00012164070394154658, 3.33694225388701e-05, 1.2162544774288353e-05, 7.1603672381082e-07, 1.1702497226528164e-05, 5.0466627657161924e-06, 9.523126802622739e-07, 2.9889347233081897e-06, 3.2234406567916567, 0.5354475822504783, 0.1312188550424844, 0.13103709245073697, 0.13105757329236303, 0.1311427863025942, 0.13111267005364427, 0.6156447703609608, 0.22233144522128817, 0.09576509216086894, 0.1266601765790008, 0.03225347850497606, 0.036223997687183665, 0.09544844878311845, 0.016100888160090848, 0.009473680968640114, 0.03622018735765729, 0.08326961787407997, 0.01109786654496731, 0.0023947080163857853, 0.009485698022477014, 0.036221162514389005, 0.0778945488972154, 0.009306042998032716, 0.0005849357445567093, 0.0023701389889276645, 0.00948535279272403, 0.03622079684057701]
+ptq_acc_list:
+[10.0, 10.21, 52.06, 77.31, 83.98, 85.6, 85.9, 86.09, 86.04, 86.09, 86.06, 86.07, 86.06, 86.06, 86.08, 10.0, 13.18, 74.37, 74.43, 74.18, 74.4, 74.31, 21.82, 68.32, 51.82, 79.4, 78.8, 82.72, 81.08, 83.47, 85.4, 82.88, 82.16, 84.38, 86.07, 85.5, 82.82, 82.46, 84.98, 86.08, 86.0, 85.34, 82.66]
+acc_loss_list:
+[0.883801998605624, 0.881361840576342, 0.3950732047408784, 0.10167325122007902, 0.02416918429003019, 0.005345108064141389, 0.0018591680223099766, -0.0003485940041831413, 0.0002323960027887058, -0.0003485940041831413, 0.0, -0.00011619800139427033, 0.0, 0.0, -0.0002323960027887058, 0.883801998605624, 0.8468510341622123, 0.13583546363002552, 0.13513827562165925, 0.13804322565651866, 0.1354868696258424, 0.13653265163839182, 0.7464559609574716, 0.20613525447362316, 0.3978619567743435, 0.07738786892865439, 0.08435974901231705, 0.038810132465721625, 0.057866604694399304, 0.03009528236114343, 0.007669068092028777, 0.03695096444341165, 0.04531722054380671, 0.01952126423425525, -0.00011619800139427033, 0.006507088078085083, 0.03764815245177793, 0.04183128050197547, 0.012549384150592589, -0.0002323960027887058, 0.0006971880083662826, 0.00836625610039506, 0.03950732047408791]
+
+
+AlexNet_BN
+Full_acc: 87.310000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[14.261531958147645, 2.5216079781917964, 0.46116271187287083, 0.10116751776414573, 0.02366857999726347, 0.005727456404133103, 0.0013995787291137347, 0.00032755864702621547, 9.083468712851506e-05, 3.9279267814657185e-05, 1.9521001999016182e-05, 1.4907377340701574e-05, 1.472309998261742e-06, 2.927860141339056e-06, 2.140239534597049e-05, 11.00797021621244, 1.7347626455421232, 0.684466797606898, 0.683273225078019, 0.7042803192576432, 0.7042800060567285, 0.7062439671835439, 2.1415266825120525, 0.8162350530415978, 0.27655595945712447, 0.48138285659911956, 0.08826544681744933, 0.13263285956507462, 0.36659157678143833, 0.04149385722275749, 0.0331260620201096, 0.1325675098211588, 0.3212629515020388, 0.027438579008588024, 0.008646288753879492, 0.033061484833924536, 0.13256908968462155, 0.30083965669229856, 0.02259296380787109, 0.002152848971541399, 0.008646190645337604, 0.03306166334274963, 0.28528407163580505]
+js_param_list:
+[4.525975088124574, 0.7756408789073711, 0.13962123018543932, 0.030527044743903838, 0.007106442539618428, 0.0017230344973903195, 0.00042786997111233664, 0.00010198592832773555, 2.910676937132983e-05, 9.735338251863014e-06, 6.254242391436015e-06, 5.110885341159888e-06, 5.134026811320913e-07, 5.412170504028867e-07, 8.626456577724872e-06, 3.615088068764246, 0.5070590858920699, 0.20256617721543332, 0.20223268982982928, 0.20818900100468749, 0.20818330171155638, 0.20904725377374708, 0.6815435977802837, 0.26233617206078796, 0.08174246558663437, 0.15525741248561856, 0.026175585917729102, 0.04077145732814419, 0.11869481379609015, 0.012033564115370663, 0.010570620047083423, 0.04074582656946896, 0.10400372156451279, 0.007843797978007105, 0.002655443829455315, 0.010547276042144588, 0.04074616626033381, 0.09741770355331004, 0.0064264679913963955, 0.0006802731478472274, 0.0026576990691152174, 0.010548301767420753, 0.08400151890829698]
+ptq_acc_list:
+[10.0, 15.54, 38.44, 76.15, 85.69, 86.96, 87.23, 87.34, 87.25, 87.34, 87.31, 87.3, 87.31, 87.3, 87.32, 10.0, 19.54, 56.21, 54.07, 52.04, 51.95, 54.64, 20.28, 62.25, 70.81, 72.49, 82.61, 83.32, 76.46, 84.89, 86.82, 83.55, 78.17, 85.69, 86.95, 86.57, 83.64, 78.4, 85.96, 87.19, 87.0, 86.62, 79.87]
+acc_loss_list:
+[0.8854655824075135, 0.822013515061276, 0.5597296987744818, 0.12782041003321493, 0.01855457564998287, 0.004008704615737126, 0.0009162753407398728, -0.0003436032527774726, 0.0006872065055549452, -0.0003436032527774726, 0.0, 0.00011453441759254514, 0.0, 0.00011453441759254514, -0.00011453441759238237, 0.8854655824075135, 0.7761997480242814, 0.3562020387126332, 0.3807124040774253, 0.4039628908487001, 0.4049937006070324, 0.37418394227465357, 0.7677242011224373, 0.2870232504867713, 0.1889817890276028, 0.16974000687206514, 0.05383117626846871, 0.04569923261940223, 0.124269843087848, 0.02771732905738176, 0.005612186462031945, 0.043064941014774996, 0.1046844576795327, 0.01855457564998287, 0.004123239033329509, 0.008475546901844109, 0.04203413125644258, 0.10205016607490547, 0.01546214637498578, 0.0013744130111098905, 0.0035505669453671086, 0.007902874813881544, 0.08521360668880996]
+
+
+VGG_16
+Full_acc: 90.060000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[7.15133747064537, 1.4625925835782612, 0.27528038309802805, 0.05999375723854592, 0.013721293740212743, 0.0033732424131763853, 0.0008308842262242227, 0.00022161440470198947, 6.398155155004749e-05, 2.3345051897377455e-05, 1.3154840854179507e-05, 1.1042119061156764e-05, 8.383129310645444e-06, 1.0039571268441182e-05, 1.0307767134309586e-05, 7.473873573128817, 1.2138168958012272, 0.8954533307613973, 0.8847200729870529, 0.889480495729074, 0.8828377520433295, 0.9202384247243933, 1.2985884732391204, 0.5001098400915878, 0.15797619371473212, 0.2963979378403137, 0.05224890302044636, 0.07668234531548482, 0.2280040599808785, 0.02401062359604003, 0.020395652708399853, 0.07667591820866455, 0.1997822583827158, 0.015853223921508276, 0.0050662161921924565, 0.020394644590516577, 0.07667526110132153, 0.18710542968170488, 0.013228032043279565, 0.0012830579038248617, 0.005075002328472146, 0.020394361774078447, 0.5904796029343199]
+js_param_list:
+[3.823939875624223, 0.5685185211098528, 0.1061155145856618, 0.023117101046099858, 0.005345663248595699, 0.0013503269855487474, 0.00032222047705326686, 8.086141579548302e-05, 3.209479985588981e-05, 9.997605488204276e-06, 8.46697402182539e-06, 3.035080993068207e-06, 4.645341375592637e-06, 5.930448596222854e-06, 5.909594486887659e-06, 4.305893745952836, 0.6018669058635838, 0.5400431259542849, 0.5416126547356545, 0.5406803652791912, 0.5390025287208834, 0.5470329327782539, 0.7519722318836962, 0.3038645326029812, 0.07427020366235762, 0.18383715082417446, 0.02218821705174367, 0.04942817720813972, 0.14148808249856845, 0.00878762265164005, 0.012693733649980042, 0.049462073173177655, 0.12408389114459065, 0.005096161222710954, 0.003090655671655611, 0.012692062625119162, 0.049462032314656366, 0.11622787586640881, 0.003961143226002479, 0.0007856883944763987, 0.003065281986746431, 0.012690010835378107, 0.3441573944792919]
+ptq_acc_list:
+[10.0, 11.51, 76.07, 88.33, 89.57, 89.74, 89.95, 89.97, 90.06, 90.06, 90.07, 90.05, 90.08, 90.08, 90.09, 10.0, 23.06, 77.66, 78.29, 74.35, 77.1, 76.48, 10.56, 56.9, 82.52, 79.68, 87.58, 86.88, 84.19, 89.07, 89.05, 86.94, 85.38, 89.28, 89.74, 89.44, 87.22, 85.69, 89.49, 89.87, 89.77, 89.54, 82.56]
+acc_loss_list:
+[0.8889629136131468, 0.872196313568732, 0.15534088385520772, 0.01920941594492565, 0.005440817232955908, 0.0035531867643793847, 0.001221407950255379, 0.0009993337774817166, 0.0, 0.0, -0.00011103708638675222, 0.00011103708638691001, -0.00022207417277366224, -0.00022207417277366224, -0.00033311125916057226, 0.8889629136131468, 0.7439484787919165, 0.13768598711969804, 0.1306906506773262, 0.17443926271374646, 0.14390406395736186, 0.15078836331334664, 0.882744836775483, 0.3681989784588053, 0.08372196313568739, 0.11525649566955358, 0.02753719742393964, 0.035309793471019396, 0.06517876970908289, 0.010992671552298569, 0.011214745725072231, 0.03464357095269825, 0.05196535642904738, 0.008660892738174563, 0.0035531867643793847, 0.006884299355984949, 0.03153453253386635, 0.048523206751054905, 0.006329113924050715, 0.0021097046413501856, 0.0032200755052188123, 0.005773928492116323, 0.0832778147901399]
+
+
+VGG_19
+Full_acc: 90.140000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[6.52908130056615, 1.3380062847274221, 0.24543462269497923, 0.05393969104397018, 0.012381510740945257, 0.003067060214909937, 0.0007555918574419032, 0.00017501117606359432, 4.4104190648609716e-05, 2.3327871023182897e-05, 1.0675439046810562e-05, 8.983655080683937e-06, 1.1246136322410839e-05, 1.669374629310751e-05, 6.347073531497991e-06, 6.932484220693536, 1.0687040042001579, 0.8323984445929756, 0.8230926166285857, 0.8200582224272311, 0.8417420472174938, 0.8317440393798151, 1.2031666133766876, 0.4699059099648023, 0.1461144158851069, 0.27725188424588565, 0.04629791970679128, 0.07348240968071523, 0.21177775297649584, 0.021228071993016986, 0.0187992093161517, 0.07344999091598375, 0.18586181536894214, 0.013847674161837173, 0.004719387421439763, 0.018766820174169, 0.07344825979729348, 0.17349586010003093, 0.011314319153131549, 0.001195520195639009, 0.004732545275858459, 0.018766792234023475, 0.5598509816229902]
+js_param_list:
+[3.9388183524565687, 0.5906007422253898, 0.11337101319621945, 0.024587731384019467, 0.005748740971112084, 0.0013975448353345961, 0.00034048951357184523, 7.219580933576352e-05, 2.420215829820016e-05, 2.245575096393826e-05, 1.1588766272794481e-05, 4.6564651206769664e-06, 1.1169147267451386e-05, 1.0010399586171422e-05, 3.3232170155925167e-06, 4.392429723980739, 0.5858373864659538, 0.5174166440347002, 0.5125834551246481, 0.5181444012623768, 0.5165617022044846, 0.5127922743171613, 0.785298571239749, 0.32687467701633877, 0.0757253679493511, 0.20164512084223385, 0.022964817771256384, 0.04912063785704141, 0.15688899149660557, 0.009258595397256264, 0.01287157719588226, 0.04911382975037919, 0.13826518313312058, 0.005405974137344003, 0.0031803287470254043, 0.012817747347664115, 0.04911231637448609, 0.12977071412854377, 0.004209915508246866, 0.000807348196237399, 0.0031895961527889497, 0.012816874513219311, 0.29721936760612694]
+ptq_acc_list:
+[10.0, 10.56, 69.87, 88.36, 89.56, 90.07, 90.03, 90.05, 90.11, 90.13, 90.15, 90.14, 90.14, 90.14, 90.14, 10.0, 17.15, 66.08, 64.32, 66.0, 58.73, 61.57, 10.58, 61.44, 80.12, 82.18, 87.95, 86.74, 84.47, 89.05, 89.36, 86.86, 86.0, 89.26, 89.82, 89.5, 87.16, 86.77, 89.53, 90.1, 89.82, 89.28, 79.38]
+acc_loss_list:
+[0.889061459951187, 0.8828489017084535, 0.2248724206789438, 0.01974706012868872, 0.006434435322831133, 0.0007765697803417727, 0.0012203239405369362, 0.0009984468604393544, 0.0003328156201464515, 0.0001109385400488697, -0.0001109385400488697, 0.0, 0.0, 0.0, 0.0, 0.889061459951187, 0.8097404038162859, 0.266918127357444, 0.28644331040603516, 0.2678056356778345, 0.34845795429332155, 0.3169514089194586, 0.882627024628356, 0.3183936099400932, 0.11116041712891053, 0.08830707787885504, 0.024295540270690013, 0.037719103616596465, 0.06290215220767696, 0.01209230086532065, 0.008653206123807423, 0.036387841136010665, 0.04592855558020857, 0.00976259152429549, 0.0035500332815620967, 0.007100066563124036, 0.03305968493454631, 0.03738628799645002, 0.006767250942977584, 0.0004437541601953212, 0.0035500332815620967, 0.009540714444197908, 0.1193698690925228]
+
+
+Inception_BN
+Full_acc: 92.850000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[12.330741426000127, 1.7774022898716264, 0.28821054098072, 0.06272624750607707, 0.014643773238671408, 0.0035362488362275716, 0.0008576211587906442, 0.00021727906401486407, 5.245980780163249e-05, 1.6362107243046427e-05, 6.144467100252098e-06, 7.116624913113964e-06, 5.504538717570937e-06, 3.849723618814388e-06, 1.0516858692556946e-05, 11.647423497409791, 1.3409273597202798, 1.3517284432634658, 1.2545879744691506, 1.2791368790498459, 1.3251419114745038, 1.3162237365748197, 2.1519596291810594, 0.8640877339820748, 0.19610413380825797, 0.5233173089245113, 0.0577209691448774, 0.13109263975912017, 0.4030272721882487, 0.023144755090275103, 0.03308999561031262, 0.13108261789492087, 0.3533817432909395, 0.013407919874326895, 0.008421085777519849, 0.03309604428632267, 0.13108322365407005, 0.3308494735980438, 0.010379484824287365, 0.0021142344319224275, 0.008399573286408415, 0.033096178092927774, 0.9498731099086688]
+js_param_list:
+[11.743628415486707, 1.8138152564298837, 0.2575987088433499, 0.056091978816504955, 0.0131183142169655, 0.003185085333115615, 0.0007642336413234433, 0.0001867811492769769, 4.84294604077158e-05, 1.6515516075695125e-05, 7.525543685547781e-06, 6.304620082507371e-06, 9.163043695968708e-06, 3.1847959617624517e-06, 1.4797626839592367e-05, 11.975132118833136, 1.3209527384787245, 1.8069678172592176, 1.6552403779967813, 1.6623214155416337, 1.7549714599696822, 1.7484103230481454, 2.1317498641722077, 0.8521769299783177, 0.18539618435504263, 0.5143901131955009, 0.05361950389674753, 0.13206790208872876, 0.3951690287541315, 0.020658682417329204, 0.03345737808203658, 0.13206631898251822, 0.3459066061409915, 0.011401936022077864, 0.008596322544040389, 0.03346616042846384, 0.13206817134822676, 0.32360442639471093, 0.008604766847197439, 0.00217485133267553, 0.008572328166388418, 0.03346495066046156, 1.2860953403025934]
+ptq_acc_list:
+[10.0, 11.24, 60.8, 88.36, 92.18, 92.92, 92.96, 92.89, 92.9, 92.87, 92.85, 92.85, 92.84, 92.85, 92.85, 10.0, 10.21, 20.71, 22.62, 19.12, 26.69, 24.87, 11.59, 15.18, 72.27, 27.03, 90.96, 79.08, 39.19, 91.77, 90.92, 77.58, 44.48, 92.16, 92.7, 91.96, 81.45, 40.77, 92.38, 92.7, 92.71, 91.93, 21.51]
+acc_loss_list:
+[0.8922994076467421, 0.8789445341949381, 0.3451803984921917, 0.04835756596661277, 0.007215939687668148, -0.0007539041464728853, -0.0011847065158858312, -0.00043080236941309916, -0.0005385029617664122, -0.0002154011847066261, 0.0, 0.0, 0.00010770059235316, 0.0, 0.0, 0.8922994076467421, 0.8900376952073236, 0.7769520732364027, 0.7563812600969305, 0.7940764674205707, 0.7125471190091546, 0.7321486268174474, 0.875175013462574, 0.8365105008077544, 0.22164781906300485, 0.7088852988691438, 0.02035541195476576, 0.14830371567043615, 0.5779213785675821, 0.01163166397415184, 0.020786214324178705, 0.16445880452342485, 0.5209477652127087, 0.007431340872374774, 0.0016155088852987773, 0.009585352719439964, 0.12277867528271397, 0.5609046849757673, 0.005061927840603111, 0.0016155088852987773, 0.0015078082929456174, 0.009908454496499597, 0.7683360258481421]
+
+

ykl/ALL/ptq_result_L2.txt

+AlexNet
+Full_acc: 86.070000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[3806.6554297042253, 1742.015153345221, 425.358251800215, 102.70556529578258, 25.13516598845947, 6.284990710624555, 1.5226463858972383, 0.37637188590897513, 0.0937218483840339, 0.02356942857993605, 0.005913245649293143, 0.0014840293248440614, 0.0004124549331586565, 0.00012464459306113474, 4.4241644032453234e-05, 3806.626823601337, 1119.5684059386515, 67.22826637442971, 65.2584135547821, 65.25849993476412, 65.25854846334343, 65.2585426561475, 663.9931633855356, 150.21247755919424, 156.37541828099458, 46.0525243022747, 55.87336577063952, 18.37679391322827, 21.364867094811032, 31.17330007882125, 4.767773367697066, 18.12624943589473, 14.65885675436464, 23.174095613967566, 1.229944644119404, 4.67310792527002, 18.126250551950505, 12.643838288241945, 20.052928560737637, 0.3380601265152883, 1.1750614507345885, 4.673106245697662, 18.12777438545329]
+js_param_list:
+[1064.8615010864344, 482.2225667072016, 116.87681175824119, 28.202862327123864, 6.8951693631537, 1.724479789021284, 0.4170550164496115, 0.10312659284974346, 0.02567170871100962, 0.006452665587276742, 0.0016207288256828358, 0.000411124521601818, 0.00011448728661560505, 3.4554701236429785e-05, 1.256408327153913e-05, 1064.8533560327176, 308.84553439789335, 18.799006637521117, 18.2646496648573, 18.264688604564057, 18.264684586733694, 18.264687473646386, 183.50218430540403, 41.5957977020034, 43.26997180631621, 12.847152788616123, 15.472483192451259, 5.139796185842727, 6.030818858640326, 8.635462553039364, 1.3329236199853485, 5.072315166766056, 4.170099277689426, 6.420378956568966, 0.3434601272612782, 1.3073140127584826, 5.0723155455971, 3.6093530667552316, 5.556115140184075, 0.09416124202888529, 0.3287037001727009, 1.3073135613517266, 5.075342519571067]
+ptq_acc_list:
+[10.0, 10.06, 51.65, 77.22, 83.96, 85.47, 86.07, 86.0, 86.06, 86.04, 86.07, 86.06, 86.09, 86.06, 86.07, 10.0, 12.28, 74.5, 74.64, 74.57, 74.5, 74.25, 20.54, 68.16, 52.38, 79.55, 79.14, 82.92, 81.3, 83.3, 85.45, 82.91, 82.06, 84.44, 85.91, 85.51, 82.82, 82.62, 85.01, 86.06, 85.98, 85.48, 82.88]
+acc_loss_list:
+[0.8838154990124317, 0.8831183920065063, 0.39990705239920993, 0.10282328337399785, 0.024514929708376897, 0.00697107005925403, 0.0, 0.0008132915069128986, 0.0001161845009874626, 0.0003485535029625529, 0.0, 0.0001161845009874626, -0.00023236900197525542, 0.0001161845009874626, 0.0, 0.8838154990124317, 0.8573254327872661, 0.1344254676426164, 0.13279888462879044, 0.1336121761357035, 0.1344254676426164, 0.1373300801673056, 0.7613570349715348, 0.20808644126873474, 0.3914255838271174, 0.07575229464389446, 0.08051585918438472, 0.03659811781108391, 0.05542000697107002, 0.03218310677355636, 0.007203439061229121, 0.03671430231207153, 0.04658998489601477, 0.018938073660973574, 0.0018589520158010526, 0.0065063320553036845, 0.03775996282095969, 0.04008365284071092, 0.012315557104682098, 0.0001161845009874626, 0.001045660508887989, 0.0068548855582664025, 0.037062855815034254]
+
+
+AlexNet_BN
+Full_acc: 87.310000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[3632.5462719527304, 1153.6671007438044, 258.53006140593516, 61.17532279693054, 14.891223739277061, 3.708468609799996, 0.9327535718553865, 0.2290387991705687, 0.0558845133577796, 0.014370449214296995, 0.003571630291049879, 0.0008625562856728075, 0.00023265116615085678, 2.582518881799345e-05, 5.500940913937269e-06, 3632.5290002264333, 707.8994232280825, 66.09579658259983, 65.1197204024375, 65.11971100025693, 65.11958369494646, 65.11959010588745, 502.53342155216836, 125.74848822462886, 106.7977490011123, 50.340123700280365, 37.91919519151391, 18.217442279952905, 30.826013119263497, 20.928424709525203, 4.712420598057065, 18.085518803241957, 24.785147412778223, 15.410974606040625, 1.2015424694844759, 4.66620720799309, 18.085515618418324, 22.628848887626468, 13.29073707534331, 0.3150020990484127, 1.1736005029204484, 4.666208204984376, 18.096500236447497]
+js_param_list:
+[1015.8729315394377, 316.80440575739027, 70.45411939495952, 16.637757426403944, 4.0480464268365335, 1.008581365757596, 0.2542549327402086, 0.06247395741987976, 0.01521756945701585, 0.0039055598906891386, 0.0009750390687994653, 0.0002388051307656589, 6.268083549538516e-05, 1.1420448279403141e-05, 9.514402586825453e-07, 1015.8657756982868, 193.51164061931982, 18.487584223270957, 18.224827841004352, 18.224824861932166, 18.22477461172962, 18.224794934502942, 138.7951197584608, 34.86550339450208, 29.34660124544504, 14.111192248397439, 10.428013148625217, 5.099791885771236, 8.714878912611212, 5.751938045273537, 1.319394969858532, 5.063761997837521, 7.033866884554059, 4.234439261320504, 0.3362723593071352, 1.3068606576090431, 5.063761194055209, 6.430008897204431, 3.651427775331254, 0.08801919927931902, 0.32871521234415546, 1.30685917092583, 5.069624667109006]
+ptq_acc_list:
+[10.0, 13.33, 38.66, 79.33, 85.48, 87.0, 87.49, 87.36, 87.24, 87.29, 87.25, 87.28, 87.31, 87.31, 87.32, 10.0, 14.81, 49.61, 54.45, 54.28, 56.42, 50.9, 20.48, 61.29, 70.85, 72.36, 81.95, 83.32, 76.64, 84.76, 86.75, 83.52, 77.85, 85.42, 86.87, 86.71, 83.64, 78.32, 85.69, 87.16, 87.07, 86.73, 79.48]
+acc_loss_list:
+[0.8854655824075135, 0.8473256213492154, 0.5572099415874471, 0.09139846523880431, 0.020959798419425017, 0.0035505669453671086, -0.002061619516664673, -0.0005726720879624001, 0.0008017409231474904, 0.0002290688351849275, 0.0006872065055549452, 0.0003436032527774726, 0.0, 0.0, -0.00011453441759238237, 0.8854655824075135, 0.8303745275455274, 0.4317947543236743, 0.37636009620891075, 0.3783071813079831, 0.3537968159431909, 0.4170198144542435, 0.7654335127705876, 0.29801855457565, 0.18852365135723292, 0.1712289543007674, 0.06139044782957278, 0.04569923261940223, 0.12220822357118316, 0.029206276486084034, 0.006413927385179272, 0.04340854426755247, 0.10834955904249235, 0.02164700492497996, 0.0050395143740693815, 0.00687206505554929, 0.04203413125644258, 0.1029664414156455, 0.01855457564998287, 0.0017180162638873633, 0.002748826022219781, 0.0066429962203642, 0.08968044897491693]
+
+
+VGG_16
+Full_acc: 90.060000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[4803.919675134635, 1287.1382930872844, 279.9264367424686, 65.25738246026505, 15.72176171355718, 3.863929102877906, 0.9633695234465877, 0.24037554092498606, 0.06003798427713084, 0.014859649305759608, 0.0036989349617955183, 0.0009297590950600473, 0.00023777455542695435, 6.177451735386848e-05, 1.687348277800094e-05, 4803.9126735699, 780.6742574825653, 90.85871885068245, 89.98441565230418, 89.98439740063434, 89.98447368346076, 89.98460526006676, 609.734857829675, 163.97601192015594, 121.10715205180763, 73.45937603752851, 42.80441037433894, 24.85925755673281, 48.867139323033896, 23.336643195538816, 6.477995998407114, 24.744689884301835, 40.82625887892399, 17.053899721476558, 1.6444280482850528, 6.436495589986837, 24.7446915417418, 37.74966086128042, 14.645385768954204, 0.42431402914077926, 1.619551796382776, 6.436509160682666, 24.763244303170193]
+js_param_list:
+[3468.943478398751, 780.3214366978973, 166.04031924610246, 38.151388298670724, 9.14175442167032, 2.2421934202373506, 0.5572632738247393, 0.1404745912003612, 0.03522789057832243, 0.008541134041069957, 0.0021640374431170254, 0.0005385031232900044, 0.0001413481065553399, 3.903370734086286e-05, 1.5885526939603433e-05, 3468.9449224664572, 471.25549248508725, 68.06340397833709, 67.6246439579611, 67.62469670868062, 67.62460844256277, 67.62469563891236, 410.2968176053128, 118.54872617685369, 75.29372245845758, 58.36820486947816, 26.416316296085885, 18.41059254585881, 41.195915783732055, 14.112047898448875, 4.859251633385958, 18.353329865236514, 35.23899741349211, 10.192427088266959, 1.2270888971031126, 4.8384185581718, 18.353331704123484, 32.851415181489216, 8.718847369799665, 0.3129867228456511, 1.2147891895642164, 4.838426537801942, 18.35858860501769]
+ptq_acc_list:
+[10.0, 12.22, 75.73, 87.93, 89.36, 89.98, 89.9, 89.98, 90.04, 90.04, 90.07, 90.07, 90.05, 90.04, 90.06, 10.0, 21.34, 73.34, 75.79, 76.82, 76.91, 76.59, 11.03, 54.96, 82.07, 79.84, 87.91, 87.24, 84.55, 89.4, 89.31, 87.16, 85.59, 89.38, 89.84, 89.41, 87.09, 85.57, 89.41, 89.89, 89.83, 89.35, 82.0]
+acc_loss_list:
+[0.8889629136131468, 0.8643126804352654, 0.15911614479236064, 0.02365089940039968, 0.0077725960470797556, 0.0008882966910948067, 0.0017765933821896134, 0.0008882966910948067, 0.00022207417277366224, 0.00022207417277366224, -0.00011103708638675222, -0.00011103708638675222, 0.00011103708638691001, 0.00022207417277366224, 0.0, 0.8889629136131468, 0.7630468576504552, 0.18565400843881855, 0.15844992227403948, 0.14701310237619375, 0.14601376859871204, 0.14956695536309125, 0.8775260937153009, 0.38974017321785476, 0.08871863202309581, 0.11347990228736396, 0.023872973573173504, 0.031312458361092685, 0.061181434599156176, 0.007328447701532273, 0.008327781479013991, 0.032200755052187495, 0.04963357761492337, 0.007550521874306094, 0.002442815900510758, 0.007217410615145522, 0.03297801465689539, 0.04985565178769719, 0.007217410615145522, 0.0018876304685765235, 0.002553852986897668, 0.007883633133466666, 0.0894958916278037]
+
+
+VGG_19
+Full_acc: 90.140000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[5144.206275389696, 1314.5031915215789, 284.75228418572925, 66.08828138577456, 15.924547195347134, 3.945508489581075, 0.9744824621662432, 0.240145204857737, 0.060970177786898, 0.015168279581215922, 0.0038762319348639397, 0.0009231650198397274, 0.00023264343031430685, 5.850058697982251e-05, 2.2746126922737273e-05, 5144.199599261901, 796.5924672633316, 98.79804986382494, 97.90326891150325, 97.90339725838194, 97.90342733252247, 97.9033895034332, 646.5794756970722, 179.30711630045755, 124.80812987710833, 83.28759482337827, 43.904681597578175, 27.154587219689308, 56.692876023912184, 23.776916070113252, 7.026013234767483, 27.04269180301487, 47.72767446760083, 17.299138622395738, 1.7806228432719848, 6.983728619409324, 27.04269089121566, 44.240986013374624, 14.849377566405725, 0.45780853948607875, 1.7553917796752694, 6.983728425473142, 27.059538268599393]
+js_param_list:
+[4036.5180986336604, 829.4606793443687, 174.56442102431618, 40.01640309397311, 9.582595067338994, 2.350277999553833, 0.5824392818690054, 0.14468797282940982, 0.03658160405467158, 0.00915174920249928, 0.002339405326979436, 0.0005618663309452209, 0.00013767900536115093, 4.960963871634183e-05, 1.221981967301472e-05, 4036.5197593916123, 499.33260286171014, 81.26284288453259, 80.81754200216422, 80.81764201292202, 80.81764351085528, 80.81763076484798, 474.4480804814884, 145.68195465957422, 81.48766551558924, 75.4339569457211, 28.164690222709147, 22.24136690419591, 54.58554288393205, 14.858657158099888, 5.747360506023278, 22.184649628754382, 47.06499217070044, 10.642578971997688, 1.4449041773182094, 5.72664701483485, 22.18464837310006, 43.9639668331577, 9.077944964333957, 0.3670860046163598, 1.4324142167409286, 5.726646439297058, 22.190228135948722]
+ptq_acc_list:
+[10.0, 10.7, 71.16, 88.29, 89.71, 90.02, 89.98, 90.13, 90.21, 90.12, 90.13, 90.14, 90.14, 90.13, 90.14, 10.0, 19.03, 68.1, 66.57, 66.96, 61.69, 65.38, 11.04, 62.01, 78.65, 81.66, 87.93, 86.94, 84.67, 89.09, 89.4, 87.17, 85.78, 89.24, 89.94, 89.56, 87.18, 86.51, 89.38, 90.05, 89.81, 89.53, 78.62]
+acc_loss_list:
+[0.889061459951187, 0.8812957621477701, 0.21056134901264703, 0.020523629909030335, 0.004770357222099033, 0.001331262480585806, 0.0017750166407809694, 0.0001109385400488697, -0.0007765697803416151, 0.00022187708009758177, 0.0001109385400488697, 0.0, 0.0, 0.0001109385400488697, 0.0, 0.889061459951187, 0.7888839582871089, 0.24450854226758384, 0.26148213889505223, 0.2571555358331485, 0.3156201464388729, 0.27468382516086093, 0.8775238517861105, 0.31207011315731087, 0.12746838251608603, 0.09407588196139344, 0.024517417350787593, 0.03550033281562018, 0.060683381406700675, 0.011648546705125329, 0.008209451963612103, 0.032948746394497436, 0.04836920346128244, 0.00998446860439323, 0.0022187708009762906, 0.006434435322831133, 0.032837807854448564, 0.040270690037719055, 0.00843132904370984, 0.0009984468604393544, 0.0036609718216108087, 0.006767250942977584, 0.12780119813623247]
+
+
+Inception_BN
+Full_acc: 92.850000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[2511.2094588241143, 514.6224944443054, 121.72162693805204, 34.91936699749214, 11.830727046720341, 4.61389921181382, 1.9815037610235493, 0.9090898451724668, 0.445693145985502, 0.22169278207806847, 0.10797906282345335, 0.05346285125971105, 0.02532186120284057, 0.012302718294603427, 0.005881463105781766, 2511.197114436539, 317.43921605650036, 53.10258916165484, 49.56627028116844, 49.55384989557891, 49.55381038799659, 49.55343630864873, 314.49013802009296, 104.54996263349719, 53.443062402939, 54.960874258820255, 19.04872998828755, 14.524522578886899, 38.56171912282987, 9.638177596411147, 3.80774697251023, 13.940455390214797, 32.05389710796874, 6.417397878853529, 1.0179720739751408, 3.5521641738404086, 13.938914251013315, 29.187816906242272, 5.153629153112564, 0.2840957063401789, 0.8982905850077583, 3.5514526481610735, 13.972900328027036]
+js_param_list:
+[2744.038379989026, 516.638349887526, 129.91342844973485, 40.73366137533873, 15.109998796040331, 6.340951315617309, 2.873664540067067, 1.3703147138534952, 0.6729131813965019, 0.33525752212052057, 0.16545026015840456, 0.07993866540433915, 0.038260218032434255, 0.018713906292927, 0.009382849493313599, 2744.0333295413316, 324.6756368084103, 60.080916610025525, 54.766434780856635, 54.747717700492004, 54.747529170172136, 54.74718325245878, 349.4332999050808, 124.12611787269181, 56.2261930410738, 67.01936850231559, 20.017832250515283, 16.607070029075818, 47.101684942696515, 9.810136391111984, 4.36989863546681, 15.72518888008714, 38.93910436603853, 6.231147896753903, 1.1879888395822027, 3.9838019674079073, 15.722349515453137, 35.26415154400586, 4.7847528600807925, 0.339524681842086, 1.0097705140107518, 3.9826169398840015, 15.75149388072123]
+ptq_acc_list:
+[10.0, 10.91, 58.96, 88.94, 91.98, 92.73, 92.86, 92.93, 92.88, 92.89, 92.86, 92.85, 92.85, 92.85, 92.85, 10.0, 11.58, 23.08, 12.96, 23.49, 14.39, 25.34, 11.25, 15.72, 69.72, 26.39, 89.61, 75.18, 44.08, 91.88, 91.65, 80.29, 54.62, 92.51, 92.64, 91.66, 83.23, 47.87, 92.44, 92.81, 92.66, 91.2, 29.07]
+acc_loss_list:
+[0.8922994076467421, 0.8824986537425956, 0.3649973074851911, 0.042110931610123824, 0.009369951534733337, 0.0012924071082389913, -0.00010770059235331305, -0.0008616047388261983, -0.0003231017770597861, -0.00043080236941309916, -0.00010770059235331305, 0.0, 0.0, 0.0, 0.0, 0.8922994076467421, 0.8752827140549273, 0.7514270328486806, 0.8604200323101776, 0.7470113085621971, 0.8450188476036617, 0.7270866989768443, 0.8788368336025848, 0.8306946688206786, 0.2491114701130856, 0.7157781367797522, 0.03489499192245552, 0.19030694668820666, 0.525255788906839, 0.010446957458266008, 0.01292407108239083, 0.13527194399569187, 0.4117393645665051, 0.0036618201400106538, 0.0022617124394183497, 0.01281637049003767, 0.10360796984383404, 0.4844372644049542, 0.004415724286483539, 0.0004308023694129461, 0.0020463112547118765, 0.01777059773828747, 0.6869143780290792]
+
+

ykl/ALL/ptq_result_nodiv.txt

+AlexNet
+Full_acc: 86.060000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[20.29526915572396, 6.947154405565474, 1.3097954986483278, 0.2806041501204682, 0.06600583186782864, 0.015990427906057616, 0.003813768549117867, 0.0009404779007737146, 0.00022623305712286078, 6.948210405950218e-05, 9.299055436569503e-06, 4.588463785228213e-05, 6.323176802503827e-06, 6.744560112952103e-06, 2.7236538125521296e-06, 20.181323343778978, 4.010774698034702, 0.8670328491284963, 0.8666524823918315, 0.8660510033487936, 0.8660729611264287, 0.8664076192754275, 3.9742171794465233, 1.3682528628687918, 0.7097116431336872, 0.7504338769105835, 0.24382982572517414, 0.2372659926125888, 0.5545490309437322, 0.12446012187624539, 0.06295130822442739, 0.23713186361064312, 0.4803567676666072, 0.08751503503961693, 0.01583786418054705, 0.06300327659273015, 0.2371345332262401, 0.44688978958546705, 0.07385774436728046, 0.003964355381077269, 0.015665392460334736, 0.06300194864679118, 0.23713405078669267]
+js_param_list:
+[6.472271200739325, 1.9073607385855067, 0.3519049896730976, 0.07560821810352604, 0.017718723069791623, 0.004282963800527071, 0.001020443650571906, 0.00024306038592283128, 6.666702871534726e-05, 2.4975186637540503e-05, 1.3491321849468107e-06, 2.3169196998707343e-05, 1.0058927602988542e-05, 1.8907509311902882e-06, 5.974142388311065e-06, 6.447438061039491, 1.0709440642338761, 0.2624273939040286, 0.2623439973305667, 0.2621623043957625, 0.2621589243725357, 0.2621859683158946, 1.2312889431290226, 0.4446631646350267, 0.1915297162811837, 0.25331989546362266, 0.06450702860498402, 0.07244799510226177, 0.19089616432803086, 0.032201968185343655, 0.018948033485787396, 0.07244055043973999, 0.1665385503147391, 0.02219587159786096, 0.004789412520292082, 0.018971298231359603, 0.07244168608599733, 0.15578948141977655, 0.01861220259767605, 0.0011699346276903527, 0.004740313570757627, 0.018970282687255417, 0.07244162541940319]
+ptq_acc_list:
+[10.0, 10.17, 54.88, 77.5, 84.15, 85.84, 85.94, 86.08, 86.11, 86.09, 86.06, 86.06, 86.07, 86.07, 86.08, 10.0, 12.87, 74.42, 73.74, 74.54, 74.54, 74.52, 20.48, 67.91, 51.8, 79.33, 78.92, 82.86, 81.23, 83.41, 85.49, 82.7, 82.14, 84.26, 86.06, 85.62, 83.09, 82.39, 84.76, 85.96, 85.99, 85.53, 82.7]
+acc_loss_list:
+[0.883801998605624, 0.8818266325819196, 0.3623053683476644, 0.09946548919358589, 0.022193818266325778, 0.002556356030676259, 0.0013943760167325651, -0.0002323960027887058, -0.000580990006971847, -0.0003485940041831413, 0.0, 0.0, -0.00011619800139427033, -0.00011619800139427033, -0.0002323960027887058, 0.883801998605624, 0.850453172205438, 0.13525447362305368, 0.14315593771787133, 0.13386009760632112, 0.13386009760632112, 0.13409249360911, 0.7620264931443179, 0.21089937253079252, 0.3980943527771323, 0.07820125493841511, 0.08296537299558449, 0.03718336044620036, 0.056123634673483594, 0.03079247036950971, 0.006623286079479518, 0.039042528468510335, 0.045549616546595416, 0.02091564025098765, 0.0, 0.005112712061352518, 0.034510806414129666, 0.04264466651173602, 0.015105740181268848, 0.0011619800139438593, 0.000813386009760718, 0.006158494073901942, 0.039042528468510335]
+
+
+AlexNet_BN
+Full_acc: 87.310000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[28.523062559045258, 5.0572556337657435, 0.9222994515743476, 0.20228252172057867, 0.04733840794454623, 0.011456054210043627, 0.0027986331414157524, 0.0006547017959301428, 0.00018235688076410206, 7.889945211402782e-05, 3.896137579417227e-05, 3.0078232088860448e-05, 3.1099101543519137e-06, 5.861285826331164e-06, 4.279552306759359e-05, 22.01594004242986, 3.427301854013008, 1.3063413452820585, 1.4045641196426313, 1.356906249917974, 1.351080683822659, 1.3664221708008975, 4.283054483794392, 1.6324676164185732, 0.5531126050153183, 0.9627616968657938, 0.17653089935503774, 0.2652652914052897, 0.7331839328597248, 0.08298583534512317, 0.06625189064409207, 0.26513542399370454, 0.642524352709163, 0.05487889254662182, 0.017293229228320214, 0.0661235968993205, 0.26514059830510045, 0.6016803287731487, 0.04518600073910273, 0.004305085389680972, 0.017293366508607133, 0.06612324530096758, 0.5719832788177283]
+js_param_list:
+[9.051924388498541, 1.5532132254895608, 0.279235384076741, 0.06104835425311557, 0.014213577461106068, 0.0034475495538411212, 0.0008562487643766872, 0.00020370005038261856, 5.8245655176918376e-05, 1.9824314206317698e-05, 1.2478366906568228e-05, 1.027524534715929e-05, 1.080271489503737e-06, 1.080119362859344e-06, 1.7222549393826723e-05, 7.2301684811811935, 1.0026643585816455, 0.3883847860336896, 0.4152343837972867, 0.40173012752731296, 0.40068117689267724, 0.40442175933627295, 1.36308850130799, 0.5246718536237667, 0.1634848500226413, 0.3105133429143706, 0.05235117714725387, 0.08154295575886782, 0.23739024316614557, 0.024066363948542654, 0.021141573431830138, 0.08149152594104668, 0.20800683060508368, 0.01568816634762468, 0.005310852279338676, 0.02109452384982742, 0.08149265449474423, 0.19483586797504632, 0.012852646679949089, 0.0013603533217732317, 0.005315723494355893, 0.021096808338497135, 0.16840667478229432]
+ptq_acc_list:
+[10.0, 13.16, 37.45, 80.39, 85.4, 87.04, 87.38, 87.26, 87.23, 87.27, 87.25, 87.32, 87.3, 87.31, 87.31, 10.0, 14.77, 55.59, 51.7, 53.81, 55.49, 52.13, 17.32, 62.24, 69.42, 72.16, 81.77, 83.47, 76.18, 84.66, 86.6, 83.71, 78.13, 85.64, 86.99, 86.76, 83.56, 78.52, 85.64, 87.18, 87.02, 86.69, 79.88]
+acc_loss_list:
+[0.8854655824075135, 0.8492727064482878, 0.5710686061161379, 0.07925781697400071, 0.02187607376016489, 0.003092429274997091, -0.0008017409231473277, 0.0005726720879624001, 0.0009162753407398728, 0.00045813767037001775, 0.0006872065055549452, -0.00011453441759238237, 0.00011453441759254514, 0.0, 0.0, 0.8854655824075135, 0.8308326652158974, 0.3633031726033673, 0.40785706104684455, 0.3836902989348299, 0.3644485167792922, 0.40293208109036766, 0.8016263887298134, 0.28713778490436376, 0.20490207307295843, 0.17351964265261718, 0.06345206734623761, 0.04398121635551487, 0.12747680678043746, 0.030351620662009, 0.008131943649066635, 0.04123239033329525, 0.10514259534990272, 0.01912724773794527, 0.0036651013629596537, 0.006299392967586727, 0.04295040659718245, 0.10067575306379574, 0.01912724773794527, 0.001488947428702273, 0.003321498110182181, 0.007101133890734217, 0.08509907227121757]
+
+
+VGG_16
+Full_acc: 90.060000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[14.262825155716163, 2.924289265562907, 0.5505950024563626, 0.11999170611395046, 0.02744225430159203, 0.006746828169761675, 0.0016618247316436267, 0.00044322956430803987, 0.00012768623230197918, 4.650674238497832e-05, 2.652061807080142e-05, 2.204594599459253e-05, 1.6918118686923485e-05, 2.011766220617495e-05, 2.067195538459942e-05, 14.917494972939796, 2.4171221602368393, 1.7598931061155216, 1.8031076352626845, 1.7405455628994735, 1.793194596897975, 1.8030161795108837, 2.5971766763412623, 1.0002184887481798, 0.31595235279138284, 0.5927951847220075, 0.10449790573630993, 0.15336463195724515, 0.456008781548831, 0.04802145042877176, 0.040792344467324326, 0.1533515894473453, 0.39956516741739173, 0.03170617607986788, 0.010133194077074878, 0.04078978612397213, 0.15334956958540794, 0.37421014652121226, 0.02645526524379531, 0.0025665850063961667, 0.010150208235922479, 0.040788831687284116, 1.2360329844220124]
+js_param_list:
+[7.646327343346346, 1.1369253976412526, 0.21223178228730832, 0.0462335545707114, 0.010691317072738505, 0.002701012583115348, 0.0006444077532065897, 0.00016180418230428466, 6.402129794029498e-05, 1.968498314365874e-05, 1.6924953659230635e-05, 6.065384498061266e-06, 9.299935263935805e-06, 1.1852805417376545e-05, 1.1821849182492628e-05, 8.602750549829853, 1.198344465680804, 1.052990177854398, 1.0729257331977795, 1.0820562338446593, 1.0805086622820688, 1.0756339188081119, 1.503943954991838, 0.6077287851397646, 0.14854021027605877, 0.36767343901811383, 0.04437681481233167, 0.09885630898936866, 0.2829765056435155, 0.01757503223494139, 0.025387881883922547, 0.09892526136629265, 0.2481673048125344, 0.010192293341661128, 0.0061815967921599645, 0.02538376152549722, 0.09892383354598411, 0.23245635974338605, 0.007921735530031428, 0.0015711629407955173, 0.006130501581707569, 0.025380028273212177, 0.6825620173760718]
+ptq_acc_list:
+[10.0, 10.79, 77.47, 88.08, 89.5, 89.93, 89.99, 90.01, 90.01, 90.03, 90.06, 90.07, 90.06, 90.09, 90.09, 10.0, 22.78, 77.33, 75.72, 77.74, 76.09, 77.69, 10.92, 55.56, 81.89, 80.03, 87.53, 86.94, 84.0, 89.09, 89.59, 87.0, 85.36, 89.28, 89.73, 89.35, 87.03, 85.68, 89.53, 89.9, 89.73, 89.45, 81.45]
+acc_loss_list:
+[0.8889629136131468, 0.8801909837885855, 0.13979569176104822, 0.021985343104596978, 0.006218076837663805, 0.0014434821230290412, 0.0007772596047080544, 0.0005551854319342345, 0.0005551854319342345, 0.00033311125916057226, 0.0, -0.00011103708638675222, 0.0, -0.00033311125916057226, -0.00033311125916057226, 0.8889629136131468, 0.7470575172107484, 0.14135021097046419, 0.15922718187874754, 0.13679769042860324, 0.1551188096824339, 0.13735287586053746, 0.8787475016655563, 0.38307794803464357, 0.09071729957805909, 0.11137019764601377, 0.028092382855873874, 0.03464357095269825, 0.06728847435043307, 0.010770597379524748, 0.005218743060182088, 0.03397734843437711, 0.05218743060182104, 0.008660892738174563, 0.0036642238507661367, 0.007883633133466666, 0.033644237175216536, 0.04863424383744165, 0.005884965578503233, 0.0017765933821896134, 0.0036642238507661367, 0.00677326226959804, 0.0956029313790806]
+
+
+VGG_19
+Full_acc: 90.140000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[13.048974717343004, 2.6749312021528913, 0.4909782092613753, 0.10788846263262551, 0.024765729210361388, 0.006134660841831837, 0.001511714235343558, 0.00035217524779276296, 9.027838452787228e-05, 4.826876430404425e-05, 2.3137291736999405e-05, 1.935483645987856e-05, 2.0907217644717642e-05, 3.387203643630215e-05, 1.4449608674907743e-05, 13.81147885113297, 2.1226757078950094, 1.6530962572212622, 1.6875658424517532, 1.6481537280948702, 1.6834940394264948, 1.6566304454586653, 2.406336585421847, 0.939816429984288, 0.2922284769134882, 0.5545077160998108, 0.09259667141740417, 0.146967465597935, 0.42356516614972445, 0.04246028388371049, 0.03759871228427055, 0.14690249068281197, 0.3717209019427504, 0.027698966422784608, 0.009442294391095476, 0.037535971730376476, 0.14689964408599956, 0.3469994616021775, 0.02262714849993608, 0.0023915509517363483, 0.009467019535196476, 0.03753423478901108, 1.103711294382473]
+js_param_list:
+[7.87702175577721, 1.1811605933279172, 0.22674393351088745, 0.04917663913573108, 0.01150250664366638, 0.0027928304104144464, 0.0006812588208013448, 0.0001419666987856605, 4.487908372373006e-05, 4.4161934990693646e-05, 2.0421922284708402e-05, 1.0125851299536251e-05, 2.2663438017443415e-05, 1.7626631301269656e-05, 7.0688205745594876e-06, 8.778726407792607, 1.1730341880807205, 1.0298268062193983, 1.0443298310858409, 1.0362971008666828, 1.0415261153131727, 1.0389181506047174, 1.5705945195303064, 0.6537306062275565, 0.15144308109334406, 0.4032963612294232, 0.045927748342518185, 0.09826016473130708, 0.31379083728006496, 0.018516901788017837, 0.02573785465867853, 0.0982350784265641, 0.2765367058530107, 0.010813489695802557, 0.006358928752244081, 0.02563535210744919, 0.09823167663379691, 0.25956214400602157, 0.008427520460296526, 0.001612199066969595, 0.0063761376736868125, 0.02563587243935141, 0.5868812546140629]
+ptq_acc_list:
+[10.0, 10.82, 67.51, 88.12, 89.54, 90.05, 90.1, 90.12, 90.16, 90.13, 90.16, 90.18, 90.16, 90.17, 90.17, 10.0, 17.7, 67.89, 58.46, 68.4, 59.88, 61.53, 10.75, 59.68, 78.68, 81.82, 88.0, 86.99, 84.53, 89.13, 89.58, 87.11, 85.86, 89.36, 89.81, 89.53, 86.96, 86.77, 89.47, 90.12, 89.88, 89.48, 79.32]
+acc_loss_list:
+[0.889061459951187, 0.8799644996671843, 0.2510539161304637, 0.022409585089860174, 0.006656312402928714, 0.0009984468604393544, 0.0004437541601953212, 0.00022187708009758177, -0.00022187708009758177, 0.0001109385400488697, -0.00022187708009758177, -0.0004437541601953212, -0.00022187708009758177, -0.0003328156201464515, -0.0003328156201464515, 0.889061459951187, 0.803638784113601, 0.24683825160860884, 0.35145329487463944, 0.2411803860661193, 0.335700022187708, 0.3173951630796539, 0.8807410694475261, 0.33791879298868427, 0.12713556689593958, 0.09230086532061246, 0.023740847570445978, 0.034945640115376146, 0.062236520967384065, 0.011204792544930165, 0.006212558242733551, 0.03361437763479034, 0.047481695140891955, 0.008653206123807423, 0.0036609718216108087, 0.006767250942977584, 0.035278455735522594, 0.03738628799645002, 0.007432882183270487, 0.00022187708009758177, 0.0028844020412691937, 0.0073219436432216175, 0.1200355003328157]
+
+
+Inception_BN
+Full_acc: 92.850000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[24.501583051204396, 3.5853632714637826, 0.5764639203863853, 0.1254596836956013, 0.029294242308528968, 0.007074992528733897, 0.0017265789356264636, 0.00044525835147108414, 0.00010715587044753592, 2.763500892255966e-05, 1.3372514338448216e-05, 1.651875332239356e-05, 1.071087798827436e-05, 1.1416507626689443e-05, 1.8697256957320165e-05, 23.2714172433988, 2.7164069977501053, 2.8011785806535356, 2.6953667618375734, 2.8191085641023816, 2.773406742070304, 2.6137332471433825, 4.303955467683129, 1.728203312661921, 0.39222141068340766, 1.0466501003638464, 0.11544128460953085, 0.26217790777428135, 0.8060525439567119, 0.04629405404231517, 0.06617748628761817, 0.26216117578349835, 0.7067635183298785, 0.02682603064554569, 0.016838761834673067, 0.06618181366908193, 0.26216038591779744, 0.6617165182206839, 0.020762169553687215, 0.004244293114838931, 0.01680512193717317, 0.06618269148789903, 1.8439418092752666]
+js_param_list:
+[23.398090191439067, 3.662370609719416, 0.5152455334173763, 0.11217440188269354, 0.02624451610255668, 0.0063719985735158444, 0.001544632694210822, 0.00038196865652848907, 0.00010483607892653654, 2.9065382911872993e-05, 2.1424251686981593e-05, 1.6622652370079758e-05, 2.136174368302968e-05, 1.0926635195267737e-05, 1.957231372594437e-05, 23.937170004690962, 2.759034655091616, 3.7002130819772976, 3.532238910924999, 3.6297386953671507, 3.620141221546079, 3.4493545951768345, 4.26354002827198, 1.704408792013372, 0.37079297092866725, 1.0288180768202424, 0.10723518968423576, 0.2641344164185636, 0.7903383537153296, 0.041324052299147755, 0.06690974766269926, 0.2641313817787517, 0.6918113077016262, 0.022814864784890004, 0.017182124843102154, 0.06691562325891631, 0.264131453339321, 0.6472324961904546, 0.017213687801549447, 0.0043651611752884645, 0.017148119773601003, 0.06691453707221243, 2.5415331853471583]
+ptq_acc_list:
+[10.0, 10.71, 69.45, 89.97, 92.08, 92.86, 92.87, 92.9, 92.9, 92.88, 92.84, 92.86, 92.85, 92.85, 92.84, 10.0, 10.36, 18.61, 23.06, 16.98, 14.41, 19.57, 11.41, 13.97, 79.08, 38.49, 90.11, 84.51, 40.49, 91.79, 91.6, 79.59, 48.48, 92.36, 92.53, 91.33, 80.13, 51.38, 92.61, 92.73, 92.63, 90.82, 29.59]
+acc_loss_list:
+[0.8922994076467421, 0.8846526655896606, 0.2520193861066235, 0.03101777059773824, 0.008292945611200819, -0.00010770059235331305, -0.0002154011847066261, -0.0005385029617664122, -0.0005385029617664122, -0.0003231017770597861, 0.00010770059235316, -0.00010770059235331305, 0.0, 0.0, 0.00010770059235316, 0.8922994076467421, 0.8884221863220247, 0.799569197630587, 0.7516424340333872, 0.8171243941841679, 0.8448034464189553, 0.7892299407646742, 0.8771136241249328, 0.8495422724824987, 0.14830371567043615, 0.5854604200323101, 0.029509962304792622, 0.08982229402261702, 0.5639203015616585, 0.011416262789445214, 0.013462574044157244, 0.14281098546041995, 0.4778675282714055, 0.005277329025309584, 0.003446418955304181, 0.016370490037695164, 0.1369951534733441, 0.4466343564889606, 0.0025848142164781357, 0.0012924071082389913, 0.0023694130317716625, 0.021863220247711375, 0.6813139472267097]
+
+

ykl/ALL/ptq_result_weightratio.txt

+AlexNet
+Full_acc: 86.060000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[15.390794522744232, 4.717097541596072, 0.861433466201176, 0.18508192637631454, 0.043397405458327654, 0.010496797691919834, 0.0024850090520978957, 0.000624175093904678, 0.00014904738718305474, 5.291579966091702e-05, 4.8962531373354375e-06, 4.193895417504035e-05, 1.592819528135614e-06, 2.967417045749432e-06, 1.0413106493855072e-06, 15.321349499009886, 2.669359363004649, 0.665240323544293, 0.6648249490573711, 0.6647401888858064, 0.6646321902074231, 0.6648930262034427, 2.970237193983619, 1.0721955650797796, 0.4770905332198318, 0.6054695109850212, 0.16107606526699011, 0.181228082062255, 0.45318591305589634, 0.07980226898451764, 0.04805752515345307, 0.18115329705371844, 0.39433974657405857, 0.055063869704067014, 0.012139171526851388, 0.048132818635071795, 0.1811559349720972, 0.3671160558876084, 0.04605680247234207, 0.003032838870538176, 0.011984056400302881, 0.04813150319087321, 0.18115602343582896]
+js_flops_weighted_list:
+[28.40515877286409, 8.70584714896398, 1.589856478399669, 0.3415861018050526, 0.08009399323423358, 0.019372827348516423, 0.004586317917020654, 0.0011519738385324405, 0.0002750809707295301, 9.766108492581001e-05, 9.036495650216035e-06, 7.740228422577126e-05, 2.9396982414617645e-06, 5.476647239052785e-06, 1.921836737145626e-06, 28.276991450362036, 4.926553753669294, 1.227763581955602, 1.2269969692746403, 1.2268405364064507, 1.2266412146883807, 1.227122612050269, 5.481852087849688, 1.978837753739401, 0.8805154487062422, 1.1174509260221974, 0.29728102741256435, 0.3344734696751785, 0.8363972240000284, 0.14728259269462676, 0.08869468240885497, 0.33433544690850636, 0.7277910893641049, 0.1016255502132089, 0.02240398272367832, 0.08883364361042133, 0.3343403154349667, 0.6775472078549858, 0.08500216053589742, 0.005597389369524905, 0.02211770316928806, 0.0888312158344373, 0.3343404787031035]
+js_param_list:
+[5.335234710880941, 1.5364356992134998, 0.2810079498291621, 0.06031047856052035, 0.014110041947520574, 0.0034040126288667374, 0.0008046306976175277, 0.00019470572779706148, 5.339504844640815e-05, 2.3103931722405794e-05, 1.118676972143684e-06, 2.186891678345004e-05, 7.638901720498192e-06, 6.725816434148269e-07, 4.19329140017799e-06, 5.313908139577299, 0.8537195505193228, 0.21603135769650977, 0.21582717195610415, 0.21581634718014503, 0.21575105452221033, 0.21583294142343779, 1.0080567176520165, 0.36710793002522313, 0.15308999403878698, 0.21012211564543096, 0.05127532080061556, 0.059764869329634644, 0.15863713336428933, 0.02542085053727128, 0.015573244047684945, 0.05975132050282547, 0.1384792977857137, 0.01742679923812338, 0.003952068904423522, 0.015598955194465434, 0.059751507791787915, 0.12954601789882095, 0.014575061413335639, 0.0009603470887152325, 0.0039045494639076486, 0.015598733274334387, 0.059751990616446676]
+ptq_acc_list:
+[10.0, 10.31, 56.17, 77.08, 84.22, 85.71, 85.98, 86.09, 86.08, 86.02, 86.06, 86.05, 86.08, 86.08, 86.08, 10.0, 16.15, 73.31, 74.31, 74.16, 74.21, 74.55, 20.62, 67.09, 53.42, 79.19, 79.39, 82.32, 81.78, 83.36, 85.47, 82.97, 82.16, 84.49, 86.01, 85.78, 82.91, 82.59, 84.78, 86.05, 86.04, 85.75, 83.02]
+acc_loss_list:
+[0.883801998605624, 0.8801998605623983, 0.3473158261677899, 0.1043458052521497, 0.021380432256565225, 0.00406693004880326, 0.0009295840111549883, -0.0003485940041831413, -0.0002323960027887058, 0.0004647920055775767, 0.0, 0.00011619800139443546, -0.0002323960027887058, -0.0002323960027887058, -0.0002323960027887058, 0.883801998605624, 0.8123402277480827, 0.14815245177782943, 0.13653265163839182, 0.1382756216593075, 0.13769463165233567, 0.13374389960492686, 0.7603997211247966, 0.22042760864513128, 0.3792702765512433, 0.07982802695793638, 0.07750406693004883, 0.04345805252149673, 0.049732744596792944, 0.03137346037648156, 0.0068556820822682244, 0.03590518243086223, 0.04531722054380671, 0.01824308621891712, 0.000580990006971847, 0.0032535440390425415, 0.03660237043922851, 0.040320706483848466, 0.014873344178480142, 0.00011619800139443546, 0.0002323960027887058, 0.003602138043225683, 0.03532419242389038]
+
+
+AlexNet_BN
+Full_acc: 87.310000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[11.591798533785282, 1.700183197789268, 0.3038094135267426, 0.06701544888451295, 0.01593624543282047, 0.003825953228443132, 0.0009532233854295787, 0.00019551855882143858, 6.55307570327729e-05, 3.47072260035146e-05, 3.505022686533898e-05, 2.3009420528141446e-05, 8.312853958041267e-07, 3.116905667386982e-06, 3.833276897941617e-05, 7.000022231570889, 1.5005345800833543, 0.745846478904552, 0.7646294650365152, 0.7972771082666167, 0.8348834173962452, 0.8141783554712251, 1.377012987292044, 0.5252011904953136, 0.18856212240816972, 0.31178739055080124, 0.05899102144452863, 0.09192247716102836, 0.23655059730355277, 0.027952782309607387, 0.021727784158718434, 0.09180928235111235, 0.20792645939896084, 0.018443659406444046, 0.006097508898828769, 0.02164334660618966, 0.09181375890446229, 0.19470294027215662, 0.01519870107475935, 0.001524288569272028, 0.006088090455227164, 0.021643227963696866, 0.33145092225344597]
+js_flops_weighted_list:
+[21.406669156099774, 3.1397422163400726, 0.5610473286712618, 0.12375797754223267, 0.029429594775694786, 0.007065419117601705, 0.0017603254218295182, 0.00036106572162803395, 0.000121016184957042, 6.409411811443945e-05, 6.472754061121002e-05, 4.249168507232945e-05, 1.5351415391158844e-06, 5.75601518763951e-06, 7.078943797961337e-05, 12.9269983048652, 2.7710494811923083, 1.377360792513532, 1.41204748662046, 1.4723381563828981, 1.541786035016542, 1.50354982782284, 2.5429411455607536, 0.9698933338562424, 0.34821921361029456, 0.5757803240896372, 0.10893920176087167, 0.16975399036310762, 0.4368399226743099, 0.051620631703546804, 0.04012487670702703, 0.1695449525818697, 0.3839794931028098, 0.034060056667721286, 0.011260319552066803, 0.039968945188197914, 0.1695532194695526, 0.35955951218252213, 0.028067565577641885, 0.0028149161672995126, 0.011242926435238788, 0.03996872609013615, 0.6120931289035025]
+js_param_list:
+[2.734479088940684, 0.3132424172357634, 0.054306647194035426, 0.011796604135106659, 0.0027847107051644014, 0.0006822415151577091, 0.0001651089941727384, 3.228843341085658e-05, 1.2759462925018916e-05, 8.564187390072141e-06, 9.326278173153569e-06, 5.965686148541275e-06, 3.5330878817685375e-08, 8.826677324214048e-07, 1.0788902521246867e-05, 1.4564256642500042, 0.31047654192531055, 0.17060282426356335, 0.1771587081095755, 0.18545945737715597, 0.19559617351750125, 0.189959581337747, 0.265386519272884, 0.10192080314098644, 0.03236375259538735, 0.06019249236191452, 0.010293599322833998, 0.016283465321462246, 0.04594274741460937, 0.004729445591036409, 0.004158369800705201, 0.016253072643470482, 0.0402070007169081, 0.0030832450417326886, 0.001062630465564137, 0.004136040389603472, 0.016254232375065597, 0.03763443684725491, 0.0025390305847557497, 0.00027616659750303706, 0.0010603967983225746, 0.004136140717399854, 0.08413230006865291]
+ptq_acc_list:
+[10.0, 15.29, 37.89, 80.02, 85.56, 87.09, 87.42, 87.42, 87.25, 87.29, 87.28, 87.32, 87.29, 87.32, 87.3, 10.0, 16.25, 50.55, 54.82, 52.0, 55.79, 53.02, 18.65, 61.89, 69.85, 72.73, 82.03, 83.5, 76.15, 84.4, 86.66, 83.66, 78.13, 85.68, 86.93, 86.54, 83.27, 78.21, 85.91, 87.23, 87.0, 87.04, 81.06]
+acc_loss_list:
+[0.8854655824075135, 0.8248768755010881, 0.5660290917420685, 0.08349559042492276, 0.020043523078685146, 0.0025197571870346908, -0.0012598785935173454, -0.0012598785935173454, 0.0006872065055549452, 0.0002290688351849275, 0.0003436032527774726, -0.00011453441759238237, 0.0002290688351849275, -0.00011453441759238237, 0.00011453441759254514, 0.8854655824075135, 0.8138815714122094, 0.42102851906998057, 0.3721223227579888, 0.40442102851906997, 0.3610124842515176, 0.3927385179246363, 0.7863933111900125, 0.2911464895201008, 0.1999770931164816, 0.16699118084984535, 0.060474172488832904, 0.0436376131027374, 0.12782041003321493, 0.033329515519413544, 0.00744473714351169, 0.04180506242125765, 0.10514259534990272, 0.018669110067575254, 0.004352307868514436, 0.008819150154621418, 0.046271904707364635, 0.10422632000916285, 0.01603481846294818, 0.0009162753407398728, 0.0035505669453671086, 0.003092429274997091, 0.07158401099530409]
+
+
+VGG_16
+Full_acc: 90.060000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[5.920568558509058, 1.2366942369029248, 0.24333976529364282, 0.05313804637395972, 0.011805343923029208, 0.0028956135593713197, 0.0006858387721491292, 0.00017475428648558512, 5.5037716377499444e-05, 1.8363316636701207e-05, 3.0147309189941645e-06, 3.5938150094587322e-06, 7.588472673062187e-07, 2.761265478140623e-06, 1.8205751891677835e-06, 6.655998976309741, 1.2806109655501847, 1.148895211956822, 1.110608810491263, 1.1338888989171876, 1.1246610427359278, 1.1214109082880812, 1.0629437809132753, 0.403389070458992, 0.13440872344714155, 0.23736481179302008, 0.046645787173690494, 0.06196931592022412, 0.1834877783166125, 0.021485302765101157, 0.017332589659448675, 0.06194557184806153, 0.16064264766655828, 0.014416940182581594, 0.004258526972833182, 0.01732019862118995, 0.06194491639496624, 0.15031629789717732, 0.012258144588710359, 0.001051457151644622, 0.004272414206520595, 0.017319316052076844, 0.8666441153492498]
+js_flops_weighted_list:
+[14.937082083214763, 3.1200725311944497, 0.6139251682302425, 0.13406269222045603, 0.02978386103568301, 0.00730538241220721, 0.0017303118668762724, 0.0004408899408617193, 0.00013885539523427596, 4.632905863884517e-05, 7.605905201638485e-06, 9.066884245605717e-06, 1.914505981707713e-06, 6.966433830287458e-06, 4.593153642344031e-06, 16.792509380208696, 3.2308706368405034, 2.898563189737484, 2.8019699122996187, 2.8607035606453954, 2.8374224781160025, 2.829222670183841, 2.6817151677984326, 1.0177157138494521, 0.33910157698362114, 0.5988508776447673, 0.11768328412449634, 0.15634322099196396, 0.4629237849197624, 0.05420555945151566, 0.04372862367851055, 0.15628331675280557, 0.40528749739950476, 0.03637269238045072, 0.01074389500234279, 0.04369736216135425, 0.15628166310137173, 0.3792350106153517, 0.03092623792799905, 0.002652735396252887, 0.01077893129107887, 0.04369513551586299, 2.1864681001458894]
+js_param_list:
+[1.770296244959528, 0.29806992793707837, 0.055665464831966924, 0.012101417041225202, 0.0028447136298671597, 0.0006880367651531245, 0.00017020795157846893, 4.522436604353743e-05, 2.085685768523233e-05, 1.3865664827935832e-05, 1.9972193056226206e-06, 5.692071212596929e-07, 1.8434384391604333e-06, 2.8346000195430685e-06, 2.562304531827042e-06, 2.6600226813451946, 0.6233435306363425, 0.7163984416773069, 0.6869824985889844, 0.7096845057337227, 0.7146866621934539, 0.7032853391142566, 0.3446638133393568, 0.140197618797341, 0.03384653692463511, 0.08580447942745584, 0.010451821497441623, 0.01977311058211095, 0.06660686501570992, 0.0044870860230231315, 0.005106004645241131, 0.019767490033555546, 0.05864023434517889, 0.002784702570814231, 0.0012759569718724024, 0.005101844752063444, 0.01976665091010429, 0.05504161767683782, 0.0022191752052733086, 0.00031513865524340187, 0.0012912743585447657, 0.005102061834463343, 0.5556901116653865]
+ptq_acc_list:
+[10.0, 13.22, 76.72, 88.11, 89.56, 89.98, 89.99, 90.0, 90.0, 90.04, 90.07, 90.06, 90.06, 90.07, 90.09, 10.0, 20.94, 77.81, 75.14, 75.06, 76.45, 77.19, 11.53, 54.29, 81.34, 80.0, 87.6, 86.8, 84.21, 88.85, 89.41, 87.05, 85.21, 89.39, 89.84, 89.38, 86.9, 85.74, 89.45, 89.86, 89.78, 89.4, 81.93]
+acc_loss_list:
+[0.8889629136131468, 0.8532089717965801, 0.14812347324006223, 0.021652231845436406, 0.00555185431934266, 0.0008882966910948067, 0.0007772596047080544, 0.0006662225183211445, 0.0006662225183211445, 0.00022207417277366224, -0.00011103708638675222, 0.0, 0.0, -0.00011103708638675222, -0.00033311125916057226, 0.8889629136131468, 0.7674883411059295, 0.13602043082389517, 0.165667332889185, 0.1665556295802798, 0.15112147457250721, 0.14290473017988012, 0.8719742393959582, 0.39717965800577393, 0.09682433932933598, 0.11170330890517435, 0.027315123251165977, 0.0361980901621142, 0.06495669553630921, 0.013435487452809326, 0.007217410615145522, 0.03342216300244287, 0.0538529868976239, 0.007439484787919184, 0.002442815900510758, 0.007550521874306094, 0.03508771929824558, 0.04796802131912067, 0.00677326226959804, 0.002220741727737096, 0.0031090384188319025, 0.007328447701532273, 0.0902731512325116]
+
+
+VGG_19
+Full_acc: 90.140000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[5.42539688208821, 1.2431814495773714, 0.22893087958060832, 0.050875650693093466, 0.011424559252241116, 0.002888330566046987, 0.0006958012938553044, 0.0001620122109056303, 4.628456898334069e-05, 4.606778419700862e-06, 6.266021337973151e-06, 3.404626571397867e-07, 4.28695999660748e-06, 3.0337623828983415e-07, 1.0465506947944175e-06, 6.098423281023068, 1.10155611081281, 0.9965644293881576, 0.9874767728771963, 0.9815533180848163, 0.9920825228870539, 1.006248223283427, 0.9964726594305288, 0.38478060783526064, 0.13371221116731546, 0.22201018739741246, 0.04292414200270462, 0.062727570196778, 0.16807050915013197, 0.020087426776406736, 0.016173171638683738, 0.06268975506483192, 0.14755320645575126, 0.013336430846658188, 0.004058001192906326, 0.01615557232713148, 0.06268919379337706, 0.13696270689396448, 0.01092882410397449, 0.001026471090943476, 0.004062249384958819, 0.016155447628954833, 0.7151647488167303]
+js_flops_weighted_list:
+[14.223174719098854, 3.2591140056975556, 0.6001632635617812, 0.1333751768731794, 0.0299505282037136, 0.0075720230575257045, 0.0018241068049697996, 0.0004247298460221305, 0.00012133923577492387, 1.2077091460700653e-05, 1.6426948704495657e-05, 8.925540311742125e-07, 1.1238658179430174e-05, 7.95328588230114e-07, 2.743628476950432e-06, 15.987575051583281, 2.887830211778206, 2.612584905045913, 2.5887607813639586, 2.5732319022268664, 2.6008351767540856, 2.6379718574679347, 2.612344321698798, 1.0087375970282038, 0.3505388053339556, 0.5820200352896387, 0.11252956873763163, 0.16444606910857248, 0.4406122296163039, 0.052661028659708, 0.042399450395679474, 0.16434693327752012, 0.386824241874752, 0.03496267465472304, 0.010638421710229424, 0.04235331218891159, 0.1643454618530067, 0.35906027752273645, 0.02865091311905997, 0.002690987956806161, 0.010649558734693455, 0.042352985281223145, 1.8748698752238893]
+js_param_list:
+[1.4984730901682581, 0.2683235732843003, 0.04958604921625643, 0.010778911066837874, 0.002515988105136827, 0.0006044757002238476, 0.00015185584462366542, 3.125311773258917e-05, 9.291434369348796e-06, 2.403748311446711e-07, 2.2523717069928906e-06, 3.05053520044847e-08, 5.350493385841079e-06, 6.700927991514511e-07, 1.7557961942484982e-07, 2.1026192189821815, 0.45659539551001177, 0.5084291166478593, 0.5101433777932717, 0.508929468342814, 0.5094047352848247, 0.5180348597519879, 0.2931382886776165, 0.11863554595409948, 0.029916657741284044, 0.0723272689470671, 0.00924226226738377, 0.01679713209566103, 0.05600879079504824, 0.004060626606009455, 0.00428490928954925, 0.01680431404310862, 0.049292483042490044, 0.0025609913043603437, 0.0010654431936954005, 0.004274791409264434, 0.01680445136301286, 0.046263886113338384, 0.00205286507975312, 0.0002642645241999461, 0.0010691611662199975, 0.00427478016792876, 0.3638833699602575]
+ptq_acc_list:
+[10.0, 10.19, 65.93, 87.82, 89.82, 90.02, 90.07, 90.04, 90.18, 90.19, 90.17, 90.15, 90.17, 90.16, 90.16, 10.0, 16.77, 59.02, 62.25, 64.35, 61.93, 61.73, 10.55, 59.85, 79.42, 81.95, 87.93, 87.04, 84.63, 89.18, 89.48, 87.21, 86.08, 89.5, 89.82, 89.43, 86.75, 86.71, 89.59, 90.05, 89.86, 89.58, 78.65]
+acc_loss_list:
+[0.889061459951187, 0.8869536276902596, 0.2685822054581761, 0.025737741291324688, 0.0035500332815620967, 0.001331262480585806, 0.0007765697803417727, 0.0011093854004880666, -0.0004437541601953212, -0.0005546927002440333, -0.0003328156201464515, -0.0001109385400488697, -0.0003328156201464515, -0.00022187708009758177, -0.00022187708009758177, 0.889061459951187, 0.8139560683381407, 0.3452407366319059, 0.30940758819613934, 0.2861104947858887, 0.31295762147770134, 0.31517639227867766, 0.8829598402485024, 0.3360328378078544, 0.11892611493232748, 0.09085866429997778, 0.024517417350787593, 0.034390947415131955, 0.061127135566895995, 0.010650099844685975, 0.0073219436432216175, 0.032504992234302275, 0.045041047259818084, 0.007100066563124036, 0.0035500332815620967, 0.007876636343465651, 0.0376081650765476, 0.03805191923674292, 0.006101619702684681, 0.0009984468604393544, 0.0031062791213667756, 0.006212558242733551, 0.12746838251608603]
+
+
+Inception_BN
+Full_acc: 92.850000
+title_list:
+['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
+js_flops_list:
+[7.715147804903006, 1.180638829948032, 0.17330341588504639, 0.03757641893210694, 0.00878982153678008, 0.0021136845718360358, 0.0005197591695297085, 0.00012868349861507722, 3.2617573467141134e-05, 1.95771371272391e-06, 2.329043574392525e-06, 1.9973666121921525e-06, 2.181171911720007e-06, 5.134343259917914e-06, 4.130203057900563e-06, 7.258357623221218, 1.135985132290838, 1.4849991181524151, 1.5395437466854156, 1.449550683059265, 1.3787184235186551, 1.3649998008469453, 1.2069771223431678, 0.48469862476839726, 0.11355577484142695, 0.2937877086812499, 0.03402380401823649, 0.07264058418816968, 0.2264499587330589, 0.013962832913529065, 0.01846691453157432, 0.07264100003421901, 0.19874527130821865, 0.008312257117463705, 0.004663653123973545, 0.018472925707157563, 0.07264083738327363, 0.1861251700828908, 0.006510681819443857, 0.0011782965721076142, 0.004656104987596719, 0.018474123863005152, 1.1891994561788195]
+js_flops_weighted_list:
+[24.492915695935718, 3.7481183848335724, 0.5501781770652018, 0.11929208413536399, 0.02790463168390087, 0.00671021467571623, 0.0016500549105994422, 0.0004085254311050457, 0.00010354947142152921, 6.215061348798991e-06, 7.393904739388733e-06, 6.34094553771341e-06, 6.924463549246638e-06, 1.6299757282582123e-05, 1.3111960763728517e-05, 23.042765459856884, 3.606358397872026, 4.714356630514774, 4.887516889022836, 4.601820156263742, 4.376952324129888, 4.333400460049073, 3.8317333189245306, 1.5387498534822412, 0.36050016024452497, 0.9326739763377017, 0.1080137651989134, 0.23060863506641904, 0.7188999989179939, 0.044327146812426284, 0.05862604219953586, 0.23060995523325012, 0.6309472349997668, 0.026388530455838718, 0.01480547951757709, 0.05864512559502185, 0.23060943887296853, 0.5908827951208971, 0.020669154364788552, 0.0037406825293873503, 0.014781516805181413, 0.05864892932385605, 3.77529540099415]
+js_param_list:
+[5.547292775694633, 1.0830498393821684, 0.11597159954861848, 0.025184683478010235, 0.005889836213039606, 0.001418800737950504, 0.0003470743045252829, 8.729454098248694e-05, 2.0691080441622126e-05, 1.3027382812223066e-06, 5.085435185989181e-06, 2.6182420731614744e-06, 3.434516785198397e-06, 6.510479031259873e-06, 4.896755871723168e-06, 6.431020460639781, 1.2042858915688233, 2.1661643185195403, 2.2217083989716477, 2.12019065583259, 2.0296487615468775, 1.996163205688216, 0.8993658060076932, 0.3600575347311997, 0.08029751063059903, 0.21764768767983095, 0.0235711444064559, 0.05512862455451699, 0.1674148989599356, 0.009286143104694698, 0.014028389319905675, 0.05513198123092901, 0.14665927416585386, 0.005281656984956287, 0.003588795191323551, 0.014033740564019261, 0.05512987087623732, 0.13726783858604585, 0.004046517151685216, 0.0009180459749539567, 0.003582893701133105, 0.01403383522314854, 1.7935260885146647]
+ptq_acc_list:
+[10.0, 10.36, 67.33, 89.38, 92.15, 92.8, 92.81, 92.9, 92.89, 92.84, 92.85, 92.85, 92.85, 92.86, 92.84, 10.0, 12.36, 17.68, 19.75, 24.11, 23.29, 22.15, 10.89, 12.39, 81.29, 33.15, 89.88, 75.48, 28.39, 91.96, 91.1, 85.62, 63.83, 92.39, 92.58, 91.25, 82.3, 57.86, 92.35, 92.82, 92.72, 91.78, 25.49]
+acc_loss_list:
+[0.8922994076467421, 0.8884221863220247, 0.27485191168551426, 0.0373721055465805, 0.007539041464727934, 0.0005385029617662591, 0.0004308023694129461, -0.0005385029617664122, -0.00043080236941309916, 0.00010770059235316, 0.0, 0.0, 0.0, -0.00010770059235331305, 0.00010770059235316, 0.8922994076467421, 0.8668820678513732, 0.8095853527194399, 0.7872913301023156, 0.7403338718362951, 0.7491653204092623, 0.7614431879375336, 0.882714054927302, 0.8665589660743134, 0.12450188476036606, 0.6429725363489499, 0.0319870759289176, 0.18707592891760896, 0.6942380183091007, 0.009585352719439964, 0.01884760366182014, 0.07786752827140539, 0.31254711900915455, 0.004954227248249799, 0.0029079159935379217, 0.017232094776521212, 0.1136241249326871, 0.3768443726440495, 0.005385029617662898, 0.0003231017770597861, 0.0014001077005923043, 0.011523963381798527, 0.7254711900915456]
+
+

ykl/ALL/ptq_weightratio.py

+from torch.serialization import load
+from model import *
+from extract_ratio import *
+from utils import *
+
+import gol
+import openpyxl
+import sys
+import argparse
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+import torch.utils.bottleneck as bn
+import os
+import os.path as osp
+from torch.utils.tensorboard import SummaryWriter
+
+
+def direct_quantize(model, test_loader,device):
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_forward(data).cpu()
+        if i % 500 == 0:
+            break
+    print('direct quantization finish')
+
+
+def full_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        # print(pred)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('\nTest set: Full Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+def quantize_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_inference(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('Test set: Quant Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+if __name__ == "__main__":
+    batch_size = 32
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(device)
+    train_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=True, download=True,
+                         transform=transforms.Compose([
+                             transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                             transforms.RandomHorizontalFlip(),
+                             transforms.ToTensor(),
+                             transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                         ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
+            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+            transforms.ToTensor(),
+            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+        ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    load_ptq = False
+    store_ptq = False
+
+    append = True
+    gol._init()
+
+    excel_path = 'ptq_result_weightratio.xlsx'
+    if os.path.exists(excel_path) and append:
+        workbook = openpyxl.load_workbook(excel_path)
+    else:
+        workbook = openpyxl.Workbook()
+    if 'Sheet' in workbook.sheetnames:
+        workbook.remove(workbook['Sheet'])
+
+    txt_path = 'ptq_result_weightratio.txt'
+    if os.path.exists(txt_path) and append:
+        ft = open(txt_path,'a')
+    else:
+        ft = open(txt_path,'w')
+    model_name_list = ['AlexNet','AlexNet_BN','VGG_16','VGG_19','Inception_BN']
+    for model_name in model_name_list:
+        if model_name in workbook.sheetnames:
+            continue
+
+        model = Model(model_name)
+        full_file = 'ckpt/cifar10_'+model_name+'.pt'
+        model.load_state_dict(torch.load(full_file))
+        model.to(device)
+
+        ptq_file_prefix = 'ckpt/cifar10_'+model_name+'_ptq_'
+
+        model.eval()
+        full_acc = full_inference(model, test_loader, device)
+        # 传入后可变
+        fold_model(model)
+    
+        Mac,Param,layer, par_ratio, flop_ratio,weight_ratio = extract_ratio(model_name)
+        par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
+
+        full_names = []
+        full_params = []
+
+        for name, param in model.named_parameters():
+            if 'conv' in name or 'fc' in name:
+                full_names.append(name)
+                full_params.append(param.data.cpu())
+    
+        #统计每个参数对应层的参数个数
+        full_par_num=[]
+        for name in full_names:
+            prefix = name.rsplit('.',1)[0]
+            cnt = 0
+            for str in full_names:
+                sprefix = str.rsplit('.',1)[0]
+                if prefix == sprefix:
+                    cnt += 1
+            full_par_num.append(cnt)
+    
+        quant_type_list = ['INT','POT','FLOAT']
+        # quant_type_list = ['INT']
+        # quant_type_list = ['POT']
+        # quant_type_list = ['INT','POT']
+        title_list = []
+        js_flops_list = []
+        js_flops_weighted_list = []
+        js_param_list = []
+        ptq_acc_list = []
+        acc_loss_list = []
+
+        for quant_type in quant_type_list:
+            num_bit_list = numbit_list(quant_type)
+            # 对一个量化类别，只需设置一次bias量化表
+            # int由于位宽大，使用量化表开销过大，直接_round即可
+            if quant_type != 'INT':
+                bias_list = build_bias_list(quant_type)
+                gol.set_value(bias_list, is_bias=True)
+
+            for num_bits in num_bit_list:
+                e_bit_list = ebit_list(quant_type,num_bits)
+                for e_bits in e_bit_list:
+                    model_ptq = Model(model_name)
+                    if quant_type == 'FLOAT':
+                        title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
+                    else:
+                        title = '%s_%d' % (quant_type, num_bits)
+                    print('\n'+model_name+': PTQ: '+title)
+                    title_list.append(title)
+
+                    # 设置量化表
+                    if quant_type != 'INT':
+                        plist = build_list(quant_type, num_bits, e_bits)
+                        gol.set_value(plist)
+                    # 判断是否需要载入
+                    if load_ptq is True and osp.exists(ptq_file_prefix + title + '.pt'):
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.load_state_dict(torch.load(ptq_file_prefix + title + '.pt'))
+                        model_ptq.to(device)
+                        print('Successfully load ptq model: ' + title)
+                    else:
+                        model_ptq.load_state_dict(torch.load(full_file))
+                        model_ptq.to(device)
+                        model_ptq.quantize(quant_type,num_bits,e_bits)
+                        model_ptq.eval()
+                        direct_quantize(model_ptq, train_loader, device)
+                        if store_ptq:
+                            torch.save(model_ptq.state_dict(), ptq_file_prefix + title + '.pt')
+
+                    model_ptq.freeze()
+                    ptq_acc = quantize_inference(model_ptq, test_loader, device)
+                    ptq_acc_list.append(ptq_acc)
+                    acc_loss = (full_acc - ptq_acc) / full_acc
+                    acc_loss_list.append(acc_loss)
+
+                    #将量化后分布反量化到全精度相同的scale
+                    model_ptq.fakefreeze()
+                    # 获取计算量/参数量下的js-div
+                    js_flops = 0.
+                    js_param = 0.
+                    for name, param in model_ptq.named_parameters():
+                        if 'conv' not in name and 'fc' not in name:
+                            continue
+                            
+                        prefix = name.rsplit('.',1)[0]
+                        layer_idx = layer.index(prefix)
+                        name_idx = full_names.index(name)
+
+                        layer_idx = layer.index(prefix)
+                        ptq_param = param.data.cpu()
+
+                        js = js_div(ptq_param,full_params[name_idx])
+                        if full_par_num[name_idx] != 1:
+                            if 'weight' in name:
+                                js *= weight_ratio[name_idx]
+                            else:
+                                js *= 1-weight_ratio[name_idx]
+
+                        js = js.item()
+                        if js < 0.:
+                            js = 0.
+                        js_flops = js_flops + js * flop_ratio[layer_idx]
+                        js_param = js_param + js * par_ratio[layer_idx]
+                    js_flops_weighted = js_flops * torch.log10(torch.tensor(Mac)).item()
+                    js_flops_list.append(js_flops)
+                    js_flops_weighted_list.append(js_flops_weighted)
+                    js_param_list.append(js_param)
+
+                    print(title + ': js_flops: %f js_flops_weighted: %f js_param: %f acc_loss: %f' % (js_flops,js_flops_weighted, js_param, acc_loss))
+        
+        # 写入xlsx
+        worksheet = workbook.create_sheet(model_name)
+        worksheet.cell(row=1,column=1,value='FP32-acc')
+        worksheet.cell(row=1,column=2,value=full_acc)
+        worksheet.cell(row=1,column=3,value='Mac')
+        worksheet.cell(row=1,column=4,value=Mac)
+        worksheet.cell(row=1,column=5,value='Param')
+        worksheet.cell(row=1,column=6,value=Param)
+
+        worksheet.cell(row=3,column=1,value='title')
+        worksheet.cell(row=3,column=2,value='js_flops')
+        worksheet.cell(row=3,column=3,value='js_flops_weighted')
+        worksheet.cell(row=3,column=4,value='js_param')
+        worksheet.cell(row=3,column=5,value='ptq_acc')
+        worksheet.cell(row=3,column=6,value='acc_loss')
+        for i in range(len(title_list)):
+            worksheet.cell(row=i+4, column=1, value=title_list[i])
+            worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
+            worksheet.cell(row=i+4, column=3, value=js_flops_weighted_list[i])
+            worksheet.cell(row=i+4, column=4, value=js_param_list[i])
+            worksheet.cell(row=i+4, column=5, value=ptq_acc_list[i])
+            worksheet.cell(row=i+4, column=6, value=acc_loss_list[i])
+
+        if 'Sheet' in workbook.sheetnames:
+            workbook.remove(workbook['Sheet'])
+        workbook.save(excel_path)
+        
+        print(model_name,file=ft)
+        print('Full_acc: %f'%full_acc,file=ft)
+        print('title_list:',file=ft)
+        print(title_list,file=ft)
+        print('js_flops_list:',file=ft)
+        print(js_flops_list, file=ft)
+        print('js_flops_weighted_list:',file=ft)
+        print(js_flops_weighted_list, file=ft)
+        print('js_param_list:',file=ft)
+        print(js_param_list, file=ft)
+        print('ptq_acc_list:',file=ft)
+        print(ptq_acc_list, file=ft)
+        print('acc_loss_list:',file=ft)
+        print(acc_loss_list, file=ft)
+        print("\n",file=ft)
+
+    ft.close()
+

ykl/ALL/ptq_weightratio.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#SBATCH -J ALL-weightratio            # The job name
+#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
+#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
+
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 3-00:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-long             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000|T4"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+python ptq_weightratio.py
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL/train.py

+from model import *
+
+import torch
+import torch.nn as nn
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+import os
+import os.path as osp
+
+
+def train(model, device, train_loader, optimizer, epoch):
+    model.train()
+    lossLayer = torch.nn.CrossEntropyLoss()
+    for batch_idx, (data, target) in enumerate(train_loader):
+        data, target = data.to(device), target.to(device)
+        optimizer.zero_grad()
+        output = model(data)
+        loss = lossLayer(output, target)
+        loss.backward()
+        optimizer.step()
+
+        if batch_idx % 50 == 0:
+            print('Train Epoch: {} [{}/{}]\tLoss: {:.6f}'.format(
+                epoch, batch_idx * len(data), len(train_loader.dataset), loss.item()
+            ))
+
+def test(model, device, test_loader):
+    model.eval()
+    test_loss = 0
+    correct = 0
+    lossLayer = torch.nn.CrossEntropyLoss(reduction='sum')
+    for data, target in test_loader:
+        data, target = data.to(device), target.to(device)
+        output = model(data)
+        test_loss += lossLayer(output, target).item()
+        pred = output.argmax(dim=1, keepdim=True)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    
+    test_loss /= len(test_loader.dataset)
+
+    print('\nTest set: Average loss: {:.4f}, Accuracy: {:.2f}%\n'.format(
+        test_loss, 100. * correct / len(test_loader.dataset)
+    ))
+
+epochs_cfg_table = {
+    'AlexNet'       : [20, 30, 20, 20, 10],
+    'AlexNet_BN'    : [15, 20, 20, 20, 10, 10],
+    'VGG_16'        : [25, 30, 30, 20, 20, 10, 10],  
+    'VGG_19'        : [30, 40, 30, 20, 20, 10, 10],
+    'Inception_BN'  : [20, 30, 30, 20, 20, 10, 10]
+}
+lr_cfg_table = {
+    'AlexNet'       : [0.01, 0.005, 0.001, 0.0005, 0.0001],
+    'AlexNet_BN'    : [0.01, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'VGG_16'        : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],  
+    'VGG_19'        : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001],
+    'Inception_BN'  : [0.01, 0.008, 0.005, 0.002, 0.001, 0.0005, 0.0001]
+}
+if __name__ == "__main__":
+    batch_size = 32
+    seed = 1
+    momentum = 0.5
+    save_model = True
+    append = True
+    torch.manual_seed(seed)
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+    train_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=True, download=True,
+                       transform=transforms.Compose([
+                           transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+                           transforms.RandomHorizontalFlip(),
+                           transforms.ToTensor(),
+                           transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+                       ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    test_loader = torch.utils.data.DataLoader(
+        datasets.CIFAR10('../data', train=False, transform=transforms.Compose([
+            transforms.Resize((32, 32), interpolation=InterpolationMode.BICUBIC),
+            transforms.ToTensor(),
+            transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
+        ])),
+        batch_size=batch_size, shuffle=True, num_workers=1, pin_memory=True
+    )
+
+    if save_model:
+        if not osp.exists('ckpt'):
+            os.makedirs('ckpt')
+
+    model_name_list = ['AlexNet', 'AlexNet_BN', 'VGG_16', 'VGG_19', 'Inception_BN']
+    for model_name in model_name_list:
+        save_path = 'ckpt/cifar10_'+model_name+'.pt'
+        if os.path.exists(save_path) and append:
+            continue
+        model = Model(model_name).to(device)
+
+        epoch_start = 1
+        epochs_cfg = epochs_cfg_table[model_name]
+        lr_cfg = lr_cfg_table[model_name]
+        for epochs,lr in zip(epochs_cfg,lr_cfg):
+            optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum)
+            epoch_end = epoch_start+epochs
+            for epoch in range(epoch_start,epoch_end):
+                train(model, device, train_loader, optimizer, epoch)
+                test(model, device, test_loader)
+            epoch_start += epochs
+        
+        if save_model:
+            torch.save(model.state_dict(), save_path)
\ No newline at end of file

ykl/ALL/train.slurm

+#!/bin/bash
+
+#- Job parameters
+
+# (TODO)
+# Please modify job name
+
+#SBATCH -J ALL              # The job name
+#SBATCH -o ret/ret-%j.out        # Write the standard output to file named 'ret-<job_number>.out'
+#SBATCH -e ret/ret-%j.err        # Write the standard error to file named 'ret-<job_number>.err'
+
+
+#- Resources
+
+# (TODO)
+# Please modify your requirements
+
+#SBATCH -p nv-gpu                    # Submit to 'nv-gpu' Partitiion
+#SBATCH -t 3-00:00:00                # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
+#SBATCH --nodes=1                    # Request N nodes
+#SBATCH --gres=gpu:1                 # Request M GPU per node
+#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
+#SBATCH --qos=gpu-long             # Request QOS Type
+
+###
+### The system will alloc 8 or 16 cores per gpu by default.
+### If you need more or less, use following:
+### #SBATCH --cpus-per-task=K            # Request K cores
+###
+### 
+### Without specifying the constraint, any available nodes that meet the requirement will be allocated
+### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
+###
+### #SBATCH --nodelist=gpu-v00           # Request a specific list of hosts 
+### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
+###
+
+# set constraint for RTX8000 to meet my cuda
+#SBATCH --constraint="Ampere|RTX8000|T4"
+
+#- Log information
+
+echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
+echo "Job run at:"
+echo "$(hostnamectl)"
+
+#- Load environments
+source /tools/module_env.sh
+module list                       # list modules loaded
+
+##- Tools
+module load cluster-tools/v1.0
+module load slurm-tools/v1.0
+module load cmake/3.15.7
+module load git/2.17.1
+module load vim/8.1.2424
+
+##- language
+module load python3/3.6.8
+
+##- CUDA
+# module load cuda-cudnn/10.2-7.6.5
+# module load cuda-cudnn/11.2-8.2.1
+module load cuda-cudnn/11.1-8.2.1
+
+##- virtualenv
+# source xxxxx/activate
+
+echo $(module list)              # list modules loaded
+echo $(which gcc)
+echo $(which python)
+echo $(which python3)
+
+cluster-quota                    # nas quota
+
+nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
+
+#- Warning! Please not change your CUDA_VISIBLE_DEVICES
+#- in `.bashrc`, `env.sh`, or your job script
+echo "Use GPU ${CUDA_VISIBLE_DEVICES}"                              # which gpus
+#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
+
+#- Job step
+# [EDIT HERE(TODO)]
+python train.py
+
+#- End
+echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"

ykl/ALL/utils.py

+import torch
+import torch.nn as nn
+
+def ebit_list(quant_type, num_bits):
+    if quant_type == 'FLOAT':
+        e_bit_list = list(range(1,num_bits-1))
+    else:
+        e_bit_list = [0]
+    return e_bit_list
+
+
+def numbit_list(quant_type):
+    if quant_type == 'INT':
+        num_bit_list = list(range(2,17))
+        # num_bit_list = [4,5]
+    elif quant_type == 'POT':
+        num_bit_list = list(range(2,9))
+        # num_bit_list = [5]
+    else:
+        num_bit_list = list(range(2,9))
+        # num_bit_list = [8]
+    
+    return num_bit_list     
+
+def build_bias_list(quant_type):
+    if quant_type == 'POT':
+        return build_pot_list(8)
+    else:
+        return build_float_list(16,7)
+    
+def build_list(quant_type, num_bits, e_bits):
+    if quant_type == 'POT':
+        return build_pot_list(num_bits)
+    else:
+        return build_float_list(num_bits,e_bits)
+
+def build_pot_list(num_bits):
+    plist = [0.]
+    for i in range(-2 ** (num_bits-1) + 2, 1): 
+        # i最高到0，即pot量化最大值为1
+        plist.append(2. ** i)
+        plist.append(-2. ** i)
+    plist = torch.Tensor(list(set(plist)))
+    # plist = plist.mul(1.0 / torch.max(plist))
+    return plist
+
+def build_float_list(num_bits,e_bits):
+    m_bits = num_bits - 1 - e_bits
+    plist = [0.]
+    # 相邻尾数的差值
+    dist_m = 2 ** (-m_bits)
+    e = -2 ** (e_bits - 1) + 1
+    for m in range(1, 2 ** m_bits):
+        frac = m * dist_m   # 尾数部分
+        expo = 2 ** e       # 指数部分
+        flt = frac * expo
+        plist.append(flt)
+        plist.append(-flt)
+
+    for e in range(-2 ** (e_bits - 1) + 2, 2 ** (e_bits - 1) + 1):
+        expo = 2 ** e
+        for m in range(0, 2 ** m_bits):
+            frac = 1. + m * dist_m
+            flt = frac * expo
+            plist.append(flt)
+            plist.append(-flt)
+    plist = torch.Tensor(list(set(plist)))
+    return plist
+
+#此处不必cfg，直接取同前缀同后缀即可。将relu一起考虑进去
+def fold_ratio(layer, par_ratio, flop_ratio):
+    idx = -1
+    for name in layer:
+        if 'conv' in name:
+            conv_idx = layer.index(name)
+            [prefix,suffix] = name.split('conv')
+            bn_name = prefix+'bn'+suffix
+            relu_name = prefix+'relu'+suffix
+            if bn_name in layer:
+                bn_idx = layer.index(bn_name)
+                par_ratio[conv_idx]+=par_ratio[bn_idx]
+                flop_ratio[conv_idx]+=flop_ratio[bn_idx]
+                if relu_name in layer:
+                    relu_idx = layer.index(relu_name)
+                    par_ratio[conv_idx]+=par_ratio[relu_idx]
+                    flop_ratio[conv_idx]+=flop_ratio[bn_idx]
+    return par_ratio,flop_ratio
+
+def fold_model(model):
+    for name, module in model.named_modules():
+        if 'conv' in name:
+            [prefix,suffix] = name.split('conv')
+            bn_name = prefix+'bn'+suffix
+            if hasattr(model,bn_name):
+                bn_layer = getattr(model,bn_name)
+                fold_bn(module,bn_layer)
+
+def fold_bn(conv, bn):
+    # 获取 BN 层的参数
+    mean = bn.running_mean
+    var = bn.running_var
+    eps = bn.eps
+    std = torch.sqrt(var + eps)
+
+    if bn.affine:
+        gamma_ = bn.weight / std
+        weight = conv.weight * gamma_.view(conv.out_channels, 1, 1, 1)
+        if conv.bias is not None:
+            bias = gamma_ * conv.bias - gamma_ * mean + bn.bias
+        else:
+            bias = bn.bias - gamma_ * mean
+    else:
+        gamma_ = 1 / std
+        weight = conv.weight * gamma_
+        if conv.bias is not None:
+            bias = gamma_ * conv.bias - gamma_ * mean
+        else:
+            bias = -gamma_ * mean
+
+    # 设置新的 weight 和 bias
+    conv.weight.data = weight.data
+    if conv.bias is not None:
+        conv.bias.data = bias.data
+    else:
+        conv.bias = torch.nn.Parameter(bias)

ykl/Inception_BN/model.py

@@ -159,9 +159,10 @@ def model_quantize(model,cfg_table,quant_type,num_bits,e_bits):
 
 # 支持选择反量化位置，进行debug。最后release时可取消
 # end_pos为-1时表示到最后才反量化，否则在i层反量化 
+# 增加了func='fakefreeze'
 def model_utils(model,cfg_table,func,x=None):
     end_flag = False
-    end_pos = 6
+    end_pos = -1
     last_qo = None
     for i in range(len(cfg_table)):
         cfg = cfg_table[i]
@@ -171,7 +172,7 @@ def model_utils(model,cfg_table,func,x=None):
                 x = last_qo.dequantize_tensor(x)
         if cfg[0] == 'Inc':
             if end_flag:
-                if func != 'freeze':
+                if func == 'inference' or func == 'forward':
                     x = inc_forward(model,cfg[1],x)
                 continue
             x,last_qo = inc_utils(model,cfg[1],func,x,last_qo)
@@ -179,7 +180,7 @@ def model_utils(model,cfg_table,func,x=None):
             if end_flag:
                 name = 'conv%d'%i
                 layer = getattr(model,name)
-                if func != 'freeze':
+                if func == 'inference' or func == 'forward':
                     x = layer(x)
                 continue
             qname = 'q_conv%d'%i
@@ -191,14 +192,16 @@ def model_utils(model,cfg_table,func,x=None):
                 if cfg[2]:
                     x = qlayer.qi.quantize_tensor(x)
                 x = qlayer.quantize_inference(x)
-            else: #freeze
+            elif func == 'freeze':
                 qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
             last_qo = qlayer.qo
         elif cfg[0] == 'R':
             if end_flag:
                 name = 'relu%d'%i
                 layer = getattr(model,name)
-                if func != 'freeze':
+                if func == 'inference' or func == 'forward':
                     x = layer(x)
                 continue
             qname = 'q_relu%d'%i
@@ -207,13 +210,15 @@ def model_utils(model,cfg_table,func,x=None):
                 x = qlayer(x)
             elif func == 'inference':
                 x = qlayer.quantize_inference(x)
-            else: #freeze
+            elif func == 'freeze':
                 qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
         elif cfg[0] == 'MP':
             if end_flag:
                 name = 'pool%d'%i
                 layer = getattr(model,name)
-                if func != 'freeze':
+                if func == 'inference' or func == 'forward':
                     x = layer(x)
                 continue
             qname = 'q_pool%d'%i
@@ -222,13 +227,15 @@ def model_utils(model,cfg_table,func,x=None):
                 x = qlayer(x)
             elif func == 'inference':
                 x = qlayer.quantize_inference(x)
-            else: #freeze
+            elif func == 'freeze':
                 qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
         elif cfg[0] == 'AAP':
             if end_flag:
                 name = 'aap%d'%i
                 layer = getattr(model,name)
-                if func != 'freeze':
+                if func == 'inference' or func == 'forward':
                     x = layer(x)
                 continue
             qname = 'q_aap%d'%i
@@ -237,11 +244,13 @@ def model_utils(model,cfg_table,func,x=None):
                 x = qlayer(x)
             elif func == 'inference':
                 x = qlayer.quantize_inference(x)
-            else: #freeze
+            elif func == 'freeze':
                 qlayer.freeze(last_qo)
+            elif func == 'fakefreeze':
+                qlayer.fakefreeze()
             last_qo = qlayer.qo
         elif cfg[0] == 'F':
-            if func != 'freeze':
+            if func == 'inference' or func == 'forward':
                 x = torch.flatten(x,start_dim=1)
     
     if func == 'inference' and not end_flag:
@@ -351,8 +360,10 @@ def inc_utils(model,inc_idx,func,x=None,qo=None):
                     tmp = qlayer(tmp)
                 elif func == 'inference':
                     tmp = qlayer.quantize_inference(tmp)
-                else: #freeze
+                elif func == 'freeze':
                     qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
             elif cfg[0] == 'R':
                 qname = qprefix+'relu%d'%j
                 qlayer = getattr(model,qname)
@@ -360,8 +371,10 @@ def inc_utils(model,inc_idx,func,x=None,qo=None):
                     tmp = qlayer(tmp)
                 elif func == 'inference':
                     tmp = qlayer.quantize_inference(tmp)
-                else: #freeze
+                elif func == 'freeze':
                     qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
             else:
                 qname = qprefix+'conv%d'%j
                 qlayer = getattr(model,qname)
@@ -369,8 +382,10 @@ def inc_utils(model,inc_idx,func,x=None,qo=None):
                     tmp = qlayer(tmp)
                 elif func == 'inference':
                     tmp = qlayer.quantize_inference(tmp)
-                else: #freeze
+                elif func == 'freeze':
                     qlayer.freeze(last_qo)
+                elif func == 'fakefreeze':
+                    qlayer.fakefreeze()
                 last_qo = qlayer.qo
         
         outs.append(tmp)
@@ -411,6 +426,8 @@ class Inception_BN(nn.Module):
     def quantize_inference(self,x):
         return model_utils(self,self.cfg_table,func='inference',x=x)
 
+    def fakefreeze(self):
+        model_utils(self,self.cfg_table,func='fakefreeze')
 
 if __name__ == "__main__":
     model = Inception_BN()

ykl/Inception_BN/module.py

@@ -8,6 +8,50 @@ from torch.autograd import Variable
 
 from function import FakeQuantize
 
+def mid_ratio(x):
+    x = x.view(-1)
+    std = torch.std(x)
+    max = 3*std#.item()
+    min = 3*(-std)#.item()
+    print("%f %f"%(max,min))
+    max = torch.max(torch.abs(max),torch.abs(min))
+    mid = max/2
+    cond = torch.logical_and(x>=-mid,x<=mid)
+    cnt = torch.sum(cond).item()
+    ratio = cnt/len(x)
+    return ratio
+
+
+def pearson_corr(tensor1, tensor2):
+    """
+    计算tensor1和tensor2的Pearson相关系数
+    """
+    # 将tensor1和tensor2展平为二维
+    tensor1 = tensor1.view(-1, tensor1.size(-1))
+    tensor2 = tensor2.view(-1, tensor2.size(-1))
+    
+    # 计算tensor1和tensor2的均值
+    tensor1_mean = torch.mean(tensor1, dim=0)
+    tensor2_mean = torch.mean(tensor2, dim=0)
+    
+    # 计算centered tensor
+    tensor1_c = tensor1 - tensor1_mean
+    tensor2_c = tensor2 - tensor2_mean
+    
+    # 计算covariance matrix
+    cov_mat = torch.matmul(tensor1_c.t(), tensor2_c) / (tensor1.size(0) - 1)
+    
+    # 计算相关系数
+    corr_mat = cov_mat / torch.std(tensor1, dim=0) / torch.std(tensor2, dim=0)
+    pearson = torch.mean(corr_mat)
+    return pearson.item()
+
+def cos_sim(a,b):
+    a = a.view(-1)
+    b = b.view(-1)
+    cossim = torch.cosine_similarity(a, b, dim=0, eps=1e-6)
+    # cossim = (cossim-0.97)/(1-0.97)
+    return cossim.item()
 
 def js_div(p_output, q_output, get_softmax=True):
     """
@@ -176,6 +220,8 @@ class QModule(nn.Module):
     def quantize_inference(self, x):
         raise NotImplementedError('quantize_inference should be implemented.')
 
+    def fakefreeze(self):
+        pass
 
 """
 QModule  量化卷积
@@ -226,7 +272,12 @@ class QConv2d(QModule):
             self.conv_module.bias.data = quantize_tensor(self.quant_type,
                                                         self.conv_module.bias.data, scale=self.qi.scale * self.qw.scale,
                                                         zero_point=0.,qmax=self.bias_qmax, is_bias=True)
-        
+
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        if self.conv_module.bias is not None:
+            self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
     def forward(self, x):  # 前向传播,输入张量,x为浮点型数据
         if hasattr(self, 'qi'):
             self.qi.update(x)
@@ -292,6 +343,11 @@ class QLinear(QModule):
                                                         self.fc_module.bias.data, scale=self.qi.scale * self.qw.scale,
                                                         zero_point=0., qmax=self.bias_qmax, is_bias=True)
 
+    def fakefreeze(self):
+        self.fc_module.weight.data = self.qw.dequantize_tensor(self.fc_module.weight.data)
+        if self.fc_module.bias is not None:
+            self.fc_module.bias.data = dequantize_tensor(self.fc_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
     def forward(self, x):
         if hasattr(self, 'qi'):
             self.qi.update(x)
@@ -483,6 +539,10 @@ class QConvBNReLU(QModule):
                                     zero_point=0., qmax=self.bias_qmax,is_bias=True)
             self.conv_module.bias = torch.nn.Parameter(bias)
 
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
     def forward(self, x):
 
         if hasattr(self, 'qi'):
@@ -601,6 +661,10 @@ class QConvBN(QModule):
                                     zero_point=0., qmax=self.bias_qmax,is_bias=True)
             self.conv_module.bias = torch.nn.Parameter(bias)
 
+    def fakefreeze(self):
+        self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
+        self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
+
     def forward(self, x):
 
         if hasattr(self, 'qi'):

ykl/Inception_BN/ptq.py

@@ -83,7 +83,7 @@ if __name__ == "__main__":
     model.to(device)
 
     load_ptq = True
-    store_ptq = False
+    store_ptq = True
     ptq_file_prefix = 'ckpt/cifar10_Inception_BN_ptq_'
 
     model.eval()
@@ -96,12 +96,18 @@ if __name__ == "__main__":
 
     full_names = []
     full_params = []
+    # full_mid_ratios = []
+    # full_params_norm = []
 
     for name, param in model.named_parameters():
         if 'conv' in name or 'fc' in name:
             full_names.append(name)
-            param_norm = F.normalize(param.data.cpu(),p=2,dim=-1)
-            full_params.append(param_norm)
+            # param_norm = F.normalize(param.data.cpu(),p=2,dim=-1)
+            full_params.append(param.data.cpu())
+            # full_mr = mid_ratio(param.data.cpu())
+            # full_mid_ratios.append(full_mr)
+            # print(name+':%f'%full_mr)
+            # full_params_norm.append(param_norm)
             writer.add_histogram(tag='Full_' + name + '_data', values=param.data)
     
     #统计每个参数对应层的参数个数
@@ -120,9 +126,10 @@ if __name__ == "__main__":
     # input()
     
     gol._init()
-    # quant_type_list = ['INT','POT','FLOAT']
+    quant_type_list = ['INT','POT','FLOAT']
     # quant_type_list = ['INT']
-    quant_type_list = ['POT']
+    # quant_type_list = ['POT']
+    # quant_type_list = ['INT','POT']
     title_list = []
     js_flops_list = []
     js_param_list = []
@@ -151,8 +158,11 @@ if __name__ == "__main__":
                 # 设置量化表
                 if quant_type != 'INT':
                     plist = build_list(quant_type, num_bits, e_bits)
+                    # list_mid_ratio = mid_ratio(plist)
                     gol.set_value(plist)
-
+                # else:
+                #     list_mid_ratio = 0.5
+                # print('list_mid_ratio:%f'%list_mid_ratio)
                 # 判断是否需要载入
                 if load_ptq is True and osp.exists(ptq_file_prefix + title + '.pt'):
                     model_ptq.quantize(quant_type,num_bits,e_bits)
@@ -174,6 +184,7 @@ if __name__ == "__main__":
                 acc_loss = (full_acc - ptq_acc) / full_acc
                 acc_loss_list.append(acc_loss)
 
+                model_ptq.fakefreeze()
                 # 获取计算量/参数量下的js-div
                 js_flops = 0.
                 js_param = 0.
@@ -188,10 +199,24 @@ if __name__ == "__main__":
                     layer_idx = layer.index(prefix)
                     ptq_param = param.data.cpu()
                     # 取L2范数
-                    ptq_norm = F.normalize(ptq_param,p=2,dim=-1)
+                    # ptq_norm = F.normalize(ptq_param,p=2,dim=-1)
                     writer.add_histogram(tag=title +':'+ name + '_data', values=ptq_param)
-                    js = js_div(ptq_norm,full_params[name_idx])
+                    js = js_div(ptq_param,full_params[name_idx])
                     js /= full_par_num[name_idx]
+                    # if 'weight' in name:
+                        #获取量化前后分布的相关系数，范围[0,1]，度量分布趋势相似度
+                        # coeff = pearson_corr(ptq_param,full_params[name_idx])
+                        # print(name+':%f'%coeff)
+                        # js /= coeff
+                        
+                        # cossim = cos_sim(ptq_param,full_params[name_idx])
+                        # print(name+':%f'%cossim)
+                        # js /= cossim
+
+                        # mr_dist = torch.abs(torch.tensor(full_mid_ratios[name_idx]-list_mid_ratio))
+                        # print(name+":%f"%mr_dist)
+                        # js /= mr_dist
+                        
                     js = js.item()
                     if js < 0.:
                         js = 0.

ykl/Inception_BN/ptq_result.txt

 title_list:
-POT_2 POT_3 POT_4 POT_5 POT_6 POT_7 POT_8
+INT_2 INT_3 INT_4 INT_5 INT_6 INT_7 INT_8 INT_9 INT_10 INT_11 INT_12 INT_13 INT_14 INT_15 INT_16 POT_2 POT_3 POT_4 POT_5 POT_6 POT_7 POT_8 FLOAT_3_E1 FLOAT_4_E1 FLOAT_4_E2 FLOAT_5_E1 FLOAT_5_E2 FLOAT_5_E3 FLOAT_6_E1 FLOAT_6_E2 FLOAT_6_E3 FLOAT_6_E4 FLOAT_7_E1 FLOAT_7_E2 FLOAT_7_E3 FLOAT_7_E4 FLOAT_7_E5 FLOAT_8_E1 FLOAT_8_E2 FLOAT_8_E3 FLOAT_8_E4 FLOAT_8_E5 FLOAT_8_E6
 js_flops_list:

ykl/Inception_BN/try.py

+import torch
+import torch.nn.functional as F
+
+
+def pearson_corr(tensor1, tensor2):
+    """
+    计算tensor1和tensor2的Pearson相关系数
+    """
+    # 将tensor1和tensor2展平为二维
+    tensor1 = tensor1.view(-1, tensor1.size(-1))
+    tensor2 = tensor2.view(-1, tensor2.size(-1))
+    
+    # 计算tensor1和tensor2的均值
+    tensor1_mean = torch.mean(tensor1, dim=0)
+    tensor2_mean = torch.mean(tensor2, dim=0)
+    
+    # 计算centered tensor
+    tensor1_c = tensor1 - tensor1_mean
+    tensor2_c = tensor2 - tensor2_mean
+    
+    # 计算covariance matrix
+    cov_mat = torch.matmul(tensor1_c.t(), tensor2_c) / (tensor1.size(0) - 1)
+    print(cov_mat)
+    print('----')
+    # 计算相关系数
+    corr_mat = cov_mat / torch.std(tensor1, dim=0) / torch.std(tensor2, dim=0)
+    print(corr_mat)
+    pearson = torch.mean(corr_mat)
+    return pearson
+
+def cos_sim(a,b):
+    a = a.view(-1)
+    b = b.view(-1)
+    cossim = torch.cosine_similarity(a, b, dim=0, eps=1e-6)
+    # cossim = (cossim-0.975)/(1-0.975)
+    return cossim
+
+def mid_ratio(x):
+    x = x.view(-1)
+    max = torch.max(x)#.item()
+    min = torch.min(x)#.item()
+    max = torch.max(torch.abs(max),torch.abs(min))
+    mid = max/2
+    cond = torch.logical_and(x>=-mid,x<=mid)
+    print(cond)
+    cnt = torch.sum(cond).item()
+    print(cnt)
+    ratio = cnt/len(x)
+    print(ratio)
+
+if __name__ == "__main__":
+    # 创建两个3维的tensor
+    x = torch.tensor([[1, 2, 3], [2, 3, 4], [3, 4, 5]]) 
+    y = torch.tensor([[2, 3, 4], [3, 4, 5], [4, 5, 6]])
+    z = torch.tensor([-3,-2,-1,1,2,3])
+    x = x.float()
+    y = y.float()
+    # x = F.normalize(x,p=2,dim=-1)
+    # y = F.normalize(y,p=2,dim=-1)
+    # 计算相关系数
+    # r = pearson_corr(x, y)
+    # r = cos_sim(x,y)
+    mid_ratio(y)
+    # 输出相关系数
+    # print(r)
\ No newline at end of file

ykl/Inception_BN/try2.py

+import torch
+from utils import *
+from module import *
+from torch.utils.tensorboard import SummaryWriter
+
+def build_int_list(num_bits):
+    plist = [0.]
+    for i in range(0,2**(num_bits-1)): 
+        # i最高到0，即pot量化最大值为1
+        plist.append(i)
+        plist.append(i)
+    plist = torch.Tensor(list(set(plist)))
+    # plist = plist.mul(1.0 / torch.max(plist))
+    return plist
+
+
+if __name__ == "__main__":
+    writer = SummaryWriter(log_dir='./log')
+    quant_type_list = ['INT','POT','FLOAT']
+    for quant_type in quant_type_list:
+        num_bit_list = numbit_list(quant_type)
+        for num_bits in num_bit_list:
+            e_bit_list = ebit_list(quant_type,num_bits)
+            for e_bits in e_bit_list:
+                if quant_type == 'FLOAT':
+                    title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
+                else:
+                    title = '%s_%d' % (quant_type, num_bits)
+                print('\nPTQ: '+title)
+                # 设置量化表
+                if quant_type != 'INT':
+                    plist = build_list(quant_type, num_bits, e_bits)
+                    list_mid_ratio = mid_ratio(plist)
+                else:
+                    plist = build_int_list(num_bits)
+                    list_mid_ratio = mid_ratio(plist)
+                writer.add_histogram(tag=title, values=plist)
+                print(list_mid_ratio)
+                
+    writer.close()
\ No newline at end of file

ykl/Inception_BN/try3.py

+from torch.serialization import load
+from model import *
+from extract_ratio import *
+from utils import *
+
+import gol
+import openpyxl
+import sys
+import argparse
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+import torch.utils.bottleneck as bn
+import os
+import os.path as osp
+from torch.utils.tensorboard import SummaryWriter
+
+
+def direct_quantize(model, test_loader,device):
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_forward(data).cpu()
+        if i % 500 == 0:
+            break
+    print('direct quantization finish')
+
+
+def full_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        # print(pred)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('\nTest set: Full Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+def quantize_inference(model, test_loader, device):
+    correct = 0
+    for i, (data, target) in enumerate(test_loader, 1):
+        data = data.to(device)
+        output = model.quantize_inference(data).cpu()
+        pred = output.argmax(dim=1, keepdim=True)
+        correct += pred.eq(target.view_as(pred)).sum().item()
+    print('Test set: Quant Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
+    return 100. * correct / len(test_loader.dataset)
+
+
+if __name__ == "__main__":
+    batch_size = 32
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(device)
+
+    model = Inception_BN()
+    writer = SummaryWriter(log_dir='./log')
+    full_file = 'ckpt/cifar10_Inception_BN.pt'
+    model.load_state_dict(torch.load(full_file))
+    model.to(device)
+
+    load_ptq = True
+    store_ptq = False
+    ptq_file_prefix = 'ckpt/cifar10_Inception_BN_ptq_'
+
+    model.eval()
+    # 传入后可变
+    fold_model(model)
+    
+    layer, par_ratio, flop_ratio = extract_ratio()
+    par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
+
+    full_names = []
+    full_params = []
+    full_mid_ratios = []
+    full_params_norm = []
+
+    for name, param in model.named_parameters():
+        if 'conv' in name or 'fc' in name:
+            full_names.append(name)
+            param_norm = F.normalize(param.data.cpu(),p=2,dim=-1)
+            full_params.append(param.data.cpu())
+            full_mr = mid_ratio(param.data.cpu())
+            full_mid_ratios.append(full_mr)
+            print(name+':%f'%full_mr)
+            full_params_norm.append(param_norm)
+            writer.add_histogram(tag='Full_' + name + '_data', values=param.data)
\ No newline at end of file

ykl/Inception_BN/typediff.py

+from torch.serialization import load
+from model import *
+from extract_ratio import *
+from utils import *
+
+import gol
+import openpyxl
+import sys
+import argparse
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+import torch.optim as optim
+from torchvision import datasets, transforms
+from torchvision.transforms.functional import InterpolationMode
+import torch.utils.bottleneck as bn
+import os
+import os.path as osp
+from torch.utils.tensorboard import SummaryWriter
+
+
+def build_int_list(num_bits):
+    plist = [0.]
+    for i in range(0,2**(num_bits-1)): 
+        # i最高到0，即pot量化最大值为1
+        plist.append(i)
+        plist.append(i)
+    plist = torch.Tensor(list(set(plist)))
+    # plist = plist.mul(1.0 / torch.max(plist))
+    return plist
+
+
+def js_div_diff(p_output, q_output, get_softmax=True):
+    """
+    Function that measures JS divergence between target and output logits:
+    """
+    KLDivLoss = nn.KLDivLoss(reduction='sum')
+    if get_softmax:
+        p_output = F.softmax(p_output)
+        q_output = F.softmax(q_output)
+    p_output = p_output.view(-1)
+    q_output = q_output.view(-1)
+    # log_mean_output = ((p_output + q_output)/2).log()
+    log_mean_output = 0.
+    log_mean_output += p_output.log()
+    log_mean_output += (q_output / q_output.size(0)).log()
+    log_mean_output /= 2.0
+    return (KLDivLoss(log_mean_output, p_output) + KLDivLoss(log_mean_output, q_output))/2
+
+
+if __name__ == "__main__":
+    batch_size = 32
+
+    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+    print(device)
+
+
+    model = Inception_BN()
+    full_file = 'ckpt/cifar10_Inception_BN.pt'
+    model.load_state_dict(torch.load(full_file))
+    model.to(device)
+
+    model.eval()
+    # 传入后可变
+    fold_model(model)
+    
+    layer, par_ratio, flop_ratio = extract_ratio()
+    par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
+
+    full_names = []
+    full_params = []
+
+    for name, param in model.named_parameters():
+        if 'conv' in name or 'fc' in name:
+            full_names.append(name)
+            param_norm = F.normalize(param.data.cpu(),p=2,dim=-1)
+            full_params.append(param_norm)
+    
+    #统计每个参数对应层的参数个数
+    full_par_num=[]
+    for name in full_names:
+        prefix = name.rsplit('.',1)[0]
+        cnt = 0
+        for str in full_names:
+            sprefix = str.rsplit('.',1)[0]
+            if prefix == sprefix:
+                cnt += 1
+        full_par_num.append(cnt)
+    
+    gol._init()
+    # quant_type_list = ['INT','POT','FLOAT']
+    # quant_type_list = ['INT']
+    # quant_type_list = ['POT']
+    quant_type_list = ['INT','POT']
+    title_list = []
+    js_flops_list = []
+    js_param_list = []
+
+    for quant_type in quant_type_list:
+        num_bit_list = numbit_list(quant_type)
+        # 对一个量化类别，只需设置一次bias量化表
+        # int由于位宽大，使用量化表开销过大，直接_round即可
+        if quant_type != 'INT':
+            bias_list = build_bias_list(quant_type)
+            gol.set_value(bias_list, is_bias=True)
+        else:
+            bias_list = build_int_list(16)
+            gol.set_value(bias_list, is_bias=True)
+
+        for num_bits in num_bit_list:
+            e_bit_list = ebit_list(quant_type,num_bits)
+            for e_bits in e_bit_list:
+                if quant_type == 'FLOAT':
+                    title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
+                else:
+                    title = '%s_%d' % (quant_type, num_bits)
+                print('\nDIFF: '+title)
+                title_list.append(title)
+
+                # 设置量化表
+                if quant_type != 'INT':
+                    plist = build_list(quant_type, num_bits, e_bits)
+                    gol.set_value(plist)
+                else:
+                    plist = build_int_list(num_bits)
+                    gol.set_value(plist)
+
+
+                # 获取计算量/参数量下的js-div
+                js_flops = 0.
+                js_param = 0.
+                for name, _ in model.named_parameters():
+                    if 'conv' not in name and 'fc' not in name:
+                        continue
+                    if 'weight' in name:
+                        plist = gol.get_value(False)
+                    else:
+                        plist = gol.get_value(True)    
+                    prefix = name.rsplit('.',1)[0]
+                    layer_idx = layer.index(prefix)
+                    name_idx = full_names.index(name)
+
+                    layer_idx = layer.index(prefix)
+                    # ptq_param = param.data.cpu()
+                    # 取L2范数
+                    plist_norm = F.normalize(plist,p=2,dim=-1)
+                    
+                    print(name)
+                    print(plist_norm)
+                    print(full_params[name_idx])
+                    js = js_div_diff(plist_norm,full_params[name_idx])
+                    js /= full_par_num[name_idx]
+                    js = js.item()
+                    if js < 0.:
+                        js = 0.
+                    js_flops = js_flops + js * flop_ratio[layer_idx]
+                    js_param = js_param + js * par_ratio[layer_idx]
+                js_flops_list.append(js_flops)
+                js_param_list.append(js_param)
+
+                print(title + ': js_flops: %f js_param: %f ' % (js_flops, js_param))
+    
+    # 写入xlsx
+    workbook = openpyxl.Workbook()
+    worksheet = workbook.active
+    worksheet.cell(row=3,column=1,value='title')
+    worksheet.cell(row=3,column=2,value='js_flops')
+    worksheet.cell(row=3,column=3,value='js_param')
+    for i in range(len(title_list)):
+        worksheet.cell(row=i+4, column=1, value=title_list[i])
+        worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
+        worksheet.cell(row=i+4, column=3, value=js_param_list[i])
+    
+
+    ft = open('diff_result.txt','w')
+
+    print('title_list:',file=ft)
+    print(" ".join(title_list),file=ft)
+    print('js_flops_list:',file=ft)
+    print(" ".join(str(i) for i in js_flops_list), file=ft)
+    print('js_param_list:',file=ft)
+    print(" ".join(str(i) for i in js_param_list), file=ft)
+
+    ft.close()
+

ykl/Inception_BN/utils.py

@@ -12,8 +12,10 @@ def ebit_list(quant_type, num_bits):
 def numbit_list(quant_type):
     if quant_type == 'INT':
         num_bit_list = list(range(2,17))
+        # num_bit_list = [4,5]
     elif quant_type == 'POT':
         num_bit_list = list(range(2,9))
+        # num_bit_list = [5]
     else:
         num_bit_list = list(range(2,9))
         # num_bit_list = [8]

ykl/MatLabScript/flops.m

 % 导入数据表
-file_data = xlsread('D:\Desktop\ptq_result.xlsx','Sheet','B4:E46');
+file_data = xlsread('D:\Desktop\ptq_result.xlsx','Inception_BN','B4:E46');
 js_flops = file_data(:,1); 
 js_param = file_data(:,2);
 ptq_acc  = file_data(:,3);
 acc_loss = file_data(:,4);
 
+x = js_flops;
+y = acc_loss;
+
 % 定义颜色向量和每个数据点所属的类别
 colors = ['r', 'g', 'm'];
 class = [ones(16,1); 2*ones(6,1); 3*ones(21,1)];
@@ -12,15 +15,25 @@ class = [ones(16,1); 2*ones(6,1); 3*ones(21,1)];
 % 指定拟合模型
 rational_model = fittype('(p1*js_flops.^2 + p2*js_flops + p3) / (q1*js_flops.^2 + q2*js_flops + q3)', 'independent', 'js_flops', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
 
-% 进行拟合
-[fitresult,gof] = fit(js_flops, acc_loss, rational_model);
+%初次拟合
+[fitresult, gof] = fit(x, y, rational_model);
+
+% 确保拟合结果是单调上升的
+tolerance = 0;
+x_range = min(x):0.1:max(x);
+y_fit = fitresult(x_range);
+while any(diff(y_fit) < tolerance)
+    % 如果拟合曲线不是单调上升的，重新拟合
+    [fitresult, gof] = fit(x, y, rational_model);
+    y_fit = fitresult(x_range);
+end
 
 % 可视化数据点和拟合曲线
-scatter(js_flops(1:15), acc_loss(1:15), [], colors(1), 'filled');
+scatter(x(1:15), y(1:15), [], colors(1), 'filled');
 hold on;
-scatter(js_flops(16:22), acc_loss(16:22), [], colors(2), 'filled');
-scatter(js_flops(23:43), acc_loss(23:43), [], colors(3), 'filled');
-plot(fitresult,'k',js_flops,acc_loss);
+scatter(x(16:22), y(16:22), [], colors(2), 'filled');
+scatter(x(23:43), y(23:43), [], colors(3), 'filled');
+plot(fitresult,'k',x,y);
 xlabel('js\_flops');
 ylabel('acc\_loss');
 legend('INT', 'POT', 'FLOAT','ALL', 'Rational-Fit', 'Location', 'Northeast');

ykl/MatLabScript/flops_all.m

+% 导入数据表
+data0 = xlsread('D:\Desktop\ptq_result.xlsx','AlexNet','B4:E46');
+data1 = xlsread('D:\Desktop\ptq_result.xlsx','AlexNet_BN','B4:E46');
+data2 = xlsread('D:\Desktop\ptq_result.xlsx','VGG_16','B4:E46');
+data3 = xlsread('D:\Desktop\ptq_result.xlsx','VGG_19','B4:E46');
+data4 = xlsread('D:\Desktop\ptq_result.xlsx','Inception_BN','B4:E46');
+file_data = vertcat(data0,data1,data2,data3,data4)
+js_flops = file_data(:,1); 
+js_param = file_data(:,2);
+ptq_acc  = file_data(:,3);
+acc_loss = file_data(:,4);
+
+% 指定横纵坐标及多项式次数
+x = js_flops;
+y = acc_loss;
+poly = 4;
+
+% 定义颜色向量和每个数据点所属的类别
+colors = ['r', 'g', 'b','m','o'];
+class = [ones(43,1); 2*ones(43,1); 3*ones(43,1);4*ones(43,1);5*ones(43,1);];
+
+% 指定拟合模型
+if poly == 2
+    rational_model = fittype('(p1*x.^2 + p2*x + p3) / (q1*x.^2 + q2*x + q3)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
+elseif poly == 3
+    rational_model = fittype('(p1*x.^3 + p2*x.^2 + p3*x + p4) / (q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3','p4', 'q1', 'q2', 'q3','q4'});
+elseif poly == 4
+    rational_model = fittype('(p0*x.^4 + p1*x.^3 + p2*x.^2 + p3*x + p4) / (q0*x.^4 + q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p0', 'p1', 'p2', 'p3','p4','q0', 'q1', 'q2', 'q3','q4'});
+end
+
+%初次拟合
+[fitresult, gof] = fit(x, y, rational_model);
+
+% 确保拟合结果是单调上升的
+tolerance = 0;
+x_range = min(x):0.1:max(x);
+y_fit = fitresult(x_range);
+while any(diff(y_fit) < tolerance)
+    % 如果拟合曲线不是单调上升的，重新拟合
+    [fitresult, gof] = fit(x, y, rational_model);
+    y_fit = fitresult(x_range);
+end
+
+% 可视化数据点和拟合曲线
+scatter(x(1:43), y(1:43), [], colors(1), 'filled');
+hold on;
+scatter(x(44:86), y(44:86), [], colors(2), 'filled');
+scatter(x(86:129), y(86:129), [], colors(3), 'filled');
+scatter(x(130:172), y(130:172), [], colors(4), 'filled');
+scatter(x(173:215), y(173:215), [], colors(5), 'filled');
+
+plot(fitresult,'k',x,y);
+xlabel('js\_flops');
+ylabel('acc\_loss')
+legend('AlexNet', 'AlexNet\_BN','VGG\_16','VGG\_19','Inception\_BN','ALL', 'Rational-Fit', 'Location', 'Northeast');
+% 获取评价指标
+SSE=gof.sse;
+R_square = gof.rsquare; 
+RMSE = gof.rmse; 
+
+% 将拟合公式和 R 方显示在图上
+text(0.65, 0.2, sprintf('Goodness of fit:\n SSE:%.4f\n R-square:%.4f\n RMSE:%.4f', SSE, R_square, RMSE), 'Units', 'normalized', 'FontSize', 11);
+
+hold off;
\ No newline at end of file

ykl/MatLabScript/flops_all_weighted.m

+% 导入数据表
+data0 = xlsread('D:\Desktop\ptq_result_weighted.xlsx','AlexNet','B4:F46');
+data1 = xlsread('D:\Desktop\ptq_result_weighted.xlsx','AlexNet_BN','B4:F46');
+data2 = xlsread('D:\Desktop\ptq_result_weighted.xlsx','VGG_16','B4:F46');
+data3 = xlsread('D:\Desktop\ptq_result_weighted.xlsx','VGG_19','B4:F46');
+data4 = xlsread('D:\Desktop\ptq_result_weighted.xlsx','Inception_BN','B4:F46');
+file_data = vertcat(data0,data1,data2,data3,data4)
+js_flops = file_data(:,1); 
+js_flops_weighted = file_data(:,2);
+js_param = file_data(:,3);
+%js_param_weighted = file_data(:,4);
+ptq_acc  = file_data(:,4);
+acc_loss = file_data(:,5);
+
+% 指定横纵坐标及多项式次数
+x = js_flops_weighted;
+y = acc_loss;
+poly = 4;
+
+% 定义颜色向量和每个数据点所属的类别
+colors = ['r', 'g', 'b','m','o'];
+class = [ones(43,1); 2*ones(43,1); 3*ones(43,1);4*ones(43,1);5*ones(43,1);];
+
+% 指定拟合模型
+if poly == 2
+    rational_model = fittype('(p1*x.^2 + p2*x + p3) / (q1*x.^2 + q2*x + q3)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
+elseif poly == 3
+    rational_model = fittype('(p1*x.^3 + p2*x.^2 + p3*x + p4) / (q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3','p4', 'q1', 'q2', 'q3','q4'});
+elseif poly == 4
+    rational_model = fittype('(p0*x.^4 + p1*x.^3 + p2*x.^2 + p3*x + p4) / (q0*x.^4 + q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p0', 'p1', 'p2', 'p3','p4','q0', 'q1', 'q2', 'q3','q4'});
+end
+
+%初次拟合
+[fitresult, gof] = fit(x, y, rational_model);
+
+% 确保拟合结果是单调上升的
+tolerance = 0;
+x_range = min(x):0.1:max(x);
+y_fit = fitresult(x_range);
+while any(diff(y_fit) < tolerance)
+    % 如果拟合曲线不是单调上升的，重新拟合
+    [fitresult, gof] = fit(x, y, rational_model);
+    y_fit = fitresult(x_range);
+end
+
+
+% 可视化数据点和拟合曲线
+scatter(x(1:43), y(1:43), [], colors(1), 'filled');
+hold on;
+scatter(x(44:86), y(44:86), [], colors(2), 'filled');
+scatter(x(86:129), y(86:129), [], colors(3), 'filled');
+scatter(x(130:172), y(130:172), [], colors(4), 'filled');
+scatter(x(173:215), y(173:215), [], colors(5), 'filled');
+
+plot(fitresult,'k',x,y);
+xlabel('js\_flops\_weighted');
+ylabel('acc\_loss')
+legend('AlexNet', 'AlexNet\_BN','VGG\_16','VGG\_19','Inception\_BN','ALL', 'Rational-Fit', 'Location', 'Northeast');
+% 获取评价指标
+SSE=gof.sse;
+R_square = gof.rsquare; 
+RMSE = gof.rmse; 
+
+% 将拟合公式和 R 方显示在图上
+text(0.65, 0.2, sprintf('Goodness of fit:\n SSE:%.4f\n R-square:%.4f\n RMSE:%.4f', SSE, R_square, RMSE), 'Units', 'normalized', 'FontSize', 11);
+
+hold off;
\ No newline at end of file

ykl/MatLabScript/param.m

 % 导入数据表
-file_data = xlsread('D:\Desktop\ptq_result.xlsx','Sheet','B4:E46');
+file_data = xlsread('D:\Desktop\ptq_result.xlsx','Inception_BN','B4:E46');
 js_flops = file_data(:,1); 
 js_param = file_data(:,2);
 ptq_acc  = file_data(:,3);
 acc_loss = file_data(:,4);
 
+x = js_param;
+y = acc_loss;
+
 % 定义颜色向量和每个数据点所属的类别
 colors = ['r', 'g', 'm'];
 class = [ones(16,1); 2*ones(6,1); 3*ones(21,1)];
@@ -12,15 +15,25 @@ class = [ones(16,1); 2*ones(6,1); 3*ones(21,1)];
 % 指定拟合模型
 rational_model = fittype('(p1*js_flops.^2 + p2*js_flops + p3) / (q1*js_flops.^2 + q2*js_flops + q3)', 'independent', 'js_flops', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
 
-% 进行拟合
-[fitresult,gof] = fit(js_param, acc_loss, rational_model);
+%初次拟合
+[fitresult, gof] = fit(x, y, rational_model);
+
+% 确保拟合结果是单调上升的
+tolerance = 0;
+x_range = min(x):0.1:max(x);
+y_fit = fitresult(x_range);
+while any(diff(y_fit) < tolerance)
+    % 如果拟合曲线不是单调上升的，重新拟合
+    [fitresult, gof] = fit(x, y, rational_model);
+    y_fit = fitresult(x_range);
+end
 
 % 可视化数据点和拟合曲线
-scatter(js_param(1:15), acc_loss(1:15), [], colors(1), 'filled');
+scatter(x(1:15), y(1:15), [], colors(1), 'filled');
 hold on;
-scatter(js_param(16:22), acc_loss(16:22), [], colors(2), 'filled');
-scatter(js_param(23:43), acc_loss(23:43), [], colors(3), 'filled');
-plot(fitresult,'k',js_param,acc_loss);
+scatter(x(16:22), y(16:22), [], colors(2), 'filled');
+scatter(x(23:43), y(23:43), [], colors(3), 'filled');
+plot(fitresult,'k',x,y);
 xlabel('js\_param');
 ylabel('acc\_loss');
 legend('INT', 'POT', 'FLOAT','ALL', 'Rational-Fit', 'Location', 'Northeast');

ykl/MatLabScript/param_all.m

+% 导入数据表
+data0 = xlsread('D:\Desktop\ptq_result.xlsx','AlexNet','B4:E46');
+data1 = xlsread('D:\Desktop\ptq_result.xlsx','AlexNet_BN','B4:E46');
+data2 = xlsread('D:\Desktop\ptq_result.xlsx','VGG_16','B4:E46');
+data3 = xlsread('D:\Desktop\ptq_result.xlsx','VGG_19','B4:E46');
+data4 = xlsread('D:\Desktop\ptq_result.xlsx','Inception_BN','B4:E46');
+file_data = vertcat(data0,data1,data2,data3,data4)
+js_flops = file_data(:,1); 
+js_param = file_data(:,2);
+ptq_acc  = file_data(:,3);
+acc_loss = file_data(:,4);
+
+% 指定横纵坐标及多项式次数
+x = js_param;
+y = acc_loss;
+poly = 4;
+
+% 定义颜色向量和每个数据点所属的类别
+colors = ['r', 'g', 'b','m','o'];
+class = [ones(43,1); 2*ones(43,1); 3*ones(43,1);4*ones(43,1);5*ones(43,1);];
+
+% 指定拟合模型
+if poly == 2
+    rational_model = fittype('(p1*x.^2 + p2*x + p3) / (q1*x.^2 + q2*x + q3)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
+elseif poly == 3
+    rational_model = fittype('(p1*x.^3 + p2*x.^2 + p3*x + p4) / (q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3','p4', 'q1', 'q2', 'q3','q4'});
+elseif poly == 4
+    rational_model = fittype('(p0*x.^4 + p1*x.^3 + p2*x.^2 + p3*x + p4) / (q0*x.^4 + q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p0', 'p1', 'p2', 'p3','p4','q0', 'q1', 'q2', 'q3','q4'});
+end
+
+%初次拟合
+[fitresult, gof] = fit(x, y, rational_model);
+
+% 确保拟合结果是单调上升的
+tolerance = 0;
+x_range = min(x):0.1:max(x);
+y_fit = fitresult(x_range);
+while any(diff(y_fit) < tolerance)
+    % 如果拟合曲线不是单调上升的，重新拟合
+    [fitresult, gof] = fit(x, y, rational_model);
+    y_fit = fitresult(x_range);
+end
+
+% 可视化数据点和拟合曲线
+scatter(x(1:43), y(1:43), [], colors(1), 'filled');
+hold on;
+scatter(x(44:86), y(44:86), [], colors(2), 'filled');
+scatter(x(86:129), y(86:129), [], colors(3), 'filled');
+scatter(x(130:172), y(130:172), [], colors(4), 'filled');
+scatter(x(173:215), y(173:215), [], colors(5), 'filled');
+
+plot(fitresult,'k',x,y);
+xlabel('js\_param');
+ylabel('acc\_loss')
+legend('AlexNet', 'AlexNet\_BN','VGG\_16','VGG\_19','Inception\_BN','ALL', 'Rational-Fit', 'Location', 'Northeast');
+% 获取评价指标
+SSE=gof.sse;
+R_square = gof.rsquare; 
+RMSE = gof.rmse; 
+
+% 将拟合公式和 R 方显示在图上
+text(0.65, 0.2, sprintf('Goodness of fit:\n SSE:%.4f\n R-square:%.4f\n RMSE:%.4f', SSE, R_square, RMSE), 'Units', 'normalized', 'FontSize', 11);
+
+hold off;
\ No newline at end of file

ykl/README.md

 # 改动说明
 
+## update: 2023/04/26
+
++ 将新框架应用于所有模型（AlexNet、AlexNet_BN、VGG_16、VGG_19、Inception_BN），并采用不同方式对单个模型拟合，以及对所有模型一同拟合。详见ALL
+
++ 更新Matlab脚本，支持单模型/多模型拟合，支持选择不同多项式次数的拟合模型。添加约束保证拟合曲线单调不下降
+
++ ALL的README所提到的POT点不协调的问题
+
+  + 关于精度的讨论可见Inception_BN部分的README
+  + 关于原因及方案讨论可见ALL的README
+
+  
+
 ## update: 2023/04/22
 
 + 添加Inception BN模型，对框架改动如下
@@ -12,7 +25,7 @@
   + 由于允许conv.bias=None,相应改变全精度模型fold_bn方法，从而保证量化前后可比参数相同。改写方式同量化层
   + 更改js_div计算方法，一个层如果同时有多个参数，例如weight和bias，应该总共加起来权重为1。当前直接简单取平均（即js除以该层参数量），后续考虑加权。PS: Inception_BN中，外层conv层有bias，Inception模块内由于后接bn层，bias为false
     + 由于named_parameters迭代器长度不固定，需要先将排成固定列表再处理，从而获得同一层参数数，改动见ptq.py。对全精度模型做此操作即可
-  + 新框架中的model_utils方法可以通过调整反量化位置来进行bug的确定。经过当前实验，可以初步判断精度问题出现在inception结构中，具体信息见Inception_BN相关部分。经过排查，量化框架本身并未出现问题，问题可能在于该模型参数分布与POT集中分布的不适配。
+  + Inception_BN框架中的model_utils方法可以通过调整反量化位置来进行bug的确定。经过当前实验，可以初步判断精度问题出现在inception结构中，具体信息见Inception_BN相关部分。经过排查，量化框架本身并未出现问题，问题可能在于该模型参数分布与POT量化点分布的不适配。
 
 ## update: 2023/04/17
 
@@ -26,6 +39,6 @@
 + ptq.py中计算js_param笔误，应由flop_ratio改为par_ratio。否则flops和param拟合没有区别
 + module.py中bias_qmax方法，应当为float类型传参num_bits为16，e_bits为7.
   + 这里主要关注e_bits，拟合离群点主要为FLOAT_7_E5 / FLOAT_8_E5 / FLOAT_8_E6，其表现为bias两极分布，与之前int量化bias溢出的问题现象相似。
-  + 原先指定e_bits为5，由于bias的scale为input和weight的scale乘积，bias量化范围应当大致为x和weight量化范围的平方倍。目前代码支持的最高x和weight量化范围大致为 $2^{2^{6}}$ ，因此bias范围应当近似取到$2^{2^7}$，即将e_bits指定为7
+  + 原先指定e_bits为5，由于bias的scale为input和weight的scale乘积，bias量化范围应当大致为x和weight量化范围的平方倍。目前代码支持的最高x和weight量化范围大致为 2的2的6次方 ，因此bias范围应当近似取到 2的2的7次方，即将e_bits指定为7
   + 改动之后，离群点消失，拟合效果显著提高