Skip to content

M
Model-Transfer-Adaptability
Commit d017afd9 by Klin
Options

feat: combine ALL-cifar10/100 and Qimera to ALL/

parent 68add01f
Showing with 7097 additions and 39 deletions
ykl/ALL/README.md 0 → 100644
# ALL
+ 逐步将不含LSTM的模型全部功能加入该文件夹，包括对不同数据集支持、模型训练及量化、生成器训练和评估。
+ 之前ResNet系列和MobileNetV2模型结构有误，在cfg.py中进行了调整。
+ 全精度模型训练策略
+ SGD优化器 lr=0.1 weight_decay=0.9 nesterov=True momentum=0.9
+ MultiStepLR：epochs=200 milestones=[60,120,160] gamma=0.2。（epoch计数达到milestones时，lr乘gamma）
+ WarmUpLR：对第一个epoch，执行warmup，逐batch增大lr。
+ 生成器训练策略
+ Adam优化器 lr=0.001 b1=0.5 b2=0.999
+ MultiStepLR: epochs=40 milesontes = [20,30] gamma=0.1
## cifar10
#### 训练全精度模型
+ slurm命令
```shell
python train_one.py $Model $Dataset
```
在ALL目录下运行脚本
```shell
# 训练单个模型
sh scripts/train_one_cifar10.sh AlexNet/...
# 训练全部模型
sh scripts/train_all_cifar10.sh
```
+ 全精度Acc
| AlexNet | AlexNet_BN | VGG_16 | VGG_19 | Inception_BN | MobileNetV2 | ResNet_18 | ResNet_50 | ResNet_152 |
| ------- | ---------- | ------ | ------ | ------------ | ----------- | --------- | --------- | ---------- |
| 88.63 | 90.14 | 93.14 | 93.03 | 94.83 | 90.28 | 94.30 | 94.58 | 94.43 |
#### 模型量化及迁移准确性评估
##### 参数量和计算量统计
+ slurm命令
```shell
python get_param_flops.py $name $Dataset > param_flops/$Dataset/$name.txt
```
在ALL目录下运行脚本
```shell
sh scripts/ptflops_cifar10.sh
```
+ 结果见param_flops/cifar10
##### 量化、相似度、准确度获取
+ slurm命令
```shell
python ptq_one.py $Model $Dataset
```
在ALL目录下运行脚本
```shell
# 量化及评估单个模型
sh scripts/ptq_one_cifar10.sh AlexNet/...
# 量化及评估全部模型
sh scripts/ptq_all_cifar10.sh
```
+ 拟合结果
![ALL-cifar10-ptq](fig/ALL-cifar10-ptq.png)
单个模型及js_flops - acc_loss拟合对应R2：
| AlexNet | AlexNet_BN | VGG_16 | VGG_19 | Inception_BN | MobileNetV2 | ResNet_18 | ResNet_50 | ResNet_152 |
| ------- | ---------- | ------ | ------ | ------------ | ----------- | --------- | --------- | ---------- |
| 0.9819 | 0.9878 | 0.9727 | 0.9648 | 0.8693 | 0.9652 | 0.9181 | 0.9011 | 0.8994 |
考虑到单个模型拟合均较好，而整体拟合较差，后期将尝试加权处理。
​
#### 基于生成器的安全性评估
+ slurm命令
```shell
#针对全精度模型
python gen_one.py --model $Model --dataset $Dataset --multi_label_prob 0.4 --multi_label_num $Label
#针对量化模型
python gen_one.py --model $Model --dataset $Dataset --quant --multi_label_prob 0.4 --multi_label_num $Label
```
在ALL目录下运行脚本
```shell
#全精度，针对单个模型训练生成器
sh scripts/gen_full_one_cifar10.sh
#全精度，针对全部模型训练生成器
sh scripts/gen_full_all_cifar10.sh
#量化，针对单个模型训练生成器
sh scripts/gen_quant_one_cifar10.sh
#量化，针对全部模型训练生成器
sh scripts/gen_quant_all_cifar10.sh
```
## cifar100
#### 训练全精度模型
+ slurm命令
```shell
python train_one.py AlexNet/AlexNet_BN/... cifar100
```
在ALL目录下运行脚本
```shell
# 训练单个模型
sh scripts/train_one_cifar100.sh AlexNet/...
# 训练全部模型
sh scripts/train_all_cifar100.sh
```
+ 全精度Acc
| AlexNet | AlexNet_BN | VGG_16 | VGG_19 | Inception_BN | MobileNetV2 | ResNet_18 | ResNet_50 | ResNet_152 |
| ------- | ---------- | ------ | ------ | ------------ | ----------- | --------- | --------- | ---------- |
| 63.43 | 66.31 | 71.12 | 70.45 | 77.97 | 66.76 | 75.73 | 77.22 | 77.41 |
#### 模型量化及迁移准确性评估
##### 参数量和计算量统计
+ slurm命令
```shell
python get_param_flops.py $name $Dataset > param_flops/$Dataset/$name.txt
```
在ALL目录下运行脚本
```shell
sh scripts/ptflops_cifar100.sh
```
+ 结果见param_flops/cifar100
##### 量化、相似度、准确度获取
+ slurm命令
```shell
python ptq_one.py $Model $Dataset
```
在ALL目录下运行脚本
```shell
# 量化及评估单个模型
sh scripts/ptq_one_cifar100.sh AlexNet/...
# 量化及评估全部模型
sh scripts/ptq_all_cifar100.sh
```
+ 拟合结果
![ALL-cifar100-ptq](fig/ALL-cifar100-ptq.png)
模型及js_flops - acc_loss拟合对应R2：
| AlexNet | AlexNet_BN | VGG_16 | VGG_19 | Inception_BN | MobileNetV2 | ResNet_18 | ResNet_50 | ResNet_152 |
| ------- | ---------- | ------ | ------ | ------------ | ----------- | --------- | --------- | ---------- |
| 0.9790 | 0.9730 | 0.9222 | 0.9115 | 0.8806 | 0.9570 | 0.8498 | 0.8620 | 0.8713 |
考虑到单个模型拟合均较好，而整体拟合较差，后期将尝试加权处理。
#### 基于生成器的安全性评估
+ slurm命令
```shell
#针对全精度模型
python gen_one.py --model $Model --dataset $Dataset --multi_label_prob 0.4 --multi_label_num $Label
#针对量化模型
python gen_one.py --model $Model --dataset $Dataset --quant --multi_label_prob 0.4 --multi_label_num $Label
```
在ALL目录下运行脚本
```shell
#全精度，针对单个模型训练生成器
sh scripts/gen_full_one_cifar100.sh
#全精度，针对全部模型训练生成器
sh scripts/gen_full_all_cifar100.sh
#量化，针对单个模型训练生成器
sh scripts/gen_quant_one_cifar100.sh
#量化，针对全部模型训练生成器
sh scripts/gen_quant_all_cifar100.sh
```
\ No newline at end of file
ykl/ALL/cancel_all.sh 0 → 100644
# name_list="AlexNet AlexNet_BN VGG_16 VGG_19 Inception_BN ResNet_18 ResNet_50 ResNet_152 MobileNetV2"
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
for name in $name_list; do
scancel -n $name
done
\ No newline at end of file
ykl/ALL/cat_one.sh 0 → 100644
cat $(ls -1 ret_one/$1/*.out | sort -n -t - -k 2 | tail -n 1)
\ No newline at end of file
ykl/ALL/cfg.py 0 → 100644
# conv: 'C',''/'B'/'BRL'/'BRS',qi,in_ch,out_ch,kernel_size,stirde,padding,bias
# relu: 'RL'
# relu6: 'RS'
# inception: 'Inc'
# maxpool: 'MP',kernel_size,stride,padding
# adaptiveavgpool: 'AAP',output_size
# view: 'VW':
# dafault: x = x.view(x.size(0),-1)
# dropout: 'D'
# MakeLayer: 'ML','BBLK'/'BTNK'/'IRES', ml_idx, blocks
# softmax: 'SM'
# class 100
ResNet_18_cfg_table = [
['C','BRL',True,3,64,3,1,1,False],
['ML','BBLK',0,2],
['ML','BBLK',1,2],
['ML','BBLK',2,2],
['ML','BBLK',3,2],
['AAP',1],
['VW'],
['FC',512,100,True],
]
ResNet_50_cfg_table = [
['C','BRL',True,3,64,3,1,1,False],
['ML','BTNK',0,3],
['ML','BTNK',1,4],
['ML','BTNK',2,6],
['ML','BTNK',3,3],
['AAP',1],
['VW'],
['FC',2048,100,True]
]
ResNet_152_cfg_table = [
['C','BRL',True,3,64,3,1,1,False],
['ML','BTNK',0,3],
['ML','BTNK',1,8],
['ML','BTNK',2,36],
['ML','BTNK',3,3],
['AAP',1],
['VW'],
['FC',2048,100,True]
]
MobileNetV2_cfg_table = [
['C','BRS',True,3,32,1,1,1,True],
['ML','IRES',0,1],
['ML','IRES',1,2],
['ML','IRES',2,3],
['ML','IRES',3,4],
['ML','IRES',4,3],
['ML','IRES',5,3],
['ML','IRES',6,1],
['C','BRS',False,320,1280,1,1,0,True],
['AAP',1],
['VW'],
['FC',1280,100,True]
]
AlexNet_cfg_table = [
['C','',True,3,32,3,1,1,True],
['RL'],
['MP',2,2,0],
['C','',False,32,64,3,1,1,True],
['RL'],
['MP',2,2,0],
['C','',False,64,128,3,1,1,True],
['RL'],
['C','',False,128,256,3,1,1,True],
['RL'],
['C','',False,256,256,3,1,1,True],
['RL'],
['MP',3,2,0],
['VW'],
['D',0.5],
['FC',2304,1024,True],
['RL'],
['D',0.5],
['FC',1024,512,True],
['RL'],
['FC',512,100,True]
]
AlexNet_BN_cfg_table = [
['C','BRL',True,3,32,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,32,64,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,64,128,3,1,1,True],
['C','BRL',False,128,256,3,1,1,True],
['C','BRL',False,256,256,3,1,1,True],
['MP',3,2,0],
['VW'],
['D',0.5],
['FC',2304,1024,True],
['RL'],
['D',0.5],
['FC',1024,512,True],
['RL'],
['FC',512,100,True]
]
VGG_16_cfg_table = [
['C','BRL',True,3,64,3,1,1,True],
['C','BRL',False,64,64,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,64,128,3,1,1,True],
['C','BRL',False,128,128,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,128,256,3,1,1,True],
['C','BRL',False,256,256,3,1,1,True],
['C','BRL',False,256,256,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,256,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['MP',2,2,0],
['VW'],
['FC',512,4096,True],
['RL'],
['D',0.5],
['FC',4096,4096,True],
['RL'],
['D',0.5],
['FC',4096,100,True]
]
VGG_19_cfg_table = [
['C','BRL',True,3,64,3,1,1,True],
['C','BRL',False,64,64,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,64,128,3,1,1,True],
['C','BRL',False,128,128,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,128,256,3,1,1,True],
['C','BRL',False,256,256,3,1,1,True],
['C','BRL',False,256,256,3,1,1,True],
['C','BRL',False,256,256,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,256,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['MP',2,2,0],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['C','BRL',False,512,512,3,1,1,True],
['MP',2,2,0],
['VW'],
['FC',512,4096,True],
['RL'],
['D',0.5],
['FC',4096,4096,True],
['RL'],
['D',0.5],
['FC',4096,100,True]
]
Inception_BN_cfg_table = [
['C','',True,3,64,3,1,1,True],
['RL'],
['C','',False,64,64,3,1,1,True],
['RL'],
['Inc',0],
['Inc',1],
['MP',3,2,1],
['Inc',2],
['Inc',3],
['Inc',4],
['Inc',5],
['Inc',6],
['MP',3,2,1],
['Inc',7],
['Inc',8],
['AAP',1],
['C','',False,1024,100,1,1,0,True],
['VW']
]
model_cfg_table = {
'AlexNet' : AlexNet_cfg_table,
'AlexNet_BN' : AlexNet_BN_cfg_table,
'VGG_16' : VGG_16_cfg_table,
'VGG_19' : VGG_19_cfg_table,
'Inception_BN' : Inception_BN_cfg_table,
'ResNet_18' : ResNet_18_cfg_table,
'ResNet_50' : ResNet_50_cfg_table,
'ResNet_152' : ResNet_152_cfg_table,
'MobileNetV2' : MobileNetV2_cfg_table
}
#每行对应一个Inc结构(channel)的参数表
inc_ch_table=[
[ 64, 64, 96,128, 16, 32, 32],#3a
[256,128,128,192, 32, 96, 64],#3b
[480,192, 96,208, 16, 48, 64],#4a
[512,160,112,224, 24, 64, 64],#4b
[512,128,128,256, 24, 64, 64],#4c
[512,112,144,288, 32, 64, 64],#4d
[528,256,160,320, 32,128,128],#4e
[832,256,160,320, 32,128,128],#5a
[832,384,192,384, 48,128,128] #5b
]
# br0,br1,br2,br3 <- br1x1,br3x3,br5x5,brM
# 每个子数组对应Inc结构中一个分支的结构，均默认含'BRL'参数，bias为False
# Conv层第2、3个参数是对应Inc结构(即ch_table中的一行)中的索引
# 由于每个Inc结构操作一致，只有权重不同，使用索引而非具体值，方便复用
# 各分支后还有Concat操作，由于只有唯一结构，未特殊说明
# conv: 'C', ('BRL' default), in_ch_idex, out_ch_idx, kernel_size, stride, padding, (bias: True default)
# maxpool: 'MP', kernel_size, stride, padding
# relu: 'RL'
inc_cfg_table = [
[
['C',0,1,1,1,0]
],
[
['C',0,2,1,1,0],
['C',2,3,3,1,1]
],
[
['C',0,4,1,1,0],
['C',4,5,5,1,2]
],
[
['MP',3,1,1],
['RL'],
['C',0,6,1,1,0]
]
]
# ml_cfg_table = []
#BasicBlock
#value: downsample,inplanes,planes,planes*expansion,stride,1(dafault stride and group)
bblk_ch_table = [
[False, 64, 64, 64,1,1], #layer1,first
[False, 64, 64, 64,1,1], # other
[True, 64,128,128,2,1], #layer2
[False,128,128,128,1,1],
[True, 128,256,256,2,1], #layer3
[False,256,256,256,1,1],
[True, 256,512,512,2,1], #layer4
[False,512,512,512,1,1]
]
#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx,bias
#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
bblk_cfg_table = [
[
['C','BRL',1,2,3,4,1,5,False],
['C','B' ,2,3,3,5,1,5,False],
],
# downsample, 仅当downsample传入为True时使用
[
['C','B' ,1,3,1,4,0,5,False]
],
# 分支交汇后动作
[
['AD',True],
['RL']
]
]
#BottleNeck
#value: downsample,inplanes,planes,planes*expansion,stride,1(dafault stride and group)
btnk_ch_table = [
[True, 64, 64, 256,1,1], #layer1,first
[False, 256, 64, 256,1,1], # other
[True, 256,128, 512,2,1], #layer2
[False, 512,128, 512,1,1],
[True, 512,256,1024,2,1], #layer3
[False,1024,256,1024,1,1],
[True, 1024,512,2048,2,1], #layer4
[False,2048,512,2048,1,1]
]
#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
btnk_cfg_table = [
[
['C','BRL',1,2,1,5,0,5,False],
['C','BRL',2,2,3,4,1,5,False],
['C','B' ,2,3,1,5,0,5,False]
],
# downsample, 仅当downsample传入为True时使用
[
['C','B' ,1,3,1,4,0,5,False]
],
# 分支交汇后动作
[
['AD',True],
['RL']
]
]
#InvertedResidual/LinearBottleNeck
#value: identity_flag, in_ch, out_ch, in_ch*expand_ratio, stride, 1(dafault stride and group)
ires_ch_table = [
[False, 32, 16, 32,1,1], #layer1,first
[ True, 16, 16, 16,1,1], # other
[False, 16, 24, 96,2,1], #layer2
[ True, 24, 24, 144,1,1],
[False, 24, 32, 144,2,1], #layer3
[ True, 32, 32, 192,1,1],
[False, 32, 64, 192,2,1], #layer4
[ True, 64, 64, 384,1,1],
[False, 64, 96, 384,1,1], #layer5
[ True, 96, 96, 576,1,1],
[False, 96,160, 576,1,1], #layer6
[ True,160,160, 960,1,1],
[False,160,320, 960,1,1], #layer6
[ True,320,320,1920,1,1]
]
#conv: 'C','B'/'BRL'/'BRS', in_ch_idx, out_ch_idx, kernel_sz, stride_idx, padding, groups_idx (bias: True default)
#add: 'AD', unconditonal. unconditonal为true或flag为true时将outs中两元素相加
ires_cfg_table = [
[
['C','BRS',1,3,1,5,0,5,True],
['C','BRS',3,3,3,4,1,3,True],
['C','B' ,3,2,1,5,0,5,True]
],
# identity_br empty
[
],
# 分支汇合后操作
[
['AD',False] #有条件的相加
]
]
\ No newline at end of file
ykl/ALL/dataloader.py 0 → 100644
import os
import torch
import torchvision.datasets as dsets
import torchvision.transforms as transforms
class DataLoader(object):
"""
data loader for CV data sets
"""
def __init__(self, dataset, batch_size):
"""
create data loader for specific data set
:params n_treads: number of threads to load data, default: 4
:params data_path_root: root path to data set, default: /lustre/datasets/
"""
self.dataset = dataset
self.batch_size = batch_size
self.n_threads = 4 #num_workers
self.data_path_root = '/lustre/datasets/'
if self.dataset in ["cifar100","cifar10"]:
self.train_loader, self.val_loader, self.test_loader = self.cifar(
dataset=self.dataset)
else:
assert False, "invalid data set"
def getloader(self):
"""d
get train_loader and test_loader
"""
return self.train_loader, self.val_loader, self.test_loader
def cifar(self, dataset):
"""
dataset: cifar
"""
if dataset == "cifar10":
norm_mean = [0.49139968, 0.48215827, 0.44653124]
norm_std = [0.24703233, 0.24348505, 0.26158768]
elif dataset == "cifar100":
norm_mean = [0.50705882, 0.48666667, 0.44078431]
norm_std = [0.26745098, 0.25568627, 0.27607843]
else:
assert False, "Invalid cifar dataset"
train_transfrom = transforms.Compose([
transforms.RandomCrop(32, padding=2),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize(norm_mean, norm_std)
])
eval_transfrom = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(norm_mean, norm_std)
])
if self.dataset == "cifar10":
data_path = self.data_path_root + 'CIFAR10'
alltrainset = dsets.CIFAR10(root=data_path,train=True,download=False,
transform=train_transfrom)
testset = dsets.CIFAR10(data_path, train=False, download=False,
transform=eval_transfrom)
elif self.dataset == "cifar100":
data_path = self.data_path_root + 'CIFAR100'
alltrainset = dsets.CIFAR100(root=data_path,train=True,download=False,
transform=train_transfrom)
testset = dsets.CIFAR100(data_path, train=False, download=False,
transform=eval_transfrom)
else:
assert False, "invalid data set"
train_size = (int)(0.8 * len(alltrainset))
val_size = (int)(0.2 * len(alltrainset))
train_idx, val_idx = torch.utils.data.random_split(range(train_size+val_size),[train_size,val_size])
trainset = torch.utils.data.Subset(alltrainset,train_idx)
valset = torch.utils.data.Subset(alltrainset,val_idx)
train_loader = torch.utils.data.DataLoader(
trainset,
batch_size=self.batch_size, shuffle=True, num_workers=self.n_threads, pin_memory=True
)
val_loader = torch.utils.data.DataLoader(
valset,
batch_size=self.batch_size, shuffle=False, num_workers=self.n_threads, pin_memory=True
)
test_loader = torch.utils.data.DataLoader(
testset,
batch_size=self.batch_size, shuffle=False, num_workers=self.n_threads, pin_memory=True
)
return train_loader, val_loader, test_loader
ykl/ALL/err.sh 0 → 100644
if [ -z $1 ];then
cat $(ls -1 ret/ret-*.err | sort -n -t - -k 2 | tail -n 1)
else
cat ret/ret-"$1".err
fi
\ No newline at end of file
ykl/ALL/err_one.sh 0 → 100644
cat $(ls -1 ret_one/$1/*.err | sort -n -t - -k 2 | tail -n 1)
\ No newline at end of file
ykl/ALL/extract_ratio.py 0 → 100644
import sys
import os
# 从get_param.py输出重定向文件val.txt中提取参数量和计算量
def extract_ratio(model_name,dataset):
fr = open('param_flops/'+dataset+'/'+model_name+'.txt','r')
lines = fr.readlines()
#跳过warning
for i in range(len(lines)):
if 'Model' in lines[i]:
head = i+1
break
Mac = lines[head].split('Mac,')[0].split(',')[-1]
if 'M' in Mac:
Mac = Mac.split('M')[0]
Mac = float(Mac)
elif 'G' in Mac:
Mac = Mac.split('G')[0]
Mac = float(Mac)
Mac *= 1024
Param = lines[head].split(',')[0]
if 'M' in Param:
Param = Param.split('M')[0]
Param = float(Param)
elif 'k' in Param:
Param = Param.split('k')[0]
Param = float(Param)
Param /= 1024
layer = []
par_ratio = []
flop_ratio = []
for line in lines:
if '(' in line and ')' in line:
layer.append(line.split(')')[0].split('(')[1])
r1 = line.split('%')[0].split(',')[-1]
r1 = float(r1)
par_ratio.append(r1)
r2 = line.split('%')[-2].split(',')[-1]
r2 = float(r2)
flop_ratio.append(r2)
return Mac, Param, layer, par_ratio, flop_ratio
\ No newline at end of file
ykl/ALL/function.py 0 → 100644
from torch.autograd import Function
class FakeQuantize(Function):
@staticmethod
def forward(ctx, x, qparam):
x = qparam.quantize_tensor(x)
x = qparam.dequantize_tensor(x)
return x
@staticmethod
def backward(ctx, grad_output):
return grad_output, None
\ No newline at end of file
ykl/ALL/gen_one.py 0 → 100644
from model import *
from dataloader import DataLoader
from utils import numbit_list, ebit_list,build_bias_list, build_list
from gen_options import GenOption
from generator import Generator,Generator_imagenet
import module
import gol
import argparse
import time
import torch
import torch.nn as nn
from torch.optim.lr_scheduler import MultiStepLR
import sys
class GenTrainer(object):
def __init__(self, option):
self.settings = option
self.set_test_loader()
self.set_teacher()
self.set_generator()
self.set_optim_G()
def set_test_loader(self):
dataloader = DataLoader(self.settings.dataset,self.settings.batchSize)
_,_,self.test_loader = dataloader.getloader()
def set_teacher(self):
self.model_teacher = Model(self.settings.model,self.settings.dataset).cuda()
if self.settings.quant:
self.model_teacher.quantize(self.settings.quant_type,self.settings.num_bits,self.settings.e_bits)
self.model_teacher.load_state_dict(torch.load(self.settings.teacher_file))
self.model_teacher.eval()
#当randemb为False，此处同时完成了latent_dim的修改
def set_generator(self):
if self.settings.randemb:
weight_t = None
else:
if self.settings.quant:
weight_t = self.model_teacher.get_quant_output_layer_weight()
else:
weight_t = self.model_teacher.get_output_layer_weight()
# 输出层如果是Conv，weight的shape有四元，后二元都是1，舍弃
if self.settings.model in ['Inception_BN']:
weight_t = weight_t.reshape(weight_t.size()[:2])
self.settings.latent_dim = weight_t.size()[1]
if self.settings.dataset in ['cifar10','cifar100']:
self.generator = Generator(self.settings, weight_t, self.settings.freeze).cuda()
elif self.settings.dataset in ['imagenet']:
self.generator = Generator_imagenet(self.settings, weight_t, self.settings.freeze).cuda()
else:
assert False, "Invalid dataset"
def set_optim_G(self):
self.optim_G = torch.optim.Adam(self.generator.parameters(), lr=self.settings.lr_G,
betas=(self.settings.b1, self.settings.b2))
self.lrs_G = MultiStepLR(self.optim_G, milestones=self.settings.milestones_G, gamma=self.settings.gamma_G)
def test_teacher(self):
correct = 0
with torch.no_grad():
for data, target in self.test_loader:
data,target = data.cuda(), target.cuda()
if self.settings.quant:
output = self.model_teacher.quantize_forward(data)
else:
output = self.model_teacher(data)
pred = output.argmax(dim=1, keepdim=True)
correct += pred.eq(target.view_as(pred)).sum().item()
print('Teacher Accuracy: {:.2f}%'.format(100. * correct / len(self.test_loader.dataset)))
def prepare_train(self):
self.log_soft = nn.LogSoftmax(dim=1)
# MSE主要用于回归问题训练
self.MSE_loss = nn.MSELoss().cuda()
self.mean_list = []
self.var_list = []
self.teacher_running_mean = []
self.teacher_running_var = []
self.model_teacher.eval()
self.generator.train()
if self.settings.quant:
for m in self.model_teacher.modules():
if isinstance(m, module.QConvBN) or isinstance(m,module.QConvBNReLU) or isinstance(m,module.QConvBNReLU6):
m.register_forward_hook(self.quant_hook_fn_forward)
else:
for m in self.model_teacher.modules():
if isinstance(m, nn.BatchNorm2d):
m.register_forward_hook(self.hook_fn_forward)
#针对量化网络的ConvBN融合层，提取其中BN层的统计信息，因此对input进行Conv才作为BN输入
def quant_hook_fn_forward(self,module, input, output):
#mean和var是针对输入的伪数据，running_mean和running_var是对train时输入的数据
weight = module.conv_module.weight.clone()
if module.conv_module.bias is not None:
bias = module.conv_module.bias.clone()
else:
bias = None
stride = module.conv_module.stride
padding = module.conv_module.padding
groups = module.conv_module.groups
input = input[0]
input = F.conv2d(input, weight, bias,
stride=stride,
padding=padding,
groups=groups)
mean = input.mean([0, 2, 3])
# use biased var in train
var = input.var([0, 2, 3], unbiased=False)
self.mean_list.append(mean)
self.var_list.append(var)
#eval状态，直接提取QConvBN层中BN的信息即可
self.teacher_running_mean.append(module.bn_module.running_mean)
self.teacher_running_var.append(module.bn_module.running_var)
def hook_fn_forward(self,module, input, output):
#mean和var是针对输入的伪数据，running_mean和running_var是对train时输入的数据
input = input[0]
mean = input.mean([0, 2, 3])
# use biased var in train
var = input.var([0, 2, 3], unbiased=False)
self.mean_list.append(mean)
self.var_list.append(var)
#eval状态，直接提取QConvBN层中BN的信息即可
self.teacher_running_mean.append(module.running_mean)
self.teacher_running_var.append(module.running_var)
def train(self, epoch):
# total_loss = 0.
correct = 0
item_len = 0
for i in range(self.settings.iters):
multi_class = torch.rand(1)
self.MERGE_PARAM = self.settings.multi_label_num
MERGE_PROB = self.settings.multi_label_prob # superpose probability
# MERGE_PROB 表示决策边界样本（即多类label）的比例
# IF分支为多类Label，由于每个元素对应多个label，无法衡量acc，因此只在else分支更新acc
# 多分类样本处于决策边界，同样能根据loss等提高生成器效果
if multi_class<MERGE_PROB:
# Get labels ranging from 0 to n_classes for n rows
z = torch.randn(self.settings.batchSize, self.MERGE_PARAM,self.settings.latent_dim).cuda()
labels = torch.randint(0, self.settings.nClasses, (self.settings.batchSize,self.MERGE_PARAM)).cuda()
linear = F.softmax(torch.randn(self.settings.batchSize,self.MERGE_PARAM),dim=1).cuda()
z = z.contiguous()
labels = labels.contiguous()
labels_loss = torch.zeros(self.settings.batchSize,self.settings.nClasses).cuda()
labels_loss.scatter_add_(1,labels,linear)
images = self.generator(z, labels, linear)
else:
z = torch.randn(self.settings.batchSize, self.settings.latent_dim).cuda()
labels = torch.randint(0, self.settings.nClasses, (self.settings.batchSize,)).cuda()
z = z.contiguous()
labels = labels.contiguous()
images = self.generator(z, labels)
labels_loss = torch.zeros(self.settings.batchSize,self.settings.nClasses).cuda()
labels_loss.scatter_(1,labels.unsqueeze(1),1.0)
self.mean_list.clear()
self.var_list.clear()
# 获取teacher模型输出，同时使用hook_fn_forward获取了mean和var列表
if self.settings.quant:
output_teacher_batch = self.model_teacher.quantize_forward(images)
else:
output_teacher_batch = self.model_teacher(images)
# teacher模型输出和label的损失，一维tensor
loss_one_hot = (-(labels_loss*self.log_soft(output_teacher_batch)).sum(dim=1)).mean()
# BN statistic loss
# 这里统计了伪数据分布和teacher模型BN层分布的loss
BNS_loss = torch.zeros(1).cuda()
for num in range(len(self.mean_list)):
BNS_loss += self.MSE_loss(self.mean_list[num], self.teacher_running_mean[num]) + self.MSE_loss(
self.var_list[num], self.teacher_running_var[num])
BNS_loss = BNS_loss / len(self.mean_list)
# loss of Generator
loss_G = loss_one_hot + 0.1 * BNS_loss
# total_loss += loss_G.item()
self.optim_G.zero_grad()
loss_G.backward()
self.optim_G.step()
if not multi_class< MERGE_PROB:
pred = output_teacher_batch.argmax(dim=1, keepdim=True)
correct += pred.eq(labels.view_as(pred)).sum().item()
item_len += self.settings.batchSize
# train_loss = total_loss/self.settings.iters
gen_acc = 100. * correct / item_len
# 对应输出的第一行，表示teacher在含噪声的label和生成器根据噪声label生成input上取得的精度
# 这里acc越高表示生成器效果越好，生成的输入接近真实数据分布
# print(
# "[Epoch %d/%d] [Batch %d/%d] [acc: %.4f%%] [G loss: %f] [Time: %5.2fs]"
# % (epoch + 1, self.settings.nEpochs, i+1, self.settings.iters, gen_acc, train_loss, (time.time()-start_time))
# )
return gen_acc
def run(self):
self.test_teacher()
self.prepare_train()
start_time = time.time()
best_gen_acc = None
for epoch in range(1,self.settings.nEpochs):
gen_acc = self.train(epoch)
if not best_gen_acc or gen_acc > best_gen_acc:
best_gen_acc = gen_acc
torch.save(self.generator, self.settings.gen_file)
time_interval = time.time()-start_time
print('>> Epoch:%d Time:%.2fs Cur acc:%.4f Best acc:%.4f'%(epoch,time_interval,gen_acc,best_gen_acc))
start_time = time.time()
def main():
#及时打印信息
sys.stdout = open(sys.stdout.fileno(), mode='w', buffering=1)
parser = argparse.ArgumentParser(description='Gen Arg')
parser.add_argument('--model', type=str)
parser.add_argument('--dataset',type=str)
parser.add_argument('--quant', action='store_true')
parser.add_argument('--freeze', action='store_true')
parser.add_argument('--randemb', action='store_true')
parser.add_argument('--multi_label_prob', type=float, default=0.0)
parser.add_argument('--multi_label_num', type=int, default=2)
parser.add_argument('--no_DM', action='store_false')
parser.add_argument('--noise_scale', type=float, default=1.0)
args = parser.parse_args()
print(args)
option = GenOption(args)
if option.quant:
gol._init()
quant_type_list = ['POT','FLOAT']
for quant_type in quant_type_list:
num_bit_list = numbit_list(quant_type)
if quant_type != 'INT':
bias_list = build_bias_list(quant_type)
gol.set_value(bias_list, is_bias=True)
for num_bits in num_bit_list:
e_bit_list = ebit_list(quant_type,num_bits)
for e_bits in e_bit_list:
if quant_type == 'FLOAT':
title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
else:
title = '%s_%d' % (quant_type, num_bits)
# 设置量化表
if quant_type != 'INT':
plist = build_list(quant_type, num_bits, e_bits)
gol.set_value(plist)
print('>'*20 + 'Gen: '+option.model+' '+title+'<'*20)
option.set(quant_type,num_bits,e_bits)
gentrainer = GenTrainer(option)
gentrainer.run()
else:
print('>'*20 + 'Gen: '+option.model+' Full'+'<'*20)
option.set()
gentrainer = GenTrainer(option)
gentrainer.run()
if __name__ == '__main__':
main()
ykl/ALL/gen_one.slurm 0 → 100644
#!/bin/bash
#- Job parameters
# (TODO)
# Please modify job name
#- Resources
# (TODO)
# Please modify your requirements
#SBATCH -p nv-gpu # Submit to 'nv-gpu' Partitiion
#SBATCH -t 1-06:00:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --gres=gpu:1 # Request M GPU per node
#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
#SBATCH --qos=gpu-normal # Request QOS Type
###
### The system will alloc 8 or 16 cores per gpu by default.
### If you need more or less, use following:
### #SBATCH --cpus-per-task=K # Request K cores
###
###
### Without specifying the constraint, any available nodes that meet the requirement will be allocated
### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
###
### #SBATCH --nodelist=gpu-v00 # Request a specific list of hosts
### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
###
# set constraint for RTX8000 to meet my cuda
#SBATCH --constraint="Ampere|RTX8000"
#- Log information
echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
echo "Job run at:"
echo "$(hostnamectl)"
#- Load environments
source /tools/module_env.sh
module list # list modules loaded
##- Tools
module load cluster-tools/v1.0
module load slurm-tools/v1.0
module load cmake/3.15.7
module load git/2.17.1
module load vim/8.1.2424
##- language
module load python3/3.6.8
##- CUDA
# module load cuda-cudnn/10.2-7.6.5
# module load cuda-cudnn/11.2-8.2.1
module load cuda-cudnn/11.1-8.2.1
##- virtualenv
# source xxxxx/activate
echo $(module list) # list modules loaded
echo $(which gcc)
echo $(which python)
echo $(which python3)
cluster-quota # nas quota
nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
#- Warning! Please not change your CUDA_VISIBLE_DEVICES
#- in `.bashrc`, `env.sh`, or your job script
echo "Use GPU ${CUDA_VISIBLE_DEVICES}" # which gpus
#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
#- Job step
# [EDIT HERE(TODO)]
if [ $Dataset = 'cifar10' ]; then
Label=2
elif [ $Dataset = 'cifar100' ]; then
Label=10
else
echo "Invalid Dataset $Dataset"
exit
fi
if [ $Quant = 'True' ]; then
python gen_one.py --model $Model --dataset $Dataset --quant --multi_label_prob 0.4 --multi_label_num $Label
else
python gen_one.py --model $Model --dataset $Dataset --multi_label_prob 0.4 --multi_label_num $Label
fi
#- End
echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"
ykl/ALL/gen_options.py 0 → 100644
import os
import os.path as osp
class GenOption(object):
def __init__(self, args):
self.model = args.model
self.dataset = args.dataset
self.batchSize = 128
self.quant = args.quant
if self.dataset == "cifar10":
self.nClasses = 10
elif self.dataset == "cifar100":
self.nClasses = 100
else:
assert False, "invalid dataset"
# ----------Generator options ---------------------------------------------
# self.nEpochs = 100
self.nEpochs = 40
#每个epoch训练多少轮，和batchsize无关
self.iters = 200
#冻结embedding层权重
self.freeze = args.freeze
self.randemb = args.randemb
# 如果不为randomemb，需要根据weight_t调整
self.latent_dim = 64
# 针对imagenet等数据集需要调整
self.img_size = 32
self.channels = 3
self.lr_G = 0.001
# self.milestones_G = [40,60,80]
self.milestones_G = [20,30]
self.gamma_G = 0.1
self.b1 = 0.5
self.b2 = 0.999
# ----------More option ---------------------------------------------
self.multi_label_prob = args.multi_label_prob
self.multi_label_num = args.multi_label_num
self.no_DM = args.no_DM
self.noise_scale = args.noise_scale
self.intermediate_dim = 100
# if self.network == "resnet20":
# self.intermediate_dim = 64
def set(self,quant_type=None,num_bits=None,e_bits=None):
if self.quant:
self.quant_type = quant_type
self.num_bits = num_bits
self.e_bits = e_bits
if quant_type == 'FLOAT':
title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
else:
title = '%s_%d' % (quant_type, num_bits)
self.teacher_file = 'ckpt_quant/'+self.dataset+'/'+self.model+'/'+title+'.pt'
gen_path = 'ckpt_quant_gen/'+self.dataset+'/'+self.model
self.gen_file = gen_path + '/' + title + '.pt'
else:
self.teacher_file = 'ckpt_full/'+self.dataset+'/'+self.model+'.pt'
gen_path = 'ckpt_full_gen/'+self.dataset
self.gen_file = gen_path +'/'+ self.model+'.pt'
if not osp.exists(self.teacher_file):
assert False, "Empty teacher file"
if not osp.exists(gen_path):
os.makedirs(gen_path)
ykl/ALL/generator.py 0 → 100644
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.nn import init
class Generator(nn.Module):
def __init__(self, options=None, teacher_weight=None, freeze=True):
super(Generator, self).__init__()
self.settings = options
# 注意这里有embedding层，两个分别是词典大小和向量长度
# 用于将标签映射为向量
if teacher_weight==None:
self.label_emb = nn.Embedding(self.settings.nClasses, self.settings.latent_dim)
else:
#当randemb为False时，要求latentdim与输出层输入通道一致
self.label_emb = nn.Embedding.from_pretrained(teacher_weight, freeze=freeze)
self.embed_normalizer = nn.BatchNorm1d(self.label_emb.weight.T.shape,affine=False,track_running_stats=False)
if not self.settings.no_DM:
self.fc_reducer = nn.Linear(in_features=self.label_emb.weight.shape[-1], out_features=self.settings.intermediate_dim)
self.init_size = self.settings.img_size // 4
self.l1 = nn.Sequential(nn.Linear(self.settings.intermediate_dim, 128 * self.init_size ** 2))
else:
self.init_size = self.settings.img_size // 4
self.l1 = nn.Sequential(nn.Linear(self.settings.latent_dim, 128 * self.init_size ** 2))
self.conv_blocks0 = nn.Sequential(
nn.BatchNorm2d(128),
)
self.conv_blocks1 = nn.Sequential(
nn.Conv2d(128, 128, 3, stride=1, padding=1),
nn.BatchNorm2d(128, 0.8),
nn.LeakyReLU(0.2, inplace=True),
)
self.conv_blocks2 = nn.Sequential(
nn.Conv2d(128, 64, 3, stride=1, padding=1),
nn.BatchNorm2d(64, 0.8),
nn.LeakyReLU(0.2, inplace=True),
nn.Conv2d(64, self.settings.channels, 3, stride=1, padding=1),
nn.Tanh(),
nn.BatchNorm2d(self.settings.channels, affine=False)
)
def forward(self, z, labels, linear=None, z2=None):
# GDFQ此处为随机噪声乘label
if linear == None:
gen_input = self.embed_normalizer(torch.add(self.label_emb(labels),self.settings.noise_scale*z).T).T
if not self.settings.no_DM:
gen_input = self.fc_reducer(gen_input)
else:
embed_norm = self.embed_normalizer(torch.add(self.label_emb(labels),self.settings.noise_scale*z).T).T
if not self.settings.no_DM:
gen_input = self.fc_reducer(embed_norm)
else:
gen_input = embed_norm
gen_input = (gen_input * linear.unsqueeze(2)).sum(dim=1)
out = self.l1(gen_input)
out = out.view(out.shape[0], 128, self.init_size, self.init_size)
img = self.conv_blocks0(out)
img = nn.functional.interpolate(img, scale_factor=2)
img = self.conv_blocks1(img)
img = nn.functional.interpolate(img, scale_factor=2)
img = self.conv_blocks2(img)
return img
class Generator_imagenet(nn.Module):
def __init__(self, options=None, teacher_weight=None, freeze=True):
super(Generator_imagenet, self).__init__()
self.settings = options
if teacher_weight==None:
self.label_emb = nn.Embedding(self.settings.nClasses, self.settings.latent_dim)
else:
self.label_emb = nn.Embedding.from_pretrained(teacher_weight, freeze=freeze)
self.embed_normalizer = nn.BatchNorm1d(self.label_emb.weight.T.shape,affine=False,track_running_stats=False)
if not self.settings.no_DM:
self.fc_reducer = nn.Linear(in_features=self.label_emb.weight.shape[-1], out_features=self.settings.intermediate_dim)
self.init_size = self.settings.img_size // 4
self.l1 = nn.Sequential(nn.Linear(self.settings.intermediate_dim, 128 * self.init_size ** 2))
else:
self.init_size = self.settings.img_size // 4
self.l1 = nn.Sequential(nn.Linear(self.settings.latent_dim, 128 * self.init_size ** 2))
self.conv_blocks0_0 = CategoricalConditionalBatchNorm2d(1000, 128)
self.conv_blocks1_0 = nn.Conv2d(128, 128, 3, stride=1, padding=1)
self.conv_blocks1_1 = CategoricalConditionalBatchNorm2d(1000, 128, 0.8)
self.conv_blocks1_2 = nn.LeakyReLU(0.2, inplace=True)
self.conv_blocks2_0 = nn.Conv2d(128, 64, 3, stride=1, padding=1)
self.conv_blocks2_1 = CategoricalConditionalBatchNorm2d(1000, 64, 0.8)
self.conv_blocks2_2 = nn.LeakyReLU(0.2, inplace=True)
self.conv_blocks2_3 = nn.Conv2d(64, self.settings.channels, 3, stride=1, padding=1)
self.conv_blocks2_4 = nn.Tanh()
self.conv_blocks2_5 = nn.BatchNorm2d(self.settings.channels, affine=False)
def forward(self, z, labels, linear=None):
if linear == None:
gen_input = self.embed_normalizer(torch.add(self.label_emb(labels),z).T).T
if not self.settings.no_DM:
gen_input = self.fc_reducer(gen_input)
else:
embed_norm = self.embed_normalizer(torch.add(self.label_emb(labels),z).T).T
if not self.settings.no_DM:
gen_input = self.fc_reducer(embed_norm)
else:
gen_input = embed_norm
gen_input = (gen_input * linear.unsqueeze(2)).sum(dim=1)
out = self.l1(gen_input)
out = out.view(out.shape[0], 128, self.init_size, self.init_size)
img = self.conv_blocks0_0(out, labels, linear=linear)
img = nn.functional.interpolate(img, scale_factor=2)
img = self.conv_blocks1_0(img)
img = self.conv_blocks1_1(img, labels, linear=linear)
img = self.conv_blocks1_2(img)
img = nn.functional.interpolate(img, scale_factor=2)
img = self.conv_blocks2_0(img)
img = self.conv_blocks2_1(img, labels, linear=linear)
img = self.conv_blocks2_2(img)
img = self.conv_blocks2_3(img)
img = self.conv_blocks2_4(img)
img = self.conv_blocks2_5(img)
return img
class ConditionalBatchNorm2d(nn.BatchNorm2d):
"""Conditional Batch Normalization"""
def __init__(self, num_features, eps=1e-05, momentum=0.1,
affine=False, track_running_stats=True):
super(ConditionalBatchNorm2d, self).__init__(
num_features, eps, momentum, affine, track_running_stats
)
def forward(self, input, weight, bias, **kwargs):
self._check_input_dim(input)
exponential_average_factor = 0.0
if self.training and self.track_running_stats:
self.num_batches_tracked += 1
if self.momentum is None: # use cumulative moving average
exponential_average_factor = 1.0 / self.num_batches_tracked.item()
else: # use exponential moving average
exponential_average_factor = self.momentum
output = F.batch_norm(input, self.running_mean, self.running_var,
self.weight, self.bias,
self.training or not self.track_running_stats,
exponential_average_factor, self.eps)
if weight.dim() == 1:
weight = weight.unsqueeze(0)
if bias.dim() == 1:
bias = bias.unsqueeze(0)
size = output.size()
weight = weight.unsqueeze(-1).unsqueeze(-1).expand(size)
bias = bias.unsqueeze(-1).unsqueeze(-1).expand(size)
return weight * output + bias
class CategoricalConditionalBatchNorm2d(ConditionalBatchNorm2d):
def __init__(self, num_classes, num_features, eps=1e-5, momentum=0.1,
affine=False, track_running_stats=True):
super(CategoricalConditionalBatchNorm2d, self).__init__(
num_features, eps, momentum, affine, track_running_stats
)
self.weights = nn.Embedding(num_classes, num_features)
self.biases = nn.Embedding(num_classes, num_features)
self._initialize()
def _initialize(self):
init.ones_(self.weights.weight.data)
init.zeros_(self.biases.weight.data)
def forward(self, input, c, linear=None,**kwargs):
weight = self.weights(c)
bias = self.biases(c)
if linear != None:
weight = (weight * linear.unsqueeze(2)).mean(dim=1)
bias = (bias * linear.unsqueeze(2)).mean(dim=1)
return super(CategoricalConditionalBatchNorm2d, self).forward(input, weight, bias)
ykl/ALL/get_param_flops.py 0 → 100644
from model import *
import sys
import torch
from ptflops import get_model_complexity_info
if __name__ == "__main__":
model_name = sys.argv[1]
dataset = sys.argv[2]
model = Model(model_name,dataset)
# full_file = 'ckpt/cifar10_'+model_name+'.pt'
# model.load_state_dict(torch.load(full_file))
flops, params = get_model_complexity_info(model, (3, 32, 32), as_strings=True, print_per_layer_stat=True)
ykl/ALL/get_param_flops.slurm 0 → 100644
#!/bin/bash
#- Job parameters
# (TODO)
# Please modify job name
#SBATCH -J ALL # The job name
#SBATCH -o ret/ret-%j.out # Write the standard output to file named 'ret-<job_number>.out'
#SBATCH -e ret/ret-%j.err # Write the standard error to file named 'ret-<job_number>.err'
#- Resources
# (TODO)
# Please modify your requirements
#SBATCH -p nv-gpu # Submit to 'nv-gpu' Partitiion
#SBATCH -t 0-01:30:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --gres=gpu:1 # Request M GPU per node
#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
#SBATCH --qos=gpu-debug # Request QOS Type
###
### The system will alloc 8 or 16 cores per gpu by default.
### If you need more or less, use following:
### #SBATCH --cpus-per-task=K # Request K cores
###
###
### Without specifying the constraint, any available nodes that meet the requirement will be allocated
### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
###
### #SBATCH --nodelist=gpu-v00 # Request a specific list of hosts
### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
###
# set constraint for RTX8000 to meet my cuda
#SBATCH --constraint="Ampere|RTX8000|T4"
#- Log information
echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
echo "Job run at:"
echo "$(hostnamectl)"
#- Load environments
source /tools/module_env.sh
module list # list modules loaded
##- Tools
module load cluster-tools/v1.0
module load slurm-tools/v1.0
module load cmake/3.15.7
module load git/2.17.1
module load vim/8.1.2424
##- language
module load python3/3.6.8
##- CUDA
# module load cuda-cudnn/10.2-7.6.5
# module load cuda-cudnn/11.2-8.2.1
module load cuda-cudnn/11.1-8.2.1
##- virtualenv
# source xxxxx/activate
echo $(module list) # list modules loaded
echo $(which gcc)
echo $(which python)
echo $(which python3)
cluster-quota # nas quota
nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
#- Warning! Please not change your CUDA_VISIBLE_DEVICES
#- in `.bashrc`, `env.sh`, or your job script
echo "Use GPU ${CUDA_VISIBLE_DEVICES}" # which gpus
#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
#- Job step
# [EDIT HERE(TODO)]
name_list="AlexNet AlexNet_BN VGG_16 VGG_19 Inception_BN ResNet_18 ResNet_50 ResNet_152 MobileNetV2"
# name_list="MobileNetV2"
for name in $name_list; do
if [ -f "param_flops/$Dataset/$name.txt" ];then
echo "$name: param_flops exists"
# elif [ ! -f "ckpt/cifar10_$name.pt" ];then
# echo "$name: ckpt not exists"
else
python get_param_flops.py $name $Dataset > param_flops/$Dataset/$name.txt
echo "$name: param_flops done"
fi
done
#- End
echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"
ykl/ALL/gol.py 0 → 100644
# -*- coding: utf-8 -*-
# 用于多个module之间共享全局变量
def _init(): # 初始化
global _global_dict
_global_dict = {}
def set_value(value,is_bias=False):
# 定义一个全局变量
if is_bias:
_global_dict[0] = value
else:
_global_dict[1] = value
def get_value(is_bias=False): # 给bias独立于各变量外的精度
if is_bias:
return _global_dict[0]
else:
return _global_dict[1]
ykl/ALL/model.py 0 → 100644
import torch.nn as nn
from cfg import *
from module import *
from model_deployment import *
class Model(nn.Module):
def __init__(self,model_name,dataset):
super(Model, self).__init__()
self.cfg_table = model_cfg_table[model_name]
adapt_dataset(self.cfg_table,dataset)
make_layers(self,self.cfg_table)
def forward(self,x):
x = model_forward(self,self.cfg_table,x)
return x
def quantize(self, quant_type, num_bits=8, e_bits=3):
model_quantize(self,self.cfg_table,quant_type,num_bits,e_bits)
def quantize_forward(self,x):
return model_utils(self,self.cfg_table,func='forward',x=x)
def freeze(self):
model_utils(self,self.cfg_table,func='freeze')
def quantize_inference(self,x):
return model_utils(self,self.cfg_table,func='inference',x=x)
def fakefreeze(self):
model_utils(self,self.cfg_table,func='fakefreeze')
def get_output_layer_weight(self):
return get_output_layer_weight(self,self.cfg_table)
def get_quant_output_layer_weight(self):
return get_quant_output_layer_weight(self,self.cfg_table)
\ No newline at end of file
ykl/ALL/model_deployment.py 0 → 100644
import torch.nn as nn
import torch.nn.functional as F
from cfg import *
from module import *
def adapt_dataset(cfg_table,dataset):
if dataset == "cifar10":
num_classes = 10
elif dataset == "cifar100":
num_classes = 100
else:
print(dataset)
assert False, "Model deployment: invalid dataset"
for i in range(len(cfg_table) - 1, -1, -1):
cfg = cfg_table[i]
if cfg[0] == 'FC':
cfg[2] = num_classes
return
elif cfg[0] == 'C':
cfg[4] = num_classes
return
#获取全精度模型输出层的权重
def get_output_layer_weight(model,cfg_table):
for i in range(len(cfg_table) - 1, -1, -1):
cfg = cfg_table[i]
if cfg[0] == 'FC':
name = 'fc%d'%i
output_layer = getattr(model,name)
return output_layer.weight.detach()
elif cfg[0] == 'C':
name = 'conv%d'%i
output_layer = getattr(model,name)
return output_layer.weight.detach()
#获取量化模型输出层权重
def get_quant_output_layer_weight(model,cfg_table):
for i in range(len(cfg_table) - 1, -1, -1):
cfg = cfg_table[i]
if cfg[0] == 'FC':
name = 'q_fc%d'%i
output_layer = getattr(model,name)
tmp_wgt = FakeQuantize.apply(output_layer.fc_module.weight,output_layer.qw)
return tmp_wgt
elif cfg[0] == 'C':
name = 'q_conv%d'%i
output_layer = getattr(model,name)
tmp_wgt = FakeQuantize.apply(output_layer.conv_module.weight,output_layer.qw)
return tmp_wgt
def make_layers(model,cfg_table):
for i in range(len(cfg_table)):
cfg = cfg_table[i]
if cfg[0] == 'Inc':
make_inc_layers(model,cfg[1])
elif cfg[0] == 'ML':
make_ml_layers(model,cfg[1],cfg[2],cfg[3])
elif cfg[0] == 'C':
name = 'conv%d'%i
layer = nn.Conv2d(cfg[3],cfg[4],kernel_size=cfg[5],stride=cfg[6],padding=cfg[7],bias=cfg[8])
model.add_module(name,layer)
if 'B' in cfg[1]:
name = 'bn%d'%i
layer = nn.BatchNorm2d(cfg[4])
model.add_module(name,layer)
if 'RL' in cfg[1]:
name = 'relu%d'%i
layer = nn.ReLU(True)
model.add_module(name,layer)
elif 'RS' in cfg[1]:
name = 'relus%d'%i
layer = nn.ReLU6(True)
model.add_module(name,layer)
elif cfg[0] == 'RL':
name = 'relu%d'%i
layer = nn.ReLU(True)
model.add_module(name,layer)
elif cfg[0] == 'RS':
name = 'relus%d'%i
layer = nn.ReLU6(True)
model.add_module(name,layer)
elif cfg[0] == 'MP':
name = 'pool%d'%i
layer = nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
model.add_module(name,layer)
elif cfg[0] == 'AAP':
name = 'aap%d'%i
layer = nn.AdaptiveAvgPool2d(cfg[1])
model.add_module(name,layer)
elif cfg[0] == 'FC':
name = 'fc%d'%i
layer = nn.Linear(cfg[1],cfg[2],bias=cfg[3])
model.add_module(name,layer)
elif cfg[0] == 'D':
name = 'drop%d'%i
layer = nn.Dropout(cfg[1])
model.add_module(name,layer)
def model_forward(model,cfg_table,x):
for i in range(len(cfg_table)):
cfg = cfg_table[i]
if cfg[0] == 'Inc':
x = inc_forward(model,cfg[1],x)
elif cfg[0] == 'ML':
x = ml_forward(model,cfg[1],cfg[2],cfg[3],x)
elif cfg[0] == 'C':
name = 'conv%d'%i
layer = getattr(model,name)
x = layer(x)
if 'B' in cfg[1]:
name = 'bn%d'%i
layer = getattr(model,name)
x = layer(x)
if 'RL' in cfg[1]:
name = 'relu%d'%i
layer = getattr(model,name)
x = layer(x)
elif 'RS' in cfg[1]:
name = 'relus%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'RL':
name = 'relu%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'RS':
name = 'relus%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'MP':
name = 'pool%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'AAP':
name = 'aap%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'FC':
name = 'fc%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'D':
name = 'drop%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'VW':
if len(cfg) == 1: #default
x = x.view(x.size(0),-1)
elif cfg[0] == 'SM':
x = F.softmax(x,dim=1)
return x
def model_quantize(model,cfg_table,quant_type,num_bits,e_bits):
for i in range(len(cfg_table)):
cfg = cfg_table[i]
if cfg[0] == 'Inc':
inc_quantize(model,cfg[1],quant_type,num_bits,e_bits)
elif cfg[0] == 'ML':
ml_quantize(model,cfg[1],cfg[2],cfg[3],quant_type,num_bits,e_bits)
elif cfg[0] == 'C':
conv_name = 'conv%d'%i
conv_layer = getattr(model,conv_name)
qname = 'q_'+conv_name
if 'B' in cfg[1]:
bn_name = 'bn%d'%i
bn_layer = getattr(model,bn_name)
if 'RL' in cfg[1]:
qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
elif 'RS' in cfg[1]:
qlayer = QConvBNReLU6(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
else:
qlayer = QConvBN(quant_type,conv_layer,bn_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
else:
qlayer = QConv2d(quant_type,conv_layer,qi=cfg[2],num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'RL':
name = 'relu%d'%i
qname = 'q_'+name
qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'RS':
name = 'relus%d'%i
qname = 'q_'+name
qlayer = QReLU6(quant_type,num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'MP':
name = 'pool%d'%i
qname = 'q_'+name
qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'AAP':
name = 'aap%d'%i
qname = 'q_'+name
qlayer = QAdaptiveAvgPool2d(quant_type,output_size=cfg[1],num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'FC':
name = 'fc%d'%i
layer = getattr(model,name)
qname = 'q_'+name
qlayer = QLinear(quant_type,layer,num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
# 增加了func='fakefreeze'
def model_utils(model,cfg_table,func,x=None):
last_qo = None
# 表示已经经过反量化，用于区别反量化不再最后，而是在softmax前的情形
done_flag = False
for i in range(len(cfg_table)):
cfg = cfg_table[i]
if cfg[0] == 'Inc':
x,last_qo = inc_utils(model,cfg[1],func,x,last_qo)
elif cfg[0] == 'ML':
x,last_qo = ml_utils(model,cfg[1],cfg[2],cfg[3],func,x,last_qo)
elif cfg[0] == 'C':
qname = 'q_conv%d'%i
qlayer = getattr(model,qname)
if func == 'forward':
x = qlayer(x)
elif func == 'inference':
# cfg[2]为True表示起始层，需要量化
if cfg[2]:
x = qlayer.qi.quantize_tensor(x)
x = qlayer.quantize_inference(x)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif func == 'fakefreeze':
qlayer.fakefreeze()
last_qo = qlayer.qo
elif cfg[0] == 'RL':
qname = 'q_relu%d'%i
qlayer = getattr(model,qname)
if func == 'forward':
x = qlayer(x)
elif func == 'inference':
x = qlayer.quantize_inference(x)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif cfg[0] == 'RS':
qname = 'q_relus%d'%i
qlayer = getattr(model,qname)
if func == 'forward':
x = qlayer(x)
elif func == 'inference':
x = qlayer.quantize_inference(x)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif cfg[0] == 'MP':
qname = 'q_pool%d'%i
qlayer = getattr(model,qname)
if func == 'forward':
x = qlayer(x)
elif func == 'inference':
x = qlayer.quantize_inference(x)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif cfg[0] == 'AAP':
qname = 'q_aap%d'%i
qlayer = getattr(model,qname)
if func == 'forward':
x = qlayer(x)
elif func == 'inference':
x = qlayer.quantize_inference(x)
elif func == 'freeze':
qlayer.freeze(last_qo)
last_qo = qlayer.qo
elif cfg[0] == 'FC':
qname = 'q_fc%d'%i
qlayer = getattr(model,qname)
if func == 'forward':
x = qlayer(x)
elif func == 'inference':
x = qlayer.quantize_inference(x)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif func == 'fakefreeze':
qlayer.fakefreeze()
last_qo = qlayer.qo
elif cfg[0] == 'D':
if func == 'forward':
name = 'drop%d'%i
layer = getattr(model,name)
x = layer(x)
elif cfg[0] == 'VW':
if func == 'inference' or func == 'forward':
if len(cfg) == 1: #default
x = x.view(x.size(0),-1)
elif cfg[0] == 'SM':
if func == 'inference':
done_flag = True
x = last_qo.dequantize_tensor(x)
x = F.softmax(x,dim=1)
elif func == 'forward':
x = F.softmax(x,dim=1)
if func == 'inference' and not done_flag:
x = last_qo.dequantize_tensor(x)
return x
def make_inc_layers(model,inc_idx):
inc_name = 'inc%d'%inc_idx
ch = inc_ch_table[inc_idx]
for i in range(4): # branch
prefix = inc_name+'_br%d_'%i
for j in range(len(inc_cfg_table[i])):
cfg = inc_cfg_table[i][j]
if cfg[0] == 'MP':
name = prefix+'pool%d'%j
layer =nn.MaxPool2d(kernel_size=cfg[1],stride=cfg[2],padding=cfg[3])
model.add_module(name,layer)
elif cfg[0] == 'RL':
name=prefix+'relu%d'%j
layer=nn.ReLU(True)
model.add_module(name,layer)
elif cfg[0] == 'C': # 'BRL' default
name=prefix+'conv%d'%j
layer=nn.Conv2d(ch[cfg[1]],ch[cfg[2]],kernel_size=cfg[3],stride=cfg[4],padding=cfg[5],bias=False)
model.add_module(name,layer)
name=prefix+'bn%d'%j
layer=nn.BatchNorm2d(ch[cfg[2]])
model.add_module(name,layer)
name=prefix+'relu%d'%j
layer=nn.ReLU(True)
model.add_module(name,layer)
def inc_forward(model,inc_idx,x):
inc_name = 'inc%d'%inc_idx
outs = []
for i in range(4):
prefix = inc_name+'_br%d_'%i
tmp = x
for j in range(len(inc_cfg_table[i])):
cfg = inc_cfg_table[i][j]
if cfg[0] == 'MP':
name = prefix+'pool%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
elif cfg[0] == 'RL':
name=prefix+'relu%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
elif cfg[0] == 'C': # 'BRL' default
name=prefix+'conv%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
name=prefix+'bn%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
name=prefix+'relu%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
outs.append(tmp)
out = torch.cat(outs,1)
return out
def inc_quantize(model,inc_idx,quant_type,num_bits,e_bits):
inc_name = 'inc%d'%inc_idx
for i in range(4):
prefix = inc_name+'_br%d_'%i
for j in range(len(inc_cfg_table[i])):
cfg = inc_cfg_table[i][j]
if cfg[0] == 'MP':
name = prefix+'pool%d'%j
qname = 'q_'+name
qlayer = QMaxPooling2d(quant_type,kernel_size=cfg[1],stride=cfg[2],padding=cfg[3],num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'RL':
name = prefix+'relu%d'%j
qname = 'q_'+name
qlayer = QReLU(quant_type, num_bits=num_bits, e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'C': # 'BRL' default
conv_name=prefix+'conv%d'%j
conv_layer=getattr(model,conv_name)
bn_name=prefix+'bn%d'%j
bn_layer=getattr(model,bn_name)
qname='q_'+conv_name
qlayer=QConvBNReLU(quant_type, conv_layer, bn_layer, num_bits=num_bits, e_bits=e_bits)
model.add_module(qname,qlayer)
qname = 'q_'+inc_name+'_concat'
qlayer = QConcat(quant_type,4,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
def inc_utils(model,inc_idx,func,x=None,qo=None):
inc_name = 'inc%d'%inc_idx
outs=[]
qos=[]
for i in range(4):
qprefix = 'q_'+inc_name+'_br%d_'%i
tmp = x
last_qo = qo
for j in range(len(inc_cfg_table[i])):
cfg = inc_cfg_table[i][j]
if cfg[0] == 'MP':
qname = qprefix+'pool%d'%j
qlayer = getattr(model,qname)
if func == 'forward':
tmp = qlayer(tmp)
elif func == 'inference':
tmp = qlayer.quantize_inference(tmp)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif cfg[0] == 'RL':
qname = qprefix+'relu%d'%j
qlayer = getattr(model,qname)
if func == 'forward':
tmp = qlayer(tmp)
elif func == 'inference':
tmp = qlayer.quantize_inference(tmp)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif cfg[0] == 'C': # 'BRL' default
qname = qprefix+'conv%d'%j
qlayer = getattr(model,qname)
if func == 'forward':
tmp = qlayer(tmp)
elif func == 'inference':
tmp = qlayer.quantize_inference(tmp)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif func == 'fakefreeze':
qlayer.fakefreeze()
last_qo = qlayer.qo
outs.append(tmp)
qos.append(last_qo)
qname = 'q_'+inc_name+'_concat'
qlayer = getattr(model,qname)
out = None
if func == 'forward':
out = qlayer(outs)
elif func == 'inference':
out = qlayer.quantize_inference(outs)
elif func == 'freeze':
qlayer.freeze(qos)
last_qo = qlayer.qo
return out,last_qo
def make_ml_layers(model,blk_type,ml_idx,blocks):
ml_name = 'ml%d'%ml_idx
if blk_type == 'BBLK':
blk_ch_table = bblk_ch_table
blk_cfg_table = bblk_cfg_table
elif blk_type == 'BTNK':
blk_ch_table = btnk_ch_table
blk_cfg_table = btnk_cfg_table
elif blk_type == 'IRES':
blk_ch_table = ires_ch_table
blk_cfg_table = ires_cfg_table
else:
raise ValueError("Make_ml_layers: Illegal blk_type")
#一个makelayer对应两行，分别表示第一个blk和其余的特征
make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0)
for i in range(1,blocks):
make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i)
# ma表示主分支，ds表示downsample
# 当cfgtable含有三个元素时，第3个表示分支合并后需经过的层。
# BasicBlock和BottleNeck合并后需经过relu层，InvertedResidual合并后无需经过relu层
def make_blk_layers(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx):
blk_name = ml_name+'_blk%d'%blk_idx
ch = blk_ch_table[ch_idx]
for i in range(2):
if i == 0:
prefix = blk_name+'_ma_'
elif i == 1:
if ch[0]: #downsample/identity_flag
prefix = blk_name+'_ds_'
else:
continue
for j in range(len(blk_cfg_table[i])):
cfg = blk_cfg_table[i][j]
if cfg[0] == 'C':
name = prefix+'conv%d'%j
layer = nn.Conv2d(ch[cfg[2]],ch[cfg[3]],kernel_size=cfg[4],stride=ch[cfg[5]],padding=cfg[6],groups=ch[cfg[7]],bias=cfg[8])
model.add_module(name,layer)
if 'B' in cfg[1]:
name = prefix+'bn%d'%j
layer=nn.BatchNorm2d(ch[cfg[3]])
model.add_module(name,layer)
if 'RL' in cfg[1]:
name = prefix+'relu%d'%j
layer = nn.ReLU(True)
model.add_module(name,layer)
elif 'RS' in cfg[1]:
name = prefix+'relus%d'%j
layer = nn.ReLU6(True)
model.add_module(name,layer)
#分支汇总
prefix = blk_name+'_'
for j in range(len(blk_cfg_table[-1])):
cfg = blk_cfg_table[-1][j]
if cfg[0] == 'RL': #当前没有blk出现汇总处有RS
name = prefix+'relu%d'%j
layer = nn.ReLU(True)
model.add_module(name,layer)
def ml_forward(model,blk_type,ml_idx,blocks,x):
ml_name = 'ml%d'%ml_idx
if blk_type == 'BBLK':
blk_ch_table = bblk_ch_table
blk_cfg_table = bblk_cfg_table
elif blk_type == 'BTNK':
blk_ch_table = btnk_ch_table
blk_cfg_table = btnk_cfg_table
elif blk_type == 'IRES':
blk_ch_table = ires_ch_table
blk_cfg_table = ires_cfg_table
else:
raise ValueError("ml_forward: Illegal blk_type")
x = blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,x)
for i in range(1,blocks):
x = blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,x)
return x
def blk_forward(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,x):
blk_name = ml_name+'_blk%d'%blk_idx
ch = blk_ch_table[ch_idx]
outs = []
for i in range(2):
tmp=x
if i == 0:
prefix = blk_name+'_ma_'
elif i == 1:
if ch[0]: #downsample/identity_flag
prefix = blk_name+'_ds_'
else:
outs.append(tmp)
continue
for j in range(len(blk_cfg_table[i])):
cfg = blk_cfg_table[i][j]
if cfg[0] == 'C':
name = prefix+'conv%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
if 'B' in cfg[1]:
name = prefix+'bn%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
if 'RL' in cfg[1]:
name = prefix+'relu%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
elif 'RS' in cfg[1]:
name = prefix+'relus%d'%j
layer = getattr(model,name)
tmp = layer(tmp)
outs.append(tmp)
#分支汇总
prefix = blk_name+'_'
for j in range(len(blk_cfg_table[-1])):
cfg = blk_cfg_table[-1][j]
if cfg[0] == 'AD':
if cfg[1] or ch[0]: #无条件加或flag为true
out = outs[0] + outs[1]
else:
out = outs[0]
elif cfg[0] == 'RL':
name = prefix+'relu%d'%j
layer = getattr(model,name)
out = layer(out)
return out
def ml_quantize(model,blk_type,ml_idx,blocks,quant_type,num_bits,e_bits):
ml_name = 'ml%d'%ml_idx
if blk_type == 'BBLK':
blk_ch_table = bblk_ch_table
blk_cfg_table = bblk_cfg_table
elif blk_type == 'BTNK':
blk_ch_table = btnk_ch_table
blk_cfg_table = btnk_cfg_table
elif blk_type == 'IRES':
blk_ch_table = ires_ch_table
blk_cfg_table = ires_cfg_table
else:
raise ValueError("ml_quantize: Illegal blk_type")
blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,quant_type,num_bits,e_bits)
for i in range(1,blocks):
blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,quant_type,num_bits,e_bits)
def blk_quantize(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,quant_type,num_bits,e_bits):
blk_name = ml_name+'_blk%d'%blk_idx
ch = blk_ch_table[ch_idx]
for i in range(2):
if i == 0:
prefix = blk_name+'_ma_'
elif i == 1:
if ch[0]: #downsample/identity_flag
prefix = blk_name+'_ds_'
else:
continue
for j in range(len(blk_cfg_table[i])):
cfg = blk_cfg_table[i][j]
if cfg[0] == 'C':
conv_name = prefix+'conv%d'%j
conv_layer = getattr(model,conv_name)
qname = 'q_'+conv_name
if 'B' in cfg[1]:
bn_name = prefix+'bn%d'%j
bn_layer = getattr(model,bn_name)
if 'RL' in cfg[1]:
qlayer = QConvBNReLU(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
elif 'RS' in cfg[1]:
qlayer = QConvBNReLU6(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
else:
qlayer = QConvBN(quant_type,conv_layer,bn_layer,num_bits=num_bits,e_bits=e_bits)
else:
qlayer = QConv2d(quant_type,conv_layer,num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
#分支汇总
prefix = blk_name+'_'
for j in range(len(blk_cfg_table[-1])):
cfg = blk_cfg_table[-1][j]
if cfg[0] == 'AD':
if cfg[1] or ch[0]: #无条件加或flag为true
qname = 'q_'+prefix+'add%d'%j
qlayer = QElementwiseAdd(quant_type,2,qi_array=False,qo=True,num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
elif cfg[0] == 'RL':
qname = 'q_'+prefix+'relu%d'%j
qlayer = QReLU(quant_type,num_bits=num_bits,e_bits=e_bits)
model.add_module(qname,qlayer)
def ml_utils(model,blk_type,ml_idx,blocks,func,x=None,qo=None):
ml_name = 'ml%d'%ml_idx
if blk_type == 'BBLK':
blk_ch_table = bblk_ch_table
blk_cfg_table = bblk_cfg_table
elif blk_type == 'BTNK':
blk_ch_table = btnk_ch_table
blk_cfg_table = btnk_cfg_table
elif blk_type == 'IRES':
blk_ch_table = ires_ch_table
blk_cfg_table = ires_cfg_table
else:
raise ValueError("ml_quantize: Illegal blk_type")
last_qo = qo
x,last_qo = blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx,0,func,x,last_qo)
for i in range(1,blocks):
x,last_qo = blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,2*ml_idx+1,i,func,x,last_qo)
return x, last_qo
def blk_utils(model,blk_ch_table,blk_cfg_table,ml_name,ch_idx,blk_idx,func,x=None,qo=None):
blk_name = ml_name+'_blk%d'%blk_idx
ch = blk_ch_table[ch_idx]
outs = []
qos = []
for i in range(2):
tmp=x
last_qo = qo
if i == 0:
qprefix = 'q_'+blk_name+'_ma_'
elif i == 1:
if ch[0]: #downsample/identity_flag
qprefix = 'q_'+blk_name+'_ds_'
else:
outs.append(tmp)
qos.append(last_qo)
continue
for j in range(len(blk_cfg_table[i])):
cfg = blk_cfg_table[i][j]
if cfg[0] == 'C':
qname = qprefix+'conv%d'%j
qlayer = getattr(model,qname)
if func == 'forward':
tmp = qlayer(tmp)
elif func == 'inference':
tmp = qlayer.quantize_inference(tmp)
elif func == 'freeze':
qlayer.freeze(last_qo)
elif func == 'fakefreeze':
qlayer.fakefreeze()
last_qo = qlayer.qo
outs.append(tmp)
qos.append(last_qo)
#分支汇总
qprefix = 'q_'+blk_name+'_'
for j in range(len(blk_cfg_table[-1])):
cfg = blk_cfg_table[-1][j]
if cfg[0] == 'AD':
if cfg[1] or ch[0]: #无条件加或flag为true
qname = qprefix+'add%d'%j
qlayer = getattr(model,qname)
out = None
if func == 'forward':
out = qlayer(outs)
elif func == 'inference':
out = qlayer.quantize_inference(outs)
elif func == 'freeze':
qlayer.freeze(qos)
last_qo = qlayer.qo
else:
out = outs[0]
last_qo = qos[0]
elif cfg[0] == 'RL':
qname = qprefix+'relu%d'%j
qlayer = getattr(model,qname)
if func == 'forward':
out = qlayer(out)
elif func == 'inference':
out = qlayer.quantize_inference(out)
elif func == 'freeze':
qlayer.freeze(last_qo)
return out,last_qo
\ No newline at end of file
ykl/ALL/module.py 0 → 100644
import math
import numpy as np
import gol
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.autograd import Variable
from function import FakeQuantize
def js_div(p_output, q_output, get_softmax=True):
"""
Function that measures JS divergence between target and output logits:
"""
KLDivLoss = nn.KLDivLoss(reduction='sum')
if get_softmax:
p_output = F.softmax(p_output)
q_output = F.softmax(q_output)
log_mean_output = ((p_output + q_output)/2).log()
return (KLDivLoss(log_mean_output, p_output) + KLDivLoss(log_mean_output, q_output))/2
# 获取最近的量化值
def get_nearest_val(quant_type, x, is_bias=False, block_size=1000000):
if quant_type == 'INT':
return x.round_()
plist = gol.get_value(is_bias)
shape = x.shape
xhard = x.view(-1)
xout = torch.zeros_like(xhard)
plist = plist.type_as(x)
n_blocks = (x.numel() + block_size - 1) // block_size
for i in range(n_blocks):
start_idx = i * block_size
end_idx = min(start_idx + block_size, xhard.numel())
block_size_i = end_idx - start_idx
xblock = xhard[start_idx:end_idx]
plist_block = plist.unsqueeze(1)
idx = (xblock.unsqueeze(0) - plist_block).abs().min(dim=0)[1]
xhard_block = plist[idx].view(xblock.shape)
xout[start_idx:end_idx] = (xhard_block - xblock).detach() + xblock
xout = xout.view(shape)
return xout
# 采用对称有符号量化时，获取量化范围最大值
def get_qmax(quant_type,num_bits=None, e_bits=None):
if quant_type == 'INT':
qmax = 2. ** (num_bits - 1) - 1
elif quant_type == 'POT':
qmax = 1
else: #FLOAT
m_bits = num_bits - 1 - e_bits
dist_m = 2 ** (-m_bits)
e = 2 ** (e_bits - 1)
expo = 2 ** e
m = 2 ** m_bits -1
frac = 1. + m * dist_m
qmax = frac * expo
return qmax
# 都采用有符号量化，zeropoint都置为0
def calcScaleZeroPoint(min_val, max_val, qmax):
scale = torch.max(max_val.abs(),min_val.abs()) / qmax
zero_point = torch.tensor(0.)
return scale, zero_point
# 将输入进行量化，输入输出都为tensor
def quantize_tensor(quant_type, x, scale, zero_point, qmax, is_bias=False):
# 量化后范围，直接根据位宽确定
qmin = -qmax
q_x = zero_point + x / scale
q_x.clamp_(qmin, qmax)
q_x = get_nearest_val(quant_type, q_x, is_bias)
return q_x
# bias使用不同精度，需要根据量化类型指定num_bits/e_bits
def bias_qmax(quant_type):
if quant_type == 'INT':
return get_qmax(quant_type, 64)
elif quant_type == 'POT':
return get_qmax(quant_type)
else:
return get_qmax(quant_type, 16, 7)
# 转化为FP32，不需再做限制
def dequantize_tensor(q_x, scale, zero_point):
return scale * (q_x - zero_point)
class QParam(nn.Module):
def __init__(self,quant_type, num_bits=8, e_bits=3):
super(QParam, self).__init__()
self.quant_type = quant_type
self.num_bits = num_bits
self.e_bits = e_bits
self.qmax = get_qmax(quant_type, num_bits, e_bits)
scale = torch.tensor([], requires_grad=False)
zero_point = torch.tensor([], requires_grad=False)
min = torch.tensor([], requires_grad=False)
max = torch.tensor([], requires_grad=False)
# 通过注册为register，使得buffer可以被记录到state_dict
self.register_buffer('scale', scale)
self.register_buffer('zero_point', zero_point)
self.register_buffer('min', min)
self.register_buffer('max', max)
# 更新统计范围及量化参数
def update(self, tensor):
if self.max.nelement() == 0 or self.max.data < tensor.max().data:
self.max.data = tensor.max().data
self.max.clamp_(min=0)
if self.min.nelement() == 0 or self.min.data > tensor.min().data:
self.min.data = tensor.min().data
self.min.clamp_(max=0)
self.scale, self.zero_point = calcScaleZeroPoint(self.min, self.max, self.qmax)
def quantize_tensor(self, tensor):
return quantize_tensor(self.quant_type, tensor, self.scale, self.zero_point, self.qmax)
def dequantize_tensor(self, q_x):
return dequantize_tensor(q_x, self.scale, self.zero_point)
# 该方法保证了可以从state_dict里恢复
def _load_from_state_dict(self, state_dict, prefix, local_metadata, strict, missing_keys, unexpected_keys,
error_msgs):
key_names = ['scale', 'zero_point', 'min', 'max']
for key in key_names:
value = getattr(self, key)
value.data = state_dict[prefix + key].data
state_dict.pop(prefix + key)
# 该方法返回值将是打印该对象的结果
def __str__(self):
info = 'scale: %.10f ' % self.scale
info += 'zp: %.6f ' % self.zero_point
info += 'min: %.6f ' % self.min
info += 'max: %.6f' % self.max
return info
# 作为具体量化层的父类，qi和qo分别为量化输入/输出
class QModule(nn.Module):
def __init__(self,quant_type, qi=False, qo=True, num_bits=8, e_bits=3):
super(QModule, self).__init__()
if qi:
self.qi = QParam(quant_type,num_bits, e_bits)
if qo:
self.qo = QParam(quant_type,num_bits, e_bits)
self.quant_type = quant_type
self.num_bits = num_bits
self.e_bits = e_bits
self.bias_qmax = bias_qmax(quant_type)
def freeze(self):
pass # 空语句
def quantize_inference(self, x):
raise NotImplementedError('quantize_inference should be implemented.')
def fakefreeze(self):
pass
"""
QModule 量化卷积
:quant_type: 量化类型
:conv_module: 卷积模块
:qi: 是否量化输入特征图
:qo: 是否量化输出特征图
:num_bits: 8位bit数
"""
class QConv2d(QModule):
def __init__(self, quant_type, conv_module, qi=False, qo=True, num_bits=8, e_bits=3):
super(QConv2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
self.conv_module = conv_module
self.qw = QParam(quant_type, num_bits,e_bits)
self.register_buffer('M', torch.tensor([], requires_grad=False)) # 将M注册为buffer
# freeze方法可以固定真量化的权重参数，并将该值更新到原全精度层上，便于散度计算
def freeze(self, qi=None, qo=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
# 这里因为在池化或者激活的输入，不需要对最大值和最小是进行额外的统计，会共享相同的输出
if qi is not None:
self.qi = qi
if qo is not None:
self.qo = qo
# 根据https://zhuanlan.zhihu.com/p/156835141, 这是式3 的系数
self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
self.conv_module.weight.data = self.qw.quantize_tensor(self.conv_module.weight.data)
self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
#考虑conv层无bias,此时forward和inference传入none亦可
if self.conv_module.bias is not None:
self.conv_module.bias.data = quantize_tensor(self.quant_type,
self.conv_module.bias.data, scale=self.qi.scale * self.qw.scale,
zero_point=0.,qmax=self.bias_qmax, is_bias=True)
def fakefreeze(self):
self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
if self.conv_module.bias is not None:
self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
def forward(self, x): # 前向传播,输入张量,x为浮点型数据
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi) # 对输入张量X完成量化
# foward前更新qw，保证量化weight时候scale正确
self.qw.update(self.conv_module.weight.data)
# 注意:此处主要为了统计各层x和weight范围，未对bias进行量化操作
tmp_wgt = FakeQuantize.apply(self.conv_module.weight, self.qw)
x = F.conv2d(x, tmp_wgt, self.conv_module.bias,
stride=self.conv_module.stride,
padding=self.conv_module.padding,
dilation=self.conv_module.dilation,
groups=self.conv_module.groups)
if hasattr(self, 'qo'):
self.qo.update(x)
x = FakeQuantize.apply(x, self.qo)
return x
# 利用公式 q_a = M(\sigma(q_w-Z_w)(q_x-Z_x) + q_b)
def quantize_inference(self, x): # 此处input为已经量化的qx
x = x - self.qi.zero_point
x = self.conv_module(x)
x = self.M * x
x = get_nearest_val(self.quant_type,x)
x = x + self.qo.zero_point
return x
class QLinear(QModule):
def __init__(self, quant_type, fc_module, qi=False, qo=True, num_bits=8, e_bits=3):
super(QLinear, self).__init__(quant_type, qi, qo, num_bits, e_bits)
self.fc_module = fc_module
self.qw = QParam(quant_type, num_bits, e_bits)
self.register_buffer('M', torch.tensor([], requires_grad=False)) # 将M注册为buffer
def freeze(self, qi=None, qo=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
if qi is not None:
self.qi = qi
if qo is not None:
self.qo = qo
self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
self.fc_module.weight.data = self.qw.quantize_tensor(self.fc_module.weight.data)
self.fc_module.weight.data = self.fc_module.weight.data - self.qw.zero_point
if self.fc_module.bias is not None:
self.fc_module.bias.data = quantize_tensor(self.quant_type,
self.fc_module.bias.data, scale=self.qi.scale * self.qw.scale,
zero_point=0., qmax=self.bias_qmax, is_bias=True)
def fakefreeze(self):
self.fc_module.weight.data = self.qw.dequantize_tensor(self.fc_module.weight.data)
if self.fc_module.bias is not None:
self.fc_module.bias.data = dequantize_tensor(self.fc_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
self.qw.update(self.fc_module.weight.data)
tmp_wgt = FakeQuantize.apply(self.fc_module.weight, self.qw)
x = F.linear(x, tmp_wgt, self.fc_module.bias)
if hasattr(self, 'qo'):
self.qo.update(x)
x = FakeQuantize.apply(x, self.qo)
return x
def quantize_inference(self, x):
x = x - self.qi.zero_point
x = self.fc_module(x)
x = self.M * x
x = get_nearest_val(self.quant_type,x)
x = x + self.qo.zero_point
return x
class QReLU(QModule):
def __init__(self,quant_type, qi=False, qo=False, num_bits=8, e_bits=3):
super(QReLU, self).__init__(quant_type, qi, qo, num_bits, e_bits)
def freeze(self, qi=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if qi is not None:
self.qi = qi
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
x = F.relu(x)
return x
def quantize_inference(self, x):
x = x.clone()
x[x < self.qi.zero_point] = self.qi.zero_point
return x
class QReLU6(QModule):
def __init__(self,quant_type, qi=False, qo=False, num_bits=8, e_bits=3):
super(QReLU6, self).__init__(quant_type, qi, qo, num_bits, e_bits)
def freeze(self, qi=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if qi is not None:
self.qi = qi
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
x = F.relu6(x)
return x
def quantize_inference(self, x):
x = x.clone()
upper = torch.tensor(6)
qupper = self.qi.quantize_tensor(upper)
x.clamp_(min=0,max=qupper.item())
return x
class QMaxPooling2d(QModule):
def __init__(self, quant_type, kernel_size=3, stride=1, padding=0, qi=False, qo=False, num_bits=8,e_bits=3):
super(QMaxPooling2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
self.kernel_size = kernel_size
self.stride = stride
self.padding = padding
def freeze(self, qi=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if qi is not None:
self.qi = qi
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
x = F.max_pool2d(x, self.kernel_size, self.stride, self.padding)
return x
def quantize_inference(self, x):
return F.max_pool2d(x, self.kernel_size, self.stride, self.padding)
class QAdaptiveAvgPool2d(QModule):
def __init__(self, quant_type, output_size, qi=False, qo=True, num_bits=8,e_bits=3):
super(QAdaptiveAvgPool2d, self).__init__(quant_type, qi, qo, num_bits, e_bits)
self.output_size = output_size
self.register_buffer('M', torch.tensor([], requires_grad=False)) # 将M注册为buffer
def freeze(self, qi=None, qo=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
if qi is not None:
self.qi = qi
if qo is not None:
self.qo = qo
self.M.data = (self.qi.scale / self.qo.scale).data
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
x = F.adaptive_avg_pool2d(x, self.output_size)
if hasattr(self, 'qo'):
self.qo.update(x)
x = FakeQuantize.apply(x, self.qo)
return x
def quantize_inference(self, x):
x = x - self.qi.zero_point
x = F.adaptive_avg_pool2d(x, self.output_size)
x = self.M * x
x = get_nearest_val(self.quant_type,x)
x = x+self.qo.zero_point
return x
class QConvBN(QModule):
def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
super(QConvBN, self).__init__(quant_type, qi, qo, num_bits, e_bits)
self.conv_module = conv_module
self.bn_module = bn_module
self.qw = QParam(quant_type, num_bits,e_bits)
self.register_buffer('M', torch.tensor([], requires_grad=False)) # 将M注册为buffer
def fold_bn(self, mean, std):
if self.bn_module.affine:
gamma_ = self.bn_module.weight / std
weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
if self.conv_module.bias is not None:
bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
else:
bias = self.bn_module.bias - gamma_ * mean
else:
gamma_ = 1 / std
weight = self.conv_module.weight * gamma_
if self.conv_module.bias is not None:
bias = gamma_ * self.conv_module.bias - gamma_ * mean
else:
bias = -gamma_ * mean
return weight, bias
def freeze(self, qi=None, qo=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
if qi is not None:
self.qi = qi
if qo is not None:
self.qo = qo
self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
weight, bias = self.fold_bn(self.bn_module.running_mean, std)
self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
if self.conv_module.bias is not None:
self.conv_module.bias.data = quantize_tensor(self.quant_type,
bias.data, scale=self.qi.scale * self.qw.scale,
zero_point=0., qmax=self.bias_qmax,is_bias=True)
else:
bias = quantize_tensor(self.quant_type,
bias, scale=self.qi.scale * self.qw.scale,
zero_point=0., qmax=self.bias_qmax,is_bias=True)
self.conv_module.bias = torch.nn.Parameter(bias)
def fakefreeze(self):
self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
if self.training:
y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias,
stride=self.conv_module.stride,
padding=self.conv_module.padding,
dilation=self.conv_module.dilation,
groups=self.conv_module.groups)
y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
# mean = y.mean(1)
# var = y.var(1)
mean = y.mean(1).detach()
var = y.var(1).detach()
self.bn_module.running_mean = \
(1 - self.bn_module.momentum) * self.bn_module.running_mean + \
self.bn_module.momentum * mean
self.bn_module.running_var = \
(1 - self.bn_module.momentum) * self.bn_module.running_var + \
self.bn_module.momentum * var
else:
mean = Variable(self.bn_module.running_mean)
var = Variable(self.bn_module.running_var)
std = torch.sqrt(var + self.bn_module.eps)
weight, bias = self.fold_bn(mean, std)
self.qw.update(weight.data)
x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias,
stride=self.conv_module.stride,
padding=self.conv_module.padding,
dilation=self.conv_module.dilation,
groups=self.conv_module.groups)
if hasattr(self, 'qo'):
self.qo.update(x)
x = FakeQuantize.apply(x, self.qo)
return x
def quantize_inference(self, x):
x = x - self.qi.zero_point
x = self.conv_module(x)
x = self.M * x
x = get_nearest_val(self.quant_type,x)
x = x + self.qo.zero_point
return x
class QConvBNReLU(QModule):
def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
super(QConvBNReLU, self).__init__(quant_type, qi, qo, num_bits, e_bits)
self.conv_module = conv_module
self.bn_module = bn_module
self.qw = QParam(quant_type, num_bits,e_bits)
self.register_buffer('M', torch.tensor([], requires_grad=False)) # 将M注册为buffer
def fold_bn(self, mean, std):
if self.bn_module.affine:
gamma_ = self.bn_module.weight / std
weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
if self.conv_module.bias is not None:
bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
else:
bias = self.bn_module.bias - gamma_ * mean
else:
gamma_ = 1 / std
weight = self.conv_module.weight * gamma_
if self.conv_module.bias is not None:
bias = gamma_ * self.conv_module.bias - gamma_ * mean
else:
bias = -gamma_ * mean
return weight, bias
def freeze(self, qi=None, qo=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
if qi is not None:
self.qi = qi
if qo is not None:
self.qo = qo
self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
weight, bias = self.fold_bn(self.bn_module.running_mean, std)
self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
if self.conv_module.bias is not None:
self.conv_module.bias.data = quantize_tensor(self.quant_type,
bias.data, scale=self.qi.scale * self.qw.scale,
zero_point=0., qmax=self.bias_qmax,is_bias=True)
else:
bias = quantize_tensor(self.quant_type,
bias, scale=self.qi.scale * self.qw.scale,
zero_point=0., qmax=self.bias_qmax,is_bias=True)
self.conv_module.bias = torch.nn.Parameter(bias)
def fakefreeze(self):
self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
if self.training:
y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias,
stride=self.conv_module.stride,
padding=self.conv_module.padding,
dilation=self.conv_module.dilation,
groups=self.conv_module.groups)
y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
# mean = y.mean(1)
# var = y.var(1)
mean = y.mean(1).detach()
var = y.var(1).detach()
self.bn_module.running_mean = \
(1 - self.bn_module.momentum) * self.bn_module.running_mean + \
self.bn_module.momentum * mean
self.bn_module.running_var = \
(1 - self.bn_module.momentum) * self.bn_module.running_var + \
self.bn_module.momentum * var
else:
mean = Variable(self.bn_module.running_mean)
var = Variable(self.bn_module.running_var)
std = torch.sqrt(var + self.bn_module.eps)
weight, bias = self.fold_bn(mean, std)
self.qw.update(weight.data)
x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias,
stride=self.conv_module.stride,
padding=self.conv_module.padding,
dilation=self.conv_module.dilation,
groups=self.conv_module.groups)
x = F.relu(x)
if hasattr(self, 'qo'):
self.qo.update(x)
x = FakeQuantize.apply(x, self.qo)
return x
def quantize_inference(self, x):
x = x - self.qi.zero_point
x = self.conv_module(x)
x = self.M * x
x = get_nearest_val(self.quant_type,x)
x = x + self.qo.zero_point
x.clamp_(min=0)
return x
class QConvBNReLU6(QModule):
def __init__(self, quant_type, conv_module, bn_module, qi=False, qo=True, num_bits=8, e_bits=3):
super(QConvBNReLU6, self).__init__(quant_type, qi, qo, num_bits, e_bits)
self.conv_module = conv_module
self.bn_module = bn_module
self.qw = QParam(quant_type, num_bits,e_bits)
self.register_buffer('M', torch.tensor([], requires_grad=False)) # 将M注册为buffer
def fold_bn(self, mean, std):
if self.bn_module.affine:
gamma_ = self.bn_module.weight / std
weight = self.conv_module.weight * gamma_.view(self.conv_module.out_channels, 1, 1, 1)
if self.conv_module.bias is not None:
bias = gamma_ * self.conv_module.bias - gamma_ * mean + self.bn_module.bias
else:
bias = self.bn_module.bias - gamma_ * mean
else:
gamma_ = 1 / std
weight = self.conv_module.weight * gamma_
if self.conv_module.bias is not None:
bias = gamma_ * self.conv_module.bias - gamma_ * mean
else:
bias = -gamma_ * mean
return weight, bias
def freeze(self, qi=None, qo=None):
if hasattr(self, 'qi') and qi is not None:
raise ValueError('qi has been provided in init function.')
if not hasattr(self, 'qi') and qi is None:
raise ValueError('qi is not existed, should be provided.')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
if qi is not None:
self.qi = qi
if qo is not None:
self.qo = qo
self.M.data = (self.qw.scale * self.qi.scale / self.qo.scale).data
std = torch.sqrt(self.bn_module.running_var + self.bn_module.eps)
weight, bias = self.fold_bn(self.bn_module.running_mean, std)
self.conv_module.weight.data = self.qw.quantize_tensor(weight.data)
self.conv_module.weight.data = self.conv_module.weight.data - self.qw.zero_point
if self.conv_module.bias is not None:
self.conv_module.bias.data = quantize_tensor(self.quant_type,
bias.data, scale=self.qi.scale * self.qw.scale,
zero_point=0., qmax=self.bias_qmax,is_bias=True)
else:
bias = quantize_tensor(self.quant_type,
bias, scale=self.qi.scale * self.qw.scale,
zero_point=0., qmax=self.bias_qmax,is_bias=True)
self.conv_module.bias = torch.nn.Parameter(bias)
def fakefreeze(self):
self.conv_module.weight.data = self.qw.dequantize_tensor(self.conv_module.weight.data)
self.conv_module.bias.data = dequantize_tensor(self.conv_module.bias.data,scale=self.qi.scale*self.qw.scale,zero_point=0.)
def forward(self, x):
if hasattr(self, 'qi'):
self.qi.update(x)
x = FakeQuantize.apply(x, self.qi)
if self.training:
y = F.conv2d(x, self.conv_module.weight, self.conv_module.bias,
stride=self.conv_module.stride,
padding=self.conv_module.padding,
dilation=self.conv_module.dilation,
groups=self.conv_module.groups)
y = y.permute(1, 0, 2, 3) # NCHW -> CNHW
y = y.contiguous().view(self.conv_module.out_channels, -1) # CNHW -> C,NHW
# mean = y.mean(1)
# var = y.var(1)
mean = y.mean(1).detach()
var = y.var(1).detach()
self.bn_module.running_mean = \
(1 - self.bn_module.momentum) * self.bn_module.running_mean + \
self.bn_module.momentum * mean
self.bn_module.running_var = \
(1 - self.bn_module.momentum) * self.bn_module.running_var + \
self.bn_module.momentum * var
else:
mean = Variable(self.bn_module.running_mean)
var = Variable(self.bn_module.running_var)
std = torch.sqrt(var + self.bn_module.eps)
weight, bias = self.fold_bn(mean, std)
self.qw.update(weight.data)
x = F.conv2d(x, FakeQuantize.apply(weight, self.qw), bias,
stride=self.conv_module.stride,
padding=self.conv_module.padding,
dilation=self.conv_module.dilation,
groups=self.conv_module.groups)
x = F.relu6(x)
if hasattr(self, 'qo'):
self.qo.update(x)
x = FakeQuantize.apply(x, self.qo)
return x
def quantize_inference(self, x):
upper = torch.tensor(6)
qupper = self.qo.quantize_tensor(upper)
x = x - self.qi.zero_point
x = self.conv_module(x)
x = self.M * x
x = get_nearest_val(self.quant_type,x)
x = x + self.qo.zero_point
x.clamp_(min=0,max=qupper.item())
return x
# 作为具体量化层的父类，qi和qo分别为量化输入/输出
# 用于处理多个层结果或qo以array形式传入
class QModule_array(nn.Module):
def __init__(self,quant_type,len,qi_array=False, qo=True, num_bits=8, e_bits=3):
super(QModule_array, self).__init__()
if qi_array:
for i in range(len):
self.add_module('qi%d'%i,QParam(quant_type,num_bits, e_bits))
if qo:
self.qo = QParam(quant_type,num_bits, e_bits)
self.quant_type = quant_type
self.num_bits = num_bits
self.e_bits = e_bits
self.bias_qmax = bias_qmax(quant_type)
self.len = len
def freeze(self):
pass # 空语句
def quantize_inference(self, x):
raise NotImplementedError('quantize_inference should be implemented.')
class QElementwiseAdd(QModule_array):
def __init__(self, quant_type, len, qi_array=False, qo=True, num_bits=8, e_bits=3):
super(QElementwiseAdd,self).__init__(quant_type,len,qi_array,qo,num_bits,e_bits)
for i in range(len):
self.register_buffer('M%d'%i,torch.tensor([], requires_grad=False))
def freeze(self, qi_array=None, qo=None):
if qi_array is None:
raise ValueError('qi_array should be provided')
elif len(qi_array) != self.len:
raise ValueError('qi_array len no match')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
for i in range(self.len):
self.add_module('qi%d'%i,qi_array[i])
if qo is not None:
self.qo = qo
for i in range(self.len):
getattr(self,'M%d'%i).data = (getattr(self,'qi%d'%i).scale / self.qo.scale).data
def forward(self,x_array):
outs=[]
for i in range(self.len):
x = x_array[i]
if hasattr(self,'qi%d'%i):
qi = getattr(self,'qi%d'%i)
qi.update(x)
x = FakeQuantize.apply(x,qi)
outs.append(x)
out = outs[0]+outs[1]
if hasattr(self,'qo'):
self.qo.update(x)
out = FakeQuantize.apply(out,self.qo)
return out
def quantize_inference(self, x_array):
outs=[]
for i in range(self.len):
qi = getattr(self,'qi%d'%i)
x = x_array[i] - qi.zero_point
x = getattr(self,'M%d'%i) * x
outs.append(x)
out = outs[0]+outs[1]
out = get_nearest_val(self.quant_type,out)
out = out + self.qo.zero_point
return out
class QConcat(QModule_array):
def __init__(self, quant_type, len, qi_array=False, qo=True, num_bits=8, e_bits=3):
super(QConcat,self).__init__(quant_type, len, qi_array, qo, num_bits, e_bits)
for i in range(len):
self.register_buffer('M%d'%i,torch.tensor([], requires_grad=False))
def freeze(self, qi_array=None, qo=None):
if qi_array is None:
raise ValueError('qi_array should be provided')
elif len(qi_array) != self.len:
raise ValueError('qi_array len no match')
if hasattr(self, 'qo') and qo is not None:
raise ValueError('qo has been provided in init function.')
if not hasattr(self, 'qo') and qo is None:
raise ValueError('qo is not existed, should be provided.')
for i in range(self.len):
self.add_module('qi%d'%i,qi_array[i])
if qo is not None:
self.qo = qo
for i in range(self.len):
getattr(self,'M%d'%i).data = (getattr(self,'qi%d'%i).scale / self.qo.scale).data
def forward(self,x_array):
outs=[]
for i in range(self.len):
x = x_array[i]
if hasattr(self,'qi%d'%i):
qi = getattr(self,'qi%d'%i)
qi.update(x)
x = FakeQuantize.apply(x,qi)
outs.append(x)
out = torch.cat(outs,1)
if hasattr(self,'qo'):
self.qo.update(x)
out = FakeQuantize.apply(out,self.qo)
return out
def quantize_inference(self, x_array):
outs=[]
for i in range(self.len):
qi = getattr(self,'qi%d'%i)
x = x_array[i] - qi.zero_point
x = getattr(self,'M%d'%i) * x
outs.append(x)
out = torch.concat(outs,1)
out = get_nearest_val(self.quant_type,out)
out = out + self.qo.zero_point
return out
\ No newline at end of file
ykl/ALL/param_flops/cifar10/AlexNet.txt 0 → 100644
Model(
3.87 M, 100.000% Params, 70.08 MMac, 100.000% MACs,
(conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.309% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
(pool2): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3): Conv2d(18.5 k, 0.478% Params, 4.73 MMac, 6.756% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu4): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool5): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv6): Conv2d(73.86 k, 1.909% Params, 4.73 MMac, 6.745% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu7): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
(conv8): Conv2d(295.17 k, 7.630% Params, 18.89 MMac, 26.955% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(conv10): Conv2d(590.08 k, 15.252% Params, 37.77 MMac, 53.887% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu11): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool12): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(drop14): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc15): Linear(2.36 M, 61.010% Params, 2.36 MMac, 3.368% MACs, in_features=2304, out_features=1024, bias=True)
(relu16): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
(drop17): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc18): Linear(524.8 k, 13.565% Params, 524.8 KMac, 0.749% MACs, in_features=1024, out_features=512, bias=True)
(relu19): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
(fc20): Linear(5.13 k, 0.133% Params, 5.13 KMac, 0.007% MACs, in_features=512, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar10/AlexNet_BN.txt 0 → 100644
Model(
3.87 M, 100.000% Params, 70.26 MMac, 100.000% MACs,
(conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.306% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(64, 0.002% Params, 65.54 KMac, 0.093% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
(pool1): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv2): Conv2d(18.5 k, 0.478% Params, 4.73 MMac, 6.739% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn2): BatchNorm2d(128, 0.003% Params, 32.77 KMac, 0.047% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool3): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv4): Conv2d(73.86 k, 1.908% Params, 4.73 MMac, 6.727% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn4): BatchNorm2d(256, 0.007% Params, 16.38 KMac, 0.023% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu4): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
(conv5): Conv2d(295.17 k, 7.627% Params, 18.89 MMac, 26.886% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn5): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu5): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(conv6): Conv2d(590.08 k, 15.247% Params, 37.77 MMac, 53.748% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn6): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool7): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(drop9): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc10): Linear(2.36 M, 60.987% Params, 2.36 MMac, 3.359% MACs, in_features=2304, out_features=1024, bias=True)
(relu11): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
(drop12): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc13): Linear(524.8 k, 13.560% Params, 524.8 KMac, 0.747% MACs, in_features=1024, out_features=512, bias=True)
(relu14): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
(fc15): Linear(5.13 k, 0.133% Params, 5.13 KMac, 0.007% MACs, in_features=512, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar10/Inception_BN.txt 0 → 100644
Model(
5.88 M, 100.000% Params, 1.46 GMac, 100.000% MACs,
(conv0): Conv2d(1.79 k, 0.030% Params, 1.84 MMac, 0.125% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(conv2): Conv2d(36.93 k, 0.628% Params, 37.81 MMac, 2.583% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu3): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc0_br0_conv0): Conv2d(4.1 k, 0.070% Params, 4.19 MMac, 0.287% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br0_bn0): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc0_br1_conv0): Conv2d(6.14 k, 0.105% Params, 6.29 MMac, 0.430% MACs, 64, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br1_bn0): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br1_relu0): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
(inc0_br1_conv1): Conv2d(110.59 k, 1.881% Params, 113.25 MMac, 7.737% MACs, 96, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc0_br1_bn1): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc0_br2_conv0): Conv2d(1.02 k, 0.017% Params, 1.05 MMac, 0.072% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br2_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.002% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br2_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc0_br2_conv1): Conv2d(12.8 k, 0.218% Params, 13.11 MMac, 0.895% MACs, 16, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc0_br2_bn1): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc0_br3_pool0): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc0_br3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc0_br3_conv2): Conv2d(2.05 k, 0.035% Params, 2.1 MMac, 0.143% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br3_bn2): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc1_br0_conv0): Conv2d(32.77 k, 0.557% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br0_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br0_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc1_br1_conv0): Conv2d(32.77 k, 0.557% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br1_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br1_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc1_br1_conv1): Conv2d(221.18 k, 3.763% Params, 226.49 MMac, 15.474% MACs, 128, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc1_br1_bn1): BatchNorm2d(384, 0.007% Params, 393.22 KMac, 0.027% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br1_relu1): ReLU(0, 0.000% Params, 196.61 KMac, 0.013% MACs, inplace=True)
(inc1_br2_conv0): Conv2d(8.19 k, 0.139% Params, 8.39 MMac, 0.573% MACs, 256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br2_bn0): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br2_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc1_br2_conv1): Conv2d(76.8 k, 1.306% Params, 78.64 MMac, 5.373% MACs, 32, 96, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc1_br2_bn1): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br2_relu1): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
(inc1_br3_pool0): MaxPool2d(0, 0.000% Params, 262.14 KMac, 0.018% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc1_br3_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.018% MACs, inplace=True)
(inc1_br3_conv2): Conv2d(16.38 k, 0.279% Params, 16.78 MMac, 1.146% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br3_bn2): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br3_relu2): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(pool6): MaxPool2d(0, 0.000% Params, 491.52 KMac, 0.034% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(inc2_br0_conv0): Conv2d(92.16 k, 1.568% Params, 23.59 MMac, 1.612% MACs, 480, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br0_bn0): BatchNorm2d(384, 0.007% Params, 98.3 KMac, 0.007% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br0_relu0): ReLU(0, 0.000% Params, 49.15 KMac, 0.003% MACs, inplace=True)
(inc2_br1_conv0): Conv2d(46.08 k, 0.784% Params, 11.8 MMac, 0.806% MACs, 480, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br1_bn0): BatchNorm2d(192, 0.003% Params, 49.15 KMac, 0.003% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br1_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
(inc2_br1_conv1): Conv2d(179.71 k, 3.057% Params, 46.01 MMac, 3.143% MACs, 96, 208, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc2_br1_bn1): BatchNorm2d(416, 0.007% Params, 106.5 KMac, 0.007% MACs, 208, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br1_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
(inc2_br2_conv0): Conv2d(7.68 k, 0.131% Params, 1.97 MMac, 0.134% MACs, 480, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br2_bn0): BatchNorm2d(32, 0.001% Params, 8.19 KMac, 0.001% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br2_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.000% MACs, inplace=True)
(inc2_br2_conv1): Conv2d(19.2 k, 0.327% Params, 4.92 MMac, 0.336% MACs, 16, 48, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc2_br2_bn1): BatchNorm2d(96, 0.002% Params, 24.58 KMac, 0.002% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br2_relu1): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
(inc2_br3_pool0): MaxPool2d(0, 0.000% Params, 122.88 KMac, 0.008% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc2_br3_relu1): ReLU(0, 0.000% Params, 122.88 KMac, 0.008% MACs, inplace=True)
(inc2_br3_conv2): Conv2d(30.72 k, 0.523% Params, 7.86 MMac, 0.537% MACs, 480, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc3_br0_conv0): Conv2d(81.92 k, 1.394% Params, 20.97 MMac, 1.433% MACs, 512, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br0_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br0_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
(inc3_br1_conv0): Conv2d(57.34 k, 0.975% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br1_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br1_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
(inc3_br1_conv1): Conv2d(225.79 k, 3.841% Params, 57.8 MMac, 3.949% MACs, 112, 224, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc3_br1_bn1): BatchNorm2d(448, 0.008% Params, 114.69 KMac, 0.008% MACs, 224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br1_relu1): ReLU(0, 0.000% Params, 57.34 KMac, 0.004% MACs, inplace=True)
(inc3_br2_conv0): Conv2d(12.29 k, 0.209% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
(inc3_br2_conv1): Conv2d(38.4 k, 0.653% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc3_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc3_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc3_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc3_br3_conv2): Conv2d(32.77 k, 0.557% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc4_br0_conv0): Conv2d(65.54 k, 1.115% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br0_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br0_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc4_br1_conv0): Conv2d(65.54 k, 1.115% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br1_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br1_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc4_br1_conv1): Conv2d(294.91 k, 5.017% Params, 75.5 MMac, 5.158% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc4_br1_bn1): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc4_br2_conv0): Conv2d(12.29 k, 0.209% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
(inc4_br2_conv1): Conv2d(38.4 k, 0.653% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc4_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc4_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc4_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc4_br3_conv2): Conv2d(32.77 k, 0.557% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc5_br0_conv0): Conv2d(57.34 k, 0.975% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br0_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br0_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
(inc5_br1_conv0): Conv2d(73.73 k, 1.254% Params, 18.87 MMac, 1.289% MACs, 512, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br1_bn0): BatchNorm2d(288, 0.005% Params, 73.73 KMac, 0.005% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br1_relu0): ReLU(0, 0.000% Params, 36.86 KMac, 0.003% MACs, inplace=True)
(inc5_br1_conv1): Conv2d(373.25 k, 6.349% Params, 95.55 MMac, 6.528% MACs, 144, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc5_br1_bn1): BatchNorm2d(576, 0.010% Params, 147.46 KMac, 0.010% MACs, 288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br1_relu1): ReLU(0, 0.000% Params, 73.73 KMac, 0.005% MACs, inplace=True)
(inc5_br2_conv0): Conv2d(16.38 k, 0.279% Params, 4.19 MMac, 0.287% MACs, 512, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc5_br2_conv1): Conv2d(51.2 k, 0.871% Params, 13.11 MMac, 0.895% MACs, 32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc5_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc5_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc5_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc5_br3_conv2): Conv2d(32.77 k, 0.557% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc6_br0_conv0): Conv2d(135.17 k, 2.299% Params, 34.6 MMac, 2.364% MACs, 528, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br0_bn0): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc6_br1_conv0): Conv2d(84.48 k, 1.437% Params, 21.63 MMac, 1.478% MACs, 528, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br1_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br1_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
(inc6_br1_conv1): Conv2d(460.8 k, 7.839% Params, 117.96 MMac, 8.059% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc6_br1_bn1): BatchNorm2d(640, 0.011% Params, 163.84 KMac, 0.011% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br1_relu1): ReLU(0, 0.000% Params, 81.92 KMac, 0.006% MACs, inplace=True)
(inc6_br2_conv0): Conv2d(16.9 k, 0.287% Params, 4.33 MMac, 0.296% MACs, 528, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc6_br2_conv1): Conv2d(102.4 k, 1.742% Params, 26.21 MMac, 1.791% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc6_br2_bn1): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc6_br3_pool0): MaxPool2d(0, 0.000% Params, 135.17 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc6_br3_relu1): ReLU(0, 0.000% Params, 135.17 KMac, 0.009% MACs, inplace=True)
(inc6_br3_conv2): Conv2d(67.58 k, 1.150% Params, 17.3 MMac, 1.182% MACs, 528, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br3_bn2): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(pool12): MaxPool2d(0, 0.000% Params, 212.99 KMac, 0.015% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(inc7_br0_conv0): Conv2d(212.99 k, 3.623% Params, 13.63 MMac, 0.931% MACs, 832, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br0_bn0): BatchNorm2d(512, 0.009% Params, 32.77 KMac, 0.002% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br0_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc7_br1_conv0): Conv2d(133.12 k, 2.265% Params, 8.52 MMac, 0.582% MACs, 832, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br1_bn0): BatchNorm2d(320, 0.005% Params, 20.48 KMac, 0.001% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br1_relu0): ReLU(0, 0.000% Params, 10.24 KMac, 0.001% MACs, inplace=True)
(inc7_br1_conv1): Conv2d(460.8 k, 7.839% Params, 29.49 MMac, 2.015% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc7_br1_bn1): BatchNorm2d(640, 0.011% Params, 40.96 KMac, 0.003% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br1_relu1): ReLU(0, 0.000% Params, 20.48 KMac, 0.001% MACs, inplace=True)
(inc7_br2_conv0): Conv2d(26.62 k, 0.453% Params, 1.7 MMac, 0.116% MACs, 832, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br2_bn0): BatchNorm2d(64, 0.001% Params, 4.1 KMac, 0.000% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br2_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(inc7_br2_conv1): Conv2d(102.4 k, 1.742% Params, 6.55 MMac, 0.448% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc7_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc7_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc7_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
(inc7_br3_conv2): Conv2d(106.5 k, 1.812% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc8_br0_conv0): Conv2d(319.49 k, 5.435% Params, 20.45 MMac, 1.397% MACs, 832, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br0_bn0): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br0_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
(inc8_br1_conv0): Conv2d(159.74 k, 2.717% Params, 10.22 MMac, 0.698% MACs, 832, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br1_bn0): BatchNorm2d(384, 0.007% Params, 24.58 KMac, 0.002% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br1_relu0): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
(inc8_br1_conv1): Conv2d(663.55 k, 11.288% Params, 42.47 MMac, 2.901% MACs, 192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc8_br1_bn1): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br1_relu1): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
(inc8_br2_conv0): Conv2d(39.94 k, 0.679% Params, 2.56 MMac, 0.175% MACs, 832, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br2_bn0): BatchNorm2d(96, 0.002% Params, 6.14 KMac, 0.000% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br2_relu0): ReLU(0, 0.000% Params, 3.07 KMac, 0.000% MACs, inplace=True)
(inc8_br2_conv1): Conv2d(153.6 k, 2.613% Params, 9.83 MMac, 0.672% MACs, 48, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc8_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc8_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc8_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
(inc8_br3_conv2): Conv2d(106.5 k, 1.812% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(aap15): AdaptiveAvgPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, output_size=1)
(conv16): Conv2d(10.25 k, 0.174% Params, 10.25 KMac, 0.001% MACs, 1024, 10, kernel_size=(1, 1), stride=(1, 1))
)
ykl/ALL/param_flops/cifar10/MobileNetV2.txt 0 → 100644
Model(
2.25 M, 100.000% Params, 68.32 MMac, 100.000% MACs,
(conv0): Conv2d(128, 0.006% Params, 147.97 KMac, 0.217% MACs, 3, 32, kernel_size=(1, 1), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(64, 0.003% Params, 73.98 KMac, 0.108% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relus0): ReLU6(0, 0.000% Params, 36.99 KMac, 0.054% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(1.06 k, 0.047% Params, 1.22 MMac, 1.787% MACs, 32, 32, kernel_size=(1, 1), stride=(1, 1))
(ml0_blk0_ma_bn0): BatchNorm2d(64, 0.003% Params, 73.98 KMac, 0.108% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relus0): ReLU6(0, 0.000% Params, 36.99 KMac, 0.054% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(320, 0.014% Params, 369.92 KMac, 0.541% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32)
(ml0_blk0_ma_bn1): BatchNorm2d(64, 0.003% Params, 73.98 KMac, 0.108% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relus1): ReLU6(0, 0.000% Params, 36.99 KMac, 0.054% MACs, inplace=True)
(ml0_blk0_ma_conv2): Conv2d(528, 0.023% Params, 610.37 KMac, 0.893% MACs, 32, 16, kernel_size=(1, 1), stride=(1, 1))
(ml0_blk0_ma_bn2): BatchNorm2d(32, 0.001% Params, 36.99 KMac, 0.054% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_conv0): Conv2d(1.63 k, 0.072% Params, 1.89 MMac, 2.762% MACs, 16, 96, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk0_ma_bn0): BatchNorm2d(192, 0.009% Params, 221.95 KMac, 0.325% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relus0): ReLU6(0, 0.000% Params, 110.98 KMac, 0.162% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(960, 0.043% Params, 277.44 KMac, 0.406% MACs, 96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96)
(ml1_blk0_ma_bn1): BatchNorm2d(192, 0.009% Params, 55.49 KMac, 0.081% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relus1): ReLU6(0, 0.000% Params, 27.74 KMac, 0.041% MACs, inplace=True)
(ml1_blk0_ma_conv2): Conv2d(2.33 k, 0.103% Params, 672.79 KMac, 0.985% MACs, 96, 24, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk0_ma_bn2): BatchNorm2d(48, 0.002% Params, 13.87 KMac, 0.020% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_conv0): Conv2d(3.6 k, 0.160% Params, 1.04 MMac, 1.523% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk1_ma_bn0): BatchNorm2d(288, 0.013% Params, 83.23 KMac, 0.122% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relus0): ReLU6(0, 0.000% Params, 41.62 KMac, 0.061% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(1.44 k, 0.064% Params, 416.16 KMac, 0.609% MACs, 144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144)
(ml1_blk1_ma_bn1): BatchNorm2d(288, 0.013% Params, 83.23 KMac, 0.122% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relus1): ReLU6(0, 0.000% Params, 41.62 KMac, 0.061% MACs, inplace=True)
(ml1_blk1_ma_conv2): Conv2d(3.48 k, 0.154% Params, 1.01 MMac, 1.472% MACs, 144, 24, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk1_ma_bn2): BatchNorm2d(48, 0.002% Params, 13.87 KMac, 0.020% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_conv0): Conv2d(3.6 k, 0.160% Params, 1.04 MMac, 1.523% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk0_ma_bn0): BatchNorm2d(288, 0.013% Params, 83.23 KMac, 0.122% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relus0): ReLU6(0, 0.000% Params, 41.62 KMac, 0.061% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(1.44 k, 0.064% Params, 116.64 KMac, 0.171% MACs, 144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144)
(ml2_blk0_ma_bn1): BatchNorm2d(288, 0.013% Params, 23.33 KMac, 0.034% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relus1): ReLU6(0, 0.000% Params, 11.66 KMac, 0.017% MACs, inplace=True)
(ml2_blk0_ma_conv2): Conv2d(4.64 k, 0.206% Params, 375.84 KMac, 0.550% MACs, 144, 32, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk0_ma_bn2): BatchNorm2d(64, 0.003% Params, 5.18 KMac, 0.008% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_conv0): Conv2d(6.34 k, 0.281% Params, 513.22 KMac, 0.751% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk1_ma_bn0): BatchNorm2d(384, 0.017% Params, 31.1 KMac, 0.046% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relus0): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(1.92 k, 0.085% Params, 155.52 KMac, 0.228% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
(ml2_blk1_ma_bn1): BatchNorm2d(384, 0.017% Params, 31.1 KMac, 0.046% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relus1): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk1_ma_conv2): Conv2d(6.18 k, 0.274% Params, 500.26 KMac, 0.732% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk1_ma_bn2): BatchNorm2d(64, 0.003% Params, 5.18 KMac, 0.008% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_conv0): Conv2d(6.34 k, 0.281% Params, 513.22 KMac, 0.751% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk2_ma_bn0): BatchNorm2d(384, 0.017% Params, 31.1 KMac, 0.046% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relus0): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk2_ma_conv1): Conv2d(1.92 k, 0.085% Params, 155.52 KMac, 0.228% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
(ml2_blk2_ma_bn1): BatchNorm2d(384, 0.017% Params, 31.1 KMac, 0.046% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relus1): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk2_ma_conv2): Conv2d(6.18 k, 0.274% Params, 500.26 KMac, 0.732% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk2_ma_bn2): BatchNorm2d(64, 0.003% Params, 5.18 KMac, 0.008% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_conv0): Conv2d(6.34 k, 0.281% Params, 513.22 KMac, 0.751% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk0_ma_bn0): BatchNorm2d(384, 0.017% Params, 31.1 KMac, 0.046% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relus0): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(1.92 k, 0.085% Params, 48.0 KMac, 0.070% MACs, 192, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=192)
(ml3_blk0_ma_bn1): BatchNorm2d(384, 0.017% Params, 9.6 KMac, 0.014% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relus1): ReLU6(0, 0.000% Params, 4.8 KMac, 0.007% MACs, inplace=True)
(ml3_blk0_ma_conv2): Conv2d(12.35 k, 0.548% Params, 308.8 KMac, 0.452% MACs, 192, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk0_ma_bn2): BatchNorm2d(128, 0.006% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_conv0): Conv2d(24.96 k, 1.107% Params, 624.0 KMac, 0.913% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk1_ma_bn0): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(3.84 k, 0.170% Params, 96.0 KMac, 0.141% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml3_blk1_ma_bn1): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk1_ma_conv2): Conv2d(24.64 k, 1.093% Params, 616.0 KMac, 0.902% MACs, 384, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk1_ma_bn2): BatchNorm2d(128, 0.006% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_conv0): Conv2d(24.96 k, 1.107% Params, 624.0 KMac, 0.913% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk2_ma_bn0): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk2_ma_conv1): Conv2d(3.84 k, 0.170% Params, 96.0 KMac, 0.141% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml3_blk2_ma_bn1): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk2_ma_conv2): Conv2d(24.64 k, 1.093% Params, 616.0 KMac, 0.902% MACs, 384, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk2_ma_bn2): BatchNorm2d(128, 0.006% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk3_ma_conv0): Conv2d(24.96 k, 1.107% Params, 624.0 KMac, 0.913% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk3_ma_bn0): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk3_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk3_ma_conv1): Conv2d(3.84 k, 0.170% Params, 96.0 KMac, 0.141% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml3_blk3_ma_bn1): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk3_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk3_ma_conv2): Conv2d(24.64 k, 1.093% Params, 616.0 KMac, 0.902% MACs, 384, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk3_ma_bn2): BatchNorm2d(128, 0.006% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk0_ma_conv0): Conv2d(24.96 k, 1.107% Params, 624.0 KMac, 0.913% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk0_ma_bn0): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk0_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml4_blk0_ma_conv1): Conv2d(3.84 k, 0.170% Params, 96.0 KMac, 0.141% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml4_blk0_ma_bn1): BatchNorm2d(768, 0.034% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk0_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml4_blk0_ma_conv2): Conv2d(36.96 k, 1.640% Params, 924.0 KMac, 1.353% MACs, 384, 96, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk0_ma_bn2): BatchNorm2d(192, 0.009% Params, 4.8 KMac, 0.007% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk1_ma_conv0): Conv2d(55.87 k, 2.479% Params, 1.4 MMac, 2.045% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk1_ma_bn0): BatchNorm2d(1.15 k, 0.051% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk1_ma_relus0): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk1_ma_conv1): Conv2d(5.76 k, 0.256% Params, 144.0 KMac, 0.211% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
(ml4_blk1_ma_bn1): BatchNorm2d(1.15 k, 0.051% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk1_ma_relus1): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk1_ma_conv2): Conv2d(55.39 k, 2.457% Params, 1.38 MMac, 2.027% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk1_ma_bn2): BatchNorm2d(192, 0.009% Params, 4.8 KMac, 0.007% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk2_ma_conv0): Conv2d(55.87 k, 2.479% Params, 1.4 MMac, 2.045% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk2_ma_bn0): BatchNorm2d(1.15 k, 0.051% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk2_ma_relus0): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk2_ma_conv1): Conv2d(5.76 k, 0.256% Params, 144.0 KMac, 0.211% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
(ml4_blk2_ma_bn1): BatchNorm2d(1.15 k, 0.051% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk2_ma_relus1): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk2_ma_conv2): Conv2d(55.39 k, 2.457% Params, 1.38 MMac, 2.027% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk2_ma_bn2): BatchNorm2d(192, 0.009% Params, 4.8 KMac, 0.007% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk0_ma_conv0): Conv2d(55.87 k, 2.479% Params, 1.4 MMac, 2.045% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk0_ma_bn0): BatchNorm2d(1.15 k, 0.051% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk0_ma_relus0): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml5_blk0_ma_conv1): Conv2d(5.76 k, 0.256% Params, 144.0 KMac, 0.211% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
(ml5_blk0_ma_bn1): BatchNorm2d(1.15 k, 0.051% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk0_ma_relus1): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml5_blk0_ma_conv2): Conv2d(92.32 k, 4.096% Params, 2.31 MMac, 3.378% MACs, 576, 160, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk0_ma_bn2): BatchNorm2d(320, 0.014% Params, 8.0 KMac, 0.012% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk1_ma_conv0): Conv2d(154.56 k, 6.857% Params, 3.86 MMac, 5.656% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk1_ma_bn0): BatchNorm2d(1.92 k, 0.085% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk1_ma_relus0): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk1_ma_conv1): Conv2d(9.6 k, 0.426% Params, 240.0 KMac, 0.351% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
(ml5_blk1_ma_bn1): BatchNorm2d(1.92 k, 0.085% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk1_ma_relus1): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk1_ma_conv2): Conv2d(153.76 k, 6.821% Params, 3.84 MMac, 5.627% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk1_ma_bn2): BatchNorm2d(320, 0.014% Params, 8.0 KMac, 0.012% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk2_ma_conv0): Conv2d(154.56 k, 6.857% Params, 3.86 MMac, 5.656% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk2_ma_bn0): BatchNorm2d(1.92 k, 0.085% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk2_ma_relus0): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk2_ma_conv1): Conv2d(9.6 k, 0.426% Params, 240.0 KMac, 0.351% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
(ml5_blk2_ma_bn1): BatchNorm2d(1.92 k, 0.085% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk2_ma_relus1): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk2_ma_conv2): Conv2d(153.76 k, 6.821% Params, 3.84 MMac, 5.627% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk2_ma_bn2): BatchNorm2d(320, 0.014% Params, 8.0 KMac, 0.012% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml6_blk0_ma_conv0): Conv2d(154.56 k, 6.857% Params, 3.86 MMac, 5.656% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
(ml6_blk0_ma_bn0): BatchNorm2d(1.92 k, 0.085% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml6_blk0_ma_relus0): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml6_blk0_ma_conv1): Conv2d(9.6 k, 0.426% Params, 240.0 KMac, 0.351% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
(ml6_blk0_ma_bn1): BatchNorm2d(1.92 k, 0.085% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml6_blk0_ma_relus1): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml6_blk0_ma_conv2): Conv2d(307.52 k, 13.643% Params, 7.69 MMac, 11.253% MACs, 960, 320, kernel_size=(1, 1), stride=(1, 1))
(ml6_blk0_ma_bn2): BatchNorm2d(640, 0.028% Params, 16.0 KMac, 0.023% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv8): Conv2d(410.88 k, 18.228% Params, 10.27 MMac, 15.036% MACs, 320, 1280, kernel_size=(1, 1), stride=(1, 1))
(bn8): BatchNorm2d(2.56 k, 0.114% Params, 64.0 KMac, 0.094% MACs, 1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relus8): ReLU6(0, 0.000% Params, 32.0 KMac, 0.047% MACs, inplace=True)
(aap9): AdaptiveAvgPool2d(0, 0.000% Params, 32.0 KMac, 0.047% MACs, output_size=1)
(fc11): Linear(12.81 k, 0.568% Params, 12.81 KMac, 0.019% MACs, in_features=1280, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar10/ResNet_152.txt 0 → 100644
Model(
58.16 M, 100.000% Params, 3.74 GMac, 100.000% MACs,
(conv0): Conv2d(1.73 k, 0.003% Params, 1.77 MMac, 0.047% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(4.1 k, 0.007% Params, 4.19 MMac, 0.112% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(36.86 k, 0.063% Params, 37.75 MMac, 1.008% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk0_ma_conv2): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn2): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ds_conv0): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ds_bn0): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.007% MACs, inplace=True)
(ml0_blk1_ma_conv0): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk1_ma_conv1): Conv2d(36.86 k, 0.063% Params, 37.75 MMac, 1.008% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk1_ma_conv2): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn2): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.007% MACs, inplace=True)
(ml0_blk2_ma_conv0): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk2_ma_conv1): Conv2d(36.86 k, 0.063% Params, 37.75 MMac, 1.008% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk2_ma_bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk2_ma_conv2): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn2): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.007% MACs, inplace=True)
(ml1_blk0_ma_conv0): Conv2d(32.77 k, 0.056% Params, 33.55 MMac, 0.896% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn0): BatchNorm2d(256, 0.000% Params, 262.14 KMac, 0.007% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk0_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ds_conv0): Conv2d(131.07 k, 0.225% Params, 33.55 MMac, 0.896% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml1_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk1_ma_conv0): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk1_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk2_ma_conv0): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk2_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk2_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk2_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk3_ma_conv0): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk3_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk3_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk3_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk4_ma_conv0): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk4_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk4_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk4_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk4_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk4_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk4_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk4_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk4_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk5_ma_conv0): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk5_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk5_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk5_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk5_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk5_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk5_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk5_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk5_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk6_ma_conv0): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk6_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk6_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk6_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk6_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk6_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk6_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk6_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk6_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk7_ma_conv0): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk7_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk7_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk7_ma_conv1): Conv2d(147.46 k, 0.254% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk7_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk7_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk7_ma_conv2): Conv2d(65.54 k, 0.113% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk7_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk7_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml2_blk0_ma_conv0): Conv2d(131.07 k, 0.225% Params, 33.55 MMac, 0.896% MACs, 512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn0): BatchNorm2d(512, 0.001% Params, 131.07 KMac, 0.004% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk0_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ds_conv0): Conv2d(524.29 k, 0.902% Params, 33.55 MMac, 0.896% MACs, 512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml2_blk0_ds_bn0): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk1_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk1_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk2_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk2_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk2_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk2_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk3_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk3_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk3_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk3_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk4_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk4_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk4_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk4_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk5_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk5_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk5_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk5_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk6_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk6_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk6_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk6_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk6_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk6_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk6_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk6_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk6_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk7_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk7_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk7_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk7_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk7_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk7_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk7_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk7_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk7_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk8_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk8_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk8_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk8_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk8_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk8_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk8_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk8_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk8_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk9_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk9_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk9_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk9_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk9_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk9_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk9_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk9_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk9_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk10_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk10_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk10_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk10_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk10_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk10_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk10_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk10_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk10_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk11_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk11_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk11_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk11_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk11_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk11_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk11_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk11_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk11_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk12_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk12_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk12_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk12_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk12_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk12_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk12_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk12_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk12_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk13_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk13_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk13_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk13_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk13_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk13_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk13_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk13_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk13_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk14_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk14_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk14_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk14_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk14_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk14_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk14_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk14_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk14_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk15_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk15_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk15_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk15_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk15_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk15_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk15_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk15_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk15_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk16_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk16_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk16_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk16_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk16_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk16_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk16_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk16_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk16_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk17_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk17_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk17_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk17_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk17_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk17_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk17_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk17_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk17_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk18_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk18_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk18_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk18_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk18_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk18_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk18_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk18_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk18_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk19_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk19_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk19_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk19_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk19_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk19_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk19_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk19_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk19_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk20_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk20_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk20_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk20_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk20_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk20_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk20_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk20_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk20_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk21_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk21_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk21_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk21_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk21_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk21_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk21_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk21_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk21_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk22_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk22_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk22_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk22_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk22_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk22_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk22_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk22_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk22_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk23_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk23_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk23_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk23_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk23_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk23_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk23_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk23_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk23_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk24_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk24_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk24_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk24_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk24_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk24_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk24_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk24_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk24_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk25_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk25_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk25_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk25_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk25_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk25_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk25_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk25_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk25_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk26_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk26_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk26_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk26_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk26_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk26_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk26_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk26_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk26_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk27_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk27_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk27_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk27_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk27_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk27_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk27_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk27_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk27_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk28_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk28_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk28_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk28_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk28_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk28_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk28_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk28_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk28_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk29_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk29_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk29_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk29_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk29_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk29_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk29_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk29_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk29_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk30_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk30_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk30_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk30_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk30_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk30_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk30_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk30_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk30_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk31_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk31_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk31_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk31_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk31_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk31_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk31_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk31_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk31_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk32_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk32_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk32_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk32_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk32_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk32_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk32_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk32_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk32_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk33_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk33_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk33_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk33_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk33_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk33_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk33_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk33_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk33_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk34_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk34_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk34_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk34_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk34_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk34_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk34_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk34_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk34_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk35_ma_conv0): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk35_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk35_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk35_ma_conv1): Conv2d(589.82 k, 1.014% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk35_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk35_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk35_ma_conv2): Conv2d(262.14 k, 0.451% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk35_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk35_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml3_blk0_ma_conv0): Conv2d(524.29 k, 0.902% Params, 33.55 MMac, 0.896% MACs, 1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn0): BatchNorm2d(1.02 k, 0.002% Params, 65.54 KMac, 0.002% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(2.36 M, 4.057% Params, 37.75 MMac, 1.008% MACs, 512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn1): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk0_ma_conv2): Conv2d(1.05 M, 1.803% Params, 16.78 MMac, 0.448% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn2): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ds_conv0): Conv2d(2.1 M, 3.606% Params, 33.55 MMac, 0.896% MACs, 1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml3_blk0_ds_bn0): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv0): Conv2d(1.05 M, 1.803% Params, 16.78 MMac, 0.448% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn0): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(2.36 M, 4.057% Params, 37.75 MMac, 1.008% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn1): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk1_ma_conv2): Conv2d(1.05 M, 1.803% Params, 16.78 MMac, 0.448% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn2): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml3_blk2_ma_conv0): Conv2d(1.05 M, 1.803% Params, 16.78 MMac, 0.448% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn0): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk2_ma_conv1): Conv2d(2.36 M, 4.057% Params, 37.75 MMac, 1.008% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk2_ma_bn1): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk2_ma_conv2): Conv2d(1.05 M, 1.803% Params, 16.78 MMac, 0.448% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn2): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(aap5): AdaptiveAvgPool2d(0, 0.000% Params, 32.77 KMac, 0.001% MACs, output_size=1)
(fc7): Linear(20.49 k, 0.035% Params, 20.49 KMac, 0.001% MACs, in_features=2048, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar10/ResNet_18.txt 0 → 100644
Model(
11.17 M, 100.000% Params, 557.22 MMac, 100.000% MACs,
(conv0): Conv2d(1.73 k, 0.015% Params, 1.77 MMac, 0.318% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(36.86 k, 0.330% Params, 37.75 MMac, 6.775% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(36.86 k, 0.330% Params, 37.75 MMac, 6.775% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk1_ma_conv0): Conv2d(36.86 k, 0.330% Params, 37.75 MMac, 6.775% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk1_ma_conv1): Conv2d(36.86 k, 0.330% Params, 37.75 MMac, 6.775% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml1_blk0_ma_conv0): Conv2d(73.73 k, 0.660% Params, 18.87 MMac, 3.387% MACs, 64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn0): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(147.46 k, 1.320% Params, 37.75 MMac, 6.775% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn1): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ds_conv0): Conv2d(8.19 k, 0.073% Params, 2.1 MMac, 0.376% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml1_blk0_ds_bn0): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml1_blk1_ma_conv0): Conv2d(147.46 k, 1.320% Params, 37.75 MMac, 6.775% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn0): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(147.46 k, 1.320% Params, 37.75 MMac, 6.775% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn1): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml2_blk0_ma_conv0): Conv2d(294.91 k, 2.639% Params, 18.87 MMac, 3.387% MACs, 128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn0): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(589.82 k, 5.279% Params, 37.75 MMac, 6.775% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn1): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ds_conv0): Conv2d(32.77 k, 0.293% Params, 2.1 MMac, 0.376% MACs, 128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml2_blk0_ds_bn0): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml2_blk1_ma_conv0): Conv2d(589.82 k, 5.279% Params, 37.75 MMac, 6.775% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn0): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(589.82 k, 5.279% Params, 37.75 MMac, 6.775% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn1): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml3_blk0_ma_conv0): Conv2d(1.18 M, 10.557% Params, 18.87 MMac, 3.387% MACs, 256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn0): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(2.36 M, 21.114% Params, 37.75 MMac, 6.775% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn1): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ds_conv0): Conv2d(131.07 k, 1.173% Params, 2.1 MMac, 0.376% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml3_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv0): Conv2d(2.36 M, 21.114% Params, 37.75 MMac, 6.775% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn0): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(2.36 M, 21.114% Params, 37.75 MMac, 6.775% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn1): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(aap5): AdaptiveAvgPool2d(0, 0.000% Params, 8.19 KMac, 0.001% MACs, output_size=1)
(fc7): Linear(5.13 k, 0.046% Params, 5.13 KMac, 0.001% MACs, in_features=512, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar10/ResNet_50.txt 0 → 100644
Model(
23.52 M, 100.000% Params, 1.31 GMac, 100.000% MACs,
(conv0): Conv2d(1.73 k, 0.007% Params, 1.77 MMac, 0.135% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(4.1 k, 0.017% Params, 4.19 MMac, 0.321% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(36.86 k, 0.157% Params, 37.75 MMac, 2.887% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk0_ma_conv2): Conv2d(16.38 k, 0.070% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn2): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ds_conv0): Conv2d(16.38 k, 0.070% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ds_bn0): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.020% MACs, inplace=True)
(ml0_blk1_ma_conv0): Conv2d(16.38 k, 0.070% Params, 16.78 MMac, 1.283% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk1_ma_conv1): Conv2d(36.86 k, 0.157% Params, 37.75 MMac, 2.887% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk1_ma_conv2): Conv2d(16.38 k, 0.070% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn2): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.020% MACs, inplace=True)
(ml0_blk2_ma_conv0): Conv2d(16.38 k, 0.070% Params, 16.78 MMac, 1.283% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk2_ma_conv1): Conv2d(36.86 k, 0.157% Params, 37.75 MMac, 2.887% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk2_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk2_ma_conv2): Conv2d(16.38 k, 0.070% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn2): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.020% MACs, inplace=True)
(ml1_blk0_ma_conv0): Conv2d(32.77 k, 0.139% Params, 33.55 MMac, 2.566% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn0): BatchNorm2d(256, 0.001% Params, 262.14 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(147.46 k, 0.627% Params, 37.75 MMac, 2.887% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk0_ma_conv2): Conv2d(65.54 k, 0.279% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ds_conv0): Conv2d(131.07 k, 0.557% Params, 33.55 MMac, 2.566% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml1_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk1_ma_conv0): Conv2d(65.54 k, 0.279% Params, 16.78 MMac, 1.283% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn0): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(147.46 k, 0.627% Params, 37.75 MMac, 2.887% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk1_ma_conv2): Conv2d(65.54 k, 0.279% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk2_ma_conv0): Conv2d(65.54 k, 0.279% Params, 16.78 MMac, 1.283% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn0): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk2_ma_conv1): Conv2d(147.46 k, 0.627% Params, 37.75 MMac, 2.887% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk2_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk2_ma_conv2): Conv2d(65.54 k, 0.279% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk3_ma_conv0): Conv2d(65.54 k, 0.279% Params, 16.78 MMac, 1.283% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn0): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk3_ma_conv1): Conv2d(147.46 k, 0.627% Params, 37.75 MMac, 2.887% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk3_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk3_ma_conv2): Conv2d(65.54 k, 0.279% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml2_blk0_ma_conv0): Conv2d(131.07 k, 0.557% Params, 33.55 MMac, 2.566% MACs, 512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn0): BatchNorm2d(512, 0.002% Params, 131.07 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(589.82 k, 2.508% Params, 37.75 MMac, 2.887% MACs, 256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk0_ma_conv2): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ds_conv0): Conv2d(524.29 k, 2.229% Params, 33.55 MMac, 2.566% MACs, 512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml2_blk0_ds_bn0): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk1_ma_conv0): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(589.82 k, 2.508% Params, 37.75 MMac, 2.887% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk1_ma_conv2): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk2_ma_conv0): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk2_ma_conv1): Conv2d(589.82 k, 2.508% Params, 37.75 MMac, 2.887% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk2_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk2_ma_conv2): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk3_ma_conv0): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk3_ma_conv1): Conv2d(589.82 k, 2.508% Params, 37.75 MMac, 2.887% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk3_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk3_ma_conv2): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk4_ma_conv0): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk4_ma_conv1): Conv2d(589.82 k, 2.508% Params, 37.75 MMac, 2.887% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk4_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk4_ma_conv2): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk5_ma_conv0): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk5_ma_conv1): Conv2d(589.82 k, 2.508% Params, 37.75 MMac, 2.887% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk5_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk5_ma_conv2): Conv2d(262.14 k, 1.115% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml3_blk0_ma_conv0): Conv2d(524.29 k, 2.229% Params, 33.55 MMac, 2.566% MACs, 1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn0): BatchNorm2d(1.02 k, 0.004% Params, 65.54 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(2.36 M, 10.031% Params, 37.75 MMac, 2.887% MACs, 512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn1): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk0_ma_conv2): Conv2d(1.05 M, 4.458% Params, 16.78 MMac, 1.283% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn2): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ds_conv0): Conv2d(2.1 M, 8.916% Params, 33.55 MMac, 2.566% MACs, 1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml3_blk0_ds_bn0): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml3_blk1_ma_conv0): Conv2d(1.05 M, 4.458% Params, 16.78 MMac, 1.283% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn0): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(2.36 M, 10.031% Params, 37.75 MMac, 2.887% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn1): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv2): Conv2d(1.05 M, 4.458% Params, 16.78 MMac, 1.283% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn2): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml3_blk2_ma_conv0): Conv2d(1.05 M, 4.458% Params, 16.78 MMac, 1.283% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn0): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk2_ma_conv1): Conv2d(2.36 M, 10.031% Params, 37.75 MMac, 2.887% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk2_ma_bn1): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk2_ma_conv2): Conv2d(1.05 M, 4.458% Params, 16.78 MMac, 1.283% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn2): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(aap5): AdaptiveAvgPool2d(0, 0.000% Params, 32.77 KMac, 0.003% MACs, output_size=1)
(fc7): Linear(20.49 k, 0.087% Params, 20.49 KMac, 0.002% MACs, in_features=2048, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar10/VGG_16.txt 0 → 100644
Model(
33.65 M, 100.000% Params, 333.36 MMac, 100.000% MACs,
(conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.550% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
(conv1): Conv2d(36.93 k, 0.110% Params, 37.81 MMac, 11.343% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
(pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.020% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3): Conv2d(73.86 k, 0.220% Params, 18.91 MMac, 5.672% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
(conv4): Conv2d(147.58 k, 0.439% Params, 37.78 MMac, 11.334% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
(pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.010% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv6): Conv2d(295.17 k, 0.877% Params, 18.89 MMac, 5.667% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn6): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
(conv7): Conv2d(590.08 k, 1.754% Params, 37.77 MMac, 11.329% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn7): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
(conv8): Conv2d(590.08 k, 1.754% Params, 37.77 MMac, 11.329% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn8): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
(pool9): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.005% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv10): Conv2d(1.18 M, 3.508% Params, 18.88 MMac, 5.664% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn10): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu10): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv11): Conv2d(2.36 M, 7.013% Params, 37.76 MMac, 11.326% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv12): Conv2d(2.36 M, 7.013% Params, 37.76 MMac, 11.326% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(pool13): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv14): Conv2d(2.36 M, 7.013% Params, 9.44 MMac, 2.832% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn14): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu14): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
(conv15): Conv2d(2.36 M, 7.013% Params, 9.44 MMac, 2.832% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn15): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu15): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
(conv16): Conv2d(2.36 M, 7.013% Params, 9.44 MMac, 2.832% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
(pool17): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.001% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(fc19): Linear(2.1 M, 6.245% Params, 2.1 MMac, 0.630% MACs, in_features=512, out_features=4096, bias=True)
(relu20): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop21): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc22): Linear(16.78 M, 49.875% Params, 16.78 MMac, 5.034% MACs, in_features=4096, out_features=4096, bias=True)
(relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc25): Linear(40.97 k, 0.122% Params, 40.97 KMac, 0.012% MACs, in_features=4096, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar10/VGG_19.txt 0 → 100644
Model(
38.96 M, 100.000% Params, 418.4 MMac, 100.000% MACs,
(conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.439% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
(conv1): Conv2d(36.93 k, 0.095% Params, 37.81 MMac, 9.038% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
(pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.016% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3): Conv2d(73.86 k, 0.190% Params, 18.91 MMac, 4.519% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
(conv4): Conv2d(147.58 k, 0.379% Params, 37.78 MMac, 9.030% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
(pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.008% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv6): Conv2d(295.17 k, 0.758% Params, 18.89 MMac, 4.515% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn6): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(conv7): Conv2d(590.08 k, 1.515% Params, 37.77 MMac, 9.026% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn7): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(conv8): Conv2d(590.08 k, 1.515% Params, 37.77 MMac, 9.026% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn8): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(conv9): Conv2d(590.08 k, 1.515% Params, 37.77 MMac, 9.026% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn9): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(pool10): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.004% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv11): Conv2d(1.18 M, 3.029% Params, 18.88 MMac, 4.513% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv12): Conv2d(2.36 M, 6.057% Params, 37.76 MMac, 9.024% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv13): Conv2d(2.36 M, 6.057% Params, 37.76 MMac, 9.024% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn13): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu13): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv14): Conv2d(2.36 M, 6.057% Params, 37.76 MMac, 9.024% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn14): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu14): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(pool15): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv16): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(conv17): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn17): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu17): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(conv18): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn18): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu18): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(conv19): Conv2d(2.36 M, 6.057% Params, 9.44 MMac, 2.256% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn19): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu19): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(pool20): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.000% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(fc22): Linear(2.1 M, 5.393% Params, 2.1 MMac, 0.502% MACs, in_features=512, out_features=4096, bias=True)
(relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc25): Linear(16.78 M, 43.074% Params, 16.78 MMac, 4.011% MACs, in_features=4096, out_features=4096, bias=True)
(relu26): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop27): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc28): Linear(40.97 k, 0.105% Params, 40.97 KMac, 0.010% MACs, in_features=4096, out_features=10, bias=True)
)
ykl/ALL/param_flops/cifar100/AlexNet.txt 0 → 100644
Model(
3.91 M, 100.000% Params, 70.13 MMac, 100.000% MACs,
(conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.308% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
(pool2): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3): Conv2d(18.5 k, 0.472% Params, 4.73 MMac, 6.752% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu4): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool5): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv6): Conv2d(73.86 k, 1.887% Params, 4.73 MMac, 6.740% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu7): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
(conv8): Conv2d(295.17 k, 7.540% Params, 18.89 MMac, 26.937% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(conv10): Conv2d(590.08 k, 15.073% Params, 37.77 MMac, 53.851% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu11): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool12): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(drop14): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc15): Linear(2.36 M, 60.290% Params, 2.36 MMac, 3.366% MACs, in_features=2304, out_features=1024, bias=True)
(relu16): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
(drop17): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc18): Linear(524.8 k, 13.405% Params, 524.8 KMac, 0.748% MACs, in_features=1024, out_features=512, bias=True)
(relu19): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
(fc20): Linear(51.3 k, 1.310% Params, 51.3 KMac, 0.073% MACs, in_features=512, out_features=100, bias=True)
)
ykl/ALL/param_flops/cifar100/AlexNet_BN.txt 0 → 100644
Model(
3.92 M, 100.000% Params, 70.31 MMac, 100.000% MACs,
(conv0): Conv2d(896, 0.023% Params, 917.5 KMac, 1.305% MACs, 3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(64, 0.002% Params, 65.54 KMac, 0.093% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.047% MACs, inplace=True)
(pool1): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.047% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv2): Conv2d(18.5 k, 0.472% Params, 4.73 MMac, 6.735% MACs, 32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn2): BatchNorm2d(128, 0.003% Params, 32.77 KMac, 0.047% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool3): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv4): Conv2d(73.86 k, 1.886% Params, 4.73 MMac, 6.723% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn4): BatchNorm2d(256, 0.007% Params, 16.38 KMac, 0.023% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu4): ReLU(0, 0.000% Params, 8.19 KMac, 0.012% MACs, inplace=True)
(conv5): Conv2d(295.17 k, 7.537% Params, 18.89 MMac, 26.868% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn5): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu5): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(conv6): Conv2d(590.08 k, 15.067% Params, 37.77 MMac, 53.713% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn6): BatchNorm2d(512, 0.013% Params, 32.77 KMac, 0.047% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.023% MACs, inplace=True)
(pool7): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.023% MACs, kernel_size=3, stride=2, padding=0, dilation=1, ceil_mode=False)
(drop9): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc10): Linear(2.36 M, 60.268% Params, 2.36 MMac, 3.357% MACs, in_features=2304, out_features=1024, bias=True)
(relu11): ReLU(0, 0.000% Params, 1.02 KMac, 0.001% MACs, inplace=True)
(drop12): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc13): Linear(524.8 k, 13.400% Params, 524.8 KMac, 0.746% MACs, in_features=1024, out_features=512, bias=True)
(relu14): ReLU(0, 0.000% Params, 512.0 Mac, 0.001% MACs, inplace=True)
(fc15): Linear(51.3 k, 1.310% Params, 51.3 KMac, 0.073% MACs, in_features=512, out_features=100, bias=True)
)
ykl/ALL/param_flops/cifar100/Inception_BN.txt 0 → 100644
Model(
5.97 M, 100.000% Params, 1.46 GMac, 100.000% MACs,
(conv0): Conv2d(1.79 k, 0.030% Params, 1.84 MMac, 0.125% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(conv2): Conv2d(36.93 k, 0.618% Params, 37.81 MMac, 2.583% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(relu3): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc0_br0_conv0): Conv2d(4.1 k, 0.069% Params, 4.19 MMac, 0.287% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br0_bn0): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc0_br1_conv0): Conv2d(6.14 k, 0.103% Params, 6.29 MMac, 0.430% MACs, 64, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br1_bn0): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br1_relu0): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
(inc0_br1_conv1): Conv2d(110.59 k, 1.852% Params, 113.25 MMac, 7.736% MACs, 96, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc0_br1_bn1): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc0_br2_conv0): Conv2d(1.02 k, 0.017% Params, 1.05 MMac, 0.072% MACs, 64, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br2_bn0): BatchNorm2d(32, 0.001% Params, 32.77 KMac, 0.002% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br2_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc0_br2_conv1): Conv2d(12.8 k, 0.214% Params, 13.11 MMac, 0.895% MACs, 16, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc0_br2_bn1): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc0_br3_pool0): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc0_br3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc0_br3_conv2): Conv2d(2.05 k, 0.034% Params, 2.1 MMac, 0.143% MACs, 64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc0_br3_bn2): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc0_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc1_br0_conv0): Conv2d(32.77 k, 0.549% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br0_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br0_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc1_br1_conv0): Conv2d(32.77 k, 0.549% Params, 33.55 MMac, 2.292% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br1_bn0): BatchNorm2d(256, 0.004% Params, 262.14 KMac, 0.018% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br1_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc1_br1_conv1): Conv2d(221.18 k, 3.704% Params, 226.49 MMac, 15.473% MACs, 128, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc1_br1_bn1): BatchNorm2d(384, 0.006% Params, 393.22 KMac, 0.027% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br1_relu1): ReLU(0, 0.000% Params, 196.61 KMac, 0.013% MACs, inplace=True)
(inc1_br2_conv0): Conv2d(8.19 k, 0.137% Params, 8.39 MMac, 0.573% MACs, 256, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br2_bn0): BatchNorm2d(64, 0.001% Params, 65.54 KMac, 0.004% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br2_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc1_br2_conv1): Conv2d(76.8 k, 1.286% Params, 78.64 MMac, 5.372% MACs, 32, 96, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc1_br2_bn1): BatchNorm2d(192, 0.003% Params, 196.61 KMac, 0.013% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br2_relu1): ReLU(0, 0.000% Params, 98.3 KMac, 0.007% MACs, inplace=True)
(inc1_br3_pool0): MaxPool2d(0, 0.000% Params, 262.14 KMac, 0.018% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc1_br3_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.018% MACs, inplace=True)
(inc1_br3_conv2): Conv2d(16.38 k, 0.274% Params, 16.78 MMac, 1.146% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc1_br3_bn2): BatchNorm2d(128, 0.002% Params, 131.07 KMac, 0.009% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc1_br3_relu2): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(pool6): MaxPool2d(0, 0.000% Params, 491.52 KMac, 0.034% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(inc2_br0_conv0): Conv2d(92.16 k, 1.544% Params, 23.59 MMac, 1.612% MACs, 480, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br0_bn0): BatchNorm2d(384, 0.006% Params, 98.3 KMac, 0.007% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br0_relu0): ReLU(0, 0.000% Params, 49.15 KMac, 0.003% MACs, inplace=True)
(inc2_br1_conv0): Conv2d(46.08 k, 0.772% Params, 11.8 MMac, 0.806% MACs, 480, 96, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br1_bn0): BatchNorm2d(192, 0.003% Params, 49.15 KMac, 0.003% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br1_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
(inc2_br1_conv1): Conv2d(179.71 k, 3.010% Params, 46.01 MMac, 3.143% MACs, 96, 208, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc2_br1_bn1): BatchNorm2d(416, 0.007% Params, 106.5 KMac, 0.007% MACs, 208, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br1_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
(inc2_br2_conv0): Conv2d(7.68 k, 0.129% Params, 1.97 MMac, 0.134% MACs, 480, 16, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br2_bn0): BatchNorm2d(32, 0.001% Params, 8.19 KMac, 0.001% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br2_relu0): ReLU(0, 0.000% Params, 4.1 KMac, 0.000% MACs, inplace=True)
(inc2_br2_conv1): Conv2d(19.2 k, 0.322% Params, 4.92 MMac, 0.336% MACs, 16, 48, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc2_br2_bn1): BatchNorm2d(96, 0.002% Params, 24.58 KMac, 0.002% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br2_relu1): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
(inc2_br3_pool0): MaxPool2d(0, 0.000% Params, 122.88 KMac, 0.008% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc2_br3_relu1): ReLU(0, 0.000% Params, 122.88 KMac, 0.008% MACs, inplace=True)
(inc2_br3_conv2): Conv2d(30.72 k, 0.515% Params, 7.86 MMac, 0.537% MACs, 480, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc2_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc2_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc3_br0_conv0): Conv2d(81.92 k, 1.372% Params, 20.97 MMac, 1.433% MACs, 512, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br0_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br0_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
(inc3_br1_conv0): Conv2d(57.34 k, 0.960% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br1_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br1_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
(inc3_br1_conv1): Conv2d(225.79 k, 3.782% Params, 57.8 MMac, 3.949% MACs, 112, 224, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc3_br1_bn1): BatchNorm2d(448, 0.008% Params, 114.69 KMac, 0.008% MACs, 224, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br1_relu1): ReLU(0, 0.000% Params, 57.34 KMac, 0.004% MACs, inplace=True)
(inc3_br2_conv0): Conv2d(12.29 k, 0.206% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
(inc3_br2_conv1): Conv2d(38.4 k, 0.643% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc3_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc3_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc3_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc3_br3_conv2): Conv2d(32.77 k, 0.549% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc3_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc3_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc4_br0_conv0): Conv2d(65.54 k, 1.098% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br0_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br0_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc4_br1_conv0): Conv2d(65.54 k, 1.098% Params, 16.78 MMac, 1.146% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br1_bn0): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br1_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc4_br1_conv1): Conv2d(294.91 k, 4.939% Params, 75.5 MMac, 5.158% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc4_br1_bn1): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc4_br2_conv0): Conv2d(12.29 k, 0.206% Params, 3.15 MMac, 0.215% MACs, 512, 24, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br2_bn0): BatchNorm2d(48, 0.001% Params, 12.29 KMac, 0.001% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br2_relu0): ReLU(0, 0.000% Params, 6.14 KMac, 0.000% MACs, inplace=True)
(inc4_br2_conv1): Conv2d(38.4 k, 0.643% Params, 9.83 MMac, 0.672% MACs, 24, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc4_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc4_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc4_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc4_br3_conv2): Conv2d(32.77 k, 0.549% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc4_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc4_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc5_br0_conv0): Conv2d(57.34 k, 0.960% Params, 14.68 MMac, 1.003% MACs, 512, 112, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br0_bn0): BatchNorm2d(224, 0.004% Params, 57.34 KMac, 0.004% MACs, 112, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br0_relu0): ReLU(0, 0.000% Params, 28.67 KMac, 0.002% MACs, inplace=True)
(inc5_br1_conv0): Conv2d(73.73 k, 1.235% Params, 18.87 MMac, 1.289% MACs, 512, 144, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br1_bn0): BatchNorm2d(288, 0.005% Params, 73.73 KMac, 0.005% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br1_relu0): ReLU(0, 0.000% Params, 36.86 KMac, 0.003% MACs, inplace=True)
(inc5_br1_conv1): Conv2d(373.25 k, 6.251% Params, 95.55 MMac, 6.527% MACs, 144, 288, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc5_br1_bn1): BatchNorm2d(576, 0.010% Params, 147.46 KMac, 0.010% MACs, 288, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br1_relu1): ReLU(0, 0.000% Params, 73.73 KMac, 0.005% MACs, inplace=True)
(inc5_br2_conv0): Conv2d(16.38 k, 0.274% Params, 4.19 MMac, 0.287% MACs, 512, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc5_br2_conv1): Conv2d(51.2 k, 0.858% Params, 13.11 MMac, 0.895% MACs, 32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc5_br2_bn1): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br2_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc5_br3_pool0): MaxPool2d(0, 0.000% Params, 131.07 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc5_br3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.009% MACs, inplace=True)
(inc5_br3_conv2): Conv2d(32.77 k, 0.549% Params, 8.39 MMac, 0.573% MACs, 512, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc5_br3_bn2): BatchNorm2d(128, 0.002% Params, 32.77 KMac, 0.002% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc5_br3_relu2): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc6_br0_conv0): Conv2d(135.17 k, 2.264% Params, 34.6 MMac, 2.364% MACs, 528, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br0_bn0): BatchNorm2d(512, 0.009% Params, 131.07 KMac, 0.009% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br0_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.004% MACs, inplace=True)
(inc6_br1_conv0): Conv2d(84.48 k, 1.415% Params, 21.63 MMac, 1.477% MACs, 528, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br1_bn0): BatchNorm2d(320, 0.005% Params, 81.92 KMac, 0.006% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br1_relu0): ReLU(0, 0.000% Params, 40.96 KMac, 0.003% MACs, inplace=True)
(inc6_br1_conv1): Conv2d(460.8 k, 7.718% Params, 117.96 MMac, 8.059% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc6_br1_bn1): BatchNorm2d(640, 0.011% Params, 163.84 KMac, 0.011% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br1_relu1): ReLU(0, 0.000% Params, 81.92 KMac, 0.006% MACs, inplace=True)
(inc6_br2_conv0): Conv2d(16.9 k, 0.283% Params, 4.33 MMac, 0.295% MACs, 528, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br2_bn0): BatchNorm2d(64, 0.001% Params, 16.38 KMac, 0.001% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br2_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc6_br2_conv1): Conv2d(102.4 k, 1.715% Params, 26.21 MMac, 1.791% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc6_br2_bn1): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(inc6_br3_pool0): MaxPool2d(0, 0.000% Params, 135.17 KMac, 0.009% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc6_br3_relu1): ReLU(0, 0.000% Params, 135.17 KMac, 0.009% MACs, inplace=True)
(inc6_br3_conv2): Conv2d(67.58 k, 1.132% Params, 17.3 MMac, 1.182% MACs, 528, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc6_br3_bn2): BatchNorm2d(256, 0.004% Params, 65.54 KMac, 0.004% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc6_br3_relu2): ReLU(0, 0.000% Params, 32.77 KMac, 0.002% MACs, inplace=True)
(pool12): MaxPool2d(0, 0.000% Params, 212.99 KMac, 0.015% MACs, kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(inc7_br0_conv0): Conv2d(212.99 k, 3.567% Params, 13.63 MMac, 0.931% MACs, 832, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br0_bn0): BatchNorm2d(512, 0.009% Params, 32.77 KMac, 0.002% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br0_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(inc7_br1_conv0): Conv2d(133.12 k, 2.230% Params, 8.52 MMac, 0.582% MACs, 832, 160, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br1_bn0): BatchNorm2d(320, 0.005% Params, 20.48 KMac, 0.001% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br1_relu0): ReLU(0, 0.000% Params, 10.24 KMac, 0.001% MACs, inplace=True)
(inc7_br1_conv1): Conv2d(460.8 k, 7.718% Params, 29.49 MMac, 2.015% MACs, 160, 320, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc7_br1_bn1): BatchNorm2d(640, 0.011% Params, 40.96 KMac, 0.003% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br1_relu1): ReLU(0, 0.000% Params, 20.48 KMac, 0.001% MACs, inplace=True)
(inc7_br2_conv0): Conv2d(26.62 k, 0.446% Params, 1.7 MMac, 0.116% MACs, 832, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br2_bn0): BatchNorm2d(64, 0.001% Params, 4.1 KMac, 0.000% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br2_relu0): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(inc7_br2_conv1): Conv2d(102.4 k, 1.715% Params, 6.55 MMac, 0.448% MACs, 32, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc7_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc7_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc7_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
(inc7_br3_conv2): Conv2d(106.5 k, 1.784% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc7_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc7_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc8_br0_conv0): Conv2d(319.49 k, 5.351% Params, 20.45 MMac, 1.397% MACs, 832, 384, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br0_bn0): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br0_relu0): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
(inc8_br1_conv0): Conv2d(159.74 k, 2.675% Params, 10.22 MMac, 0.698% MACs, 832, 192, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br1_bn0): BatchNorm2d(384, 0.006% Params, 24.58 KMac, 0.002% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br1_relu0): ReLU(0, 0.000% Params, 12.29 KMac, 0.001% MACs, inplace=True)
(inc8_br1_conv1): Conv2d(663.55 k, 11.113% Params, 42.47 MMac, 2.901% MACs, 192, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(inc8_br1_bn1): BatchNorm2d(768, 0.013% Params, 49.15 KMac, 0.003% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br1_relu1): ReLU(0, 0.000% Params, 24.58 KMac, 0.002% MACs, inplace=True)
(inc8_br2_conv0): Conv2d(39.94 k, 0.669% Params, 2.56 MMac, 0.175% MACs, 832, 48, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br2_bn0): BatchNorm2d(96, 0.002% Params, 6.14 KMac, 0.000% MACs, 48, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br2_relu0): ReLU(0, 0.000% Params, 3.07 KMac, 0.000% MACs, inplace=True)
(inc8_br2_conv1): Conv2d(153.6 k, 2.573% Params, 9.83 MMac, 0.672% MACs, 48, 128, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2), bias=False)
(inc8_br2_bn1): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br2_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(inc8_br3_pool0): MaxPool2d(0, 0.000% Params, 53.25 KMac, 0.004% MACs, kernel_size=3, stride=1, padding=1, dilation=1, ceil_mode=False)
(inc8_br3_relu1): ReLU(0, 0.000% Params, 53.25 KMac, 0.004% MACs, inplace=True)
(inc8_br3_conv2): Conv2d(106.5 k, 1.784% Params, 6.82 MMac, 0.466% MACs, 832, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(inc8_br3_bn2): BatchNorm2d(256, 0.004% Params, 16.38 KMac, 0.001% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(inc8_br3_relu2): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(aap15): AdaptiveAvgPool2d(0, 0.000% Params, 65.54 KMac, 0.004% MACs, output_size=1)
(conv16): Conv2d(102.5 k, 1.717% Params, 102.5 KMac, 0.007% MACs, 1024, 100, kernel_size=(1, 1), stride=(1, 1))
)
ykl/ALL/param_flops/cifar100/MobileNetV2.txt 0 → 100644
Model(
2.37 M, 100.000% Params, 68.43 MMac, 100.000% MACs,
(conv0): Conv2d(128, 0.005% Params, 147.97 KMac, 0.216% MACs, 3, 32, kernel_size=(1, 1), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(64, 0.003% Params, 73.98 KMac, 0.108% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relus0): ReLU6(0, 0.000% Params, 36.99 KMac, 0.054% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(1.06 k, 0.045% Params, 1.22 MMac, 1.784% MACs, 32, 32, kernel_size=(1, 1), stride=(1, 1))
(ml0_blk0_ma_bn0): BatchNorm2d(64, 0.003% Params, 73.98 KMac, 0.108% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relus0): ReLU6(0, 0.000% Params, 36.99 KMac, 0.054% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(320, 0.014% Params, 369.92 KMac, 0.541% MACs, 32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=32)
(ml0_blk0_ma_bn1): BatchNorm2d(64, 0.003% Params, 73.98 KMac, 0.108% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relus1): ReLU6(0, 0.000% Params, 36.99 KMac, 0.054% MACs, inplace=True)
(ml0_blk0_ma_conv2): Conv2d(528, 0.022% Params, 610.37 KMac, 0.892% MACs, 32, 16, kernel_size=(1, 1), stride=(1, 1))
(ml0_blk0_ma_bn2): BatchNorm2d(32, 0.001% Params, 36.99 KMac, 0.054% MACs, 16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_conv0): Conv2d(1.63 k, 0.069% Params, 1.89 MMac, 2.757% MACs, 16, 96, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk0_ma_bn0): BatchNorm2d(192, 0.008% Params, 221.95 KMac, 0.324% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relus0): ReLU6(0, 0.000% Params, 110.98 KMac, 0.162% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(960, 0.041% Params, 277.44 KMac, 0.405% MACs, 96, 96, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=96)
(ml1_blk0_ma_bn1): BatchNorm2d(192, 0.008% Params, 55.49 KMac, 0.081% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relus1): ReLU6(0, 0.000% Params, 27.74 KMac, 0.041% MACs, inplace=True)
(ml1_blk0_ma_conv2): Conv2d(2.33 k, 0.098% Params, 672.79 KMac, 0.983% MACs, 96, 24, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk0_ma_bn2): BatchNorm2d(48, 0.002% Params, 13.87 KMac, 0.020% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_conv0): Conv2d(3.6 k, 0.152% Params, 1.04 MMac, 1.520% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk1_ma_bn0): BatchNorm2d(288, 0.012% Params, 83.23 KMac, 0.122% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relus0): ReLU6(0, 0.000% Params, 41.62 KMac, 0.061% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(1.44 k, 0.061% Params, 416.16 KMac, 0.608% MACs, 144, 144, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=144)
(ml1_blk1_ma_bn1): BatchNorm2d(288, 0.012% Params, 83.23 KMac, 0.122% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relus1): ReLU6(0, 0.000% Params, 41.62 KMac, 0.061% MACs, inplace=True)
(ml1_blk1_ma_conv2): Conv2d(3.48 k, 0.147% Params, 1.01 MMac, 1.470% MACs, 144, 24, kernel_size=(1, 1), stride=(1, 1))
(ml1_blk1_ma_bn2): BatchNorm2d(48, 0.002% Params, 13.87 KMac, 0.020% MACs, 24, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_conv0): Conv2d(3.6 k, 0.152% Params, 1.04 MMac, 1.520% MACs, 24, 144, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk0_ma_bn0): BatchNorm2d(288, 0.012% Params, 83.23 KMac, 0.122% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relus0): ReLU6(0, 0.000% Params, 41.62 KMac, 0.061% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(1.44 k, 0.061% Params, 116.64 KMac, 0.170% MACs, 144, 144, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=144)
(ml2_blk0_ma_bn1): BatchNorm2d(288, 0.012% Params, 23.33 KMac, 0.034% MACs, 144, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relus1): ReLU6(0, 0.000% Params, 11.66 KMac, 0.017% MACs, inplace=True)
(ml2_blk0_ma_conv2): Conv2d(4.64 k, 0.196% Params, 375.84 KMac, 0.549% MACs, 144, 32, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk0_ma_bn2): BatchNorm2d(64, 0.003% Params, 5.18 KMac, 0.008% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_conv0): Conv2d(6.34 k, 0.267% Params, 513.22 KMac, 0.750% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk1_ma_bn0): BatchNorm2d(384, 0.016% Params, 31.1 KMac, 0.045% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relus0): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(1.92 k, 0.081% Params, 155.52 KMac, 0.227% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
(ml2_blk1_ma_bn1): BatchNorm2d(384, 0.016% Params, 31.1 KMac, 0.045% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relus1): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk1_ma_conv2): Conv2d(6.18 k, 0.261% Params, 500.26 KMac, 0.731% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk1_ma_bn2): BatchNorm2d(64, 0.003% Params, 5.18 KMac, 0.008% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_conv0): Conv2d(6.34 k, 0.267% Params, 513.22 KMac, 0.750% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk2_ma_bn0): BatchNorm2d(384, 0.016% Params, 31.1 KMac, 0.045% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relus0): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk2_ma_conv1): Conv2d(1.92 k, 0.081% Params, 155.52 KMac, 0.227% MACs, 192, 192, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=192)
(ml2_blk2_ma_bn1): BatchNorm2d(384, 0.016% Params, 31.1 KMac, 0.045% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relus1): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml2_blk2_ma_conv2): Conv2d(6.18 k, 0.261% Params, 500.26 KMac, 0.731% MACs, 192, 32, kernel_size=(1, 1), stride=(1, 1))
(ml2_blk2_ma_bn2): BatchNorm2d(64, 0.003% Params, 5.18 KMac, 0.008% MACs, 32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_conv0): Conv2d(6.34 k, 0.267% Params, 513.22 KMac, 0.750% MACs, 32, 192, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk0_ma_bn0): BatchNorm2d(384, 0.016% Params, 31.1 KMac, 0.045% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relus0): ReLU6(0, 0.000% Params, 15.55 KMac, 0.023% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(1.92 k, 0.081% Params, 48.0 KMac, 0.070% MACs, 192, 192, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), groups=192)
(ml3_blk0_ma_bn1): BatchNorm2d(384, 0.016% Params, 9.6 KMac, 0.014% MACs, 192, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relus1): ReLU6(0, 0.000% Params, 4.8 KMac, 0.007% MACs, inplace=True)
(ml3_blk0_ma_conv2): Conv2d(12.35 k, 0.521% Params, 308.8 KMac, 0.451% MACs, 192, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk0_ma_bn2): BatchNorm2d(128, 0.005% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_conv0): Conv2d(24.96 k, 1.053% Params, 624.0 KMac, 0.912% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk1_ma_bn0): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(3.84 k, 0.162% Params, 96.0 KMac, 0.140% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml3_blk1_ma_bn1): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk1_ma_conv2): Conv2d(24.64 k, 1.040% Params, 616.0 KMac, 0.900% MACs, 384, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk1_ma_bn2): BatchNorm2d(128, 0.005% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_conv0): Conv2d(24.96 k, 1.053% Params, 624.0 KMac, 0.912% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk2_ma_bn0): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk2_ma_conv1): Conv2d(3.84 k, 0.162% Params, 96.0 KMac, 0.140% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml3_blk2_ma_bn1): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk2_ma_conv2): Conv2d(24.64 k, 1.040% Params, 616.0 KMac, 0.900% MACs, 384, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk2_ma_bn2): BatchNorm2d(128, 0.005% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk3_ma_conv0): Conv2d(24.96 k, 1.053% Params, 624.0 KMac, 0.912% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk3_ma_bn0): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk3_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk3_ma_conv1): Conv2d(3.84 k, 0.162% Params, 96.0 KMac, 0.140% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml3_blk3_ma_bn1): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk3_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml3_blk3_ma_conv2): Conv2d(24.64 k, 1.040% Params, 616.0 KMac, 0.900% MACs, 384, 64, kernel_size=(1, 1), stride=(1, 1))
(ml3_blk3_ma_bn2): BatchNorm2d(128, 0.005% Params, 3.2 KMac, 0.005% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk0_ma_conv0): Conv2d(24.96 k, 1.053% Params, 624.0 KMac, 0.912% MACs, 64, 384, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk0_ma_bn0): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk0_ma_relus0): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml4_blk0_ma_conv1): Conv2d(3.84 k, 0.162% Params, 96.0 KMac, 0.140% MACs, 384, 384, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=384)
(ml4_blk0_ma_bn1): BatchNorm2d(768, 0.032% Params, 19.2 KMac, 0.028% MACs, 384, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk0_ma_relus1): ReLU6(0, 0.000% Params, 9.6 KMac, 0.014% MACs, inplace=True)
(ml4_blk0_ma_conv2): Conv2d(36.96 k, 1.560% Params, 924.0 KMac, 1.350% MACs, 384, 96, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk0_ma_bn2): BatchNorm2d(192, 0.008% Params, 4.8 KMac, 0.007% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk1_ma_conv0): Conv2d(55.87 k, 2.358% Params, 1.4 MMac, 2.041% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk1_ma_bn0): BatchNorm2d(1.15 k, 0.049% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk1_ma_relus0): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk1_ma_conv1): Conv2d(5.76 k, 0.243% Params, 144.0 KMac, 0.210% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
(ml4_blk1_ma_bn1): BatchNorm2d(1.15 k, 0.049% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk1_ma_relus1): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk1_ma_conv2): Conv2d(55.39 k, 2.338% Params, 1.38 MMac, 2.024% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk1_ma_bn2): BatchNorm2d(192, 0.008% Params, 4.8 KMac, 0.007% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk2_ma_conv0): Conv2d(55.87 k, 2.358% Params, 1.4 MMac, 2.041% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk2_ma_bn0): BatchNorm2d(1.15 k, 0.049% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk2_ma_relus0): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk2_ma_conv1): Conv2d(5.76 k, 0.243% Params, 144.0 KMac, 0.210% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
(ml4_blk2_ma_bn1): BatchNorm2d(1.15 k, 0.049% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml4_blk2_ma_relus1): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml4_blk2_ma_conv2): Conv2d(55.39 k, 2.338% Params, 1.38 MMac, 2.024% MACs, 576, 96, kernel_size=(1, 1), stride=(1, 1))
(ml4_blk2_ma_bn2): BatchNorm2d(192, 0.008% Params, 4.8 KMac, 0.007% MACs, 96, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk0_ma_conv0): Conv2d(55.87 k, 2.358% Params, 1.4 MMac, 2.041% MACs, 96, 576, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk0_ma_bn0): BatchNorm2d(1.15 k, 0.049% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk0_ma_relus0): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml5_blk0_ma_conv1): Conv2d(5.76 k, 0.243% Params, 144.0 KMac, 0.210% MACs, 576, 576, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=576)
(ml5_blk0_ma_bn1): BatchNorm2d(1.15 k, 0.049% Params, 28.8 KMac, 0.042% MACs, 576, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk0_ma_relus1): ReLU6(0, 0.000% Params, 14.4 KMac, 0.021% MACs, inplace=True)
(ml5_blk0_ma_conv2): Conv2d(92.32 k, 3.896% Params, 2.31 MMac, 3.373% MACs, 576, 160, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk0_ma_bn2): BatchNorm2d(320, 0.014% Params, 8.0 KMac, 0.012% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk1_ma_conv0): Conv2d(154.56 k, 6.523% Params, 3.86 MMac, 5.646% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk1_ma_bn0): BatchNorm2d(1.92 k, 0.081% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk1_ma_relus0): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk1_ma_conv1): Conv2d(9.6 k, 0.405% Params, 240.0 KMac, 0.351% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
(ml5_blk1_ma_bn1): BatchNorm2d(1.92 k, 0.081% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk1_ma_relus1): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk1_ma_conv2): Conv2d(153.76 k, 6.489% Params, 3.84 MMac, 5.617% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk1_ma_bn2): BatchNorm2d(320, 0.014% Params, 8.0 KMac, 0.012% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk2_ma_conv0): Conv2d(154.56 k, 6.523% Params, 3.86 MMac, 5.646% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk2_ma_bn0): BatchNorm2d(1.92 k, 0.081% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk2_ma_relus0): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk2_ma_conv1): Conv2d(9.6 k, 0.405% Params, 240.0 KMac, 0.351% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
(ml5_blk2_ma_bn1): BatchNorm2d(1.92 k, 0.081% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml5_blk2_ma_relus1): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml5_blk2_ma_conv2): Conv2d(153.76 k, 6.489% Params, 3.84 MMac, 5.617% MACs, 960, 160, kernel_size=(1, 1), stride=(1, 1))
(ml5_blk2_ma_bn2): BatchNorm2d(320, 0.014% Params, 8.0 KMac, 0.012% MACs, 160, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml6_blk0_ma_conv0): Conv2d(154.56 k, 6.523% Params, 3.86 MMac, 5.646% MACs, 160, 960, kernel_size=(1, 1), stride=(1, 1))
(ml6_blk0_ma_bn0): BatchNorm2d(1.92 k, 0.081% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml6_blk0_ma_relus0): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml6_blk0_ma_conv1): Conv2d(9.6 k, 0.405% Params, 240.0 KMac, 0.351% MACs, 960, 960, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=960)
(ml6_blk0_ma_bn1): BatchNorm2d(1.92 k, 0.081% Params, 48.0 KMac, 0.070% MACs, 960, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml6_blk0_ma_relus1): ReLU6(0, 0.000% Params, 24.0 KMac, 0.035% MACs, inplace=True)
(ml6_blk0_ma_conv2): Conv2d(307.52 k, 12.979% Params, 7.69 MMac, 11.234% MACs, 960, 320, kernel_size=(1, 1), stride=(1, 1))
(ml6_blk0_ma_bn2): BatchNorm2d(640, 0.027% Params, 16.0 KMac, 0.023% MACs, 320, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv8): Conv2d(410.88 k, 17.341% Params, 10.27 MMac, 15.010% MACs, 320, 1280, kernel_size=(1, 1), stride=(1, 1))
(bn8): BatchNorm2d(2.56 k, 0.108% Params, 64.0 KMac, 0.094% MACs, 1280, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relus8): ReLU6(0, 0.000% Params, 32.0 KMac, 0.047% MACs, inplace=True)
(aap9): AdaptiveAvgPool2d(0, 0.000% Params, 32.0 KMac, 0.047% MACs, output_size=1)
(fc11): Linear(128.1 k, 5.406% Params, 128.1 KMac, 0.187% MACs, in_features=1280, out_features=100, bias=True)
)
ykl/ALL/param_flops/cifar100/ResNet_152.txt 0 → 100644
Model(
58.34 M, 100.000% Params, 3.74 GMac, 100.000% MACs,
(conv0): Conv2d(1.73 k, 0.003% Params, 1.77 MMac, 0.047% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(4.1 k, 0.007% Params, 4.19 MMac, 0.112% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(36.86 k, 0.063% Params, 37.75 MMac, 1.008% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk0_ma_conv2): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn2): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ds_conv0): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ds_bn0): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.007% MACs, inplace=True)
(ml0_blk1_ma_conv0): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk1_ma_conv1): Conv2d(36.86 k, 0.063% Params, 37.75 MMac, 1.008% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk1_ma_conv2): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn2): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.007% MACs, inplace=True)
(ml0_blk2_ma_conv0): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk2_ma_conv1): Conv2d(36.86 k, 0.063% Params, 37.75 MMac, 1.008% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk2_ma_bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.004% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml0_blk2_ma_conv2): Conv2d(16.38 k, 0.028% Params, 16.78 MMac, 0.448% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn2): BatchNorm2d(512, 0.001% Params, 524.29 KMac, 0.014% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.007% MACs, inplace=True)
(ml1_blk0_ma_conv0): Conv2d(32.77 k, 0.056% Params, 33.55 MMac, 0.896% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn0): BatchNorm2d(256, 0.000% Params, 262.14 KMac, 0.007% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk0_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ds_conv0): Conv2d(131.07 k, 0.225% Params, 33.55 MMac, 0.896% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml1_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk1_ma_conv0): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk1_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk2_ma_conv0): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk2_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk2_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk2_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk3_ma_conv0): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk3_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk3_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk3_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk4_ma_conv0): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk4_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk4_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk4_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk4_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk4_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk4_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk4_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk4_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk5_ma_conv0): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk5_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk5_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk5_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk5_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk5_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk5_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk5_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk5_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk6_ma_conv0): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk6_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk6_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk6_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk6_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk6_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk6_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk6_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk6_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml1_blk7_ma_conv0): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk7_ma_bn0): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk7_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk7_ma_conv1): Conv2d(147.46 k, 0.253% Params, 37.75 MMac, 1.008% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk7_ma_bn1): BatchNorm2d(256, 0.000% Params, 65.54 KMac, 0.002% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk7_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml1_blk7_ma_conv2): Conv2d(65.54 k, 0.112% Params, 16.78 MMac, 0.448% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk7_ma_bn2): BatchNorm2d(1.02 k, 0.002% Params, 262.14 KMac, 0.007% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk7_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.004% MACs, inplace=True)
(ml2_blk0_ma_conv0): Conv2d(131.07 k, 0.225% Params, 33.55 MMac, 0.896% MACs, 512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn0): BatchNorm2d(512, 0.001% Params, 131.07 KMac, 0.004% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk0_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ds_conv0): Conv2d(524.29 k, 0.899% Params, 33.55 MMac, 0.896% MACs, 512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml2_blk0_ds_bn0): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk1_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk1_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk2_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk2_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk2_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk2_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk3_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk3_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk3_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk3_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk4_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk4_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk4_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk4_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk5_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk5_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk5_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk5_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk6_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk6_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk6_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk6_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk6_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk6_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk6_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk6_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk6_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk7_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk7_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk7_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk7_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk7_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk7_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk7_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk7_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk7_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk8_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk8_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk8_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk8_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk8_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk8_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk8_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk8_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk8_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk9_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk9_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk9_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk9_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk9_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk9_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk9_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk9_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk9_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk10_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk10_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk10_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk10_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk10_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk10_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk10_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk10_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk10_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk11_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk11_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk11_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk11_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk11_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk11_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk11_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk11_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk11_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk12_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk12_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk12_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk12_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk12_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk12_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk12_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk12_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk12_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk13_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk13_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk13_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk13_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk13_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk13_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk13_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk13_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk13_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk14_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk14_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk14_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk14_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk14_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk14_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk14_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk14_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk14_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk15_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk15_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk15_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk15_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk15_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk15_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk15_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk15_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk15_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk16_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk16_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk16_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk16_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk16_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk16_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk16_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk16_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk16_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk17_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk17_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk17_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk17_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk17_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk17_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk17_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk17_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk17_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk18_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk18_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk18_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk18_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk18_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk18_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk18_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk18_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk18_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk19_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk19_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk19_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk19_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk19_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk19_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk19_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk19_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk19_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk20_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk20_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk20_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk20_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk20_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk20_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk20_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk20_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk20_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk21_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk21_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk21_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk21_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk21_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk21_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk21_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk21_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk21_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk22_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk22_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk22_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk22_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk22_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk22_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk22_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk22_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk22_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk23_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk23_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk23_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk23_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk23_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk23_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk23_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk23_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk23_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk24_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk24_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk24_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk24_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk24_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk24_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk24_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk24_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk24_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk25_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk25_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk25_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk25_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk25_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk25_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk25_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk25_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk25_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk26_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk26_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk26_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk26_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk26_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk26_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk26_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk26_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk26_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk27_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk27_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk27_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk27_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk27_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk27_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk27_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk27_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk27_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk28_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk28_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk28_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk28_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk28_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk28_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk28_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk28_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk28_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk29_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk29_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk29_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk29_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk29_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk29_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk29_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk29_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk29_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk30_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk30_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk30_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk30_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk30_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk30_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk30_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk30_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk30_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk31_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk31_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk31_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk31_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk31_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk31_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk31_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk31_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk31_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk32_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk32_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk32_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk32_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk32_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk32_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk32_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk32_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk32_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk33_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk33_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk33_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk33_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk33_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk33_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk33_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk33_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk33_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk34_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk34_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk34_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk34_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk34_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk34_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk34_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk34_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk34_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml2_blk35_ma_conv0): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk35_ma_bn0): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk35_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk35_ma_conv1): Conv2d(589.82 k, 1.011% Params, 37.75 MMac, 1.008% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk35_ma_bn1): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.001% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk35_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.000% MACs, inplace=True)
(ml2_blk35_ma_conv2): Conv2d(262.14 k, 0.449% Params, 16.78 MMac, 0.448% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk35_ma_bn2): BatchNorm2d(2.05 k, 0.004% Params, 131.07 KMac, 0.004% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk35_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.002% MACs, inplace=True)
(ml3_blk0_ma_conv0): Conv2d(524.29 k, 0.899% Params, 33.55 MMac, 0.896% MACs, 1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn0): BatchNorm2d(1.02 k, 0.002% Params, 65.54 KMac, 0.002% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(2.36 M, 4.044% Params, 37.75 MMac, 1.008% MACs, 512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn1): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk0_ma_conv2): Conv2d(1.05 M, 1.797% Params, 16.78 MMac, 0.448% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn2): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ds_conv0): Conv2d(2.1 M, 3.595% Params, 33.55 MMac, 0.896% MACs, 1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml3_blk0_ds_bn0): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv0): Conv2d(1.05 M, 1.797% Params, 16.78 MMac, 0.448% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn0): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(2.36 M, 4.044% Params, 37.75 MMac, 1.008% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn1): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk1_ma_conv2): Conv2d(1.05 M, 1.797% Params, 16.78 MMac, 0.448% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn2): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(ml3_blk2_ma_conv0): Conv2d(1.05 M, 1.797% Params, 16.78 MMac, 0.448% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn0): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk2_ma_conv1): Conv2d(2.36 M, 4.044% Params, 37.75 MMac, 1.008% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk2_ma_bn1): BatchNorm2d(1.02 k, 0.002% Params, 16.38 KMac, 0.000% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.000% MACs, inplace=True)
(ml3_blk2_ma_conv2): Conv2d(1.05 M, 1.797% Params, 16.78 MMac, 0.448% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn2): BatchNorm2d(4.1 k, 0.007% Params, 65.54 KMac, 0.002% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.001% MACs, inplace=True)
(aap5): AdaptiveAvgPool2d(0, 0.000% Params, 32.77 KMac, 0.001% MACs, output_size=1)
(fc7): Linear(204.9 k, 0.351% Params, 204.9 KMac, 0.005% MACs, in_features=2048, out_features=100, bias=True)
)
ykl/ALL/param_flops/cifar100/ResNet_18.txt 0 → 100644
Model(
11.22 M, 100.000% Params, 557.26 MMac, 100.000% MACs,
(conv0): Conv2d(1.73 k, 0.015% Params, 1.77 MMac, 0.318% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(36.86 k, 0.329% Params, 37.75 MMac, 6.774% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(36.86 k, 0.329% Params, 37.75 MMac, 6.774% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk1_ma_conv0): Conv2d(36.86 k, 0.329% Params, 37.75 MMac, 6.774% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml0_blk1_ma_conv1): Conv2d(36.86 k, 0.329% Params, 37.75 MMac, 6.774% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.024% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.012% MACs, inplace=True)
(ml1_blk0_ma_conv0): Conv2d(73.73 k, 0.657% Params, 18.87 MMac, 3.387% MACs, 64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn0): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(147.46 k, 1.314% Params, 37.75 MMac, 6.774% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn1): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ds_conv0): Conv2d(8.19 k, 0.073% Params, 2.1 MMac, 0.376% MACs, 64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml1_blk0_ds_bn0): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml1_blk1_ma_conv0): Conv2d(147.46 k, 1.314% Params, 37.75 MMac, 6.774% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn0): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(147.46 k, 1.314% Params, 37.75 MMac, 6.774% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn1): BatchNorm2d(256, 0.002% Params, 65.54 KMac, 0.012% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.006% MACs, inplace=True)
(ml2_blk0_ma_conv0): Conv2d(294.91 k, 2.628% Params, 18.87 MMac, 3.387% MACs, 128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn0): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(589.82 k, 5.257% Params, 37.75 MMac, 6.774% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn1): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ds_conv0): Conv2d(32.77 k, 0.292% Params, 2.1 MMac, 0.376% MACs, 128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml2_blk0_ds_bn0): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml2_blk1_ma_conv0): Conv2d(589.82 k, 5.257% Params, 37.75 MMac, 6.774% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn0): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(589.82 k, 5.257% Params, 37.75 MMac, 6.774% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn1): BatchNorm2d(512, 0.005% Params, 32.77 KMac, 0.006% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.003% MACs, inplace=True)
(ml3_blk0_ma_conv0): Conv2d(1.18 M, 10.514% Params, 18.87 MMac, 3.387% MACs, 256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn0): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(2.36 M, 21.027% Params, 37.75 MMac, 6.774% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn1): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ds_conv0): Conv2d(131.07 k, 1.168% Params, 2.1 MMac, 0.376% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml3_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv0): Conv2d(2.36 M, 21.027% Params, 37.75 MMac, 6.774% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn0): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(2.36 M, 21.027% Params, 37.75 MMac, 6.774% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn1): BatchNorm2d(1.02 k, 0.009% Params, 16.38 KMac, 0.003% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(aap5): AdaptiveAvgPool2d(0, 0.000% Params, 8.19 KMac, 0.001% MACs, output_size=1)
(fc7): Linear(51.3 k, 0.457% Params, 51.3 KMac, 0.009% MACs, in_features=512, out_features=100, bias=True)
)
ykl/ALL/param_flops/cifar100/ResNet_50.txt 0 → 100644
Model(
23.71 M, 100.000% Params, 1.31 GMac, 100.000% MACs,
(conv0): Conv2d(1.73 k, 0.007% Params, 1.77 MMac, 0.135% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk0_ma_conv0): Conv2d(4.1 k, 0.017% Params, 4.19 MMac, 0.321% MACs, 64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk0_ma_conv1): Conv2d(36.86 k, 0.156% Params, 37.75 MMac, 2.886% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk0_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk0_ma_conv2): Conv2d(16.38 k, 0.069% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ma_bn2): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_ds_conv0): Conv2d(16.38 k, 0.069% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk0_ds_bn0): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk0_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.020% MACs, inplace=True)
(ml0_blk1_ma_conv0): Conv2d(16.38 k, 0.069% Params, 16.78 MMac, 1.283% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk1_ma_conv1): Conv2d(36.86 k, 0.156% Params, 37.75 MMac, 2.886% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk1_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk1_ma_conv2): Conv2d(16.38 k, 0.069% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk1_ma_bn2): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk1_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.020% MACs, inplace=True)
(ml0_blk2_ma_conv0): Conv2d(16.38 k, 0.069% Params, 16.78 MMac, 1.283% MACs, 256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn0): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk2_ma_conv1): Conv2d(36.86 k, 0.156% Params, 37.75 MMac, 2.886% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml0_blk2_ma_bn1): BatchNorm2d(128, 0.001% Params, 131.07 KMac, 0.010% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_ma_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml0_blk2_ma_conv2): Conv2d(16.38 k, 0.069% Params, 16.78 MMac, 1.283% MACs, 64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml0_blk2_ma_bn2): BatchNorm2d(512, 0.002% Params, 524.29 KMac, 0.040% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml0_blk2_relu1): ReLU(0, 0.000% Params, 262.14 KMac, 0.020% MACs, inplace=True)
(ml1_blk0_ma_conv0): Conv2d(32.77 k, 0.138% Params, 33.55 MMac, 2.566% MACs, 256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn0): BatchNorm2d(256, 0.001% Params, 262.14 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu0): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk0_ma_conv1): Conv2d(147.46 k, 0.622% Params, 37.75 MMac, 2.886% MACs, 128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml1_blk0_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk0_ma_conv2): Conv2d(65.54 k, 0.276% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk0_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_ds_conv0): Conv2d(131.07 k, 0.553% Params, 33.55 MMac, 2.566% MACs, 256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml1_blk0_ds_bn0): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk0_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk1_ma_conv0): Conv2d(65.54 k, 0.276% Params, 16.78 MMac, 1.283% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn0): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk1_ma_conv1): Conv2d(147.46 k, 0.622% Params, 37.75 MMac, 2.886% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk1_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk1_ma_conv2): Conv2d(65.54 k, 0.276% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk1_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk1_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk2_ma_conv0): Conv2d(65.54 k, 0.276% Params, 16.78 MMac, 1.283% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn0): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk2_ma_conv1): Conv2d(147.46 k, 0.622% Params, 37.75 MMac, 2.886% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk2_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk2_ma_conv2): Conv2d(65.54 k, 0.276% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk2_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk2_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml1_blk3_ma_conv0): Conv2d(65.54 k, 0.276% Params, 16.78 MMac, 1.283% MACs, 512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn0): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk3_ma_conv1): Conv2d(147.46 k, 0.622% Params, 37.75 MMac, 2.886% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml1_blk3_ma_bn1): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.005% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_ma_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml1_blk3_ma_conv2): Conv2d(65.54 k, 0.276% Params, 16.78 MMac, 1.283% MACs, 128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml1_blk3_ma_bn2): BatchNorm2d(1.02 k, 0.004% Params, 262.14 KMac, 0.020% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml1_blk3_relu1): ReLU(0, 0.000% Params, 131.07 KMac, 0.010% MACs, inplace=True)
(ml2_blk0_ma_conv0): Conv2d(131.07 k, 0.553% Params, 33.55 MMac, 2.566% MACs, 512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn0): BatchNorm2d(512, 0.002% Params, 131.07 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk0_ma_conv1): Conv2d(589.82 k, 2.488% Params, 37.75 MMac, 2.886% MACs, 256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml2_blk0_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk0_ma_conv2): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk0_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_ds_conv0): Conv2d(524.29 k, 2.212% Params, 33.55 MMac, 2.566% MACs, 512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml2_blk0_ds_bn0): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk0_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk1_ma_conv0): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk1_ma_conv1): Conv2d(589.82 k, 2.488% Params, 37.75 MMac, 2.886% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk1_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk1_ma_conv2): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk1_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk1_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk2_ma_conv0): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk2_ma_conv1): Conv2d(589.82 k, 2.488% Params, 37.75 MMac, 2.886% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk2_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk2_ma_conv2): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk2_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk2_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk3_ma_conv0): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk3_ma_conv1): Conv2d(589.82 k, 2.488% Params, 37.75 MMac, 2.886% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk3_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk3_ma_conv2): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk3_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk3_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk4_ma_conv0): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk4_ma_conv1): Conv2d(589.82 k, 2.488% Params, 37.75 MMac, 2.886% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk4_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk4_ma_conv2): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk4_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk4_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml2_blk5_ma_conv0): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn0): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu0): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk5_ma_conv1): Conv2d(589.82 k, 2.488% Params, 37.75 MMac, 2.886% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml2_blk5_ma_bn1): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.003% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_ma_relu1): ReLU(0, 0.000% Params, 16.38 KMac, 0.001% MACs, inplace=True)
(ml2_blk5_ma_conv2): Conv2d(262.14 k, 1.106% Params, 16.78 MMac, 1.283% MACs, 256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml2_blk5_ma_bn2): BatchNorm2d(2.05 k, 0.009% Params, 131.07 KMac, 0.010% MACs, 1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml2_blk5_relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.005% MACs, inplace=True)
(ml3_blk0_ma_conv0): Conv2d(524.29 k, 2.212% Params, 33.55 MMac, 2.566% MACs, 1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn0): BatchNorm2d(1.02 k, 0.004% Params, 65.54 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu0): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml3_blk0_ma_conv1): Conv2d(2.36 M, 9.953% Params, 37.75 MMac, 2.886% MACs, 512, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(ml3_blk0_ma_bn1): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk0_ma_conv2): Conv2d(1.05 M, 4.423% Params, 16.78 MMac, 1.283% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk0_ma_bn2): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_ds_conv0): Conv2d(2.1 M, 8.847% Params, 33.55 MMac, 2.566% MACs, 1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
(ml3_blk0_ds_bn0): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk0_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml3_blk1_ma_conv0): Conv2d(1.05 M, 4.423% Params, 16.78 MMac, 1.283% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn0): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv1): Conv2d(2.36 M, 9.953% Params, 37.75 MMac, 2.886% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk1_ma_bn1): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk1_ma_conv2): Conv2d(1.05 M, 4.423% Params, 16.78 MMac, 1.283% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk1_ma_bn2): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk1_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(ml3_blk2_ma_conv0): Conv2d(1.05 M, 4.423% Params, 16.78 MMac, 1.283% MACs, 2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn0): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu0): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk2_ma_conv1): Conv2d(2.36 M, 9.953% Params, 37.75 MMac, 2.886% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(ml3_blk2_ma_bn1): BatchNorm2d(1.02 k, 0.004% Params, 16.38 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_ma_relu1): ReLU(0, 0.000% Params, 8.19 KMac, 0.001% MACs, inplace=True)
(ml3_blk2_ma_conv2): Conv2d(1.05 M, 4.423% Params, 16.78 MMac, 1.283% MACs, 512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
(ml3_blk2_ma_bn2): BatchNorm2d(4.1 k, 0.017% Params, 65.54 KMac, 0.005% MACs, 2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(ml3_blk2_relu1): ReLU(0, 0.000% Params, 32.77 KMac, 0.003% MACs, inplace=True)
(aap5): AdaptiveAvgPool2d(0, 0.000% Params, 32.77 KMac, 0.003% MACs, output_size=1)
(fc7): Linear(204.9 k, 0.864% Params, 204.9 KMac, 0.016% MACs, in_features=2048, out_features=100, bias=True)
)
ykl/ALL/param_flops/cifar100/VGG_16.txt 0 → 100644
Model(
34.02 M, 100.000% Params, 333.73 MMac, 100.000% MACs,
(conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.550% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
(conv1): Conv2d(36.93 k, 0.109% Params, 37.81 MMac, 11.331% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.039% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.020% MACs, inplace=True)
(pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.020% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3): Conv2d(73.86 k, 0.217% Params, 18.91 MMac, 5.665% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
(conv4): Conv2d(147.58 k, 0.434% Params, 37.78 MMac, 11.321% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.020% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.010% MACs, inplace=True)
(pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.010% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv6): Conv2d(295.17 k, 0.868% Params, 18.89 MMac, 5.661% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn6): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
(conv7): Conv2d(590.08 k, 1.735% Params, 37.77 MMac, 11.316% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn7): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
(conv8): Conv2d(590.08 k, 1.735% Params, 37.77 MMac, 11.316% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn8): BatchNorm2d(512, 0.002% Params, 32.77 KMac, 0.010% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.005% MACs, inplace=True)
(pool9): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.005% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv10): Conv2d(1.18 M, 3.469% Params, 18.88 MMac, 5.658% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn10): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu10): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv11): Conv2d(2.36 M, 6.937% Params, 37.76 MMac, 11.314% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv12): Conv2d(2.36 M, 6.937% Params, 37.76 MMac, 11.314% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.005% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(pool13): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv14): Conv2d(2.36 M, 6.937% Params, 9.44 MMac, 2.828% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn14): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu14): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
(conv15): Conv2d(2.36 M, 6.937% Params, 9.44 MMac, 2.828% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn15): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu15): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
(conv16): Conv2d(2.36 M, 6.937% Params, 9.44 MMac, 2.828% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.001% MACs, inplace=True)
(pool17): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.001% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(fc19): Linear(2.1 M, 6.177% Params, 2.1 MMac, 0.630% MACs, in_features=512, out_features=4096, bias=True)
(relu20): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop21): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc22): Linear(16.78 M, 49.334% Params, 16.78 MMac, 5.028% MACs, in_features=4096, out_features=4096, bias=True)
(relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc25): Linear(409.7 k, 1.204% Params, 409.7 KMac, 0.123% MACs, in_features=4096, out_features=100, bias=True)
)
ykl/ALL/param_flops/cifar100/VGG_19.txt 0 → 100644
Model(
39.33 M, 100.000% Params, 418.77 MMac, 100.000% MACs,
(conv0): Conv2d(1.79 k, 0.005% Params, 1.84 MMac, 0.438% MACs, 3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn0): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu0): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
(conv1): Conv2d(36.93 k, 0.094% Params, 37.81 MMac, 9.030% MACs, 64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn1): BatchNorm2d(128, 0.000% Params, 131.07 KMac, 0.031% MACs, 64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu1): ReLU(0, 0.000% Params, 65.54 KMac, 0.016% MACs, inplace=True)
(pool2): MaxPool2d(0, 0.000% Params, 65.54 KMac, 0.016% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv3): Conv2d(73.86 k, 0.188% Params, 18.91 MMac, 4.515% MACs, 64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn3): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu3): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
(conv4): Conv2d(147.58 k, 0.375% Params, 37.78 MMac, 9.022% MACs, 128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn4): BatchNorm2d(256, 0.001% Params, 65.54 KMac, 0.016% MACs, 128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu4): ReLU(0, 0.000% Params, 32.77 KMac, 0.008% MACs, inplace=True)
(pool5): MaxPool2d(0, 0.000% Params, 32.77 KMac, 0.008% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv6): Conv2d(295.17 k, 0.751% Params, 18.89 MMac, 4.511% MACs, 128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn6): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu6): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(conv7): Conv2d(590.08 k, 1.500% Params, 37.77 MMac, 9.018% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn7): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu7): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(conv8): Conv2d(590.08 k, 1.500% Params, 37.77 MMac, 9.018% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn8): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu8): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(conv9): Conv2d(590.08 k, 1.500% Params, 37.77 MMac, 9.018% MACs, 256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn9): BatchNorm2d(512, 0.001% Params, 32.77 KMac, 0.008% MACs, 256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu9): ReLU(0, 0.000% Params, 16.38 KMac, 0.004% MACs, inplace=True)
(pool10): MaxPool2d(0, 0.000% Params, 16.38 KMac, 0.004% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv11): Conv2d(1.18 M, 3.001% Params, 18.88 MMac, 4.509% MACs, 256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn11): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu11): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv12): Conv2d(2.36 M, 6.000% Params, 37.76 MMac, 9.016% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn12): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu12): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv13): Conv2d(2.36 M, 6.000% Params, 37.76 MMac, 9.016% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn13): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu13): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(conv14): Conv2d(2.36 M, 6.000% Params, 37.76 MMac, 9.016% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn14): BatchNorm2d(1.02 k, 0.003% Params, 16.38 KMac, 0.004% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu14): ReLU(0, 0.000% Params, 8.19 KMac, 0.002% MACs, inplace=True)
(pool15): MaxPool2d(0, 0.000% Params, 8.19 KMac, 0.002% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(conv16): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn16): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu16): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(conv17): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn17): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu17): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(conv18): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn18): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu18): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(conv19): Conv2d(2.36 M, 6.000% Params, 9.44 MMac, 2.254% MACs, 512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn19): BatchNorm2d(1.02 k, 0.003% Params, 4.1 KMac, 0.001% MACs, 512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu19): ReLU(0, 0.000% Params, 2.05 KMac, 0.000% MACs, inplace=True)
(pool20): MaxPool2d(0, 0.000% Params, 2.05 KMac, 0.000% MACs, kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(fc22): Linear(2.1 M, 5.343% Params, 2.1 MMac, 0.502% MACs, in_features=512, out_features=4096, bias=True)
(relu23): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop24): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc25): Linear(16.78 M, 42.671% Params, 16.78 MMac, 4.007% MACs, in_features=4096, out_features=4096, bias=True)
(relu26): ReLU(0, 0.000% Params, 4.1 KMac, 0.001% MACs, inplace=True)
(drop27): Dropout(0, 0.000% Params, 0.0 Mac, 0.000% MACs, p=0.5, inplace=False)
(fc28): Linear(409.7 k, 1.042% Params, 409.7 KMac, 0.098% MACs, in_features=4096, out_features=100, bias=True)
)
ykl/ALL/ptq_one.py 0 → 100644
from torch.serialization import load
from model import *
from extract_ratio import *
from utils import *
from dataloader import DataLoader
import gol
import openpyxl
import sys
import torch
import os
import os.path as osp
def direct_quantize(model, test_loader,device):
with torch.no_grad():
for i, (data, target) in enumerate(test_loader,1):
data = data.to(device)
output = model.quantize_forward(data)
if i % 100 == 0:
break
print('direct quantization finish')
def full_inference(model, test_loader, device):
correct = 0
with torch.no_grad():
for data, target in test_loader:
data,target = data.to(device), target.to(device)
output = model(data)
pred = output.argmax(dim=1, keepdim=True)
# print(pred)
correct += pred.eq(target.view_as(pred)).sum().item()
print('\nTest set: Full Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
return 100. * correct / len(test_loader.dataset)
def quantize_inference(model, test_loader, device):
correct = 0
with torch.no_grad():
for data, target in test_loader:
data,target = data.to(device), target.to(device)
output = model.quantize_inference(data)
pred = output.argmax(dim=1, keepdim=True)
correct += pred.eq(target.view_as(pred)).sum().item()
print('Test set: Quant Model Accuracy: {:.2f}%'.format(100. * correct / len(test_loader.dataset)))
return 100. * correct / len(test_loader.dataset)
if __name__ == "__main__":
batch_size = 128
model_name = sys.argv[1]
dataset = sys.argv[2]
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(device)
dataloader = DataLoader(dataset,batch_size)
train_loader, val_loader, test_loader = dataloader.getloader()
load_ptq = False
store_ptq = True
append = False
gol._init()
ckpt_full_path = 'ckpt_full/'+dataset
ckpt_quant_path = 'ckpt_quant/'+dataset
ptq_result_path = 'ptq_result/'+dataset
if not osp.exists(ckpt_quant_path) and store_ptq:
os.makedirs(ckpt_quant_path)
if not osp.exists(ptq_result_path):
os.makedirs(ptq_result_path)
excel_path = ptq_result_path+'/'+model_name+'.xlsx'
txt_path = ptq_result_path+'/'+model_name+'.txt'
if append and os.path.exists(excel_path) and os.path.exists(txt_path):
print("> Exit: "+model_name+"has been quantized")
sys.exit()
else:
workbook = openpyxl.Workbook()
ft = open(txt_path,'w')
full_file = ckpt_full_path+'/'+model_name+'.pt'
ptq_file_dir = ckpt_quant_path+'/'+model_name
if not osp.exists(ptq_file_dir):
os.makedirs(ptq_file_dir)
model = Model(model_name,dataset)
model.load_state_dict(torch.load(full_file))
model.to(device)
model.eval()
full_acc = full_inference(model, test_loader, device)
# 传入后可变
fold_model(model)
Mac,Param,layer, par_ratio, flop_ratio = extract_ratio(model_name,dataset)
par_ratio, flop_ratio = fold_ratio(layer, par_ratio, flop_ratio)
full_names = []
full_params = []
for name, param in model.named_parameters():
if 'conv' in name or 'fc' in name:
full_names.append(name)
full_params.append(param.data.cpu())
quant_type_list = ['INT','POT','FLOAT']
title_list = []
js_flops_list = []
js_flops_wt_log_list = []
js_flops_wt_cbrt_list = []
js_param_list = []
ptq_acc_list = []
acc_loss_list = []
for quant_type in quant_type_list:
num_bit_list = numbit_list(quant_type)
# 对一个量化类别，只需设置一次bias量化表
# int由于位宽大，使用量化表开销过大，直接_round即可
if quant_type != 'INT':
bias_list = build_bias_list(quant_type)
gol.set_value(bias_list, is_bias=True)
for num_bits in num_bit_list:
e_bit_list = ebit_list(quant_type,num_bits)
for e_bits in e_bit_list:
model_ptq = Model(model_name,dataset)
if quant_type == 'FLOAT':
title = '%s_%d_E%d' % (quant_type, num_bits, e_bits)
else:
title = '%s_%d' % (quant_type, num_bits)
print('\n'+model_name+': PTQ: '+title)
title_list.append(title)
# 设置量化表
if quant_type != 'INT':
plist = build_list(quant_type, num_bits, e_bits)
gol.set_value(plist)
# 判断是否需要载入
ptq_file = ptq_file_dir +'/' + title + '.pt'
if load_ptq is True and osp.exists(ptq_file):
model_ptq.quantize(quant_type,num_bits,e_bits)
model_ptq.load_state_dict(torch.load(ptq_file))
model_ptq.to(device)
model_ptq.eval()
print('Successfully load ptq model: ' + title)
else:
model_ptq.load_state_dict(torch.load(full_file))
model_ptq.to(device)
model_ptq.quantize(quant_type,num_bits,e_bits)
model_ptq.eval()
direct_quantize(model_ptq, train_loader, device)
if store_ptq:
torch.save(model_ptq.state_dict(), ptq_file)
model_ptq.freeze()
ptq_acc = quantize_inference(model_ptq, test_loader, device)
ptq_acc_list.append(ptq_acc)
acc_loss = (full_acc - ptq_acc) / full_acc
acc_loss_list.append(acc_loss)
#将量化后分布反量化到全精度相同的scale
model_ptq.fakefreeze()
# 获取计算量/参数量下的js-div
js_flops = 0.
js_param = 0.
for name, param in model_ptq.named_parameters():
if 'conv' not in name and 'fc' not in name:
continue
prefix = name.rsplit('.',1)[0]
layer_idx = layer.index(prefix)
name_idx = full_names.index(name)
layer_idx = layer.index(prefix)
ptq_param = param.data.cpu()
js = js_div(ptq_param,full_params[name_idx])
js = js.item()
if js < 0.:
js = 0.
js_flops = js_flops + js * flop_ratio[layer_idx]
js_param = js_param + js * par_ratio[layer_idx]
js_flops_wt_log = js_flops * torch.log10(torch.tensor(Mac)).item()
js_flops_wt_cbrt = js_flops * torch.pow(torch.tensor(Mac),1/3).item()
js_flops_list.append(js_flops)
js_flops_wt_log_list.append(js_flops_wt_log)
js_flops_wt_cbrt_list.append(js_flops_wt_cbrt)
js_param_list.append(js_param)
print(title + ': js_flops: %f js_flops_wt_log: %f js_flops_wt_cbrt: %f js_param: %f acc_loss: %f' % (js_flops,js_flops_wt_log, js_flops_wt_cbrt, js_param, acc_loss))
# 写入xlsx
worksheet = workbook.active
worksheet.cell(row=1,column=1,value='FP32-acc')
worksheet.cell(row=1,column=2,value=full_acc)
worksheet.cell(row=1,column=3,value='Mac')
worksheet.cell(row=1,column=4,value=Mac)
worksheet.cell(row=1,column=5,value='Param')
worksheet.cell(row=1,column=6,value=Param)
worksheet.cell(row=3,column=1,value='title')
worksheet.cell(row=3,column=2,value='js_flops')
worksheet.cell(row=3,column=3,value='js_flops_wt_log')
worksheet.cell(row=3,column=4,value='js_flops_wt_cbrt')
worksheet.cell(row=3,column=5,value='js_param')
worksheet.cell(row=3,column=6,value='ptq_acc')
worksheet.cell(row=3,column=7,value='acc_loss')
for i in range(len(title_list)):
worksheet.cell(row=i+4, column=1, value=title_list[i])
worksheet.cell(row=i+4, column=2, value=js_flops_list[i])
worksheet.cell(row=i+4, column=3, value=js_flops_wt_log_list[i])
worksheet.cell(row=i+4, column=4, value=js_flops_wt_cbrt_list[i])
worksheet.cell(row=i+4, column=5, value=js_param_list[i])
worksheet.cell(row=i+4, column=6, value=ptq_acc_list[i])
worksheet.cell(row=i+4, column=7, value=acc_loss_list[i])
workbook.save(excel_path)
print(model_name,file=ft)
print('Full_acc: %f'%full_acc,file=ft)
print('title_list:',file=ft)
print(title_list,file=ft)
print('js_flops_list:',file=ft)
print(js_flops_list, file=ft)
print('js_flops_wt_log_list:',file=ft)
print(js_flops_wt_log_list, file=ft)
print('js_flops_wt_cbrt_list:',file=ft)
print(js_flops_wt_cbrt_list, file=ft)
print('js_param_list:',file=ft)
print(js_param_list, file=ft)
print('ptq_acc_list:',file=ft)
print(ptq_acc_list, file=ft)
print('acc_loss_list:',file=ft)
print(acc_loss_list, file=ft)
print("\n",file=ft)
ft.close()
ykl/ALL/ptq_one.slurm 0 → 100644
#!/bin/bash
#- Job parameters
# (TODO)
# Please modify job name
#- Resources
# (TODO)
# Please modify your requirements
#SBATCH -p nv-gpu # Submit to 'nv-gpu' Partitiion
#SBATCH -t 1-06:00:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --gres=gpu:1 # Request M GPU per node
#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
#SBATCH --qos=gpu-normal # Request QOS Type
###
### The system will alloc 8 or 16 cores per gpu by default.
### If you need more or less, use following:
### #SBATCH --cpus-per-task=K # Request K cores
###
###
### Without specifying the constraint, any available nodes that meet the requirement will be allocated
### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
###
### #SBATCH --nodelist=gpu-v00 # Request a specific list of hosts
### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
###
# set constraint for RTX8000 to meet my cuda
#SBATCH --constraint="Ampere|RTX8000"
#- Log information
echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
echo "Job run at:"
echo "$(hostnamectl)"
#- Load environments
source /tools/module_env.sh
module list # list modules loaded
##- Tools
module load cluster-tools/v1.0
module load slurm-tools/v1.0
module load cmake/3.15.7
module load git/2.17.1
module load vim/8.1.2424
##- language
module load python3/3.6.8
##- CUDA
# module load cuda-cudnn/10.2-7.6.5
# module load cuda-cudnn/11.2-8.2.1
module load cuda-cudnn/11.1-8.2.1
##- virtualenv
# source xxxxx/activate
echo $(module list) # list modules loaded
echo $(which gcc)
echo $(which python)
echo $(which python3)
cluster-quota # nas quota
nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
#- Warning! Please not change your CUDA_VISIBLE_DEVICES
#- in `.bashrc`, `env.sh`, or your job script
echo "Use GPU ${CUDA_VISIBLE_DEVICES}" # which gpus
#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
#- Job step
# [EDIT HERE(TODO)]
python ptq_one.py $Model $Dataset
#- End
echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"
ykl/ALL/ptq_result/cifar10/AlexNet.txt 0 → 100644
AlexNet
Full_acc: 88.630000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[16.134064351484934, 9.803080945516005, 2.5798806807263825, 0.5670208264725348, 0.13436593638690267, 0.03258542668624173, 0.007852610824269553, 0.0018014206200349513, 0.0003162150510672301, 6.92716001480207e-05, 9.684179880150801e-06, 1.1595438673905888e-07, 7.945963039257542e-07, 1.5685770022075827e-06, 0.0, 16.01436476484219, 6.952820969558956, 0.7167166887613639, 0.7119351064288386, 0.7146290586824979, 0.7120276276007171, 0.7120483250193009, 4.308314078011601, 1.0381514277725061, 1.2567698279608643, 0.39416582491303853, 0.4671888875734605, 0.19757403359526637, 0.23172120832374465, 0.2652704329374882, 0.0509185267481933, 0.19694534563825358, 0.1841072727534976, 0.19769045007362226, 0.012649947461768432, 0.050778713986116, 0.1970719456861238, 0.16609038257453898, 0.1711650729485768, 0.003183632319308628, 0.012617062910949925, 0.05085217397639437, 0.19707866413312908]
js_flops_wt_log_list:
[29.77693314521759, 18.09250785000869, 4.761412430188898, 1.0464902627132961, 0.24798497251680535, 0.060139469559962806, 0.014492731802459045, 0.003324691175181796, 0.0005836046162958227, 0.00012784725296326294, 1.7873064751410352e-05, 2.1400472606308148e-07, 1.4665022094855177e-06, 2.8949563798883818e-06, 0.0, 29.556016300501824, 12.832085001704433, 1.3227680552386436, 1.313943195343322, 1.318915137589619, 1.3141139518675014, 1.3141521509000589, 7.951398821447238, 1.9160060965387782, 2.319486914819017, 0.727470100581621, 0.8622410304309428, 0.36464146054157476, 0.42766328299354595, 0.48958153227246276, 0.09397492992485672, 0.36348115778974827, 0.3397872868794326, 0.3648563181011757, 0.023346667749278273, 0.0937168923232625, 0.36371481008463735, 0.3065354215926674, 0.31590123994497216, 0.00587569286112845, 0.023285976218000038, 0.09385246964412312, 0.36372720961044547]
js_flops_wt_cbrt_list:
[66.51840319691642, 40.41667844505815, 10.636473214803948, 2.3377444848778883, 0.5539712336459476, 0.13434497987851984, 0.0323751735196939, 0.007426995487327126, 0.0013037087125469166, 0.0002855967429135439, 3.992646662773776e-05, 4.780630894688359e-07, 3.2760051138911077e-06, 6.46701508096138e-06, 0.0, 66.02489919238619, 28.665470679527296, 2.9549187757267994, 2.9352050064876867, 2.9463117802241916, 2.935586457836906, 2.9356717902865914, 17.76255298707487, 4.280147503296049, 5.181479404123357, 1.6250884276160402, 1.9261519054168204, 0.8145690349079002, 0.955352875057603, 1.0936714537606507, 0.2099296878109677, 0.8119770458034453, 0.7590475365480169, 0.8150490031339678, 0.05215389546926388, 0.2093532601063716, 0.8124989994070458, 0.6847665160208967, 0.7056887271293883, 0.013125653501376717, 0.05201831724404789, 0.2096561249732598, 0.8125266985879229]
js_param_list:
[4.110435615455709, 2.6180759555986155, 0.6928868501213382, 0.15211415355264937, 0.03600861379914816, 0.008708675296167655, 0.002021576962555789, 0.00045613222041422045, 7.770756870467822e-05, 3.4782638420791655e-05, 2.5772797483716303e-06, 4.235692471765695e-08, 1.1409637429282833e-05, 2.70695872828825e-05, 0.0, 4.111555910135532, 1.8469226254888855, 0.1678692583099073, 0.16668128604242521, 0.1669457541337397, 0.16677503235751284, 0.16686573854576636, 1.1061334469811006, 0.2569252001359494, 0.3323236930888579, 0.09191654990892967, 0.12427202662450998, 0.04625754039946728, 0.051159436500936975, 0.07107262999829092, 0.011936849865612175, 0.04620882915913972, 0.039402812238512885, 0.05317898697834904, 0.0030091857478168147, 0.011968180918548683, 0.04620752119636976, 0.035451652338101006, 0.04611954068644799, 0.0007129095603291375, 0.002958953813729707, 0.01198034544585294, 0.04631816751034479]
ptq_acc_list:
[10.0, 10.07, 56.8, 83.06, 87.31, 88.19, 88.66, 88.49, 88.67, 88.67, 88.64, 88.62, 88.63, 88.64, 88.62, 10.0, 14.35, 69.55, 70.05, 70.34, 69.65, 70.2, 23.93, 67.71, 67.14, 79.55, 84.03, 84.79, 83.3, 87.06, 88.01, 85.19, 84.89, 87.49, 88.63, 88.02, 85.03, 85.91, 87.84, 88.71, 88.54, 88.0, 85.21]
acc_loss_list:
[0.8871713866636579, 0.8863815863703036, 0.35913347624957687, 0.06284553762834247, 0.014893376960397081, 0.004964458986799027, -0.00033848584000903913, 0.001579600586708796, -0.00045131445334543896, -0.00045131445334543896, -0.00011282861333639983, 0.00011282861333623949, 0.0, -0.00011282861333639983, 0.00011282861333623949, 0.8871713866636579, 0.8380909398623492, 0.2152769942457407, 0.2096355635789236, 0.20636353379216962, 0.21414870811237718, 0.2079431343788784, 0.7300011282861333, 0.2360374590996277, 0.24246869005979912, 0.1024483809093986, 0.0519011621347173, 0.043326187521155246, 0.06013765090827032, 0.017714092293805633, 0.006995374026853102, 0.038813042987701656, 0.04219790138779189, 0.012862461920343005, 0.0, 0.006882545413516862, 0.04061830080108309, 0.03068938282748504, 0.008913460453570936, -0.0009026289066907176, 0.001015457520026957, 0.007108202640189501, 0.03858738576102902]
ykl/ALL/ptq_result/cifar10/AlexNet_BN.txt 0 → 100644
AlexNet_BN
Full_acc: 90.140000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[129.5632809293304, 63.91782325281653, 13.416712728751037, 2.920907552518548, 0.6862891592953664, 0.16581168442674293, 0.040531296185229805, 0.009538300129160873, 0.0018645872176961651, 7.766948420581876e-05, 5.4083427000152004e-05, 1.4807942822291448e-05, 3.7134854647260264e-05, 1.7350501366308888e-08, 1.9435654536586355e-05, 129.4265988239714, 39.99630282031915, 5.162327959855199, 5.145162687613236, 5.146356356056539, 5.1463004531161625, 5.1466908388526775, 26.85518625091552, 7.402168766954119, 6.97771328797255, 3.3635156759931935, 2.520031279406507, 1.4570953805417208, 2.253481546269108, 1.391032559450782, 0.3753237383602544, 1.4550462880254835, 1.8839926307821282, 1.0208799909895032, 0.09454382183390929, 0.3746936069209011, 1.4554437659578396, 1.74061611055598, 0.8778935424903659, 0.023484665753534216, 0.09407697570154169, 0.3749941970027924, 1.4554437457886777]
js_flops_wt_log_list:
[239.26557052810506, 118.03756695418507, 24.776753125040646, 5.394063865937431, 1.2673758033005071, 0.3062057937250771, 0.07484947615128858, 0.017614456857701688, 0.0034433484644837377, 0.0001434328717043312, 9.987630696358105e-05, 2.734594911698515e-05, 6.857723978525448e-05, 3.204131277458857e-08, 3.58919821876073e-05, 239.01315856628122, 73.86149952885403, 9.53331326366811, 9.501614015032349, 9.503818372309038, 9.503715135891841, 9.504436064424242, 49.59369208524899, 13.669645600720976, 12.885800196268002, 6.211431907437539, 4.653762373806726, 2.6908299561304028, 4.1615228016395145, 2.568831204125649, 0.6931134172226585, 2.6870458802221227, 3.4791845995371484, 1.8852674286479463, 0.1745948490092372, 0.6919497483400621, 2.687779905970586, 3.2144100070277535, 1.6212131847870317, 0.04336932431686495, 0.17373272043848342, 0.6925048505026727, 2.68777986872403]
js_flops_wt_cbrt_list:
[534.6275121931167, 263.749316822964, 55.36247381612953, 12.05277858779172, 2.8318908200488986, 0.6842022500647543, 0.16724758658505331, 0.03935866421433108, 0.007693998008646585, 0.0003204939147606846, 0.00022316884707281035, 6.110321979340532e-05, 0.00015323257340593775, 7.159478607085111e-08, 8.01989233230953e-05, 534.0635096962268, 165.03999991643525, 21.301734060036175, 21.23090359997825, 21.23582913123718, 21.235598454383958, 21.237209334824865, 110.81474096538965, 30.54417149924457, 28.792706306815326, 13.879148514768039, 10.39861019182436, 6.012531272282353, 9.29872433156496, 5.739930876283026, 1.5487288919150721, 6.004075935042113, 7.774072144210402, 4.212540203641092, 0.39012386764010776, 1.5461287292659038, 6.005716080588003, 7.182445938347171, 3.6225235824957904, 0.09690668788595468, 0.38819748244412927, 1.5473690785880172, 6.0057159973623495]
js_param_list:
[36.759534463320726, 18.13745281968715, 3.806673814006264, 0.8287388043320857, 0.19462808316219096, 0.04699609089246882, 0.011453273265140763, 0.0026919390034946533, 0.0005254148223667485, 2.1769278082333976e-05, 1.4501728680342828e-05, 3.352003308381146e-06, 9.417038242733876e-06, 2.999740900122561e-10, 4.641088806572213e-06, 36.73241494803418, 11.344235549515417, 1.4616139412820612, 1.4571897381462697, 1.4575153880062297, 1.457484752859762, 1.457611889119609, 7.617841173316666, 2.099910215271149, 1.9783228519470004, 0.9540300630137106, 0.7146220926588001, 0.41208680428748734, 0.6392469460225422, 0.3943444750039264, 0.10617885590492321, 0.4115579668357343, 0.5343759382953008, 0.28945540778254714, 0.02660113971284551, 0.10616542858231626, 0.4117781257561503, 0.4936858946415845, 0.24889073908249335, 0.006604708170802468, 0.026651549591417394, 0.10621597545076181, 0.4117778645202598]
ptq_acc_list:
[10.0, 12.43, 51.07, 76.42, 88.85, 89.84, 90.11, 90.06, 90.12, 90.11, 90.16, 90.14, 90.13, 90.13, 90.13, 10.0, 18.85, 71.18, 69.78, 71.07, 67.47, 65.35, 23.77, 67.99, 73.63, 82.79, 85.66, 87.83, 85.33, 87.98, 89.38, 88.05, 86.81, 88.68, 89.94, 89.44, 87.83, 87.04, 88.97, 90.1, 89.78, 89.43, 87.79]
acc_loss_list:
[0.889061459951187, 0.8621033947193256, 0.43343687597071223, 0.15220767694697138, 0.014311071666296941, 0.003328156201464357, 0.0003328156201464515, 0.0008875083203904847, 0.00022187708009758177, 0.0003328156201464515, -0.00022187708009758177, 0.0, 0.0001109385400488697, 0.0001109385400488697, 0.0001109385400488697, 0.889061459951187, 0.7908808520079875, 0.2103394719325493, 0.22587086753938318, 0.2115597958730864, 0.251497670290659, 0.2750166407810074, 0.7362990903039717, 0.24572886620812076, 0.18315952962059026, 0.08153982693587745, 0.04970046594186825, 0.02562680275127582, 0.053361437763479055, 0.02396272465054356, 0.00843132904370984, 0.023186154870201946, 0.0369425338362547, 0.016197026847126622, 0.0022187708009762906, 0.007765697803416938, 0.02562680275127582, 0.034390947415131955, 0.012979809185711134, 0.0004437541601953212, 0.0039937874417572605, 0.007876636343465651, 0.02607055691147098]
ykl/ALL/ptq_result/cifar10/Inception_BN.txt 0 → 100644
Inception_BN
Full_acc: 94.830000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[6.113248686239302, 3.764291730695215, 1.2273413109260587, 0.3167888219320939, 0.07969457696563005, 0.01970450334396353, 0.004871218829937093, 0.001183148165746224, 0.00029042342437181855, 7.24095354551304e-05, 1.7601770787312008e-05, 2.7202899449706604e-05, 1.495770654424544e-05, 8.577113022936626e-06, 2.177044284867923e-05, 6.118561499111071, 2.772135571636355, 0.2556906139158136, 0.25094002577961333, 0.2509444914539089, 0.2518256066410849, 0.25083778784403044, 1.8783781939787385, 0.5031133984499222, 0.518643254519312, 0.17766229471348774, 0.18815315001956281, 0.07009551975633653, 0.09368101696460907, 0.10506705155177981, 0.01826264326837488, 0.06941806017783317, 0.06982901798266375, 0.07785436955092967, 0.004692887032668605, 0.017943409832527922, 0.0694242399513946, 0.06219218659454411, 0.067285345156737, 0.001260196623730233, 0.00451446593527846, 0.017954262015044287, 0.07213500494430332]
js_flops_wt_log_list:
[19.40744214974005, 11.95031933882503, 3.8963825475316494, 1.0056945252645246, 0.25300261309342176, 0.06255495700643283, 0.01546442857028373, 0.0037560846547887443, 0.0009219935416847589, 0.00022987513555570486, 5.587951117655057e-05, 8.635976131050368e-05, 4.74855251846151e-05, 2.722935600168197e-05, 6.911359767041957e-05, 19.424308486078264, 8.800567995683062, 0.8117289272024263, 0.7966474592035733, 0.7966616361690461, 0.7994588709782346, 0.7963228892535891, 5.963198620895848, 1.5972103665853983, 1.6465122678705368, 0.564016104007569, 0.5973209273353428, 0.22252893910388685, 0.2974047042061225, 0.33355141095843566, 0.05797755185882371, 0.22037824014603383, 0.22168288849793003, 0.24716059344440758, 0.014898287028105372, 0.0569640965331131, 0.22039785878156587, 0.19743860023498577, 0.21360761040100576, 0.0040006867588086945, 0.014331861989044426, 0.05699854844488611, 0.22900359649674132]
js_flops_wt_cbrt_list:
[69.90201047760932, 43.042835897079485, 14.03404794721903, 3.6223253275680514, 0.9112685316728247, 0.22531136387535944, 0.05570000619399983, 0.013528720934373475, 0.0033208498942762215, 0.0008279676430400681, 0.0002012676449379108, 0.00031105185801373245, 0.00017103406277752095, 9.807509479313954e-05, 0.000248934372248929, 69.96275989581373, 31.698015199359514, 2.9236971846412323, 2.8693765314639834, 2.8694275942666634, 2.8795027157294295, 2.868207490646061, 21.478336468132163, 5.7528557817442305, 5.930432094666331, 2.03148149599949, 2.151439298333795, 0.801508004650656, 1.0711966362751792, 1.2013903761114775, 0.20882439871427963, 0.7937615855230823, 0.7984606871388542, 0.8902266594072794, 0.05366086926354702, 0.2051741204215598, 0.7938322482129894, 0.7111372532157904, 0.7693750317605966, 0.014409732388954927, 0.05162071122985431, 0.2052982098239215, 0.8248285208434767]
js_param_list:
[6.991313494407524, 4.563520311259842, 1.730100873344829, 0.5178502000194385, 0.14109104440127346, 0.03575389135026652, 0.008973823797161662, 0.0022236950592541354, 0.0005452716164764403, 0.0001323646059161847, 3.279936359774982e-05, 3.4568934853126346e-05, 2.6141171889124126e-05, 6.634560903245657e-06, 1.6425160009501393e-05, 6.994022422861251, 3.5140164619769116, 0.29871736184397835, 0.2866335115447693, 0.28699350664588935, 0.28734478542409403, 0.28626051365850613, 2.463102016163771, 0.7263018370697303, 0.6920269760465765, 0.24327092880965767, 0.24335987559893493, 0.0812922909172416, 0.11047689546422382, 0.132003950065328, 0.02133323668622415, 0.07946108170203432, 0.07408133408938296, 0.0965569393768569, 0.005603507470415825, 0.02061485999713829, 0.07946771607462498, 0.06332189527093476, 0.08311185438573757, 0.001634361398031487, 0.005173953535644472, 0.020621452860440773, 0.08165409435089002]
ptq_acc_list:
[10.0, 9.55, 34.21, 82.41, 93.32, 94.56, 94.73, 94.83, 94.84, 94.82, 94.84, 94.82, 94.83, 94.79, 94.82, 10.0, 10.89, 31.27, 30.05, 26.78, 33.34, 33.21, 9.83, 18.77, 54.81, 57.25, 85.38, 86.4, 76.32, 92.07, 93.64, 87.98, 78.72, 93.16, 94.52, 93.38, 87.85, 82.33, 93.69, 94.86, 94.55, 93.35, 84.06]
acc_loss_list:
[0.8945481387746493, 0.8992934725297902, 0.6392491827480755, 0.1309712116418855, 0.015923231045027998, 0.002847200253084425, 0.0010545186122534465, 0.0, -0.00010545186122540458, 0.00010545186122540458, -0.00010545186122540458, 0.00010545186122540458, 0.0, 0.0004218074449013186, 0.00010545186122540458, 0.8945481387746493, 0.8851629231255932, 0.6702520299483287, 0.6831171570178214, 0.717599915638511, 0.648423494674681, 0.6497943688706106, 0.8963408204154804, 0.802066856480017, 0.4220183486238532, 0.3962880944848676, 0.09965200885795637, 0.0888959190129705, 0.19519139512812406, 0.029104713698196828, 0.012548771485816701, 0.07223452493936512, 0.16988294843403987, 0.017610460824633575, 0.0032690076979858935, 0.015290519877675872, 0.07360539913529478, 0.13181482653168827, 0.012021512179689978, -0.00031635558367606385, 0.0029526521143098295, 0.015606875461351935, 0.11357165453970258]
ykl/ALL/ptq_result/cifar10/MobileNetV2.txt 0 → 100644
MobileNetV2
Full_acc: 90.280000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[4.181518320770741, 2.3934210931112023, 0.8299774534186838, 0.2227311420117117, 0.05614742050371703, 0.014290348874781582, 0.0036461960969134915, 0.000919992986083523, 0.00023874379576946226, 4.354552985272698e-05, 2.111792137977542e-05, 7.578662429780003e-06, 1.0082542326278948e-05, 6.938405912729428e-06, 2.554517014849906e-06, 4.124005216043576, 1.7683233451569575, 0.17269766998466235, 0.1685604939603026, 0.16863805065435566, 0.16864278104488645, 0.1686450926024181, 1.2850773503660626, 0.36631267601475503, 0.3312366451478319, 0.1322264829154868, 0.11832443639651827, 0.04694827624094409, 0.07169034199458685, 0.06489754441128845, 0.012182574844289919, 0.04637128062280152, 0.05391533589489863, 0.04761867810804146, 0.003093513338869911, 0.012058348678954865, 0.0464763144196043, 0.04803550849410842, 0.04095358476999055, 0.0008107319199455053, 0.0030124874335516116, 0.012077514806342487, 0.046491362011174764]
js_flops_wt_log_list:
[7.671195559309975, 4.3908455858802204, 1.522633375385822, 0.408610943788798, 0.10300513110162186, 0.026216329194274284, 0.006689121309854875, 0.0016877711797611856, 0.00043798692375020655, 7.988635935773382e-05, 3.8741837840502496e-05, 1.3903419073414885e-05, 1.849690662258048e-05, 1.2728837839124767e-05, 4.6863837671495325e-06, 7.5656850151151485, 3.2440738392584327, 0.31682214388703994, 0.3092322963934859, 0.30937457786193223, 0.30938325598983935, 0.30938749665280235, 2.3575359252954122, 0.6720181422152397, 0.607669484243806, 0.24257581355699895, 0.21707184362533663, 0.0861288605212676, 0.1315193647299196, 0.11905765234247143, 0.022349516820783066, 0.085070334434786, 0.09891026502053393, 0.0873587448434969, 0.005675198329259245, 0.02212161797286107, 0.08526302396375653, 0.08812344014345505, 0.0751313120080912, 0.0014873265228048712, 0.0055265524266503, 0.022156779151176316, 0.08529062949093077]
js_flops_wt_cbrt_list:
[17.094245535934448, 9.784419126731523, 3.3929872571773076, 0.9105354892587192, 0.2295333222701242, 0.05841962505491415, 0.014905822854631552, 0.003760975030847295, 0.0009759959784923138, 0.00017801619464330698, 8.633106579523528e-05, 3.0981932032942016e-05, 4.1217911995204584e-05, 2.8364532976221542e-05, 1.0442986907568484e-05, 16.859129231682825, 7.228989838185455, 0.7059962788008196, 0.6890833066790686, 0.6894003621286436, 0.689419700189848, 0.6894291499468467, 5.253457255157136, 1.4975036210213808, 1.3541111405707198, 0.5405481435015572, 0.4837159169232869, 0.1919267835238493, 0.29307352368234907, 0.2653042444738815, 0.04980294468967078, 0.1895679980270374, 0.2204084543553786, 0.19466741820359862, 0.01264642948487274, 0.04929510222448852, 0.18999738117793727, 0.19637144062312817, 0.16742020001621957, 0.0033143106020909513, 0.012315191734842485, 0.049373454263724983, 0.19005889644710453]
js_param_list:
[4.030988811477383, 2.6377155916634014, 0.9571509985992305, 0.2599560316091801, 0.06566988129420442, 0.016743355093560855, 0.004276155012300976, 0.0010829703857362888, 0.000283256236338334, 5.140716293103421e-05, 2.5045455943604417e-05, 8.728291604610918e-06, 1.2029853163492895e-05, 7.871773015655838e-06, 2.0612893902350045e-06, 4.00818872899252, 2.0021755056510933, 0.16381022541877208, 0.15898214355022486, 0.15896989829952274, 0.15896360522517555, 0.15910114970898098, 1.3734457949762167, 0.37735137624057996, 0.37440928537236706, 0.12277905416255215, 0.13459771225157058, 0.04491960983112498, 0.05893901887388952, 0.07449106085824778, 0.011663613429120305, 0.044226769459113614, 0.041073894430956756, 0.054942229701478704, 0.0029454662020147538, 0.011508924331293099, 0.04435076087239326, 0.035529757714568046, 0.047355473808008926, 0.0007854130619828617, 0.00284540157682428, 0.011529839479715592, 0.0443690622016044]
ptq_acc_list:
[10.0, 10.0, 10.11, 23.2, 60.66, 81.25, 88.13, 89.74, 90.31, 90.33, 90.22, 90.24, 90.27, 90.29, 90.29, 10.0, 9.93, 13.08, 10.17, 14.67, 13.4, 11.16, 10.32, 11.16, 13.12, 17.31, 26.89, 46.32, 29.37, 45.5, 69.14, 46.94, 33.53, 58.66, 85.74, 69.16, 46.22, 41.52, 68.06, 89.36, 85.76, 69.04, 47.97]
acc_loss_list:
[0.8892334957908729, 0.8892334957908729, 0.8880150642445724, 0.743021710234825, 0.3280903854674347, 0.10002215330084184, 0.023814798404962402, 0.005981391227292936, -0.0003322995126273941, -0.0005538325210456043, 0.0006645990252547882, 0.0004430660168365779, 0.00011076650420918382, -0.00011076650420918382, -0.00011076650420918382, 0.8892334957908729, 0.8900088613203366, 0.8551174124944617, 0.8873504652193177, 0.8375055383252105, 0.8515728843597695, 0.8763845813026141, 0.8856889676561809, 0.8763845813026141, 0.8546743464776251, 0.8082631812140009, 0.7021488701816571, 0.486929552503323, 0.6746787771377935, 0.49601240584847145, 0.23416038989809482, 0.48006202924235714, 0.6285999113867966, 0.3502436863092601, 0.050287992910943796, 0.2339388568896766, 0.4880372175454143, 0.540097474523704, 0.24612317235268052, 0.010190518387239717, 0.05006645990252543, 0.23526805494018602, 0.46865307930881706]
ykl/ALL/ptq_result/cifar10/ResNet_152.txt 0 → 100644
ResNet_152
Full_acc: 94.430000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[3.884172415043158, 2.775511735393752, 1.3896416659957913, 0.5171567223295069, 0.16011101697021182, 0.04528250830769957, 0.012116614037137473, 0.002973614759913081, 0.0006224399470396359, 9.679236642071944e-05, 1.1784369331787502e-05, 3.994741357296106e-06, 3.6639747865761664e-06, 1.3498340435669076e-06, 4.614709735462933e-06, 3.8780042573600237, 2.3153381148091436, 0.18806555157212965, 0.16490888876731125, 0.16498365739813015, 0.16496728841354316, 0.1649734334538275, 1.7868294140660492, 0.6446346772041729, 0.5124607663605131, 0.2157441225234644, 0.17191056943469762, 0.049341958451529666, 0.08319942570848018, 0.08552067122239285, 0.013037865238501452, 0.045747333600017814, 0.04459985617094451, 0.059072960425739444, 0.003530917722374488, 0.01177231940997834, 0.045803614820307634, 0.0333373893878054, 0.04967783323369072, 0.001134618076642093, 0.002813572616105592, 0.011803836861528427, 0.046179169966594866]
js_flops_wt_log_list:
[13.917656310639067, 9.945134842559874, 4.979324560199374, 1.8530612833361322, 0.5737052498256691, 0.1622549980194992, 0.04341590737950558, 0.010654971975092903, 0.0022303091447123053, 0.00034682365904244926, 4.2225417585030516e-05, 1.4313843805032342e-05, 1.3128650420593548e-05, 4.836687017807287e-06, 1.653529689433717e-05, 13.895554743167136, 8.29625379142151, 0.6738711444719903, 0.5908968478185702, 0.5911647566535448, 0.5911061037727526, 0.5911281225066176, 6.4025164213611605, 2.309836671642123, 1.8362348672447073, 0.7730482140012364, 0.6159850711806752, 0.17680070451110294, 0.2981178198394443, 0.30643524085559926, 0.0467169085262852, 0.16392054680871423, 0.15980893826625964, 0.21166855448364263, 0.01265188412615988, 0.042182240647348636, 0.16412221207921132, 0.11945358707467839, 0.1780042014937353, 0.004065531275950717, 0.010081513509628322, 0.0422951730848384, 0.1654678906158989]
js_flops_wt_cbrt_list:
[60.769968400189846, 43.4243752416846, 21.741691950410775, 8.09119532186387, 2.5050230530752384, 0.7084692194101948, 0.18957094934879698, 0.04652380370510617, 0.009738408049580422, 0.0015143686788943328, 0.0001843728020762785, 6.249987902432453e-05, 5.732485796381723e-05, 2.11188802678704e-05, 7.219961804837e-05, 60.67346425273241, 36.224711222337284, 2.9423867956890386, 2.5800883401795582, 2.581258135658923, 2.5810020340831423, 2.581098176546451, 27.95590808708141, 10.085656551095425, 8.017724562754617, 3.3754329384299333, 2.6896334034361185, 0.7719815022345297, 1.301699803962514, 1.3380169396239565, 0.20398442033136766, 0.7157416613379496, 0.6977887592384837, 0.9242282666120947, 0.05524310856576221, 0.1841842745627388, 0.7166222113272576, 0.5215814034916492, 0.777236444012937, 0.017751710608103387, 0.04401985820973823, 0.18467738205903209, 0.7224979737635525]
js_param_list:
[3.568588468131594, 2.680659686169375, 1.370448805783652, 0.5080205845238838, 0.15602262084226592, 0.04386650663535348, 0.011678703718883257, 0.0028642960407596766, 0.0006012492270979219, 9.517512872689344e-05, 1.2580916022049693e-05, 4.4070128155525026e-06, 4.8943921026917e-06, 1.094550449411584e-06, 6.327577696693385e-06, 3.5680112778935893, 2.258210450697765, 0.17154228069419739, 0.14948686146848364, 0.14950086848763475, 0.14951546514002018, 0.14953705413251314, 1.7274918605421477, 0.6187124547057584, 0.49866474560833446, 0.20105756264027386, 0.16727924473969336, 0.045065829576603544, 0.07348986380475629, 0.08357807672727313, 0.011862448424735762, 0.041637405836526524, 0.0368579783966308, 0.057960172682577256, 0.0032272262474772936, 0.010654655754564828, 0.04168746344712044, 0.026365193691402146, 0.048826423125494955, 0.0010453741381514332, 0.0025317087046215253, 0.010685502933311481, 0.04209708538181569]
ptq_acc_list:
[10.0, 9.98, 14.74, 29.62, 77.93, 90.36, 93.98, 94.4, 94.32, 94.45, 94.44, 94.43, 94.43, 94.41, 94.43, 10.0, 10.2, 31.26, 18.95, 26.93, 24.13, 27.14, 9.86, 16.49, 19.68, 60.62, 69.62, 83.73, 88.39, 83.55, 91.03, 84.9, 90.16, 85.86, 93.7, 90.93, 84.02, 90.77, 88.29, 94.45, 93.88, 90.7, 83.43]
acc_loss_list:
[0.8941014508101239, 0.8943132479085036, 0.8439055384941226, 0.686328497299587, 0.17473260616329556, 0.04310070952027965, 0.004765434713544454, 0.0003176956475696403, 0.0011648840410887814, -0.00021179709837971004, -0.00010589854918977977, 0.0, 0.0, 0.00021179709837986052, 0.0, 0.8941014508101239, 0.8919834798263263, 0.6689611352324473, 0.7993222492851848, 0.7148152070316636, 0.7444668008048291, 0.7125913374986763, 0.8955840304987822, 0.8253732923858944, 0.7915916551943237, 0.3580429948109712, 0.2627343005400826, 0.11331144763316744, 0.06396272371068523, 0.11521762151858529, 0.036005506724557934, 0.10092131737795193, 0.0452186805040772, 0.09075505665572389, 0.007730594090860997, 0.037064492216456635, 0.11024038970666113, 0.03875886900349476, 0.06502170920258393, -0.00021179709837971004, 0.005824420205443305, 0.039500158847823826, 0.1164884041088637]
ykl/ALL/ptq_result/cifar10/ResNet_18.txt 0 → 100644
ResNet_18
Full_acc: 94.300000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[12.80299527868102, 8.405820796720942, 2.773593692871517, 0.6783118175964616, 0.16203377803458088, 0.03944124196845232, 0.009749781421022428, 0.0024065085260245142, 0.000584502400796242, 0.00015822609902129157, 4.550811461774384e-05, 3.714601548751021e-05, 1.2238319532654754e-05, 2.101353999820815e-05, 2.4071849591002673e-05, 12.765614169275102, 6.274365648075481, 0.5195606412340176, 0.5125058172339328, 0.5125955837634969, 0.513058063588398, 0.512922534838037, 4.092833325985446, 1.020161932363568, 1.1507773637393666, 0.3315181105925415, 0.4243942154730236, 0.14301510987061236, 0.16600666733254404, 0.24131034691764322, 0.03714508796661997, 0.14192365314916092, 0.12076065057100295, 0.18032132485975225, 0.009558370804247743, 0.0367419926655379, 0.14192339487510483, 0.10650084355725027, 0.15635458408031847, 0.002502898104198746, 0.009325720410743874, 0.03673861294533459, 0.1434812548685684]
js_flops_wt_log_list:
[35.15736757144959, 23.0826088003134, 7.616362486402792, 1.8626623989315165, 0.44494908959611157, 0.10830670567131667, 0.026773160631477548, 0.006608336797105143, 0.0016050592305853972, 0.0004344931011535307, 0.00012496650028168306, 0.00010200395234728985, 3.3606752865312386e-05, 5.7703743039298546e-05, 6.61019430042871e-05, 35.05471804491256, 17.229575936373514, 1.4267274213510628, 1.4073546858993933, 1.407601187191224, 1.408871169163763, 1.408499003589284, 11.239029814371138, 2.8013919600696155, 3.1600654291623482, 0.9103576012483854, 1.1653978700927246, 0.3927233179851927, 0.4558587499005055, 0.6626446687443921, 0.10200140535011001, 0.38972614862683996, 0.33161197734422065, 0.49516718244999586, 0.026247541956742025, 0.10089449486877407, 0.389725439399372, 0.2924541657721706, 0.429353871054398, 0.006873025157614916, 0.025608677751763537, 0.10088521406667347, 0.39400336462100005]
js_flops_wt_cbrt_list:
[105.3546771711894, 69.17073052991391, 22.82364882248317, 5.581765907800925, 1.3333611396457614, 0.32455837281579836, 0.08023005958706406, 0.01980293855854231, 0.0048098167968720264, 0.0013020280973002254, 0.0003744821129634769, 0.00030567116402825174, 0.00010070801210305022, 0.00017291849872205618, 0.00019808504864455312, 105.04707144080852, 51.63118105711691, 4.275416997084549, 4.217363495630392, 4.218102176192021, 4.221907880372181, 4.22079262668108, 33.67955110416199, 8.394819236204002, 9.469641674753754, 2.728032211027894, 3.4923011835310827, 1.176858258794328, 1.3660536823135505, 1.985720774255815, 0.3056635315436782, 1.1678767612594254, 0.9937283486369863, 1.4838476898694546, 0.07865495912826637, 0.3023464971786463, 1.1678746359458398, 0.876385700939615, 1.2866275720219258, 0.020596119581433345, 0.07674050031860292, 0.30231868577020654, 1.1806941233483357]
js_param_list:
[19.23987607919351, 13.276544208589291, 4.691457176297082, 1.1543934993178218, 0.2736012569973487, 0.0664575269557867, 0.016349216801729644, 0.004069234416321323, 0.0009458122127482155, 0.00026014598915303995, 5.009872385009695e-05, 5.014557788914478e-05, 7.804331203843108e-06, 2.2664059048005836e-05, 1.9617915834909515e-05, 19.25939607497878, 10.160138472445075, 0.7742284675560568, 0.7633418730269013, 0.7633980535964743, 0.7648033812948744, 0.7638633216930796, 6.678662066483134, 1.6460420449142454, 1.8710294843466486, 0.5030177091990324, 0.6915414154170487, 0.213498908615622, 0.23540381405596134, 0.3947354518206468, 0.055709824268573675, 0.2117091373957813, 0.164203782259264, 0.2957156620243196, 0.014283730206225535, 0.055013288179935435, 0.21169700527137067, 0.14266891367121662, 0.2567414534824632, 0.003789666974358319, 0.01390540812098994, 0.05501216634759848, 0.21523209186116166]
ptq_acc_list:
[10.0, 10.0, 47.44, 87.34, 93.06, 94.26, 94.32, 94.36, 94.34, 94.33, 94.32, 94.29, 94.29, 94.3, 94.29, 10.0, 10.51, 59.23, 46.37, 51.24, 52.19, 55.34, 12.97, 47.09, 52.32, 82.3, 85.6, 89.33, 87.28, 91.6, 93.24, 89.5, 90.04, 92.52, 94.07, 93.11, 88.85, 90.35, 92.14, 94.26, 94.06, 93.4, 88.87]
acc_loss_list:
[0.8939554612937434, 0.8939554612937434, 0.49692470837751856, 0.07380699893955454, 0.013149522799575769, 0.00042417815482494214, -0.00021208907741247107, -0.0006362672322375639, -0.00042417815482509284, -0.00031813361611878193, -0.00021208907741247107, 0.00010604453870616018, 0.00010604453870616018, 0.0, 0.00010604453870616018, 0.8939554612937434, 0.8885471898197242, 0.371898197242842, 0.508271474019088, 0.456627783669141, 0.44655355249204665, 0.4131495227995758, 0.8624602332979852, 0.5006362672322375, 0.4451749734888653, 0.12725344644750797, 0.0922587486744433, 0.052704135737009536, 0.07444326617179212, 0.02863202545068932, 0.011240721102863227, 0.050901378579003155, 0.045174973488865226, 0.018875927889713692, 0.0024390243902439445, 0.012619300106044515, 0.05779427359490989, 0.041887592788971396, 0.022905620360551396, 0.00042417815482494214, 0.002545068928950105, 0.009544008483563007, 0.05758218451749727]
ykl/ALL/ptq_result/cifar10/ResNet_50.txt 0 → 100644
ResNet_50
Full_acc: 94.580000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[6.406746146714387, 4.463006928866337, 1.8606068397680064, 0.5476336925833593, 0.1427087137002757, 0.03631802870036436, 0.00917632768694844, 0.002274022378465238, 0.0005732790341203692, 0.00012735280067719076, 3.204993011158073e-05, 1.9430616000983667e-05, 8.168679372916183e-06, 2.0007283482548982e-05, 1.7010192348812227e-05, 6.396918480840575, 3.565691183029724, 0.2766922472718606, 0.26466369414337404, 0.2648690090212167, 0.2648278963412444, 0.26487935208725366, 2.5332916627514512, 0.7347056021801005, 0.6964089202042065, 0.2270771187947045, 0.2466501363272561, 0.07526408085311341, 0.09674318659527918, 0.13500442926006606, 0.019829886388482706, 0.07347626738449571, 0.06167650705362234, 0.09913797982030731, 0.005215974569042684, 0.019155984700123193, 0.0734822388392775, 0.051459046305026596, 0.08547780129374767, 0.001491210776670565, 0.004814321284266958, 0.019164106642206327, 0.07388189174626741]
js_flops_wt_log_list:
[20.037555659152776, 13.958372580472153, 5.81918063520768, 1.7127634441383166, 0.4463316835616854, 0.11358722584747057, 0.02869961952030047, 0.007112167227356747, 0.0017929710794461944, 0.00039830497002396066, 0.0001002384430062209, 6.077063780824273e-05, 2.5548127528124786e-05, 6.257420649881966e-05, 5.320059015246238e-05, 20.006818932983744, 11.151953566936534, 0.8653747437783619, 0.8277545856994077, 0.8283967226279223, 0.8282681397881504, 0.8284290713048725, 7.923050411367082, 2.297844187930709, 2.1780685827982373, 0.7101998897056941, 0.7714158984634639, 0.23539378253696655, 0.30257121815897786, 0.4222359843175668, 0.06201938443084247, 0.22980226836342907, 0.19289767608208944, 0.31006110482528065, 0.01631333259512982, 0.05991170882140292, 0.22982094451428883, 0.1609418386324988, 0.2673379219065577, 0.004663868093538958, 0.015057133291290924, 0.0599371107747228, 0.23107088749388688]
js_flops_wt_cbrt_list:
[70.65798877740484, 49.22110011414127, 20.520047804752092, 6.039679802910221, 1.5738895314656962, 0.40054011904995357, 0.10120283274446663, 0.025079477790698124, 0.006322514211051091, 0.0014045339950974607, 0.00035346860173350895, 0.0002142941543016582, 9.008979632410289e-05, 0.00022065403863414924, 0.00018760006289624153, 70.54960254059787, 39.32488689651173, 3.0515517947525157, 2.918892664439246, 2.921157017669767, 2.920703598849422, 2.921271088092916, 27.938877204988312, 8.102837072787844, 7.680475008910493, 2.5043621432745575, 2.7202267992925315, 0.8300638824262214, 1.0669501859762627, 1.488921400834605, 0.21869758186270258, 0.8103466498234491, 0.680210803335682, 1.093361607459697, 0.057525343461852785, 0.211265332036853, 0.8104125071206315, 0.5675256414180223, 0.9427077936634397, 0.01644609477414201, 0.05309563567599876, 0.2113549063823037, 0.814820153368865]
js_param_list:
[5.240355990704543, 4.026883911008927, 1.7722860998248184, 0.5206632931861254, 0.13381661760780209, 0.03360286902449438, 0.008406931184356226, 0.0020096596608478574, 0.0005025902311521465, 0.00010877738484837579, 3.021143538787174e-05, 2.0939698634905692e-05, 1.1682259678940405e-05, 1.5547544580351305e-05, 3.1162163148950856e-05, 5.242284273107739, 3.290895328280436, 0.22128279530762301, 0.2106870639518734, 0.21088019070867536, 0.2108256334718985, 0.21091756435296746, 2.3009659412555465, 0.6533229753308406, 0.6409359887303046, 0.1843546909332463, 0.22723686953672997, 0.06016384523609278, 0.06770352251673334, 0.12537713629483233, 0.0157725155248299, 0.05854852646267403, 0.03771974331281288, 0.09249785038246981, 0.004149155918620373, 0.015155177067758561, 0.05854862828091227, 0.029469211436662226, 0.07993427309685415, 0.001198247749728423, 0.003782568004670307, 0.015159839003777256, 0.05938070132577549]
ptq_acc_list:
[10.0, 9.93, 17.05, 79.83, 92.88, 93.86, 94.43, 94.58, 94.57, 94.62, 94.61, 94.57, 94.57, 94.56, 94.55, 10.0, 10.25, 45.02, 54.21, 50.11, 44.66, 44.98, 9.45, 48.56, 22.24, 84.14, 67.5, 89.19, 87.72, 88.84, 92.9, 88.84, 90.69, 91.76, 94.26, 93.09, 88.79, 90.68, 92.67, 94.52, 94.17, 93.02, 86.98]
acc_loss_list:
[0.8942694015648128, 0.8950095157538592, 0.8197293296680059, 0.15595263269190104, 0.017974201733981846, 0.007612603087333462, 0.001585958976527717, 0.0, 0.00010573059843524124, -0.0004229223937408147, -0.00031719179530557344, 0.00010573059843524124, 0.00010573059843524124, 0.0002114611968703322, 0.00031719179530557344, 0.8942694015648128, 0.8916261366039332, 0.5240008458447875, 0.4268344258828505, 0.4701839712412772, 0.5278071473884542, 0.5244237682385282, 0.9000845844787482, 0.4865722139987312, 0.7648551490801438, 0.11038274476633536, 0.2863184605624868, 0.05698879255656588, 0.07253119052653838, 0.060689363501797364, 0.01776274053711136, 0.060689363501797364, 0.0411292027912878, 0.029816028758722704, 0.0033833791499259165, 0.01575385916684283, 0.06121801649397327, 0.041234933389722894, 0.02019454430112071, 0.0006343835906111469, 0.004334954535842637, 0.01649397335588922, 0.08035525481074217]
ykl/ALL/ptq_result/cifar10/VGG_16.txt 0 → 100644
VGG_16
Full_acc: 93.140000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[9.707856921981262, 6.486572902463099, 2.005316166527318, 0.5190581585685807, 0.13491936372039182, 0.03478897771537315, 0.00883023282337944, 0.002240868859371342, 0.0005592475775348325, 0.00015758524814800832, 3.412366999010219e-05, 4.357281474141023e-05, 1.7183449624302408e-05, 2.2050730420072442e-05, 1.2750526261498446e-05, 9.618822632146273, 4.733930080796053, 0.3982828800947237, 0.38795379558723697, 0.38807410856449526, 0.38820750558191597, 0.38849372381062763, 2.945696330723041, 0.7538307098720384, 0.8813729995052929, 0.25061212424988527, 0.3224955863954204, 0.10919356120067138, 0.12068196405865428, 0.1806574511886002, 0.028349744707636428, 0.107615886201107, 0.08391949583889775, 0.1337665599720689, 0.007379243854310278, 0.027778513044400217, 0.1076221064514893, 0.07263421124744922, 0.11556745962299968, 0.002023641857142568, 0.006992225258738289, 0.027775958632076332, 0.10781419241433256]
js_flops_wt_log_list:
[24.492082857031388, 16.36506205871, 5.05923913999218, 1.3095388126801188, 0.3403898782001609, 0.08776957999731655, 0.02227791320939133, 0.00565351819835672, 0.0014109332385774155, 0.00039757394301389985, 8.609106618487422e-05, 0.00010993044062529271, 4.3352356277017214e-05, 5.563208449061639e-05, 3.216847427574575e-05, 24.26745704916457, 11.943295900429012, 1.00483323747073, 0.9787738511790116, 0.9790773904082635, 0.9794139395384995, 0.9801360433590581, 7.4317369099623924, 1.9018496414544075, 2.223627800174493, 0.6322727004975406, 0.8136284544057278, 0.2754859048577079, 0.3044701510156383, 0.4557831145237226, 0.07152395239601561, 0.2715055673721703, 0.21172162526959745, 0.33748145411143227, 0.018617193614872664, 0.07008278434636478, 0.27152126052555925, 0.1832498289194585, 0.2915666989543422, 0.005105473271435212, 0.01764077919239614, 0.07007633978514076, 0.27200587678591814]
js_flops_wt_cbrt_list:
[67.31231269171278, 44.97658206308593, 13.904455940362705, 3.5990440892923963, 0.9355035279053602, 0.2412197218213951, 0.06122704503422324, 0.015537730580019836, 0.0038777093763990877, 0.0010926641774995415, 0.0002366066128726786, 0.00030212506779851554, 0.00011914655763151456, 0.00015289529636100708, 8.840956532378686e-05, 66.69496696795294, 32.82411189412296, 2.761612782321529, 2.6899930034378103, 2.6908272292418336, 2.691752177128986, 2.693736756327458, 20.424861439757734, 5.226909385595537, 6.111261776674915, 1.737693685374158, 2.236119045388184, 0.7571260263239646, 0.8367842837251345, 1.2526421580194735, 0.19657138499536034, 0.7461867475778887, 0.5818807786555136, 0.9275102203192557, 0.05116618155187639, 0.1926105803971694, 0.7462298775335647, 0.5036309021544991, 0.8013213463742126, 0.014031522565561129, 0.04848267303583488, 0.19259286861974306, 0.7475617626755179]
js_param_list:
[2.643630196821214, 2.030373814928824, 0.9088964101943003, 0.32912985562027364, 0.1053558533306981, 0.03044398804029105, 0.008108920237150345, 0.0020933207876840157, 0.0005270093236228446, 0.00014486326739074796, 3.850613408698056e-05, 2.9745078920157153e-05, 8.500018620847096e-06, 8.889711732320838e-06, 4.48826134820024e-06, 2.6438652865631758, 1.639671497228674, 0.12136619008732417, 0.10628808321731797, 0.1059608428068582, 0.10599506180130609, 0.10598393528866858, 1.155448097083798, 0.3949031250272791, 0.3548814434177675, 0.13151808439711926, 0.1215374057695574, 0.03170310969775694, 0.04723244874718427, 0.06115033337293262, 0.008514393294068574, 0.029246228243453496, 0.022230083061500646, 0.042553726414550706, 0.0024868175480420402, 0.007685811908287718, 0.029233005849203148, 0.015231422863962578, 0.03600678903667483, 0.0009166167737045329, 0.0018873743186338101, 0.007551743090941612, 0.02964848057488445]
ptq_acc_list:
[10.0, 9.21, 28.48, 88.17, 92.41, 93.08, 93.03, 93.03, 93.13, 93.12, 93.15, 93.15, 93.15, 93.13, 93.13, 10.0, 10.0, 70.23, 77.7, 75.37, 75.91, 73.03, 10.14, 80.5, 72.02, 87.72, 90.24, 89.89, 90.07, 91.82, 92.7, 89.79, 90.77, 91.86, 92.96, 92.77, 89.7, 91.07, 92.15, 93.11, 93.0, 92.74, 89.88]
acc_loss_list:
[0.8926347433970367, 0.9011165986686709, 0.6942237491947605, 0.053360532531672736, 0.007837663732016362, 0.0006441915396178041, 0.00118101782263259, 0.00118101782263259, 0.00010736525660301821, 0.00021473051320588385, -0.00010736525660301821, -0.00010736525660301821, -0.00010736525660301821, 0.00010736525660301821, 0.00010736525660301821, 0.8926347433970367, 0.8926347433970367, 0.24597380287738885, 0.16577195619497528, 0.1907880609834657, 0.18499033712690577, 0.21591153102855914, 0.8911316298045953, 0.13570968434614558, 0.2267554219454585, 0.05819196907880612, 0.031135924414859413, 0.03489370839596306, 0.032961133777109804, 0.014172213871591233, 0.00472407129053036, 0.035967360961992635, 0.025445565814902347, 0.013742752845179312, 0.0019325746188534124, 0.00397251449430969, 0.036933648271419345, 0.02222460811681348, 0.01062916040369331, 0.00032209576980890206, 0.001503113592441492, 0.004294610264118592, 0.035001073652566085]
ykl/ALL/ptq_result/cifar10/VGG_19.txt 0 → 100644
VGG_19
Full_acc: 93.030000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[8.024123945511475, 5.350393071050288, 1.6599776394933907, 0.4271921735523944, 0.11304354416339005, 0.028849968810954116, 0.007251873760070392, 0.0018494235336889505, 0.00046018644573646793, 0.00012088018248667003, 4.6490608984934174e-05, 3.556251850440877e-05, 1.507652496529077e-05, 2.990267330188534e-05, 2.2260680017889275e-05, 7.917801775320006, 3.952287197775858, 0.33107678229570486, 0.32296706134234826, 0.3225707008926969, 0.32259356442551, 0.32319079923547345, 2.4576681446325686, 0.6236871239818896, 0.7417313324450194, 0.210823575265336, 0.27251394272920365, 0.09013414900124771, 0.10135054029778329, 0.15296487310804363, 0.023425130447645638, 0.08880010294945571, 0.07059554430005066, 0.11349445945455026, 0.006082203908415393, 0.022908628679488032, 0.08880467313561308, 0.06120504745935862, 0.09817263037279855, 0.0016877448067475632, 0.00580373047691887, 0.022905164997284837, 0.08924816431939135]
js_flops_wt_log_list:
[21.03597567608492, 14.026545360514485, 4.351783382753069, 1.1199234000976295, 0.2963540021948062, 0.07563283497166308, 0.019011451101551422, 0.004848433141567074, 0.0012064209058474681, 0.0003168984671445291, 0.00012187938850577066, 9.32303986477589e-05, 3.952449072364471e-05, 7.839259618867692e-05, 5.835841103269244e-05, 20.757242371220233, 10.361282791976521, 0.8679481008247283, 0.8466877247546312, 0.8456486295419392, 0.8457085683867765, 0.8472742741286887, 6.443002084894146, 1.635052905296829, 1.9445166068541972, 0.552693307249777, 0.7144202544194523, 0.23629492500991045, 0.26569972185622004, 0.4010114215391915, 0.061411124460687735, 0.232797601129218, 0.18507268367490068, 0.297536117919991, 0.015945054481116862, 0.0600570677804322, 0.2328095822907123, 0.160454636338076, 0.25736853999303827, 0.004424577554293664, 0.015215010881195454, 0.060047987420374094, 0.23397223503858472]
js_flops_wt_cbrt_list:
[60.01523598044835, 40.01746544890526, 12.41555469892648, 3.195120025418971, 0.8454922961187685, 0.21577903058021877, 0.05423930611820875, 0.013832486955068099, 0.003441895753782187, 0.0009041052614043009, 0.0003477195626653206, 0.0002659845429350617, 0.00011276261554548698, 0.00022365257651120555, 0.00016649542971057857, 59.22001519655658, 29.56054149309425, 2.4762393193388936, 2.415583873329297, 2.4126193545755146, 2.412790358952621, 2.417257287467072, 18.38176163072123, 4.664774644299732, 5.547668661783456, 1.5768234271690889, 2.0382273120300165, 0.6741448984262876, 0.7580362210281995, 1.144077910390928, 0.17520476269246188, 0.6641671003325944, 0.5280088242786575, 0.8488648496573153, 0.045490935250189185, 0.17134166490000602, 0.6642012823570211, 0.45777400641009275, 0.7342675186293222, 0.012623235077069183, 0.0434081348324691, 0.1713157588065561, 0.6675183083938123]
js_param_list:
[1.729679907104299, 1.308784745637505, 0.5535505208524657, 0.20063692566649705, 0.06507369517760643, 0.018303461147078606, 0.004763944487067647, 0.0012536692137142807, 0.0003007278450856389, 8.02359855308368e-05, 2.499211218568465e-05, 2.2779966931979102e-05, 4.960790979259166e-06, 3.0272066618613143e-05, 1.177184214829974e-05, 1.7302509008662255, 1.044541644560887, 0.07959334884952608, 0.07096552625488958, 0.07042699382358332, 0.070429719663752, 0.0704527543255789, 0.7183265498423228, 0.24376553224008424, 0.22645431114772974, 0.08293533850874694, 0.07735725902284186, 0.020660066763632034, 0.03002364437629356, 0.039148687541727886, 0.005542332367563041, 0.01915219569302639, 0.014648462521770654, 0.02746807197884521, 0.0015743723613774814, 0.004954252490325559, 0.01917171551658501, 0.010468788332101183, 0.023267441347928894, 0.0005739400486149861, 0.0012381338226764296, 0.00495153646237609, 0.020080275191084195]
ptq_acc_list:
[10.0, 10.0, 21.69, 88.51, 92.42, 92.85, 92.92, 93.02, 93.05, 93.01, 93.03, 93.04, 93.01, 93.03, 93.02, 10.0, 10.0, 71.57, 69.01, 72.3, 76.18, 67.87, 10.0, 76.95, 72.3, 86.7, 88.97, 90.42, 88.94, 91.08, 92.53, 90.46, 89.87, 91.83, 92.93, 92.58, 90.54, 90.05, 92.09, 93.05, 92.94, 92.65, 90.72]
acc_loss_list:
[0.8925077931849941, 0.8925077931849941, 0.7668494034182523, 0.048586477480382625, 0.006557024615715354, 0.0019348597226701797, 0.0011824142749650589, 0.0001074922068150609, -0.00021498441362996905, 0.00021498441362996905, 0.0, -0.0001074922068150609, 0.00021498441362996905, 0.0, 0.0001074922068150609, 0.8925077931849941, 0.8925077931849941, 0.23067827582500278, 0.2581962807696442, 0.2228313447275073, 0.1811243684832849, 0.2704503923465548, 0.8925077931849941, 0.17284746855852948, 0.2228313447275073, 0.06804256691389872, 0.043641835966892424, 0.028055465978716537, 0.043964312587337455, 0.020960980328926182, 0.005374610340750295, 0.0276254971514566, 0.033967537353541834, 0.01289906481780074, 0.001074922068149998, 0.004837149306675297, 0.026765559496936417, 0.0320326776308718, 0.01010426744061053, -0.00021498441362996905, 0.0009674298613350899, 0.004084703858970176, 0.02483069977426639]
ykl/ALL/ptq_result/cifar100/AlexNet.txt 0 → 100644
AlexNet
Full_acc: 63.430000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[32.80662471491742, 19.26523828551374, 4.849584590989981, 1.0460649099520598, 0.24170127284938484, 0.058772295645415626, 0.014377242624187672, 0.00347165887456444, 0.0007720974983424043, 0.00011322701598531815, 3.0873078161106628e-06, 6.786069409177032e-07, 9.039998308193465e-07, 1.0437515133697176e-06, 3.603692457586344e-05, 32.303368012514916, 12.791516864645015, 1.43887990253494, 1.4346128620973575, 1.4336919250710196, 1.4351721938795325, 1.436032353963674, 7.990586625406007, 2.0190135143560317, 2.2728831352231116, 0.797604178003875, 0.8348188628651314, 0.40246230456662013, 0.48577802808872705, 0.4681184898166976, 0.10263848459243573, 0.4016551899570668, 0.38926690427531274, 0.3475839397146749, 0.025419922222255847, 0.10237432423667726, 0.4016763028910776, 0.3571251216484937, 0.3003118006287682, 0.006486648196178182, 0.025284006630165317, 0.10244492415131133, 0.4016767746332344]
js_flops_wt_log_list:
[60.55787563587672, 35.56177797404382, 8.95186698110082, 1.9309352691539414, 0.44615731576892875, 0.10848800818304534, 0.0265390078494265, 0.0064083485638333606, 0.0014252177628663603, 0.00020900618738575958, 5.698873456271385e-06, 1.252643180785703e-06, 1.668696789301794e-06, 1.926664961441785e-06, 6.652069866158131e-05, 59.62891213957548, 23.611910527667412, 2.6560339855094, 2.648157439037526, 2.646457477813412, 2.6491899117407836, 2.6507776845718816, 14.749854802923602, 3.7269048666847815, 4.195523783192983, 1.4723006416521542, 1.5409953725865306, 0.7429067269145634, 0.8966995436827769, 0.8641017335832122, 0.18946077627355873, 0.7414168709304759, 0.7185492864052002, 0.6416065406661063, 0.0469227328926972, 0.18897316164965394, 0.7414558433771462, 0.6592186453549727, 0.5543467160214665, 0.01197372903099127, 0.04667184577479591, 0.18910348230554175, 0.7414567141678208]
js_flops_wt_cbrt_list:
[135.28911007959496, 79.44666559779789, 19.998887092838274, 4.313799178754134, 0.996736189505954, 0.24236725491567077, 0.05928937758538526, 0.014316548676407583, 0.0031840027541122913, 0.0004669295412964682, 1.2731548295898529e-05, 2.798463113117714e-06, 3.727946220814434e-06, 4.3042591127581355e-06, 0.00014861033398712203, 133.21376243254895, 52.750106060104976, 5.933703427797338, 5.916106856723945, 5.912309064301979, 5.918413448752454, 5.92196060708772, 32.95186149014397, 8.326078771270227, 9.372995221280219, 3.289188094648606, 3.442655317824498, 1.6596881726952752, 2.0032684766397435, 1.9304434530964634, 0.423264184020578, 1.656359765149312, 1.605272517989314, 1.4333788462113763, 0.10482756716443789, 0.42217483027695374, 1.6564468314073717, 1.4727251070398153, 1.2384363404197607, 0.02674986742727353, 0.10426707367692871, 0.4224659726820484, 1.65644877679423]
js_param_list:
[10.633487354669809, 5.6581753167686095, 1.3769097489050328, 0.2948281877502868, 0.06801710994680957, 0.01642581996399826, 0.00394480664498228, 0.0009184215094422542, 0.00019503660111395502, 3.1039601339482914e-05, 1.2021317312487235e-06, 4.781109200724387e-08, 2.3491908263983952e-07, 2.2857143889837285e-07, 1.0081107789685352e-05, 10.499482666951511, 3.655615931956932, 0.4290935794901198, 0.42769394266416566, 0.42775598521919167, 0.42799670478476576, 0.42826653507472656, 2.430364313549843, 0.6463998908180691, 0.6440670431149257, 0.27628641438686263, 0.23562670724262494, 0.11822378333829507, 0.178385483615454, 0.13167158536202225, 0.030551749492768808, 0.11803024064023392, 0.14693848218973501, 0.0973862219274645, 0.007626292573081045, 0.030544545167243798, 0.11810513618955643, 0.13518328139410563, 0.0840129802687004, 0.0018703401130295563, 0.0076815799744418675, 0.030577140777086634, 0.11810482023853337]
ptq_acc_list:
[1.0, 1.05, 15.51, 46.84, 58.16, 61.81, 62.92, 63.32, 63.26, 63.3, 63.4, 63.46, 63.4, 63.44, 63.46, 1.0, 1.48, 23.71, 23.74, 23.91, 24.09, 24.27, 2.01, 15.31, 20.41, 37.76, 44.37, 50.67, 47.77, 55.26, 60.21, 49.56, 51.36, 58.57, 62.23, 59.76, 49.98, 53.6, 58.68, 62.94, 62.41, 59.92, 50.19]
acc_loss_list:
[0.9842345893110516, 0.9834463187766042, 0.7554784802144097, 0.2615481633296547, 0.08308371433075837, 0.025539965316096445, 0.008040359451363676, 0.0017341951757843203, 0.002680119817121263, 0.0020495033895633384, 0.0004729623206684713, -0.0004729623206684713, 0.0004729623206684713, -0.00015765410688945312, -0.0004729623206684713, 0.9842345893110516, 0.9766671921803564, 0.6262021125650323, 0.6257291502443638, 0.6230490304272426, 0.620211256503232, 0.6173734825792211, 0.9683115245152136, 0.7586315623521992, 0.6782279678385621, 0.4046980923853067, 0.30048872773135743, 0.20116664039098214, 0.24688633138893262, 0.12880340532870885, 0.05076462241841398, 0.21866624625571493, 0.19028850701560776, 0.07661989594828944, 0.01891849282673818, 0.05785905722844083, 0.21204477376635666, 0.1549739870723632, 0.07488570077250513, 0.007725051237584771, 0.016080718902727466, 0.055336591518209015, 0.20873403752167746]
ykl/ALL/ptq_result/cifar100/AlexNet_BN.txt 0 → 100644
AlexNet_BN
Full_acc: 66.310000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[241.1384276914563, 99.98147824274622, 19.42161089999079, 4.2483987797795875, 0.9939560702348172, 0.24045008970243287, 0.05912125034451196, 0.014689748820453062, 0.003601008552514204, 0.0008469610059717723, 0.0002395469492320963, 8.695931102907095e-05, 4.090230358090943e-05, 1.1586028240138518e-06, 6.621935138340083e-06, 240.70765382584733, 59.3008317859552, 9.58974223529778, 9.571211753180789, 9.57135100324931, 9.572260706126064, 9.572165888914057, 46.418899415948076, 14.312297969369594, 10.33315582616085, 7.224776908621552, 3.656449895009492, 2.6872992734284935, 5.1396656171000945, 1.961869601983462, 0.6999806409255603, 2.6848463357604007, 4.395753773426759, 1.4168188822971277, 0.17674767655889811, 0.6990753028729249, 2.684888051638414, 4.089754602362963, 1.2107357111816763, 0.04464851748152792, 0.1761601349758236, 0.699069783060451, 2.6848880242881825]
js_flops_wt_log_list:
[445.38678730553505, 184.66749497750308, 35.8720464666592, 7.846864980557503, 1.8358538084654064, 0.44411541530379106, 0.10919795739118272, 0.027132216528500955, 0.006651124192931286, 0.0015643514185366696, 0.00044244729945672455, 0.00016061533010863595, 7.554725208960292e-05, 2.1399591698877653e-06, 1.2230827103113853e-05, 444.59114063115163, 109.52964737535414, 17.71241741168079, 17.678191295269887, 17.67844849252063, 17.680128729244238, 17.679953600237045, 85.73649865371665, 26.435058371092257, 19.085514989050058, 13.344286131148968, 6.753525297783298, 4.963487575909172, 9.49305002500038, 3.6236066043270547, 1.292876178312042, 4.9589569582141335, 8.119032166195442, 2.616888632066397, 0.32645597211495214, 1.291204003493001, 4.959034008152071, 7.553846480019178, 2.2362495013414856, 0.08246657303620343, 0.3253707727935533, 1.29119380830525, 4.959033957635727]
js_flops_wt_cbrt_list:
[995.2650688683347, 412.6595407520374, 80.15997738297519, 17.534670623081663, 4.102414393943586, 0.992424050277442, 0.2440146759644711, 0.060629879739258, 0.014862658180716394, 0.0034957128650431038, 0.0009886964645458246, 0.00035891237041170144, 0.00016881852627162043, 4.781970797738801e-06, 2.733110933249731e-05, 993.4871101036265, 244.75587319875916, 39.58031723776304, 39.50383526955543, 39.504410004699736, 39.50816467584423, 39.507773331088686, 191.5875025241476, 59.07200424469488, 42.64865265734496, 29.819254261366115, 15.091484552709368, 11.091451172024863, 21.213249598311855, 8.097341859702041, 2.8890720051002567, 11.081327015537088, 18.142857710077177, 5.847721393736902, 0.7295012668316512, 2.885335349155654, 11.081499191977821, 16.87988901206384, 4.997141983991121, 0.18428049917859016, 0.7270762712814195, 2.885312566903176, 11.081499079093573]
js_param_list:
[69.3560949829066, 28.5426675012072, 5.522735288295837, 1.207103235392773, 0.2824179558544593, 0.06826091373566213, 0.016624240426034907, 0.004108488795195486, 0.0010013302421040524, 0.00023600544443462334, 6.636711423177611e-05, 2.4026399863847474e-05, 1.153604925500107e-05, 2.785574395615664e-07, 1.6950574832108597e-06, 69.232898547365, 16.888658826263583, 2.7536312354547046, 2.74878463099763, 2.7485373779579456, 2.749323165916103, 2.7487724986785835, 13.31595086878375, 4.123251773604103, 2.942787898195962, 2.0898047525977494, 1.0401807681229005, 0.7715535174760516, 1.4898706490686615, 0.5574576422502738, 0.2008454562539802, 0.7707317586831185, 1.2748800638071396, 0.4022879195280152, 0.05066406130679979, 0.20071430418594033, 0.7709156545819815, 1.1864994537288847, 0.3436921142756107, 0.01276939868000909, 0.05059806805560356, 0.20065317452110798, 0.770915718123234]
ptq_acc_list:
[1.0, 1.17, 9.2, 39.86, 63.26, 65.59, 66.35, 66.38, 66.43, 66.42, 66.37, 66.35, 66.33, 66.3, 66.31, 1.0, 1.57, 24.31, 19.52, 24.83, 21.77, 23.08, 1.85, 14.61, 24.97, 34.6, 52.32, 53.26, 44.05, 59.83, 63.08, 53.26, 53.17, 63.05, 65.34, 62.85, 53.55, 53.1, 63.51, 66.59, 65.16, 62.73, 53.43]
acc_loss_list:
[0.9849193183531896, 0.9823556024732317, 0.8612577288493439, 0.39888402955813607, 0.04599607902277189, 0.010858090785703496, -0.0006032272658722974, -0.0010556477152766277, -0.0018096817976173208, -0.0016588749811491392, -0.0009048408988086604, -0.0006032272658722974, -0.0003016136329361487, 0.0001508068164681815, 0.0, 0.9849193183531896, 0.9763233298145078, 0.6333886291660383, 0.7056250942542603, 0.6255466747096969, 0.6716935605489369, 0.6519378675916152, 0.9721007389534008, 0.7796712411400996, 0.6234353792791435, 0.47820841502035893, 0.21097873623887803, 0.19680289549087623, 0.33569597345800034, 0.09772281707133168, 0.04871060171919776, 0.19680289549087623, 0.19816015683908914, 0.0491630221686021, 0.014628261197406105, 0.05217915849796412, 0.19242949781330124, 0.19921580455436586, 0.042225908611069285, -0.004222590861106939, 0.017342783893832085, 0.05398884029558144, 0.19423917961091844]
ykl/ALL/ptq_result/cifar100/Inception_BN.txt 0 → 100644
Inception_BN
Full_acc: 77.970000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[70.18077844226828, 29.36227542929503, 12.161101101718925, 5.065398765832169, 1.6821996542354316, 0.45803271592311523, 0.11603422763898134, 0.0288871764679315, 0.007208607902993315, 0.00180579208804947, 0.0004463495713655097, 0.00013489078545568422, 4.34243651964441e-05, 4.1922292832075194e-05, 1.0219350463342103e-05, 70.19003219002651, 22.937027525975946, 4.4711333504858635, 4.233725367593571, 4.233461443162384, 4.233666233223974, 4.232194323541173, 23.26242350082805, 9.555320973314766, 5.52811458444971, 4.331244872017863, 1.6915712923382122, 1.2708596966700276, 2.467333263488592, 0.7447567711400319, 0.323998128340025, 1.235048645934723, 1.875071614546571, 0.48111423516727964, 0.08305712734144913, 0.3102554738876757, 1.2350540447475817, 1.6737970608085262, 0.3948749903470355, 0.02521308851508495, 0.07478309123137465, 0.3102657446092667, 1.2443084039786505]
js_flops_wt_log_list:
[222.7996058311718, 93.21503034245619, 38.607273844431205, 16.08088245036336, 5.340399867483631, 1.4540948509056302, 0.36836838739479133, 0.09170675608742397, 0.022884827370496623, 0.005732762935387398, 0.0014170049231493518, 0.00042823141174625385, 0.00013785728320462188, 0.00013308872494494705, 3.24428897145421e-05, 222.8289832675405, 72.8170989999781, 14.194296076757228, 13.440608155596763, 13.439770287158384, 13.440420424503866, 13.435747622286135, 73.85011824748811, 30.334826624873365, 17.549844526420152, 13.750198724821592, 5.3701515647665214, 4.034538313332126, 7.832926490144382, 2.3643441798895544, 1.0285815701456884, 3.9208506603120137, 5.952701379193942, 1.5273706609454798, 0.26367754310914154, 0.984953413504819, 3.9208677996684527, 5.31372455060874, 1.2535909996250503, 0.08004280242585766, 0.23741035108199804, 0.9849860194800286, 3.9502471772511383]
js_flops_wt_cbrt_list:
[802.4828960485347, 335.7432668042923, 139.0562465657799, 57.92035884283493, 19.2351307612407, 5.23738021317789, 1.3267946737450742, 0.33031074241674535, 0.08242690769283896, 0.020648349828030675, 0.005103789166061572, 0.0015424101950049617, 0.0004965363895269067, 0.0004793609262763131, 0.00011685327717337656, 802.5887081307766, 262.273697789369, 51.12522430438173, 48.41058006866456, 48.40756222180424, 48.409903896054, 48.393073328323155, 265.9944417032693, 109.2604245426934, 63.211288046151296, 49.52566793272978, 19.342290861945838, 14.5316594157482, 28.212749797723863, 8.51592962947608, 3.704760222323414, 14.122177555579773, 21.44052734863197, 5.50130610313498, 0.9497176515542196, 3.5476196862806635, 14.12223928836579, 19.13905121272033, 4.5152024936784745, 0.2882993426263779, 0.8551080931110243, 3.5477371269615703, 14.228058362500606]
js_param_list:
[246.70260764269344, 100.62643666806441, 43.72041273286525, 18.753320192360555, 6.289573009569436, 1.7169640898369907, 0.43540581609867723, 0.10845241739328468, 0.027019458716378127, 0.006762965621613173, 0.0016879170485830052, 0.00043371921759512296, 0.00012008073325100211, 4.7503815910586614e-05, 1.2734235943685835e-05, 246.71314287492342, 80.0213016753408, 16.197068144626368, 15.289168263642832, 15.289478810280226, 15.289283379832112, 15.287813473838234, 83.92812042207213, 35.22094818557763, 19.785877569323443, 16.01243214027354, 5.97448938931926, 4.61478017839912, 9.085196111361512, 2.5711981845670953, 1.1758063770231035, 4.478291792506832, 6.883519397315181, 1.6350958915968825, 0.30166172201289365, 1.1234140233672545, 4.4782912558469805, 6.137256566908287, 1.333289400268289, 0.0923696804420931, 0.27017583150520874, 1.1234237332888781, 4.484176731603195]
ptq_acc_list:
[1.0, 0.81, 5.89, 58.69, 74.23, 77.49, 77.78, 77.83, 77.91, 77.96, 77.98, 77.98, 77.98, 77.97, 77.97, 1.0, 1.18, 6.68, 9.12, 4.97, 6.84, 11.72, 1.06, 2.26, 31.69, 9.79, 59.74, 62.0, 12.83, 69.8, 72.8, 53.61, 27.04, 73.54, 76.73, 73.8, 46.95, 30.24, 74.78, 77.77, 76.56, 73.55, 39.05]
acc_loss_list:
[0.9871745543157625, 0.9896113889957676, 0.9244581249198409, 0.24727459279209954, 0.04796716685904829, 0.006156213928434064, 0.002436834680005101, 0.0017955623957932611, 0.0007695267410542809, 0.0001282544568424409, -0.0001282544568424409, -0.0001282544568424409, -0.0001282544568424409, 0.0, 0.0, 0.9871745543157625, 0.9848659740925997, 0.9143260228292932, 0.8830319353597537, 0.9362575349493395, 0.9122739515198153, 0.8496857765807362, 0.9864050275747082, 0.9710144927536232, 0.5935616262665128, 0.8744388867513148, 0.2338078748236501, 0.2048223675772733, 0.8354495318712325, 0.10478389124022063, 0.06630755418750804, 0.31242785686802615, 0.6531999486982173, 0.05681672438117215, 0.01590355264845447, 0.05348210850327051, 0.39784532512504805, 0.6121585225086572, 0.04091317173271768, 0.002565089136847542, 0.01808387841477487, 0.056688469924329896, 0.49916634603052457]
ykl/ALL/ptq_result/cifar100/MobileNetV2.txt 0 → 100644
MobileNetV2
Full_acc: 66.760000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[23.04290375294078, 12.195740403207688, 3.0512408587319224, 0.6733236683858506, 0.1585792380855503, 0.03854275118017126, 0.009526560589591077, 0.002406190395999627, 0.0005716111569045701, 0.00015879783539462844, 4.9114381702608734e-05, 3.922291692015658e-05, 7.424914040443347e-06, 1.2573186742079053e-05, 1.2281579872435388e-05, 22.95893605264291, 8.288965407292613, 0.9385526716692725, 0.933877547297082, 0.933937329336115, 0.9335094132829027, 0.9330596314618789, 5.622180924527127, 1.4790523998771123, 1.4828957433589072, 0.6271754101133044, 0.5466238259136381, 0.26010245611104454, 0.40183201303519406, 0.3074738449441434, 0.06722173255693384, 0.2594933168426587, 0.32968148652012863, 0.22800798186071702, 0.016899992162402767, 0.06720311993765539, 0.25961538673717793, 0.30279759385838295, 0.19695360532471415, 0.004261626929233912, 0.01684844599346723, 0.06721579250348103, 0.25961622622699565]
js_flops_wt_log_list:
[42.28940991315112, 22.38219065771204, 5.599779298744056, 1.2357149481632452, 0.2910320016529671, 0.0707354516303456, 0.01748358757885591, 0.004415952654080296, 0.0010490474110574233, 0.00029143318161312456, 9.013700021150835e-05, 7.198372346689528e-05, 1.3626547972980495e-05, 2.3074897753826706e-05, 2.2539727248578296e-05, 42.13530847977999, 15.212295274209568, 1.7224755648358587, 1.713895558900396, 1.714005273682058, 1.7132199422161658, 1.712394481675051, 10.318088218065112, 2.714425833308902, 2.721479316224132, 1.1510215157820676, 1.0031894977388274, 0.4773521385946671, 0.7374608208092391, 0.5642903171331476, 0.12336845363117883, 0.47623421784599934, 0.6050468150068724, 0.41845086503681705, 0.031015652530062825, 0.12333429488562042, 0.4764582462003564, 0.5557082434004782, 0.36145842723432153, 0.007821136174484785, 0.030921052600690235, 0.12335755216849578, 0.476459786871159]
js_flops_wt_cbrt_list:
[94.251017672818, 49.88351106266855, 12.480308870086084, 2.7540557235744374, 0.6486272186696845, 0.15764912103049236, 0.0389659237445771, 0.009841897356733216, 0.002338027091942219, 0.0006495213342317704, 0.0002008896321172774, 0.00016043116247227985, 3.036968395772038e-05, 5.142735735638774e-05, 5.023461513435799e-05, 93.90756958617176, 33.903861833924, 3.838906128880643, 3.8197837459314146, 3.8200282688473557, 3.8182779892851, 3.8164382734742484, 22.996071995039255, 6.049679995823044, 6.06540019490512, 2.5652982495750756, 2.2358228992730407, 1.0638815945467281, 1.643589565279969, 1.2576419666753722, 0.27495305154032246, 1.061390068454123, 1.3484765611818335, 0.9326074768313611, 0.06912503202925661, 0.2748769214515385, 1.0618893636778937, 1.2385149751966922, 0.8055876088887364, 0.01743107897028479, 0.06891419580256407, 0.27492875529324523, 1.0618927973931274]
js_param_list:
[25.8108611712771, 13.863871611607545, 3.4961598229771615, 0.7717539478451231, 0.18164238305161576, 0.044155355562212346, 0.010918187937328976, 0.002759935002440996, 0.0006553540957858193, 0.00018275677197526676, 5.609276171226996e-05, 4.516510795633014e-05, 8.233874491560544e-06, 1.3550136002370482e-05, 1.2972172482835081e-05, 25.759903734262128, 9.459737726699748, 1.0400577346142854, 1.034052150602228, 1.0345227526362717, 1.0343056546267266, 1.0339732857520283, 6.3612540090144085, 1.6581543505644003, 1.6932406337329557, 0.6954004493340186, 0.6252461971116032, 0.2901426123571005, 0.44219841701871376, 0.35239800493850254, 0.07491850844896765, 0.2894406725343766, 0.3616160176519872, 0.26163185985759274, 0.018814230951148023, 0.0748961967013584, 0.28957845173590435, 0.33182338739327644, 0.2260843653483685, 0.004770132536079175, 0.01875459195097388, 0.07490902103458702, 0.2895795470034575]
ptq_acc_list:
[1.0, 0.98, 1.0, 7.46, 36.08, 56.08, 63.09, 66.36, 66.75, 66.91, 66.78, 66.78, 66.76, 66.8, 66.73, 1.0, 0.96, 1.43, 1.59, 1.94, 1.47, 1.16, 0.9, 0.99, 1.88, 2.19, 13.18, 13.15, 3.69, 22.5, 38.98, 13.04, 4.46, 38.58, 58.16, 38.16, 13.4, 5.41, 42.44, 64.31, 57.7, 38.28, 12.39]
acc_loss_list:
[0.9850209706411025, 0.9853205512282803, 0.9850209706411025, 0.8882564409826244, 0.4595566207309767, 0.15997603355302586, 0.05497303774715401, 0.005991611743559102, 0.00014979029358905206, -0.0022468544038345035, -0.00029958058717789124, -0.00029958058717789124, 0.0, -0.0005991611743557825, 0.0004493708807669433, 0.9850209706411025, 0.9856201318154585, 0.9785799880167764, 0.9761833433193529, 0.9709406830437388, 0.9779808268424206, 0.9826243259436789, 0.9865188735769921, 0.9851707609346915, 0.9718394248052726, 0.9671959257040145, 0.8025763930497304, 0.8030257639304973, 0.944727381665668, 0.6629718394248053, 0.41611743559017383, 0.8046734571599761, 0.933193529059317, 0.42210904733373283, 0.128819652486519, 0.42840023966446983, 0.799281006590773, 0.9189634511683643, 0.36428999400838835, 0.03669862192929902, 0.13571000599161176, 0.4266027561414021, 0.8144098262432594]
ykl/ALL/ptq_result/cifar100/ResNet_152.txt 0 → 100644
ResNet_152
Full_acc: 77.410000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[8.624846585002025, 6.593320189498599, 3.2865562850438548, 1.121692437823039, 0.3256337895358447, 0.08930726037224679, 0.023638846806671138, 0.006031718631730127, 0.0014690428233190406, 0.00032910234437374166, 5.349880408999319e-05, 1.388662438345417e-05, 7.256994415791253e-06, 3.5221448690200414e-06, 5.784124541249686e-06, 8.624407360657392, 5.53942031790266, 0.401196982352419, 0.3600293854527351, 0.3601361834054045, 0.360122642741003, 0.36013362927582715, 4.168566367467982, 1.3818541848223522, 1.1624080711102005, 0.42905497308288926, 0.3976260611096333, 0.10673165272344699, 0.15844890531091957, 0.20604619901271168, 0.028255663289472852, 0.1003812104969247, 0.08276610784675194, 0.14635292068779623, 0.007787948435601907, 0.02593794114616229, 0.10042432327764016, 0.06127605428717364, 0.12453358892161101, 0.0025519256943184214, 0.006475075487039712, 0.02596948038864452, 0.10099312501869116]
js_flops_wt_log_list:
[30.904305390035958, 23.624997692698855, 11.776295162299805, 4.019216493944046, 1.1668017486398081, 0.3200032395811853, 0.08470204467775697, 0.021612682936981643, 0.0052638325326086486, 0.0011792301758490357, 0.0001916953957736358, 4.97581581948668e-05, 2.6003056336027082e-05, 1.2620449487116065e-05, 2.0725510822142925e-05, 30.902731573835574, 19.848693597161958, 1.4375576356036845, 1.2900470713028551, 1.2904297466944459, 1.2903812281702352, 1.2904205948098662, 14.936688645903967, 4.95142067877595, 4.165107595061511, 1.5373775969910952, 1.4247624119990443, 0.3824378275069219, 0.5677496185219855, 0.7382988898397282, 0.10124489040933961, 0.3596831032348983, 0.29656516758089935, 0.524407629873025, 0.027905555703938077, 0.09294009423468376, 0.35983758372661645, 0.2195626178534272, 0.4462252198251433, 0.009143987688666524, 0.0232013066322807, 0.09305310475282284, 0.3618756979746301]
js_flops_wt_cbrt_list:
[134.94036783669262, 103.1560437472783, 51.419942331612795, 17.549482030519133, 5.094715935751651, 1.3972601653082544, 0.3698424837926182, 0.09436948504898007, 0.022983965800764235, 0.005148983343417946, 0.0008370175900033289, 0.00021726371406672975, 0.00011353958429342496, 5.5105852552341056e-05, 9.049574221606752e-05, 134.93349593537314, 86.66721291015962, 6.2769427653064485, 5.632853550055239, 5.634524461517302, 5.634312610533393, 5.634484500785099, 65.21946488369484, 21.619852615216974, 18.18650003174682, 6.712796027078982, 6.221073780128754, 1.6698741637172771, 2.4790184214009114, 3.223703704420286, 0.4420750630372289, 1.5705180764492759, 1.2949203127477373, 2.2897702303398, 0.12184664576188162, 0.4058130524055926, 1.571192599112476, 0.9586968560659233, 1.9483950388552576, 0.039926251265970096, 0.1013060417226291, 0.40630650083275, 1.5800918085543045]
js_param_list:
[19.827877795117903, 15.663340180243965, 7.5804395348947375, 2.321882904040722, 0.609373871793933, 0.15789342226986147, 0.0405012160330352, 0.010238829721414756, 0.0025284544545387047, 0.0005959459808172158, 0.0001226338118733685, 3.138607898364615e-05, 1.012212078064238e-05, 2.220401846451086e-06, 9.187307669841549e-06, 19.82839760777665, 13.120045638457718, 0.8715193165800089, 0.81467755801544, 0.8147589968779452, 0.8147523571261951, 0.8148063147901086, 9.630136751746925, 2.8807522074753673, 2.586014264876044, 0.8159900078582137, 0.9050262871081463, 0.236087797356497, 0.2961045501582039, 0.4903981235828728, 0.06222394170153005, 0.22754289305360137, 0.15906869740711185, 0.3582177899643967, 0.016697198362036113, 0.05906751022155645, 0.2275801112326192, 0.12116151596661229, 0.30827770898137696, 0.0050545028845387875, 0.014876618045174561, 0.05909252817213956, 0.22825845612782902]
ptq_acc_list:
[1.0, 1.06, 2.85, 32.68, 61.79, 75.11, 77.13, 77.26, 77.4, 77.43, 77.4, 77.42, 77.42, 77.41, 77.4, 1.0, 1.3, 5.88, 6.42, 8.98, 7.71, 4.99, 1.0, 2.12, 6.37, 32.12, 40.26, 53.67, 40.49, 54.46, 68.6, 53.02, 52.09, 62.72, 74.53, 68.77, 51.68, 56.72, 65.31, 76.96, 74.72, 68.73, 52.12]
acc_loss_list:
[0.9870817723808294, 0.986306678723679, 0.9631830512853637, 0.5778323214055031, 0.20178271541144552, 0.029711923524092458, 0.0036171037333677966, 0.0019377341428754875, 0.000129182276191589, -0.0002583645523835452, 0.000129182276191589, -0.0001291822761917726, -0.0001291822761917726, 0.0, 0.000129182276191589, 0.9870817723808294, 0.9832063040950781, 0.9240408215992766, 0.9170649786849244, 0.8839943159798475, 0.9004004650561944, 0.9355380441803385, 0.9870817723808294, 0.9726133574473582, 0.9177108900658829, 0.5850665288722388, 0.4799121560521896, 0.3066787236791112, 0.47694096369978034, 0.2964733238599664, 0.11380958532489346, 0.31507557163157207, 0.32708952331740077, 0.18976876372561682, 0.03720449554321142, 0.11161348662963443, 0.3323859966412608, 0.2672781294406407, 0.1563105541919648, 0.005813202428626829, 0.03475003229556902, 0.11213021573440116, 0.32670197648882576]
ykl/ALL/ptq_result/cifar100/ResNet_18.txt 0 → 100644
ResNet_18
Full_acc: 75.730000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[111.75808501564241, 88.20479725393312, 36.698824971050385, 9.050154929605284, 2.1281321136945013, 0.5159263456723158, 0.12707464239406224, 0.03153296914966042, 0.007850445768696248, 0.0019940126009431384, 0.0004914783771314171, 0.00015001133646329762, 4.143660934511198e-05, 2.3161348426108868e-05, 2.25244912894178e-05, 111.6828586085451, 71.35946902355816, 4.783651268058645, 4.686391020506018, 4.686164626912448, 4.68602988451295, 4.686399080266112, 49.22642273685793, 12.022923911843062, 13.30069209811163, 3.357707074414986, 4.936656244672016, 1.3290554150715084, 1.3956863821050531, 2.833099026495589, 0.34720870462788045, 1.3154295672568588, 0.8951666584316648, 2.1320931144378434, 0.08918929000773806, 0.34205401404885305, 1.3154359528147965, 0.7519163944224915, 1.8546206287353129, 0.02398073576751127, 0.08609101632583506, 0.34206529869278834, 1.3172414550760627]
js_flops_wt_log_list:
[306.8941819928494, 242.21548800967824, 100.77710143308975, 24.852250229958916, 5.843974188655556, 1.4167636623471473, 0.34895433671846743, 0.08659136179397035, 0.02155777930631189, 0.0054756742294235425, 0.0013496281230642772, 0.0004119398286675554, 0.00011378733205517517, 6.360240584014263e-05, 6.185355921321843e-05, 306.68760591679046, 195.95723987543016, 13.136183772694684, 12.869101493064464, 12.868479803128588, 12.868109792686232, 12.869123625633044, 135.17861133847896, 33.01564623777243, 36.52447177148207, 9.220458330419321, 13.556344310807189, 3.6496632380059837, 3.832635737224184, 7.779854210273969, 0.9534552365733172, 3.6122458697820146, 2.458179552273165, 5.854851326384168, 0.24491896219961773, 0.9393001573371668, 3.6122634048943922, 2.0648060206203844, 5.092895790788852, 0.06585249099362203, 0.23641092300885574, 0.9393311456237781, 3.6172214187998737]
js_flops_wt_cbrt_list:
[919.669015715166, 725.8465377294758, 301.9984839066013, 74.47467514332305, 17.51262260837163, 4.245612068599507, 1.0457105745552682, 0.2594881139595929, 0.06460214249386642, 0.016408939055964845, 0.004044427168544208, 0.0012344590383058871, 0.0003409862089678714, 0.00019059716804107867, 0.00018535640379596947, 919.0499697127109, 587.224563034227, 39.365168547799776, 38.56480375877875, 38.562940742126905, 38.56183193277406, 38.5648700833908, 405.0894012656733, 98.93790322619982, 109.45279179120422, 27.630924060258064, 40.62426256522004, 10.936936555763687, 11.485249779449587, 23.31386935232568, 2.8572168858133784, 10.82480802344113, 7.3664204209087405, 17.545218099376743, 0.7339480319680696, 2.8147983958180745, 10.824860570867857, 6.187598957711563, 15.261867881105113, 0.19733999250564907, 0.7084520125341904, 2.8148912583378576, 10.83971823854584]
js_param_list:
[329.83768296470447, 262.1417987881206, 109.55299249322502, 26.93282115565066, 6.316488838356098, 1.5287235827582208, 0.37620206575502757, 0.09333173232737417, 0.02321419905363232, 0.005836263908306144, 0.0014272749789561508, 0.0003676273824454489, 8.131745788957567e-05, 3.453414103245251e-05, 2.0099298974261437e-05, 329.8244057909673, 212.4179431572911, 14.133527084977164, 13.85056402492579, 13.850298724802842, 13.850215263119646, 13.850486422094308, 146.66402164618708, 35.70100773677067, 39.574763597496606, 9.920061084599604, 14.70086921946221, 3.9319568358919845, 4.104552466888791, 8.444664426043065, 1.0270930440949402, 3.8925516097678514, 2.6252280884444676, 6.358440551995419, 0.26367797100700413, 1.0122385924908912, 3.892559286273842, 2.2027063731430716, 5.532037851339746, 0.07087185402434304, 0.25474678164834264, 1.0122448873828584, 3.8940094241892598]
ptq_acc_list:
[1.0, 1.3, 8.17, 54.06, 72.3, 74.86, 75.73, 75.66, 75.72, 75.75, 75.74, 75.75, 75.72, 75.72, 75.75, 1.0, 1.0, 17.57, 12.17, 17.16, 17.58, 16.33, 0.98, 27.12, 16.22, 53.44, 52.97, 59.31, 60.67, 64.39, 71.15, 60.59, 63.83, 68.9, 74.37, 71.45, 59.82, 64.84, 70.03, 75.38, 74.2, 71.33, 42.82]
acc_loss_list:
[0.986795193450416, 0.9828337514855408, 0.8921167304898983, 0.28614815792948634, 0.045292486465073376, 0.011488181698138181, 0.0, 0.0009243364584709809, 0.00013204806549590802, -0.0002640961309916284, -0.00013204806549572037, -0.0002640961309916284, 0.00013204806549590802, 0.00013204806549590802, -0.0002640961309916284, 0.986795193450416, 0.986795193450416, 0.7679915489238083, 0.8392975042915621, 0.7734055196091378, 0.7678595008583124, 0.7843655090452926, 0.9870592895814075, 0.6418856463752806, 0.7858180377657468, 0.29433513799022853, 0.300541397068533, 0.2168229235441701, 0.19886438663673578, 0.14974250627228314, 0.06047801399709492, 0.1999207711607025, 0.15713719794005024, 0.09018882873365902, 0.0179585369074343, 0.05651657203221974, 0.21008847220388224, 0.1438003433249703, 0.0752673973326291, 0.00462168229235453, 0.02020335402086361, 0.058101148818169884, 0.43457018354681104]
ykl/ALL/ptq_result/cifar100/ResNet_50.txt 0 → 100644
ResNet_50
Full_acc: 77.220000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[16.230424870213202, 12.204187758693397, 5.29429635921478, 1.4908381553301968, 0.37157801037319915, 0.09192073799491478, 0.022930675380617136, 0.005731395335657435, 0.0014247125738302258, 0.00035592105675043935, 9.548260041071398e-05, 2.870062295469733e-05, 1.1098739056292632e-05, 1.0766992929636802e-05, 8.956828437643308e-06, 16.211792687445943, 9.911577435363476, 0.68758256159107, 0.661879016508269, 0.6619088164187588, 0.6618630686394162, 0.6620601794295361, 7.0007053819521, 1.9351723520873367, 1.8967959925020923, 0.5538402277504298, 0.6782420491659223, 0.1878712898930471, 0.22001917846113037, 0.37781511542137663, 0.04923951600613683, 0.1841581381401746, 0.13399604203141785, 0.2803057866278194, 0.012898884164729169, 0.04784685495564067, 0.18416269296757956, 0.10979746411287197, 0.24261402057227638, 0.0036747320386184604, 0.012081043881936837, 0.04785413704937075, 0.18624724831630168]
js_flops_wt_log_list:
[50.76181173112005, 38.16946791546908, 16.55828958171227, 4.662702693712519, 1.162136737443321, 0.28748866610675383, 0.07171732322759744, 0.017925347815090226, 0.004455890219883035, 0.0011131684979525595, 0.00029862864493101593, 8.976324591884024e-05, 3.471209823116324e-05, 3.367453855182686e-05, 2.801311995806178e-05, 50.703538250213995, 30.999165564402713, 2.1504635165246087, 2.0700738454747736, 2.0701670468208864, 2.070023967377146, 2.070640445435992, 21.8952055430095, 6.052389594816304, 5.932364792276039, 1.732174825674706, 2.1212503975219326, 0.5875808627003773, 0.6881256777680949, 1.1816437284638612, 0.15400009927171776, 0.5759677156807752, 0.4190821813172123, 0.8766763459198321, 0.04034218048802728, 0.14964445247775793, 0.5759819612284456, 0.3433994024445278, 0.7587926584856064, 0.011492986622242065, 0.03778432665528994, 0.14966722774543517, 0.5825015568028737]
js_flops_wt_cbrt_list:
[179.00025255724583, 134.59614942498237, 58.389129858921805, 16.441985250542505, 4.098017041044216, 1.013764916686499, 0.2528952086751361, 0.06320975703297317, 0.015712707004061577, 0.003925341422558433, 0.0010530475773131054, 0.00031653014622368445, 0.0001224045032727016, 0.00011874577955282762, 9.878204454108128e-05, 178.79476407202736, 109.31167103498355, 7.583131875040239, 7.299655558293794, 7.299984212136593, 7.2994796742627095, 7.301653547189349, 77.20857842409859, 21.34240739447029, 20.919166591292463, 6.108129727745521, 7.480118300442819, 2.0719733838162444, 2.426522338565419, 4.166804111486148, 0.5430471396390045, 2.0310222006609986, 1.477800215157279, 3.09140438410335, 0.14225773765359542, 0.5276879188078264, 2.0310724344202726, 1.2109217080582222, 2.6757137477076927, 0.04052746420705786, 0.1332380343283926, 0.5277682307298649, 2.054062339914097]
js_param_list:
[40.049153605609696, 31.39668030594216, 13.77279867201038, 3.7750910178593595, 0.9170836138077584, 0.22357907670487892, 0.055217801968629146, 0.013710141941906864, 0.003432156112417536, 0.0008518860902329481, 0.00023256616362080597, 5.744133253044784e-05, 2.3188684723552593e-05, 1.0295064945079603e-05, 9.884394929287985e-06, 40.04755820488529, 25.65781606298512, 1.671142319907154, 1.6161062312111112, 1.616222272272229, 1.6160775260317466, 1.616476549384266, 17.96723061318741, 4.813296913597177, 4.859380577109968, 1.31499808442326, 1.7503227182262473, 0.4602354426497577, 0.49841330645385123, 0.9844889370734478, 0.1203035827269011, 0.45247572595218377, 0.2932056876912121, 0.7343828456239584, 0.03140441293544373, 0.11739646772009306, 0.45247647003816643, 0.23677366819451418, 0.6369008096290816, 0.00879301422182398, 0.02966711507172892, 0.11739732521037508, 0.4544248629851162]
ptq_acc_list:
[1.0, 0.93, 2.13, 50.78, 74.06, 76.35, 76.92, 77.08, 77.16, 77.23, 77.26, 77.25, 77.22, 77.22, 77.23, 1.0, 1.46, 6.58, 10.99, 11.21, 8.87, 8.82, 1.04, 3.09, 5.61, 39.54, 46.2, 61.59, 57.67, 67.05, 72.0, 60.39, 61.47, 70.72, 76.11, 72.7, 60.55, 63.06, 72.38, 77.06, 76.11, 72.17, 56.86]
acc_loss_list:
[0.987049987049987, 0.9879564879564878, 0.9724164724164724, 0.3423983423983424, 0.04092204092204088, 0.011266511266511325, 0.003885003885003848, 0.0018130018130018204, 0.0007770007770008065, -0.00012950012950019575, -0.000518000518000599, -0.00038850038850040325, 0.0, 0.0, -0.00012950012950019575, 0.987049987049987, 0.9810929810929812, 0.9147889147889148, 0.8576793576793578, 0.8548303548303547, 0.885133385133385, 0.8857808857808859, 0.9865319865319865, 0.9599844599844599, 0.9273504273504274, 0.48795648795648794, 0.40170940170940167, 0.20240870240870235, 0.2531727531727531, 0.13170163170163174, 0.06759906759906759, 0.21794871794871792, 0.20396270396270397, 0.08417508417508418, 0.014374514374514367, 0.058534058534058485, 0.2158767158767159, 0.18337218337218333, 0.06267806267806272, 0.002072002072002028, 0.014374514374514367, 0.06539756539756536, 0.26366226366226364]
ykl/ALL/ptq_result/cifar100/VGG_16.txt 0 → 100644
VGG_16
Full_acc: 71.120000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[17.085091741976537, 10.737790353470139, 2.894559789187618, 0.6980086345515658, 0.1735258041248935, 0.04306407130299703, 0.010661987353550572, 0.0026460739045473755, 0.0006612781375472983, 0.00018993777483660915, 6.312506202859814e-05, 1.4584962500067828e-05, 4.039319183628637e-05, 4.244454054819202e-05, 5.737979970726031e-05, 16.659716127015596, 7.51794604216787, 0.6892428468558738, 0.6838481955811394, 0.6799609280072476, 0.6775429548584105, 0.6778664314152565, 4.4673742785800306, 1.0965438484427747, 1.3545090072143018, 0.38027145012718827, 0.49868859247230296, 0.1853740828776368, 0.19811604513819048, 0.2815195242548014, 0.0482649761621508, 0.18411857523338, 0.14729052151548197, 0.20940355467255276, 0.01239995188485554, 0.047822924478066936, 0.18415117919637156, 0.13076871472442161, 0.18109643332552047, 0.003281099053322425, 0.012141069956625443, 0.0478168013309649, 0.18540608661872723]
js_flops_wt_log_list:
[43.11244020026036, 27.09568970938158, 7.304118567348855, 1.7613517077250642, 0.4378741985421196, 0.10866767512103638, 0.026904408775665356, 0.006677090453953316, 0.0016686661442210822, 0.0004792880882983481, 0.00015928948482958638, 3.6803625822056625e-05, 0.00010192799042810856, 0.00010710435412621077, 0.0001447919168911639, 42.039049372770954, 18.97074970779466, 1.739232160253733, 1.7256193225824068, 1.7158102098569499, 1.709708707777862, 1.7105249670010387, 11.272951257514373, 2.767013593307365, 3.4179616624923597, 0.9595751899414768, 1.2583884503640903, 0.46777208945883336, 0.49992509713876426, 0.7103850443378872, 0.12179161399791187, 0.4646039473703128, 0.3716722097136807, 0.5284079456454802, 0.03128998030537321, 0.12067614285597668, 0.4646862200572842, 0.32998116011107315, 0.4569778886843545, 0.008279509929697877, 0.03063671886444535, 0.12066069173535875, 0.46785284754868056]
js_flops_wt_cbrt_list:
[118.50838567942922, 74.48120383383315, 20.077715299966496, 4.841640754422958, 1.2036378400030139, 0.29870799922971936, 0.07395540676550048, 0.018354127185979976, 0.004586864720971276, 0.0013174772143673265, 0.00043785829832819836, 0.00010116658354437908, 0.00028018181167834965, 0.0002944107094791285, 0.0003980070775510936, 115.55782631470767, 52.14719724878091, 4.780838075414054, 4.743418819870842, 4.716455323751353, 4.699683386040251, 4.701927136623268, 30.987326376456235, 7.606025373945598, 9.395365167330555, 2.637700537704877, 3.4590847355790886, 1.2858217936669962, 1.3742046598927844, 1.9527214053354454, 0.33478335944662085, 1.2771131378180203, 1.0216604156590647, 1.4524989151101513, 0.08601055836103007, 0.3317171288252808, 1.2773392907166545, 0.9070591784584807, 1.2561504667242127, 0.022758891666247515, 0.08421485952256556, 0.3316746564587698, 1.286043783209084]
js_param_list:
[9.916218225439012, 6.831230002329063, 2.236457302571373, 0.6336840228224254, 0.1715918605221316, 0.04376155997323663, 0.010956176784660656, 0.0027281584863837905, 0.0006961723657893088, 0.00018288844981070661, 5.6039130730449534e-05, 1.721553433174565e-05, 2.1350037613513175e-05, 1.0913037521762206e-05, 7.180516631236933e-05, 9.914057032541649, 5.028618075456426, 0.40764191231362745, 0.39307882833608027, 0.39338364455575237, 0.3934834575726964, 0.3934762329555188, 3.1764720071659625, 0.8709418556460512, 0.9547245500418231, 0.28441481512852407, 0.3420416262434062, 0.11137141938346057, 0.1255291491963517, 0.1876768559757988, 0.02887925286961113, 0.10935800529406944, 0.08230194805704694, 0.13786717133836088, 0.007533318011244774, 0.028575264180214287, 0.10970283789838954, 0.06959342086207226, 0.11903771558622628, 0.0021533746488378892, 0.007354876507847977, 0.028523505859202897, 0.109801740087647]
ptq_acc_list:
[1.0, 1.04, 14.77, 55.52, 69.06, 70.61, 71.17, 71.09, 71.08, 71.14, 71.11, 71.11, 71.11, 71.12, 71.11, 1.0, 1.4, 26.27, 22.67, 31.1, 26.62, 30.99, 1.09, 18.72, 42.31, 51.42, 62.98, 61.7, 58.27, 67.99, 69.24, 61.62, 61.04, 68.82, 70.56, 68.83, 61.38, 62.45, 69.46, 71.19, 70.7, 68.93, 61.82]
acc_loss_list:
[0.9859392575928009, 0.9853768278965128, 0.7923228346456693, 0.21934758155230596, 0.028965129358830177, 0.007170978627671612, -0.000703037120359915, 0.000421822272215989, 0.0005624296962880519, -0.00028121484814392603, 0.00014060742407206292, 0.00014060742407206292, 0.00014060742407206292, 0.0, 0.00014060742407206292, 0.9859392575928009, 0.9803149606299212, 0.6306242969628797, 0.6812429696287964, 0.562710911136108, 0.6257030371203599, 0.5642575928008999, 0.9846737907761529, 0.7367829021372329, 0.40508998875140606, 0.2769966254218223, 0.11445444319460077, 0.13245219347581555, 0.18068053993250843, 0.044010123734533314, 0.026434195725534442, 0.13357705286839155, 0.14173228346456698, 0.032339707536558085, 0.007874015748031527, 0.03219910011248602, 0.13695163104611927, 0.12190663667041621, 0.023340832395950657, -0.000984251968503841, 0.005905511811023645, 0.030793025871765994, 0.13076490438695168]
ykl/ALL/ptq_result/cifar100/VGG_19.txt 0 → 100644
VGG_19
Full_acc: 70.450000
title_list:
['INT_2', 'INT_3', 'INT_4', 'INT_5', 'INT_6', 'INT_7', 'INT_8', 'INT_9', 'INT_10', 'INT_11', 'INT_12', 'INT_13', 'INT_14', 'INT_15', 'INT_16', 'POT_2', 'POT_3', 'POT_4', 'POT_5', 'POT_6', 'POT_7', 'POT_8', 'FLOAT_3_E1', 'FLOAT_4_E1', 'FLOAT_4_E2', 'FLOAT_5_E1', 'FLOAT_5_E2', 'FLOAT_5_E3', 'FLOAT_6_E1', 'FLOAT_6_E2', 'FLOAT_6_E3', 'FLOAT_6_E4', 'FLOAT_7_E1', 'FLOAT_7_E2', 'FLOAT_7_E3', 'FLOAT_7_E4', 'FLOAT_7_E5', 'FLOAT_8_E1', 'FLOAT_8_E2', 'FLOAT_8_E3', 'FLOAT_8_E4', 'FLOAT_8_E5', 'FLOAT_8_E6']
js_flops_list:
[12.933096217004081, 8.282459855818454, 2.3511338583732004, 0.5607665105783252, 0.13750860793240308, 0.034055243514816445, 0.008466258330629493, 0.002134299490155856, 0.0005518020792059293, 0.0001387761061248467, 4.1164875592213196e-05, 2.0749240583301132e-05, 1.2657347050655076e-05, 1.619567712066784e-05, 3.111866443620645e-05, 12.766474700217593, 5.913951424122345, 0.5312434033572428, 0.5240513815367216, 0.5250262623839118, 0.524969353101955, 0.5245434967946389, 3.598951343967536, 0.8750073227860701, 1.0683690264640484, 0.2987686569023001, 0.3948113124704691, 0.14247788072680156, 0.15534309310691274, 0.22380186120048573, 0.03708847445554712, 0.1414851885762422, 0.11450165272776922, 0.16679070383810646, 0.009543419974557965, 0.03673265812289985, 0.14151062836765657, 0.10154254677462951, 0.14450689459197366, 0.002492146654501212, 0.009299251796759863, 0.03672766674028512, 0.14268339780267283]
js_flops_wt_log_list:
[33.91026349361435, 21.716408149567414, 6.164615750818413, 1.4703161418612436, 0.3605442230838099, 0.08929201960225876, 0.02219832327692665, 0.005596081315030784, 0.0014468116209941759, 0.0003638675724939018, 0.00010793330186305248, 5.440400377962453e-05, 3.31872558910926e-05, 4.2464671212858927e-05, 8.159238073352639e-05, 33.4733859321139, 15.506236690387817, 1.3929072733104273, 1.3740499671485007, 1.3766060810015497, 1.3764568662494123, 1.3753402813768338, 9.436362826573575, 2.294247902950426, 2.8012375836328856, 0.7833641464655517, 1.0351856517181814, 0.3735735354002334, 0.4073058091257852, 0.5868030328029054, 0.09724507730099655, 0.370970720743272, 0.3002205465190931, 0.43732116583184005, 0.025022614889541793, 0.09631213553724963, 0.3710374232571642, 0.2662420861303997, 0.37889356036917476, 0.006534347870060304, 0.024382411870329482, 0.09629904825352244, 0.3741123961709489]
js_flops_wt_cbrt_list:
[96.7596643074306, 61.965674873360506, 17.590136117515375, 4.195405215270027, 1.0287781455942455, 0.25478616064643805, 0.06334077318118503, 0.015967877972444203, 0.004128337333322346, 0.001038261002410069, 0.00030797726056679467, 0.00015523657442791447, 9.469663188879727e-05, 0.00012116884117563355, 0.00023281598420215054, 95.513076347352, 44.24554993847292, 3.974526479343794, 3.9207189760693337, 3.9280126002665967, 3.927586830371614, 3.9244007631957634, 26.92575065228598, 6.54641498050621, 7.993061106373206, 2.2352539919590195, 2.95380235470544, 1.065956030871699, 1.162207818553189, 1.6743858236101101, 0.27747944326550134, 1.0585291504374177, 0.8566503561628492, 1.2478537422273757, 0.07139961673438815, 0.27481738397788974, 1.0587194796235617, 0.7596960985967759, 1.0811362087018963, 0.01864513103810209, 0.06957285920302152, 0.27478004067678624, 1.0674936180771977]
js_param_list:
[5.441795197362784, 3.615214283911254, 1.1790201233925872, 0.341252094418348, 0.09401382071186312, 0.024464816483120978, 0.006208213391891069, 0.0015985597051018019, 0.0004015855469499279, 0.00011155739733848337, 2.4537871906840675e-05, 2.351879757638181e-05, 6.264552940483891e-06, 1.5177568923359899e-05, 9.624882805967266e-06, 5.45133398076672, 2.650677666315291, 0.2256276208836905, 0.21693865571525875, 0.21749322883768124, 0.21758511964463856, 0.21728026335671044, 1.7035465065053872, 0.4813913827879723, 0.5064766680531055, 0.1623796404313223, 0.17965458985791877, 0.0599836624875662, 0.07432381074360364, 0.09737465344971145, 0.015601066392873146, 0.05887208831188401, 0.04897613855857195, 0.07075391119873932, 0.004133567706307015, 0.015265638217273907, 0.05909795862509216, 0.04153595037687257, 0.06084168404200329, 0.0011923608298122395, 0.0038164235757733693, 0.01519637035247201, 0.061059609830074656]
ptq_acc_list:
[1.0, 0.99, 8.49, 51.4, 68.16, 69.66, 70.64, 70.57, 70.43, 70.44, 70.43, 70.48, 70.43, 70.46, 70.45, 1.0, 1.02, 18.99, 17.97, 21.57, 25.97, 25.82, 1.11, 24.97, 40.24, 49.39, 60.11, 61.9, 56.17, 66.15, 68.72, 62.13, 59.45, 67.82, 70.01, 68.48, 61.86, 60.86, 68.44, 70.47, 69.91, 68.29, 61.0]
acc_loss_list:
[0.985805535841022, 0.9859474804826118, 0.8794889992902768, 0.27040454222853094, 0.03250532292405971, 0.011213626685592707, -0.0026969481902057875, -0.0017033356990772226, 0.0002838892831795035, 0.0001419446415898526, 0.0002838892831795035, -0.00042583392476935606, 0.0002838892831795035, -0.0001419446415896509, 0.0, 0.985805535841022, 0.9855216465578425, 0.7304471256210079, 0.7449254790631654, 0.6938254080908446, 0.6313697657913414, 0.6334989354151881, 0.9842441447835344, 0.6455642299503194, 0.42881476224272536, 0.29893541518807665, 0.14677075940383255, 0.12136266855926195, 0.20269694819020584, 0.06103619588360535, 0.024556422995031992, 0.11809794180269695, 0.15613910574875797, 0.03733144073811227, 0.006245564229950287, 0.02796309439318664, 0.12193044712562105, 0.13612491128459905, 0.028530872959545848, -0.0002838892831795035, 0.007665010645848207, 0.030660042583392427, 0.13413768630234213]
ykl/ALL/scripts/gen_full_all_cifar10.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
if [ ! -d "ckpt_full_gen/cifar10" ]; then
mkdir -p "ckpt_full_gen/cifar10"
fi
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar10,Quant=False gen_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/gen_full_all_cifar100.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
if [ ! -d "ckpt_full_gen/cifar100" ]; then
mkdir -p "ckpt_full_gen/cifar100"
fi
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar100,Quant=False gen_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/gen_full_one_cifar10.sh 0 → 100644
#!/bin/bash
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
if [ ! -d "ckpt_full_gen/cifar10/$1" ]; then
mkdir -p "ckpt_full_gen/cifar10/$1"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar10,Quant=False gen_one.slurm
ykl/ALL/scripts/gen_full_one_cifar100.sh 0 → 100644
#!/bin/bash
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
if [ ! -d "ckpt_full_gen/cifar100/$1" ]; then
mkdir -p "ckpt_full_gen/cifar100/$1"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar100,Quant=False gen_one.slurm
ykl/ALL/scripts/gen_quant_all_cifar10.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
if [ ! -d "ckpt_quant_gen/cifar10" ]; then
mkdir -p "ckpt_quant_gen/cifar10"
fi
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar10,Quant=True gen_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/gen_quant_all_cifar100.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
if [ ! -d "ckpt_quant_gen/cifar100" ]; then
mkdir -p "ckpt_quant_gen/cifar100"
fi
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar100,Quant=True gen_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/gen_quant_one_cifar10.sh 0 → 100644
#!/bin/bash
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
if [ ! -d "ckpt_quant_gen/cifar10/$1" ]; then
mkdir -p "ckpt_quant_gen/cifar10/$1"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar10,Quant=True gen_one.slurm
ykl/ALL/scripts/gen_quant_one_cifar100.sh 0 → 100644
#!/bin/bash
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
if [ ! -d "ckpt_quant_gen/cifar100/$1" ]; then
mkdir -p "ckpt_quant_gen/cifar100/$1"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar100,Quant=True gen_one.slurm
ykl/ALL/scripts/ptflops_cifar10.sh 0 → 100644
#!/bin/bash
if [ ! -d "param_flops/cifar10" ]; then
mkdir -p "param_flops/cifar10"
fi
sbatch --export=Dataset=cifar10 get_param_flops.slurm
\ No newline at end of file
ykl/ALL/scripts/ptflops_cifar100.sh 0 → 100644
#!/bin/bash
if [ ! -d "param_flops/cifar100" ]; then
mkdir -p "param_flops/cifar100"
fi
sbatch --export=Dataset=cifar100 get_param_flops.slurm
\ No newline at end of file
ykl/ALL/scripts/ptq_all_cifar10.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
if [ ! -d "ckpt_quant/cifar10/$name" ]; then
mkdir -p "ckpt_quant/cifar10/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar10 ptq_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/ptq_all_cifar100.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
if [ ! -d "ckpt_quant/cifar100/$name" ]; then
mkdir -p "ckpt_quant/cifar100/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar100 ptq_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/ptq_one_cifar10.sh 0 → 100644
#!/bin/bash
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
if [ ! -d "ckpt_quant/cifar10/$1" ]; then
mkdir -p "ckpt_quant/cifar10/$1"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar10 ptq_one.slurm
ykl/ALL/scripts/ptq_one_cifar100.sh 0 → 100644
#!/bin/bash
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
if [ ! -d "ckpt_quant/cifar100/$1" ]; then
mkdir -p "ckpt_quant/cifar100/$1"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar100 ptq_one.slurm
ykl/ALL/scripts/train_all_cifar10.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
if [ ! -d "ckpt_full/cifar10" ]; then
mkdir -p "ckpt_full/cifar10"
fi
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar10 train_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/train_all_cifar100.sh 0 → 100644
#!/bin/bash
name_list="ResNet_152 ResNet_50 ResNet_18 MobileNetV2 Inception_BN VGG_19 VGG_16 AlexNet_BN AlexNet"
if [ ! -d "ckpt_full/cifar100" ]; then
mkdir -p "ckpt_full/cifar100"
fi
for name in $name_list; do
if [ ! -d "ret_one/$name" ]; then
mkdir -p "ret_one/$name"
fi
sbatch --job-name=$name -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$name,Dataset=cifar100 train_one.slurm
done
\ No newline at end of file
ykl/ALL/scripts/train_one_cifar10.sh 0 → 100644
#!/bin/bash
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
if [ ! -d "ckpt_full/cifar10" ]; then
mkdir -p "ckpt_full/cifar10"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar10 train_one.slurm
\ No newline at end of file
ykl/ALL/scripts/train_one_cifar100.sh 0 → 100644
#!/bin/bash
if [ ! -d "ckpt_full/cifar100" ]; then
mkdir -p "ckpt_full/cifar100"
fi
if [ ! -d "ret_one/$1" ]; then
mkdir -p "ret_one/$1"
fi
sbatch --job-name=$1 -o "ret_one/%x/%j.out" -e "ret_one/%x/%j.err" --export=Model=$1,Dataset=cifar100 train_one.slurm
\ No newline at end of file
ykl/ALL/sync.sh 0 → 100755
#!/bin/bash
# 指定本地源目录和远程目标目录
local_dir="/lustre/S/youkunlin/WorkSpace/ALL/"
remote_dir="/lustre/S/youkunlin/Model-Transfer-Adaptability/ykl/ALL"
# 读取.gitignore文件，排除指定的文件和文件夹
exclude_args=$(while read line; do echo "--exclude=$line"; done < ${local_dir}/.gitignore)
# 使用rsync同步文件夹
rsync -avz --delete ${exclude_args} ${local_dir}/ ${remote_dir}
\ No newline at end of file
ykl/ALL/train_one.py 0 → 100644
from model import *
from dataloader import DataLoader
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.optim.lr_scheduler import _LRScheduler,MultiStepLR
import os
import os.path as osp
import time
import sys
class WarmUpLR(_LRScheduler):
"""warmup_training learning rate scheduler
Args:
optimizer: optimzier(e.g. SGD)
total_iters: totoal_iters of warmup phase
"""
def __init__(self, optimizer, total_iters, last_epoch=-1):
self.total_iters = total_iters
super().__init__(optimizer, last_epoch)
def get_lr(self):
"""we will use the first m batches, and set the learning
rate to base_lr * m / total_iters
"""
return [base_lr * self.last_epoch / (self.total_iters + 1e-8) for base_lr in self.base_lrs]
def train(model, device, train_loader, optimizer, epoch):
model.train()
total_loss = 0.
lossLayer = nn.CrossEntropyLoss()
#对第一个epoch，使用warmup策略
if epoch == 1:
warmup_scheduler = WarmUpLR(optimizer, len(train_loader))
else:
warmup_scheduler = None
start_time = time.time()
for batch_idx, (data, targets) in enumerate(train_loader):
data,targets = data.to(device), targets.to(device)
optimizer.zero_grad()
output = model(data)
loss = lossLayer(output, targets)
loss.backward()
total_loss += loss.item()
optimizer.step()
if warmup_scheduler is not None:
warmup_scheduler.step()
if batch_idx % 50 == 0 and batch_idx > 0:
cur_loss = total_loss / 50
elapsed = time.time() - start_time
lr = optimizer.param_groups[0]['lr']
print('| epoch {:3d} | {:5d}/{:5d} batches | lr {:02.7f} | ms/batch {:5.2f} | '
'loss {:5.2f}'.format(
epoch, batch_idx, len(train_loader.dataset) // len(data), lr,
elapsed * 1000 / 50, cur_loss))
total_loss = 0.
start_time = time.time()
def evaluate(model, device, eval_loader):
model.eval()
total_loss = 0
correct = 0
lossLayer = nn.CrossEntropyLoss()
with torch.no_grad():
for data, targets in eval_loader:
data,targets = data.to(device), targets.to(device)
output = model(data)
total_loss += len(data) * lossLayer(output, targets).item()
pred = output.argmax(dim=1, keepdim=True)
correct += pred.eq(targets.view_as(pred)).sum().item()
test_loss = total_loss / len(eval_loader.dataset)
test_acc = 100. * correct / len(eval_loader.dataset)
return test_loss,test_acc
if __name__ == "__main__":
model_name = sys.argv[1]
dataset = sys.argv[2]
batch_size = 128
optim_type = 'sgd'
lr = 0.1
momentum = 0.9
weight_decay = 5e-4
nesterov = True
epochs = 200
milestones = [60, 120, 160]
gamma = 0.2
print('model: '+model_name+' dataset: '+dataset)
print('optim_type: '+optim_type+' lr: '+str(lr)+' weight_decay: '+str(weight_decay)+' nesterov: '+str(nesterov)+' momentum: '+str(momentum))
print('epochs: '+str(epochs)+' milestones: '+str(milestones)+' gamma: '+str(gamma))
save_model = True
append = False
ckpt_path = 'ckpt_full/'+dataset
if save_model:
if not osp.exists(ckpt_path):
os.makedirs(ckpt_path)
save_path = ckpt_path+'/'+model_name+'.pt'
if os.path.exists(save_path) and append:
print('Append: Model '+model_name+' exists!')
sys.exit()
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
dataloader = DataLoader(dataset,batch_size)
train_loader, val_loader, test_loader = dataloader.getloader()
print('>>>>>>>>>>>>>>>>>>>>>>>> Train: '+model_name+' <<<<<<<<<<<<<<<<<<<<<<<<')
model = Model(model_name,dataset).to(device)
best_val_acc = None
optimizer = optim.SGD(model.parameters(), lr=lr, momentum=momentum,weight_decay = weight_decay,nesterov=nesterov)
lr_scheduler = MultiStepLR(optimizer, milestones=milestones, gamma=gamma)
for epoch in range(1, epochs + 1):
epoch_start_time = time.time()
train(model, device, train_loader, optimizer, epoch)
val_loss, val_acc = evaluate(model, device, val_loader)
if not best_val_acc or val_acc > best_val_acc:
best_val_acc = val_acc
if save_model:
torch.save(model.state_dict(), save_path)
print('-' * 89)
print('| end of epoch {:3d} | time: {:5.2f}s | val loss {:5.2f} | '
'val acc {:.2f} | best val acc {:.2f}'.format(epoch, (time.time() - epoch_start_time),
val_loss, val_acc, best_val_acc))
print('-' * 89)
lr_scheduler.step(epoch)
print('>>>>>>>>>>>>>>>>>>>>>>>> Test: '+model_name+' <<<<<<<<<<<<<<<<<<<<<<<<')
model = Model(model_name,dataset).to(device)
model.load_state_dict(torch.load(save_path))
test_loss,test_acc = evaluate(model, device, test_loader)
print('=' * 89)
print('| Test on {:s} | test loss {:5.2f} | test acc {:.2f}'.format(
model_name, test_loss, test_acc))
print('=' * 89)
\ No newline at end of file
ykl/ALL/train_one.slurm 0 → 100644
#!/bin/bash
#- Job parameters
# (TODO)
# Please modify job name
#- Resources
# (TODO)
# Please modify your requirements
#SBATCH -p nv-gpu # Submit to 'nv-gpu' Partitiion
#SBATCH -t 1-06:00:00 # Run for a maximum time of 0 days, 12 hours, 00 mins, 00 secs
#SBATCH --nodes=1 # Request N nodes
#SBATCH --gres=gpu:1 # Request M GPU per node
#SBATCH --gres-flags=enforce-binding # CPU-GPU Affinity
#SBATCH --qos=gpu-normal # Request QOS Type
###
### The system will alloc 8 or 16 cores per gpu by default.
### If you need more or less, use following:
### #SBATCH --cpus-per-task=K # Request K cores
###
###
### Without specifying the constraint, any available nodes that meet the requirement will be allocated
### You can specify the characteristics of the compute nodes, and even the names of the compute nodes
###
### #SBATCH --nodelist=gpu-v00 # Request a specific list of hosts
### #SBATCH --constraint="Volta|RTX8000" # Request GPU Type: Volta(V100 or V100S) or RTX8000
###
# set constraint for RTX8000 to meet my cuda
#SBATCH --constraint="Ampere|RTX8000"
#- Log information
echo "Job start at $(date "+%Y-%m-%d %H:%M:%S")"
echo "Job run at:"
echo "$(hostnamectl)"
#- Load environments
source /tools/module_env.sh
module list # list modules loaded
##- Tools
module load cluster-tools/v1.0
module load slurm-tools/v1.0
module load cmake/3.15.7
module load git/2.17.1
module load vim/8.1.2424
##- language
module load python3/3.6.8
##- CUDA
# module load cuda-cudnn/10.2-7.6.5
# module load cuda-cudnn/11.2-8.2.1
module load cuda-cudnn/11.1-8.2.1
##- virtualenv
# source xxxxx/activate
echo $(module list) # list modules loaded
echo $(which gcc)
echo $(which python)
echo $(which python3)
cluster-quota # nas quota
nvidia-smi --format=csv --query-gpu=name,driver_version,power.limit # gpu info
#- Warning! Please not change your CUDA_VISIBLE_DEVICES
#- in `.bashrc`, `env.sh`, or your job script
echo "Use GPU ${CUDA_VISIBLE_DEVICES}" # which gpus
#- The CUDA_VISIBLE_DEVICES variable is assigned and specified by SLURM
#- Job step
# [EDIT HERE(TODO)]
python train_one.py $Model $Dataset
#- End
echo "Job end at $(date "+%Y-%m-%d %H:%M:%S")"
ykl/ALL/utils.py 0 → 100644
import torch
import torch.nn as nn
def ebit_list(quant_type, num_bits):
if quant_type == 'FLOAT':
e_bit_list = list(range(1,num_bits-1))
else:
e_bit_list = [0]
return e_bit_list
def numbit_list(quant_type):
if quant_type == 'INT':
num_bit_list = list(range(2,17))
# num_bit_list = [4,5]
elif quant_type == 'POT':
num_bit_list = list(range(2,9))
# num_bit_list = [5]
else:
num_bit_list = list(range(2,9))
# num_bit_list = [8]
return num_bit_list
def build_bias_list(quant_type):
if quant_type == 'POT':
return build_pot_list(8)
else:
return build_float_list(16,7)
def build_list(quant_type, num_bits, e_bits):
if quant_type == 'POT':
return build_pot_list(num_bits)
else:
return build_float_list(num_bits,e_bits)
def build_pot_list(num_bits):
plist = [0.]
for i in range(-2 ** (num_bits-1) + 2, 1):
# i最高到0，即pot量化最大值为1
plist.append(2. ** i)
plist.append(-2. ** i)
plist = torch.Tensor(list(set(plist)))
# plist = plist.mul(1.0 / torch.max(plist))
return plist
def build_float_list(num_bits,e_bits):
m_bits = num_bits - 1 - e_bits
plist = [0.]
# 相邻尾数的差值
dist_m = 2 ** (-m_bits)
e = -2 ** (e_bits - 1) + 1
for m in range(1, 2 ** m_bits):
frac = m * dist_m # 尾数部分
expo = 2 ** e # 指数部分
flt = frac * expo
plist.append(flt)
plist.append(-flt)
for e in range(-2 ** (e_bits - 1) + 2, 2 ** (e_bits - 1) + 1):
expo = 2 ** e
for m in range(0, 2 ** m_bits):
frac = 1. + m * dist_m
flt = frac * expo
plist.append(flt)
plist.append(-flt)
plist = torch.Tensor(list(set(plist)))
return plist
#此处不必cfg，直接取同前缀同后缀即可。将relu一起考虑进去
def fold_ratio(layer, par_ratio, flop_ratio):
idx = -1
for name in layer:
if 'conv' in name:
conv_idx = layer.index(name)
[prefix,suffix] = name.split('conv')
bn_name = prefix+'bn'+suffix
relu_name = prefix+'relu'+suffix
relus_name = prefix+'relus'+suffix
if bn_name in layer:
bn_idx = layer.index(bn_name)
par_ratio[conv_idx]+=par_ratio[bn_idx]
flop_ratio[conv_idx]+=flop_ratio[bn_idx]
if relu_name in layer:
relu_idx = layer.index(relu_name)
par_ratio[conv_idx]+=par_ratio[relu_idx]
flop_ratio[conv_idx]+=flop_ratio[relu_idx]
elif relus_name in layer:
relus_idx = layer.index(relus_name)
par_ratio[conv_idx]+=par_ratio[relus_idx]
flop_ratio[conv_idx]+=flop_ratio[relus_idx]
return par_ratio,flop_ratio
def fold_model(model):
for name, module in model.named_modules():
if 'conv' in name:
[prefix,suffix] = name.split('conv')
bn_name = prefix+'bn'+suffix
if hasattr(model,bn_name):
bn_layer = getattr(model,bn_name)
fold_bn(module,bn_layer)
def fold_bn(conv, bn):
# 获取 BN 层的参数
mean = bn.running_mean
var = bn.running_var
eps = bn.eps
std = torch.sqrt(var + eps)
if bn.affine:
gamma_ = bn.weight / std
weight = conv.weight * gamma_.view(conv.out_channels, 1, 1, 1)
if conv.bias is not None:
bias = gamma_ * conv.bias - gamma_ * mean + bn.bias
else:
bias = bn.bias - gamma_ * mean
else:
gamma_ = 1 / std
weight = conv.weight * gamma_
if conv.bias is not None:
bias = gamma_ * conv.bias - gamma_ * mean
else:
bias = -gamma_ * mean
# 设置新的 weight 和 bias
conv.weight.data = weight.data
if conv.bias is not None:
conv.bias.data = bias.data
else:
conv.bias = torch.nn.Parameter(bias)
ykl/MatLabScript/flops.m
% 导入数据表
file_data = xlsread('D:\Desktop\ptq_result.xlsx','Inception_BN','B4:E46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\AlexNet.xlsx','Sheet','B4:G46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\AlexNet_BN.xlsx','Sheet','B4:G46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\VGG_16.xlsx','Sheet','B4:G46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\VGG_19.xlsx','Sheet','B4:G46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\Inception_BN.xlsx','Sheet','B4:G46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\MobileNetV2.xlsx','Sheet','B4:G46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_18.xlsx','Sheet','B4:G46');
% file_data = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_50.xlsx','Sheet','B4:G46');
file_data = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_152.xlsx','Sheet','B4:G46');
js_flops = file_data(:,1);
js_param = file_data(:,2);
ptq_acc = file_data(:,3);
acc_loss = file_data(:,4);
js_param = file_data(:,4);
ptq_acc = file_data(:,5);
acc_loss = file_data(:,6);
x = js_flops;
y = acc_loss;
......@@ -20,7 +29,7 @@ rational_model = fittype('(p1*js_flops.^2 + p2*js_flops + p3) / (q1*js_flops.^2
% 确保拟合结果是单调上升的
tolerance = 0;
x_range = min(x):0.1:max(x);
x_range = min(x):0.5:max(x);
y_fit = fitresult(x_range);
while any(diff(y_fit) < tolerance)
% 如果拟合曲线不是单调上升的，重新拟合
......
ykl/MatLabScript/flops_all.m
% 导入数据表
data0 = xlsread('D:\Desktop\ptq_result.xlsx','AlexNet','B4:E46');
data1 = xlsread('D:\Desktop\ptq_result.xlsx','AlexNet_BN','B4:E46');
data2 = xlsread('D:\Desktop\ptq_result.xlsx','VGG_16','B4:E46');
data3 = xlsread('D:\Desktop\ptq_result.xlsx','VGG_19','B4:E46');
data4 = xlsread('D:\Desktop\ptq_result.xlsx','Inception_BN','B4:E46');
file_data = vertcat(data0,data1,data2,data3,data4)
data0 = xlsread('D:\Desktop\ptq_result\cifar100\AlexNet.xlsx','Sheet','B4:G46');
data1 = xlsread('D:\Desktop\ptq_result\cifar100\AlexNet_BN.xlsx','Sheet','B4:G46');
data2 = xlsread('D:\Desktop\ptq_result\cifar100\VGG_16.xlsx','Sheet','B4:G46');
data3 = xlsread('D:\Desktop\ptq_result\cifar100\VGG_19.xlsx','Sheet','B4:G46');
data4 = xlsread('D:\Desktop\ptq_result\cifar100\Inception_BN.xlsx','Sheet','B4:G46');
data5 = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_18.xlsx','Sheet','B4:G46');
data6 = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_50.xlsx','Sheet','B4:G46');
data7 = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_152.xlsx','Sheet','B4:G46');
data8 = xlsread('D:\Desktop\ptq_result\cifar100\MobileNetV2.xlsx','Sheet','B4:G46');
file_data = vertcat(data0,data1,data2,data3,data4,data5,data6,data7,data8);
js_flops = file_data(:,1);
js_param = file_data(:,2);
ptq_acc = file_data(:,3);
acc_loss = file_data(:,4);
js_flops_wt_log = file_data(:,2);
js_flops_wt_cbrt = file_data(:,3);
js_param = file_data(:,4);
ptq_acc = file_data(:,5);
acc_loss = file_data(:,6);
% 指定拟合类别（横坐标）及多项式次数
% fit_type 1/2/3对应js_flops/js_flops_wt_log/js_flops_wt_cbrt
fit_type = 1;
poly = 2;
switch fit_type
case 1
x = js_flops;
xlabel('js\_flops');
case 2
x = js_flops_wt_log;
xlabel('js\_flops\_wt\_log');
case 3
x = js_flops_wt_cbrt;
xlabel('js\_flops\_wt\_cbrt');
end
% 指定横纵坐标及多项式次数
x = js_flops;
y = acc_loss;
poly = 4;
ylabel('acc\_loss');
% 定义颜色向量和每个数据点所属的类别
colors = ['r', 'g', 'b','m','o'];
class = [ones(43,1); 2*ones(43,1); 3*ones(43,1);4*ones(43,1);5*ones(43,1);];
% colors = ['r', 'g', 'b','m','o','c','dr','db','lm'];
colors = [
1 0 0;
0 1 0;
0 0 1;
0 1 1;
1 0 1;
1 1 0;
1 0.5 0;
0 1 0.5;
0.5 0 1;
];
class = [ones(43,1); 2*ones(43,1); 3*ones(43,1);4*ones(43,1);5*ones(43,1);
6*ones(43,1); 7*ones(43,1); 8*ones(43,1); 9*ones(43,1);];
% 指定拟合模型
if poly == 2
rational_model = fittype('(p1*x.^2 + p2*x + p3) / (q1*x.^2 + q2*x + q3)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
elseif poly == 3
rational_model = fittype('(p1*x.^3 + p2*x.^2 + p3*x + p4) / (q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3','p4', 'q1', 'q2', 'q3','q4'});
elseif poly == 4
rational_model = fittype('(p0*x.^4 + p1*x.^3 + p2*x.^2 + p3*x + p4) / (q0*x.^4 + q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p0', 'p1', 'p2', 'p3','p4','q0', 'q1', 'q2', 'q3','q4'});
switch poly
case 2
rational_model = fittype('(p1*x.^2 + p2*x + p3) / (q1*x.^2 + q2*x + q3)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
case 3
rational_model = fittype('(p1*x.^3 + p2*x.^2 + p3*x + p4) / (q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3','p4', 'q1', 'q2', 'q3','q4'});
case 4
rational_model = fittype('(p0*x.^4 + p1*x.^3 + p2*x.^2 + p3*x + p4) / (q0*x.^4 + q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p0', 'p1', 'p2', 'p3','p4','q0', 'q1', 'q2', 'q3','q4'});
end
%初次拟合
......@@ -33,30 +68,39 @@ end
% 确保拟合结果是单调上升的
tolerance = 0;
x_range = min(x):0.1:max(x);
x_range = min(x):0.5:max(x);
y_fit = fitresult(x_range);
while any(diff(y_fit) < tolerance)
SSE=gof.sse;
R_square = gof.rsquare;
RMSE = gof.rmse;
% R_square = 0;
while any(diff(y_fit) < tolerance) % || R_square < 0.7
% while R_square < 0.85
% 如果拟合曲线不是单调上升的，重新拟合
[fitresult, gof] = fit(x, y, rational_model);
y_fit = fitresult(x_range);
% 获取评价指标
SSE=gof.sse;
R_square = gof.rsquare;
RMSE = gof.rmse;
end
% 可视化数据点和拟合曲线
scatter(x(1:43), y(1:43), [], colors(1), 'filled');
hold on;
scatter(x(44:86), y(44:86), [], colors(2), 'filled');
scatter(x(86:129), y(86:129), [], colors(3), 'filled');
scatter(x(130:172), y(130:172), [], colors(4), 'filled');
scatter(x(173:215), y(173:215), [], colors(5), 'filled');
for i = 1:9
color = colors(i,:);
scatter(x(43*i-42:43*i), y(43*i-42:43*i), [], color, 'filled');
end
% scatter(x(44:86), y(44:86), [], colors(2), 'filled');
% scatter(x(86:129), y(86:129), [], colors(3), 'filled');
% scatter(x(130:172), y(130:172), [], colors(4), 'filled');
% scatter(x(173:215), y(173:215), [], colors(5), 'filled');
plot(fitresult,'k',x,y);
xlabel('js\_flops');
ylabel('acc\_loss')
legend('AlexNet', 'AlexNet\_BN','VGG\_16','VGG\_19','Inception\_BN','ALL', 'Rational-Fit', 'Location', 'Northeast');
% 获取评价指标
SSE=gof.sse;
R_square = gof.rsquare;
RMSE = gof.rmse;
legend('AlexNet', 'AlexNet\_BN','VGG\_16','VGG\_19','Inception\_BN',...
'MobileNetV2','ResNet\_18','ResNet\_50','ResNet\_152',...
'ALL', 'Rational-Fit', 'Location', 'Northeast');
% 将拟合公式和 R 方显示在图上
text(0.65, 0.2, sprintf('Goodness of fit:\n SSE:%.4f\n R-square:%.4f\n RMSE:%.4f', SSE, R_square, RMSE), 'Units', 'normalized', 'FontSize', 11);
......
ykl/MatLabScript/flops_lstm.m 0 → 100644
% 导入数据表
file_data = xlsread('D:\Desktop\ptq_result.xlsx','PTB_LSTM','B4:G46');
js_flops = file_data(:,1);
js_param = file_data(:,4);
ptq_acc = file_data(:,5);
ppl_ratio = file_data(:,6);
x = js_flops;
y = ppl_ratio;
% 定义颜色向量和每个数据点所属的类别
colors = ['r', 'g', 'm'];
class = [ones(16,1); 2*ones(6,1); 3*ones(21,1)];
% 指定拟合模型
rational_model = fittype('(p1*js_flops.^2 + p2*js_flops + p3) / (q1*js_flops.^2 + q2*js_flops + q3)', 'independent', 'js_flops', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
%初次拟合
[fitresult, gof] = fit(x, y, rational_model);
% 确保拟合结果是单调上升的
tolerance = 0;
x_range = min(x):5:max(x);
y_fit = fitresult(x_range);
while any(diff(y_fit) < tolerance)
% 如果拟合曲线不是单调上升的，重新拟合
[fitresult, gof] = fit(x, y, rational_model);
y_fit = fitresult(x_range);
end
% 可视化数据点和拟合曲线
scatter(x(1:15), y(1:15), [], colors(1), 'filled');
hold on;
scatter(x(16:22), y(16:22), [], colors(2), 'filled');
scatter(x(23:43), y(23:43), [], colors(3), 'filled');
plot(fitresult,'k',x,y);
xlabel('js\_flops');
ylabel('loss\_ratio');
legend('INT', 'POT', 'FLOAT','ALL', 'Rational-Fit', 'Location', 'Northeast');
% 获取评价指标
SSE=gof.sse;
R_square = gof.rsquare;
RMSE = gof.rmse;
% 将拟合公式和 R 方显示在图上
text(0.65, 0.2, sprintf('Goodness of fit:\n SSE:%.4f\n R-square:%.4f\n RMSE:%.4f', SSE, R_square, RMSE), 'Units', 'normalized', 'FontSize', 11);
hold off;
\ No newline at end of file
ykl/MatLabScript/flops_sep.m 0 → 100644
% 导入数据表
data0 = xlsread('D:\Desktop\ptq_result\cifar100\AlexNet.xlsx','Sheet','B4:G46');
data1 = xlsread('D:\Desktop\ptq_result\cifar100\AlexNet_BN.xlsx','Sheet','B4:G46');
data2 = xlsread('D:\Desktop\ptq_result\cifar100\VGG_16.xlsx','Sheet','B4:G46');
data3 = xlsread('D:\Desktop\ptq_result\cifar100\VGG_19.xlsx','Sheet','B4:G46');
data4 = xlsread('D:\Desktop\ptq_result\cifar100\Inception_BN.xlsx','Sheet','B4:G46');
data5 = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_18.xlsx','Sheet','B4:G46');
data6 = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_50.xlsx','Sheet','B4:G46');
data7 = xlsread('D:\Desktop\ptq_result\cifar100\ResNet_152.xlsx','Sheet','B4:G46');
data8 = xlsread('D:\Desktop\ptq_result\cifar100\MobileNetV2.xlsx','Sheet','B4:G46');
file_data = vertcat(data0,data1,data2,data3,data4,data5,data6,data7,data8);
js_flops = file_data(:,1);
js_flops_wt_log = file_data(:,2);
js_flops_wt_cbrt = file_data(:,3);
js_param = file_data(:,4);
ptq_acc = file_data(:,5);
acc_loss = file_data(:,6);
% 指定拟合类别（横坐标）及多项式次数
% fit_type 1/2/3对应js_flops/js_flops_wt_log/js_flops_wt_cbrt
fit_type = 1;
poly = 2;
switch fit_type
case 1
x = js_flops;
xlabel('js\_flops');
case 2
x = js_flops_wt_log;
xlabel('js\_flops\_wt\_log');
case 3
x = js_flops_wt_cbrt;
xlabel('js\_flops\_wt\_cbrt');
end
y = acc_loss;
ylabel('acc\_loss');
% 定义颜色向量和每个数据点所属的类别
% colors = ['r', 'g', 'b','m','o','c','dr','db','lm'];
colors = [
1 0 0;
0 1 0;
0 0 1;
0 1 1;
1 0 1;
1 1 0;
1 0.5 0;
0 1 0.5;
0.5 0 1;
];
class = [ones(43,1); 2*ones(43,1); 3*ones(43,1);4*ones(43,1);5*ones(43,1);
6*ones(43,1); 7*ones(43,1); 8*ones(43,1); 9*ones(43,1);];
% 指定拟合模型
switch poly
case 2
rational_model = fittype( '(x<10)*((p1*x.^2 + p2*x + p3) / (q1*x.^2 + q2*x + q3)) + (x>=10)*((p1*10^2 + p2*10 + p3) / (q1*10^2 + q2*10 + q3))',...
'independent', 'x', 'coefficients', {'p1', 'p2', 'p3', 'q1', 'q2', 'q3'});
case 3
rational_model = fittype('(p1*x.^3 + p2*x.^2 + p3*x + p4) / (q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p1', 'p2', 'p3','p4', 'q1', 'q2', 'q3','q4'});
case 4
rational_model = fittype('(p0*x.^4 + p1*x.^3 + p2*x.^2 + p3*x + p4) / (q0*x.^4 + q1*x.^3 + q2*x.^2 + q3*x + q4)', 'independent', 'x', 'coefficients', {'p0', 'p1', 'p2', 'p3','p4','q0', 'q1', 'q2', 'q3','q4'});
end
%初次拟合
[fitresult, gof] = fit(x, y, rational_model);
% 确保拟合结果是单调上升的
tolerance = 0;
x_range = min(x):0.5:max(x);
y_fit = fitresult(x_range);
SSE=gof.sse;
R_square = gof.rsquare;
RMSE = gof.rmse;
% R_square = 0;
while any(diff(y_fit) < tolerance)
% while R_square < 0.85
% 如果拟合曲线不是单调上升的，重新拟合
[fitresult, gof] = fit(x, y, rational_model);
y_fit = fitresult(x_range);
% 获取评价指标
SSE=gof.sse;
R_square = gof.rsquare;
RMSE = gof.rmse;
end
% 可视化数据点和拟合曲线
hold on;
for i = 1:9
color = colors(i,:);
scatter(x(43*i-42:43*i), y(43*i-42:43*i), [], color, 'filled');
end
% scatter(x(44:86), y(44:86), [], colors(2), 'filled');
% scatter(x(86:129), y(86:129), [], colors(3), 'filled');
% scatter(x(130:172), y(130:172), [], colors(4), 'filled');
% scatter(x(173:215), y(173:215), [], colors(5), 'filled');
plot(fitresult,'k',x,y);
legend('AlexNet', 'AlexNet\_BN','VGG\_16','VGG\_19','Inception\_BN',...
'ResNet\_18','ResNet\_50','ResNet\_152','MobileNetV2',...
'ALL', 'Rational-Fit', 'Location', 'Northeast');
% 将拟合公式和 R 方显示在图上
text(0.65, 0.2, sprintf('Goodness of fit:\n SSE:%.4f\n R-square:%.4f\n RMSE:%.4f', SSE, R_square, RMSE), 'Units', 'normalized', 'FontSize', 11);
hold off;
\ No newline at end of file
ykl/README.md
# 改动说明
## update: 2023/06/05
+ 解决之前ResNet系列和MobileNetV2的模型结构错误
+ 将准确性和安全性框架融合入ALL文件夹，并添加部署说明（安全性评估尚未添加）
+ 改动matlab脚本适配当前框架
+ 针对准确性，单个模型拟合较好，整体较差，需要后期数据处理。
## update: 2023/05/29
+ GDFQ：结合之前框架，训练了所有模型的生成器。后续将进一步引入评估和决策边界样本增强。
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment