Skip to content
Projects
Groups
Snippets
Help
This project
Loading...
Sign in / Register
Toggle navigation
M
Model-Transfer-Adaptability
Overview
Overview
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
haoyifan
Model-Transfer-Adaptability
Commits
718adf69
Commit
718adf69
authored
Apr 24, 2023
by
Zhihong Ma
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
fix: mobilenet v2 curve
parent
efd05aee
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
1 additions
and
0 deletions
+1
-0
mzh/new_mzh/MobileNetV2/fig/defreeze.png
+0
-0
mzh/new_mzh/MobileNetV2/readme.md
+1
-0
No files found.
mzh/new_mzh/MobileNetV2/fig/defreeze.png
0 → 100644
View file @
718adf69
64.4 KB
mzh/new_mzh/MobileNetV2/readme.md
View file @
718adf69
...
...
@@ -7,6 +7,7 @@
我仔细检查了是否成功fold了BN,以及相应的ratio,计算js散度时是否使用的是匹配的两个权值参数,发现都没有问题。但在具体查看权值参数的数据时,发现了一些层的权值参数数据异常,也是这些异常层的js非常大干扰了整体的js计算。
<br>
从tensorboard来观察,这些层的数据分布相似度应该与全精度模型很像,但js计算结果没能反应出这一点,对这些层的数据重点观察,他们有很多1或-1的值,因此我想到可能是对量化前后的模型权值参数先进行的normalize操作导致了数据分布变得不合理,进而导致了问题。
<br>
考虑到normalize操作的本意应该是为了将量化前后的模型权值参数归一到同一个scale进而方便使用js散度计算距离,则可以考虑将量化后的模型的权值参数通过dequantize来恢复到与全精度模型相近的scale,而后再使用js散度计算距离。我将上述过程命名为fakefreeze. 经过实践,效果很好,重新计算的js散度反映的权值参数相似程度与tensorboard直接对数据分布的观察比较一致。
<img
src =
"fig/defreeze.png"
class=
"h-90 auto"
>
## update: <br>2023.4.23<br>
1.
实现了MobileNetV2的PTQ量化
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment