feat data record before 2023.3.30 for LeNet

mzh/data_record/README.md

+- ptq_analysis_data
+
+  数据都是Wasserstein距离算出来的结果，越大，相似度越低。
+
+  csv中前面的列分别对应一个层的weight或bias权重的相似度，最后一列的sum是加权求和后的结果
+
+  1. fakefreeze.csv: INT量化对应的数据。之所以叫fakefreeze是因为使用fakefreeze方法将INT量化后的数据反量化回FP32，便于通过Wasserstein距离衡量与全精度模型的相似度
+  2. PoT.csv: PoT量化的数据
+  3. mode3_bit8_x.csv: 是FP8量化的数据，其中x=2,3,4,5,6,7.，x代表指数位位宽 
+  4. trial_data.csv是之前用Pearson系数度量相似度时的结果，因为nan的问题不太好解决，已弃用。
+
+- qat_analysis_data
+
+  都有scratch_loss.csv和wasserstein_distance.csv. 前者代表从0开始的QAT量化的loss数据，后者代表量化模型各个层的weight和bias的训练梯度与全精度模型训练梯度的距离(相似度)
+
+  第一行5，10，15，20，25，30代表epoch，对于loss来说就是若干epoch的loss差 (loss减小量，认为其越大，收敛速度越快)，对于distance则是这些epoch的梯度相似度的平均值
+
+  1. mode1: INT量化
+  2. mode2: PoT量化
+  3. mode3: FP8量化
+
+  
\ No newline at end of file

mzh/data_record/ptq_analysis_data/PoT.csv

+,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+0,16.81457976,21.35748664,45.37735622,80.00641067,319.2686728,339.0942679,116.5587458,116.8333445,24.69601563,18.13536118,0.004275832
+0,16.814579755832288,21.357486635446552,45.37735621680935,80.00641067139804,319.26867281826156,339.094267936227,116.55874582763597,116.83334448760434,24.696015631530567,18.135361181339253,0.004275832061357

mzh/data_record/ptq_analysis_data/fakefreeze.csv

+bit_nums,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+2,68.22259833,100.3982969,220.8686779,536.2932538,1731.957568,2057.146386,473.7986472,647.6118065,94.60077135,128.5511945,0.021896189
+3,30.50904583,74.84542993,109.7717294,536.2932538,921.9989655,2057.146386,195.6498429,647.6118065,43.86690759,128.5511945,0.012968514
+4,14.1248777,124.6109976,50.66098556,341.7691645,429.0019149,2057.146386,92.92714985,647.6118065,19.16205574,128.5511945,0.00799247
+5,6.917819789,171.4374247,24.76906054,72.43782381,207.9410647,1693.836094,44.77978193,457.2613676,9.434641769,87.35781269,0.004893503
+6,3.497701885,200.4068802,11.82132822,18.98553748,101.6827932,470.2345973,22.02404145,149.4768515,4.807946158,12.54096353,0.001976173
+7,1.593871809,217.5861913,5.978719549,8.55026323,50.4931325,109.0827829,11.05211889,27.53274504,2.322869264,4.694798753,0.00091747
+8,0.824684315,226.1787109,3.005121785,137.5807588,25.24494274,26.05569958,5.46757911,8.437261568,1.188152966,0.991986744,0.00070737
+10,0.202968472,232.6236711,0.746211185,412.8853103,6.258679201,231.9680682,1.353214248,109.1300321,0.29020193,24.73140668,0.001227567
+12,0.051461387,234.2349415,0.187390658,502.550738,1.569309379,1284.990258,0.340222851,425.835733,0.072910558,95.82387505,0.002948755
+18,0.003193582,234.7384641,0.011594186,534.1850694,0.09787436,2002.568571,0.021376261,631.8790353,0.00447766,126.3787593,0.004071902
+24,0.0008284,234.7636402,0.002888851,535.7662166,0.02446819,2043.400856,0.005298984,643.6399055,0.001181837,127.9852662,0.004135547
+28,1.42E-05,234.7719012,5.20E-05,536.2850189,0.000418704,2056.930684,8.95E-05,647.5495068,1.97E-05,128.5423516,0.004156675
+32,1.03E-05,234.7720241,3.06E-05,536.2927391,0.000281262,2057.132904,6.46E-05,647.6079128,1.38E-05,128.5506419,0.004156993
+48,1.03E-05,234.7720318,3.06E-05,536.2932216,0.000281262,2057.145543,6.46E-05,647.6115631,1.38E-05,128.55116,0.004157013
+0,1.03E-05,234.7720323,3.06E-05,536.2932538,0.000281262,2057.146386,6.46E-05,647.6118065,1.38E-05,128.5511945,0.004157014
+0,1.03E-05,234.7720323,3.06E-05,536.2932538,0.000281262,2057.146386,6.46E-05,647.6118065,1.38E-05,128.5511945,0.004157014
+0,1.03E-05,234.7720323,3.06E-05,536.2932538,0.000281262,2057.146386,6.46E-05,647.6118065,1.38E-05,128.5511945,0.004157014
+0,1.03E-05,234.7720323,3.06E-05,536.2932538,0.000281262,2057.146386,6.46E-05,647.6118065,1.38E-05,128.5511945,0.004157014

mzh/data_record/ptq_analysis_data/mode3_bit8_2.csv

+,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+0,75.82670590966163,50.7095322906971,258.78640179320803,536.2932537887245,1870.4519468609028,2057.146385598171,691.8076705166582,647.6118064931361,140.89196097698215,128.55119453393854,0.024936932915293347

mzh/data_record/ptq_analysis_data/mode3_bit8_3.csv

+,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+0,12.152955909661637,11.522032290697096,97.30291220987469,33.74637878872454,1787.1347984234033,1859.7791980981708,479.52987103749143,395.21336899313604,26.96571469722022,24.957444533938542,0.019288045735227046

mzh/data_record/ptq_analysis_data/mode3_bit8_4.csv

+,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+0,4.336237159661637,7.959532290697099,11.850282001541348,26.373331913724538,105.14049422418414,105.01259653567102,33.078471297908095,30.086415868136115,6.778017059571412,5.035569533938541,0.001315061858831
+0,4.336237159661637,7.959532290697099,11.850282001541348,26.373331913724538,105.14049422418414,105.01259653567102,33.078471297908095,30.086415868136115,6.778017059571412,5.035569533938541,0.001315061858831
+0,4.336237159661637,7.959532290697099,11.850282001541348,26.37333191372454,105.14049422418414,105.01259653567102,33.07847129790809,30.086415868136115,6.778017059571412,5.035569533938541,0.001315061858831

mzh/data_record/ptq_analysis_data/mode3_bit8_5.csv

+,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+0,8.154977394036633,10.334532290697098,22.55606894815592,47.902628788724535,159.37403708612268,171.09536020754604,58.62780334441688,55.2849021962611,12.90157256087349,8.895432815188542,0.0021463790569357373
+0,8.154977394036633,10.334532290697098,22.55606894815592,47.902628788724535,159.37403708612268,171.09536020754604,58.62780334441688,55.2849021962611,12.90157256087349,8.895432815188542,0.0021463790569357373
+0,8.154977394036633,10.334532290697098,22.55606894815592,47.90262878872454,159.3740370861227,171.09536020754604,58.62780334441688,55.2849021962611,12.90157256087349,8.895432815188542,0.0021463790569357373

mzh/data_record/ptq_analysis_data/mode3_bit8_6.csv

+,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+0,16.002682472161634,25.7720322906971,41.80056996032226,71.49637878872454,297.6255039707358,322.8899158716086,110.9594735611362,121.04320297751107,24.348992226982123,16.864182815188542,0.004034826849009954

mzh/data_record/ptq_analysis_data/mode3_bit8_7.csv

+,conv_layers.conv1.weight,conv_layers.conv1.bias,conv_layers.conv2.weight,conv_layers.conv2.bias,fc_layers.fc1.weight,fc_layers.fc1.bias,fc_layers.fc2.weight,fc_layers.fc2.bias,fc_layers.fc3.weight,fc_layers.fc3.bias,sum
+0,29.11317726382831,49.52203229069712,74.01636481607582,163.51200378872454,522.2396780697042,560.7878650903585,191.12617392429536,186.38792954001107,42.74461048405059,27.530686721438546,0.00707688707157158

mzh/data_record/qat_analysis_data/mode1/scratch_loss.csv

+bit_num,5,10,15,20,25,30
+2,5.03E-05,5.03E-05,4.20E-05,4.24E-05,4.15E-05,4.08E-05
+3,0.000166416,0.000384808,0.001027823,0.012617111,0.023335695,0.029314995
+4,0.000247955,0.001880646,0.025078058,0.043783903,0.06728721,0.07710004
+5,0.00041151,0.003772736,0.033952475,0.0826118,0.11236,0.13291907
+6,0.000445366,0.003655195,0.039137125,0.09958696,0.13628888,0.15648198
+7,0.000458241,0.004037857,0.043150425,0.10892105,0.14628649,0.17344975
+8,0.000464916,0.004246473,0.045412302,0.114221096,0.14996433,0.1722641
+10,0.000468493,0.00434351,0.0470829,0.11749983,0.15200114,0.17576337
+12,0.000468731,0.004367828,0.047480345,0.1181581,0.15229583,0.18007684
+16,0.000469446,0.00437355,0.04760027,0.11837077,0.15241933,0.18089294
+18,0.000469208,0.00437212,0.04760146,0.11837149,0.15237403,0.18175483
+24,0.000468969,0.00437212,0.047600508,0.11838174,0.1523714,0.1806283
+28,0.000469446,0.004373312,0.047596455,0.1183753,0.15237498,0.1814084
+32,0.000469208,0.004372597,0.04759693,0.118377924,0.1524148,0.18114114
+48,0.000469446,0.00437212,0.0475955,0.118382215,0.15243077,0.18084097
+56,0.000469685,0.004375935,0.0476079,0.118377924,0.1523726,0.18089223
+63,0.000468969,0.00437212,0.04759884,0.11839533,0.15237617,0.18109322
+64,0.000469446,0.004373074,0.047595024,0.1183579,0.15236902,0.18139434

mzh/data_record/qat_analysis_data/mode1/wasserstein_distance.csv

+bit_num,5,10,15,20,25,30
+2,9.04E-07,1.96E-06,4.56E-06,6.13E-06,6.38E-06,6.31E-06
+3,7.24E-07,1.69E-06,4.15E-06,5.24E-06,5.05E-06,4.72E-06
+4,3.47E-07,8.17E-07,2.28E-06,3.09E-06,3.34E-06,3.20E-06
+5,1.31E-07,2.41E-07,9.99E-07,1.67E-06,1.56E-06,1.60E-06
+6,8.17E-08,2.00E-07,7.24E-07,1.05E-06,1.07E-06,1.09E-06
+7,4.40E-08,1.04E-07,3.71E-07,5.29E-07,5.81E-07,6.40E-07
+8,2.17E-08,4.43E-08,1.83E-07,2.62E-07,3.23E-07,3.40E-07
+10,9.22E-09,1.54E-08,7.45E-08,1.29E-07,1.99E-07,2.33E-07
+12,8.24E-09,1.28E-08,6.04E-08,1.15E-07,1.85E-07,2.39E-07
+16,7.82E-09,1.30E-08,5.92E-08,1.14E-07,1.89E-07,2.47E-07
+18,7.93E-09,1.30E-08,5.93E-08,1.14E-07,1.88E-07,2.51E-07
+24,7.81E-09,1.29E-08,5.93E-08,1.14E-07,1.88E-07,2.44E-07
+28,7.66E-09,1.29E-08,5.93E-08,1.14E-07,1.87E-07,2.48E-07
+32,7.63E-09,1.27E-08,5.90E-08,1.14E-07,1.89E-07,2.47E-07
+48,7.69E-09,1.27E-08,5.90E-08,1.14E-07,1.88E-07,2.44E-07
+56,7.96E-09,1.33E-08,5.94E-08,1.14E-07,1.89E-07,2.45E-07
+63,7.71E-09,1.28E-08,5.93E-08,1.14E-07,1.87E-07,2.45E-07
+64,7.71E-09,1.28E-08,5.90E-08,1.14E-07,1.87E-07,2.47E-07

mzh/data_record/qat_analysis_data/mode2/scratch_loss.csv

+,5,10,15,20,25,30
+0,0.0004386902,0.005204439,0.035360813,0.087856054,0.12195778,0.14326978

mzh/data_record/qat_analysis_data/mode2/wasserstein_distance.csv

+,5,10,15,20,25,30
+0,3.813864466754601e-07,5.483315760236439e-06,5.883244292717652e-06,7.282206948262632e-06,5.59871330093482e-06,8.178037484545082e-06

mzh/data_record/qat_analysis_data/mode3/scratch_loss_3.csv

+,5,10,15,20,25,30
+0,0.0,0.0,0.0,0.0,0.0,0.0

mzh/data_record/qat_analysis_data/mode3/scratch_loss_4.csv

+,5,10,15,20,25,30
+0,0.00016069412,0.00060224533,0.0040824413,0.039744854,0.100259304,0.14057279

mzh/data_record/qat_analysis_data/mode3/scratch_loss_5.csv

+,5,10,15,20,25,30
+0,8.273125e-05,0.00023508072,0.0006353855,0.0034022331,0.035773993,0.089517355

mzh/data_record/qat_analysis_data/mode3/scratch_loss_6.csv

+,5,10,15,20,25,30
+0,2.6464462e-05,4.887581e-05,7.200241e-05,0.0001308918,0.00022411346,0.00040555

mzh/data_record/qat_analysis_data/mode3/scratch_loss_7.csv

+,5,10,15,20,25,30
+0,3.33786e-06,7.867813e-06,1.1444092e-05,1.7642975e-05,2.6226044e-05,3.361702e-05

mzh/data_record/qat_analysis_data/mode3/wasserstein_distance_3.csv

+,5,10,15,20,25,30
+0,9.050764626906219e-07,1.9604800559142703e-06,4.557128377450547e-06,6.128364154870417e-06,6.382632338553727e-06,6.316248566041126e-06

mzh/data_record/qat_analysis_data/mode3/wasserstein_distance_4.csv

+,5,10,15,20,25,30
+0,2.5703666997834155e-07,1.106935500984309e-06,4.7830658285725055e-06,5.971455908572133e-06,5.435577131578025e-06,4.0157894803005465e-06

mzh/data_record/qat_analysis_data/mode3/wasserstein_distance_5.csv

+,5,10,15,20,25,30
+0,3.906529858129073e-07,1.2216483709957484e-06,3.633368319474145e-06,5.18533765255752e-06,6.756451408860653e-06,7.019801240006222e-06

mzh/data_record/qat_analysis_data/mode3/wasserstein_distance_6.csv

+,5,10,15,20,25,30
+0,6.006697717580949e-07,1.5584335175024169e-06,4.075986918359249e-06,5.552694011939173e-06,5.675200282716066e-06,5.331318593962554e-06

mzh/data_record/qat_analysis_data/mode3/wasserstein_distance_7.csv

+,5,10,15,20,25,30
+0,7.794453519194253e-07,1.8238894247457043e-06,4.403918245038333e-06,5.964581519268091e-06,6.209193608756802e-06,6.1263434493206575e-06

mzh/data_record/qat_analysis_data/scratch_loss.csv

+,5,10,15,20,25,30
+2,5.03E-05,5.03E-05,4.20E-05,4.24E-05,4.15E-05,4.08E-05
+3,0.000166416,0.000384808,0.001027823,0.012617111,0.023335695,0.029314995
+4,0.000247955,0.001880646,0.025078058,0.043783903,0.06728721,0.07710004
+5,0.00041151,0.003772736,0.033952475,0.0826118,0.11236,0.13291907
+6,0.000445366,0.003655195,0.039137125,0.09958696,0.13628888,0.15648198
+7,0.000458241,0.004037857,0.043150425,0.10892105,0.14628649,0.17344975
+8,0.000464916,0.004246473,0.045412302,0.114221096,0.14996433,0.1722641
+10,0.000468493,0.00434351,0.0470829,0.11749983,0.15200114,0.17576337
+12,0.000468731,0.004367828,0.047480345,0.1181581,0.15229583,0.18007684
+16,0.000469446,0.00437355,0.04760027,0.11837077,0.15241933,0.18089294
+18,0.000469208,0.00437212,0.04760146,0.11837149,0.15237403,0.18175483
+24,0.000468969,0.00437212,0.047600508,0.11838174,0.1523714,0.1806283
+28,0.000469446,0.004373312,0.047596455,0.1183753,0.15237498,0.1814084
+32,0.000469208,0.004372597,0.04759693,0.118377924,0.1524148,0.18114114
+48,0.000469446,0.00437212,0.0475955,0.118382215,0.15243077,0.18084097
+56,0.000469685,0.004375935,0.0476079,0.118377924,0.1523726,0.18089223
+63,0.000468969,0.00437212,0.04759884,0.11839533,0.15237617,0.18109322
+64,0.000469446,0.004373074,0.047595024,0.1183579,0.15236902,0.18139434

mzh/data_record/qat_analysis_data/wasserstein_distance.csv

+,5,10,15,20,25,30
+2,9.04E-07,1.96E-06,4.56E-06,6.13E-06,6.38E-06,6.31E-06
+3,7.24E-07,1.69E-06,4.15E-06,5.24E-06,5.05E-06,4.72E-06
+4,3.47E-07,8.17E-07,2.28E-06,3.09E-06,3.34E-06,3.20E-06
+5,1.31E-07,2.41E-07,9.99E-07,1.67E-06,1.56E-06,1.60E-06
+6,8.17E-08,2.00E-07,7.24E-07,1.05E-06,1.07E-06,1.09E-06
+7,4.40E-08,1.04E-07,3.71E-07,5.29E-07,5.81E-07,6.40E-07
+8,2.17E-08,4.43E-08,1.83E-07,2.62E-07,3.23E-07,3.40E-07
+10,9.22E-09,1.54E-08,7.45E-08,1.29E-07,1.99E-07,2.33E-07
+12,8.24E-09,1.28E-08,6.04E-08,1.15E-07,1.85E-07,2.39E-07
+16,7.82E-09,1.30E-08,5.92E-08,1.14E-07,1.89E-07,2.47E-07
+18,7.93E-09,1.30E-08,5.93E-08,1.14E-07,1.88E-07,2.51E-07
+24,7.81E-09,1.29E-08,5.93E-08,1.14E-07,1.88E-07,2.44E-07
+28,7.66E-09,1.29E-08,5.93E-08,1.14E-07,1.87E-07,2.48E-07
+32,7.63E-09,1.27E-08,5.90E-08,1.14E-07,1.89E-07,2.47E-07
+48,7.69E-09,1.27E-08,5.90E-08,1.14E-07,1.88E-07,2.44E-07
+56,7.96E-09,1.33E-08,5.94E-08,1.14E-07,1.89E-07,2.45E-07
+63,7.71E-09,1.28E-08,5.93E-08,1.14E-07,1.87E-07,2.45E-07
+64,7.71E-09,1.28E-08,5.90E-08,1.14E-07,1.87E-07,2.47E-07