haoyifan

AAAI2021/tex/appendix.tex

+\begin{figure}[t]
+  \centering \includegraphics[width=0.99\columnwidth]{fig/Appendix_Figure1_MIS.pdf}
+  \caption{Compositionality of symbolic language under different parameters
+    ($[\mu-\sigma,\mu+\sigma]$, where $\mu$ is the mean value and $\sigma$ is
+    the standard deviation).}
+  \label{fig:exp1}
+\end{figure}
+
+
+\begin{figure}[t]
+  \centering \includegraphics[width=0.99\columnwidth]{fig/Appendix_Figure2_Ratio.pdf}
+  \caption{The ratio of high compositional language. (a) $MIS>0.99$. (b)
+    $MIS>0.9$. }
+  \label{fig:exp2}
+\end{figure}
+
+
+\begin{table}[b]
+  \centering
+  \small
+  \caption{The Chi-square test between high-compositionality and agent capacity.}
+  \label{tab:exp10}
+  \begin{tabular}{cccc}
+    \toprule
+    \multicolumn{4}{c}{$H_0$: $\mathit{MIS} > 0.90$ is independent with $h_{\mathit{size}}$}\\
+    \midrule
+    Configuration & $\chi^2$ & $df$ & $p$-value \\
+    \midrule
+    $|M_0|=3,|M_1|=5,|V|=10$ & 87.20 & 10 & $1.72\times 10^{-13}$ \\
+    $|M_0|=4,|M_1|=4,|V|=10$ & 71.47 & 10 & $1.70\times 10^{-10}$ \\
+    \bottomrule
+    \multicolumn{4}{c}{\vspace{1em}}\\
+    \toprule
+    \multicolumn{4}{c}{$H_0$: $\mathit{MIS} > 0.99$ is independent with $h_{\mathit{size}}$}\\
+    \midrule
+    Configuration & $\chi^2$ & $df$ & $p$-value \\
+    \midrule
+    $|M_0|=3,|M_1|=5,|V|=10$ & 34.15 & 10 & $6.39\times 10^{-4}$ \\
+    $|M_0|=4,|M_1|=4,|V|=10$ & 38.26 & 10 & $1.39\times 10^{-4}$ \\
+    \bottomrule
+    \end{tabular}
+  \end{table}
+
+
+
+\section{Appendix}
+\label{sec:exp}
+
+We exploit the relationship between agent capacity and the compositionality of
+symbolic language that emerged in our natural referential game.
+For various configuration of
+vocabulary size, we fix $|M_0|=|M_1|=3$ and train the speaker-listener agents to emerge symbolic
+language when varying the agent capacities, i.e., hidden layer size
+($h_{size}$), from 6 to 100. 
+
+Figure~\ref{fig:exp1} reports the experimental results. It can be observed that
+the mean value of MIS decreases as the value of $h_{size}$ increases. Taking the
+configuration of vocabulary size $|V|=10$ as an example, the mean value of MIS
+is around 0.8 when $h_{size}\le 20$; MIS significantly decreases to 0.75 when
+$h_{size}$ increases from 20 to 40; MIS further reduces to 0.7 when $h_{size}$
+increases from 40 to 100.
+For different vocabulary sizes, the MIS shares the
+similar behavior.
+It is because symbols in low-compositional languages carry semantic information
+about more concepts. As a result, higher capacity is required to characterize the
+complex semantic information for low-compositional language to emerge.
+In summary, lower agent capacity improves the possibility of
+emerging high compositional symbolic language.
+
+
+\subsection{Ratio of high compositional language.}
+We further breakdown our results to investigate the importance of agent capacity
+to the compositionality of symbolic language. Figure~\ref{fig:exp2} reports the
+ratio of high compositional symbolic language in all emerged languages,
+Figure~\ref{fig:exp2} (a) and (b) for $\mathit{MIS}>0.99$ and $\mathit{MIS}>0.9$, respectively. It
+can be observed that the ratio of high compositional symbolic languages
+decreases drastically with the increase of $h_{size}$.
+Taking vocabulary size $|V|=4$ as an example, symbolic languages with
+compositionality $\mathit{MIS}>0.99$ take $>$10\% mainly over all the emerged symbolic
+languages, when $h_{size}<20$; the ratio reduces to 0\%$\sim$5\% when $h_{size}$
+increases to 40; the ratio reduces around 3\% when $h_{size}$ goes beyond 40. 
+$\mathit{MIS}>0.9$ reports similar results.
+Notably, when $h_{size}$ is large enough (e.g., $>40$), high compositional
+symbolic language is hard to emerge in a natural referential game, for
+easy-to-emerge low compositional symbolic language is sufficient in scenarios of
+referential game.
+On the other side, agents are enforced to use compositionality to express
+more meanings, for the constraint from low capacity.
+
+
+
+Additionally, we also perform $\chi^2$ test to check the statistical
+significance between the high compositionality and agent
+capacity. Table~\ref{tab:exp10} reports the $\chi^2$ test results for
+$\mathit{MIS}>0.99$ and $\mathit{MIS}>0.9$, respectively. It can be observed that
+for different vocabulary sizes, the p-value is always less than 0.05, which means
+the high compositionality has a statistical significance related to agent
+capacity.