Commit 44b88f72 by YZhao

..

parent a6c27cd8
No preview for this file type
......@@ -28,7 +28,7 @@ emerging high compositional symbolic language.
\begin{figure}[t]
\centering \includegraphics[width=0.99\columnwidth]{fig/Figure6_Compostionality_of_symbolic_language.pdf}
\centering \includegraphics[width=\columnwidth]{fig/Figure6_Compostionality_of_symbolic_language.pdf}
\caption{Compositionality of symbolic language under different parameters
($[\mu-\sigma,\mu+\sigma]$, where $\mu$ is the mean value and $\sigma$ is
the standard deviation).}
......@@ -39,14 +39,14 @@ emerging high compositional symbolic language.
We further breakdown our results to investigate the importance of agent capacity
to the compositionality of symbolic language. Figure~\ref{fig:exp2} reports the
ratio of high compositional symbolic language in all emerged languages,
Figure~\ref{fig:exp2} (a) and (b) for $MIS>0.99$ and $MIS>0.9$, respectively. It
Figure~\ref{fig:exp2} (a) and (b) for $\mathit{MIS}>0.99$ and $\mathit{MIS}>0.9$, respectively. It
can be observed that the ratio of high compositional symbolic languages
decreases drastically with the increase of $h_{size}$.
Taking vocabulary size $|V|=4$ as an example, symbolic languages with
compositionality $MIS>0.99$ take $>$10\% mainly over all the emerged symbolic
compositionality $\mathit{MIS}>0.99$ take $>$10\% mainly over all the emerged symbolic
languages, when $h_{size}<20$; the ratio reduces to 0\%$\sim$5\% when $h_{size}$
increases to 40; the ratio reduces around 3\% when $h_{size}$ goes beyond 40.
$MIS>0.9$ reports similar results.
$\mathit{MIS}>0.9$ reports similar results.
Notably, when $h_{size}$ is large enough (e.g., $>40$), high compositional
symbolic language is hard to emerge in a natural referential game, for
easy-to-emerge low compositional symbolic language is sufficient in scenarios of
......@@ -57,7 +57,7 @@ more meanings, for the constraint from low capacity.
\begin{figure}[t]
\centering
\includegraphics[width=0.99\columnwidth]{fig/Figure7_The_ratio_of_high_compositional_language.pdf}
\includegraphics[width=\columnwidth]{fig/Figure7_The_ratio_of_high_compositional_language.pdf}
\caption{The ratio of high compositional language. (a) $h_{size}>0.99$. (b)
$h_{size}>0.9$. }
\label{fig:exp2}
......@@ -72,14 +72,14 @@ more meanings, for the constraint from low capacity.
\centering
\includegraphics[width=0.8\columnwidth]{fig/Figure8_Three_artificial_languages_with_different_MIS.pdf}
\caption{Three pre-defined language for teaching. (a) LA: high compositionality
($MIS=1$). (b) LB: mediate compositionality ($MIS=0.83$). (c) LC: low compositionality ($MIS=0.41$).}
($\mathit{MIS}=1$). (b) LB: mediate compositionality ($\mathit{MIS}=0.83$). (c) LC: low compositionality ($\mathit{MIS}=0.41$).}
\label{fig:bench}
\end{figure}
\begin{figure*}[t]
\centering
\includegraphics[width=1.99\columnwidth]{fig/Figure9.pdf}
\includegraphics[width=\textwidth]{fig/Figure9.pdf}
\caption{Accuracy of Listeners when varying $h_{size}$ from 1 to 8. Each curve
represents an average accuracy trend from 50 repeated training, with the
range of [$\mu - \sigma$, $\mu + \sigma$], where $\mu$ is the average
......@@ -100,7 +100,7 @@ more meanings, for the constraint from low capacity.
We further breakdown the learning process to investigate the language teaching
scenario, where the Speaker teaches the Listener its fixed symbolic language.
We define three symbolic languages in different compositionality for Speaker to
teach, i.e., high (LA, $MIS=1$), mediate (LB, $MIS=0.83$), low (LC, $MIS=0.41$), see
teach, i.e., high (LA, $\mathit{MIS}=1$), mediate (LB, $\mathit{MIS}=0.83$), low (LC, $\mathit{MIS}=0.41$), see
Figure~\ref{fig:bench}.
Figure~\ref{fig:exp3} reports the accuracy of Listener, i.e., ratio of the correctly
......
......@@ -39,7 +39,7 @@ vocabulary can express almost infinite concepts.}
%
\begin{figure}[t]
\centering
\includegraphics[width=0.99\columnwidth]{fig/Figure1_motivation.pdf}
\includegraphics[width=\columnwidth]{fig/Figure1_motivation.pdf}
\caption{The distribution of compositionality when training for 100 symbolic
languages without
any induction. It can be observed that high compositional symbolic language
......
......@@ -45,7 +45,7 @@ $t=\hat{t}$ if $t$ expresses the same meaning as $\hat{t}$, e.g., ``red circle''
\begin{figure*}[t]
\centering
\includegraphics[width=1.8\columnwidth]{fig/Figure3_The_architecture_of_agents.pdf}
\includegraphics[width=\textwidth]{fig/Figure3_The_architecture_of_agents.pdf}
\caption{The architecture of agents. \emph{Left:} speaker. \emph{Right:} listener.}
\label{fig:agents}
\end{figure*}
......
......@@ -33,20 +33,20 @@ R\left(c_0,s_0\right) & R\left(c_0,s_0\right)
\end{equation}
Each column of $M$ correspond to the semantic information carried by one symbol. In a perfectly compositional language, each symbol represents one specific concept exclusively. Therefore, the similarity between the columns of $M$ and a one-hot vector is align with the compositionality of the emergent language.
Finally, we define \emph{raw mutual information similarity} ($MIS_0$)
Finally, we define \emph{raw mutual information similarity} ($\mathit{MIS}_0$)
as the average cosine similarity of $M$ columns and one-hot vectors, as
Equation~\ref{eq:mis2}. Furthermore, $MIS$ is the normalized raw mutual
Equation~\ref{eq:mis2}. Furthermore, $\mathit{MIS}$ is the normalized mutual
information similarity into the $[0,1]$ value range, which can be computed with
following formula:
\begin{equation}\label{eq:mis2}\begin{aligned}
MIS_0 &= \frac{1}{2}\sum_{j=0}^1\frac{\max_{i=0,1}RI\left(c_i,s_j\right)}{\epsilon + \sqrt{\sum_{i=0}^{1}RI^2\left(c_i,s_j\right)}}, \epsilon > 0\\
MIS &= 2MIS_0 - 1
\mathit{MIS}_0 &= \frac{1}{2}\sum_{j=0}^1\frac{\max_{i=0,1}R\left(c_i,s_j\right)}{\epsilon + \sqrt{\sum_{i=0}^{1}R^2\left(c_i,s_j\right)}}, \epsilon > 0\\
\mathit{MIS} &= 2\mathit{MIS}_0 - 1
\end{aligned}\end{equation}
Generalized to $m$ symbols and $n$ objects, $MIS$ can be computed with
Generalized to $m$ symbols and $n$ objects, MIS can be computed with
following formula:
\begin{equation}\label{eq:mis2}\begin{aligned}
MIS_0 &= \frac{1}{m}\sum_{j=0}^{m-1}\frac{\max_{i\in[0,n-1]}R\left(c_i,s_j\right)}{\epsilon + \sqrt{\sum_{i=0}^{n-1}R^2\left(c_i,s_j\right)}}, \epsilon > 0\\
MIS &= \frac{n\cdot MIS_0 - 1}{n-1}
\mathit{MIS}_0 &= \frac{1}{m}\sum_{j=0}^{m-1}\frac{\max_{i\in[0,n-1]}R\left(c_i,s_j\right)}{\epsilon + \sqrt{\sum_{i=0}^{n-1}R^2\left(c_i,s_j\right)}}, \epsilon > 0\\
\mathit{MIS} &= \frac{n\cdot \mathit{MIS}_0 - 1}{n-1}
\end{aligned}\end{equation}
\begin{figure}[t]
......@@ -56,5 +56,5 @@ MIS &= \frac{n\cdot MIS_0 - 1}{n-1}
\label{fig:unilateral}
\end{figure}
MIS is a bilateral metric. Unilateral metrics, e.g. \emph{topographic similarity (topo)}\cite{} and \emph{posdis}\cite{}, only take the policy of the speaker into consideration. We provide an example to illustrate the inadequacy of unilateral metrics, shown in Figure~\ref{fig:unilateral}. In this example, the speaker only uses $s_1$ to represent shape. From the perspective of speaker, the language is perfectly compositional (i.e. both topo and posdis are 1). However, the listener cannot distinguish the shape depend only on $s_1$, showing the non-compositionality in this language. The bilateral metric MIS addresses such defect by taking the policy of the listener into account, thus $MIS < 1$.
MIS is a bilateral metric. Unilateral metrics, e.g. \emph{topographic similarity (topo)}\cite{} and \emph{posdis}\cite{}, only take the policy of the speaker into consideration. We provide an example to illustrate the inadequacy of unilateral metrics, shown in Figure~\ref{fig:unilateral}. In this example, the speaker only uses $s_1$ to represent shape. From the perspective of speaker, the language is perfectly compositional (i.e. both topo and posdis are 1). However, the listener cannot distinguish the shape depend only on $s_1$, showing the non-compositionality in this language. The bilateral metric MIS addresses such defect by taking the policy of the listener into account, thus $\mathit{MIS} < 1$.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment