Commit fa918e0d by Zidong Du

~

parent 9b50b663
...@@ -186,12 +186,14 @@ ...@@ -186,12 +186,14 @@
In this paper, \note{we are the first to successfully achieve high compositional In this paper, \note{we are the first to successfully achieve high compositional
symbolic language} in a \emph{natural} manner without handcrafted inductions. symbolic language} in a \emph{natural} manner without handcrafted inductions.
Initially, by investigating the compositionality of emerged symbolic Initially, by investigating the emerged symbolic
language after removing the \emph{deliberately handcrafted} language after removing the \emph{deliberately handcrafted}
inductions, we observe that the agent capacity plays a key role in inductions, we observe the difficulty in naturally generating high compositional
compositionality. We further reveal and characterize the \note{quantitative relationship} symbolic language.
%the agent capacity plays a key role in compositionality.
Further we reveal and characterize the \note{quantitative relationship}
between the agent capacity and the compositionality of symbolic language, with between the agent capacity and the compositionality of symbolic language, with
a novel mutual information-based metric for the compositionality. a novel mutual information-based metric for more reasonable measuring the compositionality.
% both theoretically and experimentally. % both theoretically and experimentally.
%The theoretical analysis is built on the MSC %The theoretical analysis is built on the MSC
%(Markov Series Channel) model for the language transmission process and a %(Markov Series Channel) model for the language transmission process and a
......
...@@ -40,7 +40,9 @@ vocabulary can express infinite concepts.} ...@@ -40,7 +40,9 @@ vocabulary can express infinite concepts.}
\begin{figure}[t] \begin{figure}[t]
\centering \centering
\includegraphics[width=0.99\columnwidth]{fig/Figure1_motivation.pdf} \includegraphics[width=0.99\columnwidth]{fig/Figure1_motivation.pdf}
\caption{} \caption{The distribution of compositionality when training 50 times without
any induction. It can be observed that high compositional symbolic language
seldom emerged (e.g., 0 for compositionality $>0.99$).}
\label{fig:induction} \label{fig:induction}
\end{figure} \end{figure}
...@@ -49,8 +51,8 @@ through \emph{deliberately handcrafted} inductions, e.g., small vocabulary ...@@ -49,8 +51,8 @@ through \emph{deliberately handcrafted} inductions, e.g., small vocabulary
sizes~\cite{}, memoryless~\cite{}, carefully constructed rewards~\cite{}, and sizes~\cite{}, memoryless~\cite{}, carefully constructed rewards~\cite{}, and
ease-of-teaching~\cite{}. \note{The possible intuition is that high compositional symbolic ease-of-teaching~\cite{}. \note{The possible intuition is that high compositional symbolic
language cannot emerge without induction in existing multi-agent environment.} language cannot emerge without induction in existing multi-agent environment.}
Figure~\ref{fig:induction} reports the compositionality when training two agents in the widely-used Figure~\ref{fig:induction} reports the compositionality when training two agents
listener-speaker referential game, and it can be observed that \note{the compositionality in the widely-used listener-speaker referential game for 50 times, and it can be observed that \note{the compositionality
of emerged symbolic language is extremely low without any induction}. of emerged symbolic language is extremely low without any induction}.
Though such unnatural inductions are useful, they prevent us from better understanding the mystery of Though such unnatural inductions are useful, they prevent us from better understanding the mystery of
the emergence of language and even intelligence among our pre-human ancestors. the emergence of language and even intelligence among our pre-human ancestors.
...@@ -62,10 +64,10 @@ environment and agents are sufficient for achieving high compositionality. ...@@ -62,10 +64,10 @@ environment and agents are sufficient for achieving high compositionality.
In this paper, we are the first work to achieve high compositional In this paper, we are the first work to achieve high compositional
symbolic language without any deliberately handcrafted induction. The key observation symbolic language without any deliberately handcrafted induction. The key observation
is that the internal \emph{agent capacity} plays a crucial role in the is that the internal \emph{agent capacity} plays a crucial role in the
compositionality of symbolic language, compositionality of symbolic language.
by %thoroughly %by thoroughly
analyzing the compositionality after removing the inductions in %analyzing the compositionality after removing the inductions in
the most widely-used listener-speaker referential game framework. %the most widely-used listener-speaker referential game framework.
Concretely, the relationship between the agent capacity and the compositionality Concretely, the relationship between the agent capacity and the compositionality
of symbolic language is characterized, with a novel mutual information-based of symbolic language is characterized, with a novel mutual information-based
metric for the compositionality. metric for the compositionality.
...@@ -180,12 +182,23 @@ In this paper, we made the following contributions: ...@@ -180,12 +182,23 @@ In this paper, we made the following contributions:
\item To our best knowledge, we are the first work to successfully achieve \item To our best knowledge, we are the first work to successfully achieve
high compositional symbolic high compositional symbolic
language naturally, without any deliberately handcrafted induction. language naturally, without any deliberately handcrafted induction.
\item We thoroughly analyze the compositionality of emerged symbolic language \item We analyze the compositionality of emerged symbolic language
after removing deliberately handcrafted inductions, and confirm that the agent after removing deliberately handcrafted inductions.
capacity acts as a key factor for compositionality. \item We propose a novel mutual information-based metric to measure the
\item We experimentally verified the conclusion for relationship between agent compositionality quantitatively, which is more reasonable.
\item We experimentally exploited the relationship between agent
capacity. Both theoretical analysis and capacity. Both theoretical analysis and
experimental results lead to a counter-intuitive conclusion that lower agent experimental results lead to a counter-intuitive conclusion that lower agent
capacity facilitates the emergence of symbolic language with higher capacity facilitates the emergence of symbolic language with higher
compositionality. compositionality.
\end{itemize} \end{itemize}
The rest of this paper is arranged as follows.
Section~\ref{sec:relatedwork} summarizes the related works.
Section~\ref{sec:thory}
introduces the experimental setup used in this study. Section~\ref{sec:mis}
describes our proposed the novel mutual information-based metric for measuring
the compositionality of symbolic language. Section~\ref{sec:exp} gives the
experimental results of the exploration for the relationship between agent
capacity and compositionality. Section~\ref{sec:con} concludes this paper.
\section{conclusion} \section{conclusion}
\label{sec:con}
In this paper, we are the first work to achieve high compositional In this paper, we are the first work to achieve high compositional
symbolic language without any deliberately handcrafted induction. symbolic language without any deliberately handcrafted induction.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment