Commit 239786c8 by Zidong Du

abstract

parent f38a07bb
......@@ -166,14 +166,21 @@
\maketitle
\begin{abstract}
The natural emergence of symbolic languages with high compositionality has
The natural emergence of symbolic languages with high compositionality has
attracted extensive attentions from a broad range of communities. Existing
studies only investigated the impacts of \emph{deliberately designed} external
environmental factors (e.g., small vocabulary sizes, carefully constructed
distractors, and ease-of-teaching), which may be too ideal to exist in the
real world, without considering the importance of internal capacity of agents.
studies achieve high compositionality through \emph{deliberately handcrafted}
inductions (e.g., small vocabulary sizes, carefully constructed distractors,
and ease-of-teaching) in multi-agent learning, which are unnatural.
Yet, few studies investigate the emergence of symbolic language with high compositionality in
\emph{``natural''} environments, i.e., without any deliberately handcrafted
inductions.
In this paper, we first reveal and characterize the quantitative relationship
In this paper, we are the first to successfully achieve high compositional symbolic
language in a purely \emph{natural} environment.
Initially, by thoroughly investigating the compositionality of symbolic
language emerged after removing the \emph{deliberately handcrafted}
inductions, we observe that the agent capacity plays the key role in
compositionality. We further reveal and characterize the quantitative relationship
between the agent capacity and the compositionality of symbolic language both
theoretically and experimentally. The theoretical analysis is built on the MSC
(Markov Series Channel) model for the language transmission process and a
......@@ -182,7 +189,27 @@
with eliminated external environment factors. Both theoretical analysis and
experimental results lead to a counter-intuitive conclusion that lower agent
capacity facilitates the emergence of symbolic language with higher
compositionality.
compositionality. Based on our conclusion, we are able to generate higher
compositional symbolic language with a high probability.
% The natural emergence of symbolic languages with high compositionality has
% attracted extensive attentions from a broad range of communities. Existing
% studies only investigated the impacts of \emph{deliberately designed} external
% environmental factors (e.g., small vocabulary sizes, carefully constructed
% distractors, and ease-of-teaching), which may be too ideal to exist in the
% real world, without considering the importance of internal capacity of agents.
%
% In this paper, we first reveal and characterize the quantitative relationship
% between the agent capacity and the compositionality of symbolic language both
% theoretically and experimentally. The theoretical analysis is built on the MSC
% (Markov Series Channel) model for the language transmission process and a
% novel mutual information-based metric for the compositionality. The
% experiments are conducted on a listener-speaker referential game framework
% with eliminated external environment factors. Both theoretical analysis and
% experimental results lead to a counter-intuitive conclusion that lower agent
% capacity facilitates the emergence of symbolic language with higher
% compositionality.
\end{abstract}
......
......@@ -14,30 +14,35 @@ learning~\cite{}. For example, \note{XXXX}
To evaluate the emerged symbolic language, compositionality is widely used and
taken as an important metric.
It
is a concept in the philosophy of language [1], which describes and quantifies
how complex expressions can be assembled out of simpler parts [2]. For example,
Figure1(a) shows a perfect compositional language (with maximum
compostionality). In this example, each shape is represented by a unique value
of symbol $s_0$ and each color is represented by symbol $s_1$. Figure1(b) shows a
language with low compostionality. Colors and shapes are ambiguous if only we
extract information from a single symbol.
\begin{figure}[t]
\centering
\includegraphics[width=0.9\columnwidth]{fig/occupy}
\caption{(a): The correspondence between symbol sequences ($s_0$, $s_1$) and (shape,
color) pairs in a perfectly compostional language. $s_0$, $s_1$ in {a, b, c}, shape
in {circle, square} and color in {red, blue, green}; (b): The correspondence
between symbol sequences ($s_0$, $s_1$) and (shape, color) pairs in a language with
low compostionality.}
\label{fig:symbols}
\end{figure}
taken as an important metric. Roughly, compositionality is a principle that the
meaning of a complex expression (e.g, phase), which is assembled out of the
given set of simple components (e.g., symbols),
is determined by its constituent components and the rules that combines them~\cite{}.
For example, the expression "AAAI is a conference'' is consists of two
meaningful words "''
%It
%is a concept in the philosophy of language [1], which describes and quantifies
%how complex expressions can be assembled out of simpler parts [2]. For example,
%Figure1(a) shows a perfect compositional language (with maximum
%compostionality). In this example, each shape is represented by a unique value
%of symbol $s_0$ and each color is represented by symbol $s_1$. Figure1(b) shows a
%language with low compostionality. Colors and shapes are ambiguous if only we
%extract information from a single symbol.
%
%
%\begin{figure}[t]
% \centering
% \includegraphics[width=0.9\columnwidth]{fig/occupy}
% \caption{(a): The correspondence between symbol sequences ($s_0$, $s_1$) and (shape,
%color) pairs in a perfectly compostional language. $s_0$, $s_1$ in {a, b, c}, shape
%in {circle, square} and color in {red, blue, green}; (b): The correspondence
%between symbol sequences ($s_0$, $s_1$) and (shape, color) pairs in a language with
%low compostionality.}
% \label{fig:symbols}
% \end{figure}
Prior studies focus on investigating how to affect the compositionality of the
emergent language. Researchers have found that various environmental pressures
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment