Skip to content

A
AAAI21_Emergent_language
Commit 74e3af09 by Zidong Du
Options

~

parent 264bb84e
Showing with 6 additions and 164 deletions
AAAI2021/paper.tex
......@@ -182,170 +182,12 @@
\end{abstract}
\section{Introduction}
\input{tex/introduction.tex}
\input{tex/relatedwork.tex}
\input{tex/theory.tex}
\input{tex/experiments.tex}
\input{tex/last.tex}
The emergence and evolution of human language has always been an important and
controversial issue. The problem covers many fields, including artificial
intelligence in computer science. Computer scientists induce the emergence and
evolution of languages in multi-agent systems by setting up pure communication
scenarios, such as referential games and communication-action policies.
Researchers have confirmed that agents can master a symbolic language to
complete appointed tasks. Such symbolic language is a communication protocol
using symbols or characters to represent concepts. Moreover, people not only
care about the emergence of language, but also try to make the emergent language
similar to human natural language.
Compositionality is a widely accepted metric used to measure the hierarchical
complexity of language structure, and it is also a key feature to distinguish
human language from animal language. Syntactic languages with high
compositionality, such as human natural language, are able to express complex
meanings through the combination of symbols and to produce certain syntax. In
contrast, non-syntactic languages with low compositionality, such as animal
languages, are almost impossible to extract specific concepts from a single
symbol. Researchers have recognized the importance of compositionality and found
that various environmental pressures would affect compositionality.
Besides environmental pressures, we suggest that the impact of internal factors
from agents themselves on compositionality is equally significant. A biological
hypothesis show that the cranial capacity of animals is not big enough to master
languages with high compositionality. In neuron network based multi-agent
systems, this hypothesis corresponds to a point of view that it’s difficult for
agents with insufficient characterization capacity (i.e. number of neural nodes)
to master languages with high compositionality. However, combine theoretical
analysis and environmental results, we hold the complete opposite view -- within
the range afforded by the need for successful communication, lower
characterization capacity facilitates the emergence of symbolic language with
higher compositionality.
For theoretical analysis, we use the MSC (Markov Series Channel) to model
language transmission process and the probability distribution of symbols and
concepts to model agents. Our methodology has the certain generalization ability
cause it does not depend on the specific structure or algorithm of agents’
model. Combine the MSC model with mutual information theory, we certify the
characterization capacity’s impact on compositionality
theoretically. Specifically, we prove that a symbol of emergent languages with
lower compositionality need carry more complex semantic information (i.e. mutual
information between original concepts received by speaker and predicted concepts
outputted by listener). So agents use such languages require more neural nodes
in to characterize the semantic information.
In terms of experiments, in order to examine the relationship between capacity
and compostionality in 'natural' environments, we avoid imposing any
environmental pressures on agents through the following settings: a). Scenarios:
a listener-speaker referential games for pure communication; b). Models: the
listener and the speaker don’t share any parameters, and are not connected
together to form an Auto-Encoder structure; c). Rewards: the only criterion for
each of agents to receive a positive reward is whether the forecast output from
the listener is correct. Under an experimental framework with such settings, the
experimental results show that the effect of characterization capacity on
compositionality is consistent with the theoretical analysis.
In addition, as a by-product of theoretical analysis, we propose ‘bilateral’
metrics for measuring compositionality and the degree of alignment between
symbols and concepts. For the degree of alignment between symbols and concepts,
the metric should be higher only if speaker and listener ‘bilateral’ correspond
a symbol to the same concept more stably. For compositionality, we hold the view
that a single symbol of symbolic languages with higher composionality should be
used to ground or transmit a certain concept ‘bilaterally’ and more exclusively
between listener and speaker.
To sum up, our contributions are as follows:
\begin{itemize}
\item We explore a novel factor (i.e. characterization capacity of agents) in compositionality, and show its impact both theoretically and experimentally.
\item We offer a methodology with the certain generalization ability to quantificationally analyze the process of language transmission.
\item We propose novel ‘bilateral’ metrics for measuring communication.
\end{itemize}
\section{Related work}
\section{Experimental Framework}
\section{Compositionality and Characterization Capacity}
\section{Theoretical Analysis}
\section{Experiments}
\section{Discussion}
\subsection{References}
The AAAI style includes a set of definitions for use in formatting references with BibTeX. These definitions make the bibliography style fairly close to the one specified below. To use these definitions, you also need the BibTeX style file ``aaai21.bst," available in the AAAI Author Kit on the AAAI web site. Then, at the end of your paper but before \textbackslash end{document}, you need to put the following lines:
\begin{quote}
\begin{small}
\textbackslash bibliography\{bibfile1,bibfile2,...\}
\end{small}
\end{quote}
Please note that the aaai21.sty class already sets the bibliographystyle for you, so you do not have to place any \textbackslash bibliographystyle command in the document yourselves. The aaai21.sty file is incompatible with the hyperref and navigator packages. If you use either, your references will be garbled and your paper will be returned to you.
References may be the same size as surrounding text. However, in this section (only), you may reduce the size to \textbackslash small if your paper exceeds the allowable number of pages. Making it any smaller than 9 point with 10 point linespacing, however, is not allowed. A more precise and exact method of reducing the size of your references minimally is by means of the following command: \begin{quote}
\textbackslash fontsize\{9.8pt\}\{10.8pt\}
\textbackslash selectfont\end{quote}
\noindent You must reduce the size equally for both font size and line spacing, and may not reduce the size beyond \{9.0pt\}\{10.0pt\}.
The list of files in the \textbackslash bibliography command should be the names of your BibTeX source files (that is, the .bib files referenced in your paper).
The following commands are available for your use in citing references:
\begin{quote}
{\em \textbackslash cite:} Cites the given reference(s) with a full citation. This appears as ``(Author Year)'' for one reference, or ``(Author Year; Author Year)'' for multiple references.\smallskip\\
{\em \textbackslash shortcite:} Cites the given reference(s) with just the year. This appears as ``(Year)'' for one reference, or ``(Year; Year)'' for multiple references.\smallskip\\
{\em \textbackslash citeauthor:} Cites the given reference(s) with just the author name(s) and no parentheses.\smallskip\\
{\em \textbackslash citeyear:} Cites the given reference(s) with just the date(s) and no parentheses.
\end{quote}
Formatted bibliographies should look like the following examples.
\smallskip \noindent \textit{Book with Multiple Authors}\\
Engelmore, R., and Morgan, A. eds. 1986. \textit{Blackboard Systems.} Reading, Mass.: Addison-Wesley.
\smallskip \noindent \textit{Journal Article}\\
Robinson, A. L. 1980a. New Ways to Make Microcircuits Smaller. \textit{Science} 208: 1019--1026.
\smallskip \noindent \textit{Magazine Article}\\
Hasling, D. W.; Clancey, W. J.; and Rennels, G. R. 1983. Strategic Explanations in Consultation. \textit{The International Journal of Man-Machine Studies} 20(1): 3--19.
\smallskip \noindent \textit{Proceedings Paper Published by a Society}\\
Clancey, W. J. 1983. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. In \textit{Proceedings of the Eighth International Joint Conference on Artificial Intelligence,} 556--560. Menlo Park, Calif.: International Joint Conferences on Artificial Intelligence, Inc.
\smallskip \noindent \textit{Proceedings Paper Published by a Press or Publisher}\\
Clancey, W. J. 1984. Classification Problem Solving. In \textit{Proceedings of the Fourth National Conference on Artificial Intelligence,} 49--54. Menlo Park, Calif.: AAAI Press.
\smallskip \noindent \textit{University Technical Report}\\
Rice, J. 1986. Poligon: A System for Parallel Problem Solving, Technical Report, KSL-86-19, Dept. of Computer Science, Stanford Univ.
\smallskip \noindent \textit{Dissertation or Thesis}\\
Clancey, W. J. 1979. Transfer of Rule-Based Expertise through a Tutorial Dialogue. Ph.D. diss., Dept. of Computer Science, Stanford Univ., Stanford, Calif.
\smallskip \noindent \textit{Forthcoming Publication}\\
Clancey, W. J. 2021. The Engineering of Qualitative Models. Forthcoming.
For the most up to date version of the AAAI reference style, please consult the \textit{AI Magazine} Author Guidelines at \url{https://aaai.org/ojs/index.php/aimagazine/about/submissions#authorGuidelines}
\section{ Acknowledgments}
AAAI is especially grateful to Peter Patel Schneider for his work in implementing the original aaai.sty file, liberally using the ideas of other style hackers, including Barbara Beeton. We also acknowledge with thanks the work of George Ferguson for his guide to using the style and BibTeX files --- which has been incorporated into this document --- and Hans Guesgen, who provided several timely modifications, as well as the many others who have, from time to time, sent in suggestions on improvements to the AAAI style. We are especially grateful to Francisco Cruz, Marc Pujol-Gonzalez, and Mico Loretan for the improvements to the Bib\TeX{} and \LaTeX{} files made in 2020.
The preparation of the \LaTeX{} and Bib\TeX{} files that implement these instructions was supported by Schlumberger Palo Alto Research, AT\&T Bell Laboratories, Morgan Kaufmann Publishers, The Live Oak Press, LLC, and AAAI Press. Bibliography style changes were added by Sunil Issar. \verb+\+pubnote was added by J. Scott Penberthy. George Ferguson added support for printing the AAAI copyright slug. Additional changes to aaai21.sty and aaai21.bst have been made by Francisco Cruz, Marc Pujol-Gonzalez, and Mico Loretan.
\bigskip
\noindent Thank you for reading these instructions carefully. We look forward to receiving your electronic files!
\bibliography{ref.bib}
\end{document}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment