Recent advance on symbolic language in neural network based multi-agent systems have shown great progress in compositionality, which is taken as a distinguished feature of human language different from animal language. However, these efforts only explored environmental pressures, without realizing the importance of characterization capacity of agents.
In this work, we explore the relationship between the characterization capacity of agents and the compositionality of symbolic languages. By both proving with mutual information theory and verifying with extensive experiments, we made the counter-intuitive conclusion that symbolic languages with higher compositionality require lower characterization capacity of agents and are easier-to-teach.
\end{abstract}
\section{Introduction}
The emergence and evolution of human language has always been an important and controversial issue. The problem covers many fields, including artificial intelligence in computer science. Computer scientists induce the emergence and evolution of languages in multi-agent systems by setting up pure communication scenarios, such as referential games and communication-action policies.
Researchers have confirmed that agents can master a symbolic language to complete appointed tasks. Such symbolic language is a communication protocol using symbols or characters to represent concepts. Moreover, people not only care about the emergence of language, but also try to make the emergent language similar to human natural language.
Compositionality is a widely accepted metric used to measure the hierarchical complexity of language structure, and it is also a key feature to distinguish human language from animal language. Syntactic languages with high compositionality, such as human natural language, are able to express complex meanings through the combination of symbols and to produce certain syntax. In contrast, non-syntactic languages with low compositionality, such as animal languages, are almost impossible to extract specific concepts from a single symbol. Researchers have recognized the importance of compositionality and found that various environmental pressures would affect compositionality.
Besides environmental pressures, we suggest that the impact of internal factors from agents themselves on compositionality is equally significant. A biological hypothesis show that the cranial capacity of animals is not big enough to master languages with high compositionality. In neuron network based multi-agent systems, this hypothesis corresponds to a point of view that it’s difficult for agents with insufficient characterization capacity (i.e. number of neural nodes) to master languages with high compositionality. However, combine theoretical analysis and environmental results, we hold the complete opposite view -- within the range afforded by the need for successful communication, lower characterization capacity facilitates the emergence of symbolic language with higher compositionality.
For theoretical analysis, we use the MSC (Markov Series Channel) to model language transmission process and the probability distribution of symbols and concepts to model agents. Our methodology has the certain generalization ability cause it does not depend on the specific structure or algorithm of agents’ model. Combine the MSC model with mutual information theory, we certify the characterization capacity’s impact on compositionality theoretically. Specifically, we prove that a symbol of emergent languages with lower compositionality need carry more complex semantic information (i.e. mutual information between original concepts received by speaker and predicted concepts outputted by listener). So agents use such languages require more neural nodes in to characterize the semantic information.
In terms of experiments, in order to examine the relationship between capacity and compostionality in 'natural' environments, we avoid imposing any environmental pressures on agents through the following settings: a). Scenarios: a listener-speaker referential games for pure communication; b). Models: the listener and the speaker don’t share any parameters, and are not connected together to form an Auto-Encoder structure; c). Rewards: the only criterion for each of agents to receive a positive reward is whether the forecast output from the listener is correct. Under an experimental framework with such settings, the experimental results show that the effect of characterization capacity on compositionality is consistent with the theoretical analysis.
In addition, as a by-product of theoretical analysis, we propose ‘bilateral’ metrics for measuring compositionality and the degree of alignment between symbols and concepts. For the degree of alignment between symbols and concepts, the metric should be higher only if speaker and listener ‘bilateral’ correspond a symbol to the same concept more stably. For compositionality, we hold the view that a single symbol of symbolic languages with higher composionality should be used to ground or transmit a certain concept ‘bilaterally’ and more exclusively between listener and speaker.
To sum up, our contributions are as follows:
\begin{itemize}
\item We explore a novel factor (i.e. characterization capacity of agents) in compositionality, and show its impact both theoretically and experimentally.
\item We offer a methodology with the certain generalization ability to quantificationally analyze the process of language transmission.
\item We propose novel ‘bilateral’ metrics for measuring communication.
\end{itemize}
\section{Related work}
\section{Experimental Framework}
\section{Compositionality and Characterization Capacity}
\section{Theoretical Analysis}
\section{Experiments}
\section{Discussion}
\subsection{References}
The AAAI style includes a set of definitions for use in formatting references with BibTeX. These definitions make the bibliography style fairly close to the one specified below. To use these definitions, you also need the BibTeX style file ``aaai21.bst," available in the AAAI Author Kit on the AAAI web site. Then, at the end of your paper but before \textbackslash end{document}, you need to put the following lines:
Please note that the aaai21.sty class already sets the bibliographystyle for you, so you do not have to place any \textbackslash bibliographystyle command in the document yourselves. The aaai21.sty file is incompatible with the hyperref and navigator packages. If you use either, your references will be garbled and your paper will be returned to you.
References may be the same size as surrounding text. However, in this section (only), you may reduce the size to \textbackslash small if your paper exceeds the allowable number of pages. Making it any smaller than 9 point with 10 point linespacing, however, is not allowed. A more precise and exact method of reducing the size of your references minimally is by means of the following command: \begin{quote}
\textbackslash fontsize\{9.8pt\}\{10.8pt\}
\textbackslash selectfont\end{quote}
\noindent You must reduce the size equally for both font size and line spacing, and may not reduce the size beyond \{9.0pt\}\{10.0pt\}.
The list of files in the \textbackslash bibliography command should be the names of your BibTeX source files (that is, the .bib files referenced in your paper).
The following commands are available for your use in citing references:
\begin{quote}
{\em\textbackslash cite:} Cites the given reference(s) with a full citation. This appears as ``(Author Year)'' for one reference, or ``(Author Year; Author Year)'' for multiple references.\smallskip\\
{\em\textbackslash shortcite:} Cites the given reference(s) with just the year. This appears as ``(Year)'' for one reference, or ``(Year; Year)'' for multiple references.\smallskip\\
{\em\textbackslash citeauthor:} Cites the given reference(s) with just the author name(s) and no parentheses.\smallskip\\
{\em\textbackslash citeyear:} Cites the given reference(s) with just the date(s) and no parentheses.
\end{quote}
Formatted bibliographies should look like the following examples.
\smallskip\noindent\textit{Book with Multiple Authors}\\
Engelmore, R., and Morgan, A. eds. 1986. \textit{Blackboard Systems.} Reading, Mass.: Addison-Wesley.
\smallskip\noindent\textit{Journal Article}\\
Robinson, A. L. 1980a. New Ways to Make Microcircuits Smaller. \textit{Science} 208: 1019--1026.
\smallskip\noindent\textit{Magazine Article}\\
Hasling, D. W.; Clancey, W. J.; and Rennels, G. R. 1983. Strategic Explanations in Consultation. \textit{The International Journal of Man-Machine Studies} 20(1): 3--19.
\smallskip\noindent\textit{Proceedings Paper Published by a Society}\\
Clancey, W. J. 1983. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. In \textit{Proceedings of the Eighth International Joint Conference on Artificial Intelligence,} 556--560. Menlo Park, Calif.: International Joint Conferences on Artificial Intelligence, Inc.
\smallskip\noindent\textit{Proceedings Paper Published by a Press or Publisher}\\
Clancey, W. J. 1984. Classification Problem Solving. In \textit{Proceedings of the Fourth National Conference on Artificial Intelligence,} 49--54. Menlo Park, Calif.: AAAI Press.
Rice, J. 1986. Poligon: A System for Parallel Problem Solving, Technical Report, KSL-86-19, Dept. of Computer Science, Stanford Univ.
\smallskip\noindent\textit{Dissertation or Thesis}\\
Clancey, W. J. 1979. Transfer of Rule-Based Expertise through a Tutorial Dialogue. Ph.D. diss., Dept. of Computer Science, Stanford Univ., Stanford, Calif.
Clancey, W. J. 2021. The Engineering of Qualitative Models. Forthcoming.
For the most up to date version of the AAAI reference style, please consult the \textit{AI Magazine} Author Guidelines at \url{https://aaai.org/ojs/index.php/aimagazine/about/submissions#authorGuidelines}
\section{ Acknowledgments}
AAAI is especially grateful to Peter Patel Schneider for his work in implementing the original aaai.sty file, liberally using the ideas of other style hackers, including Barbara Beeton. We also acknowledge with thanks the work of George Ferguson for his guide to using the style and BibTeX files --- which has been incorporated into this document --- and Hans Guesgen, who provided several timely modifications, as well as the many others who have, from time to time, sent in suggestions on improvements to the AAAI style. We are especially grateful to Francisco Cruz, Marc Pujol-Gonzalez, and Mico Loretan for the improvements to the Bib\TeX{} and \LaTeX{} files made in 2020.
The preparation of the \LaTeX{} and Bib\TeX{} files that implement these instructions was supported by Schlumberger Palo Alto Research, AT\&T Bell Laboratories, Morgan Kaufmann Publishers, The Live Oak Press, LLC, and AAAI Press. Bibliography style changes were added by Sunil Issar. \verb+\+pubnote was added by J. Scott Penberthy. George Ferguson added support for printing the AAAI copyright slug. Additional changes to aaai21.sty and aaai21.bst have been made by Francisco Cruz, Marc Pujol-Gonzalez, and Mico Loretan.
\bigskip
\noindent Thank you for reading these instructions carefully. We look forward to receiving your electronic files!
