From: François Fleuret <francois@fleuret.org>
Date: Tue, 25 Jun 2024 06:27:37 +0000 (+0200)
Subject: Update.
X-Git-Url: https://fleuret.org/cgi-bin/gitweb/gitweb.cgi?a=commitdiff_plain;ds=sidebyside;h=7d0b423ece4608825eed573eda70fd6b601cf80b;p=culture.git

Update.
---

diff --git a/report/culture.tex b/report/culture.tex
index fbeb1b4..7bf330e 100644
--- a/report/culture.tex
+++ b/report/culture.tex
@@ -190,6 +190,46 @@ present in the original quizzes:
 \includegraphics[scale=0.35]{pics/occlusions_1.png}
 \end{center}
 
+\section{Various thoughts}
+
+\begin{itemize}
+
+\item The whole process can be envisioned as natural selection of
+  quizzes in the representation landscape of GPTs. There probably is a
+  subtle relation between the temperature (mutation rate) and the
+  number of models used to validate with the ``all but one'' criterion
+  (survival).
+
+\item The setup does not push toward any specific invariance or
+  property in the generated quizzes, their consistency is entirely due
+  to the statistics of the ``world quizzes'' that remain in the
+  training set, and to the GPTs' inductive biased.
+
+\item The GPTs obviously get a sense of objectness and 2d topology
+  early on, since they rapidly increase the number of birds and
+  ``discover'' occlusion even though they never was in the world
+  quizzes.
+
+\item There may not be so many problems that can be cast as pairs of
+  patterns that are each a deterministic function of the other, which
+  is probably critical here.
+
+\item This overall process probably fight the ``simplicity bias'': If
+  a model is lacking a ``cue'' that the others have, there will
+  rapidly be quizzes that requires this cue, they will be added to the
+  training data, and that model will catch up.
+
+\item The randomness of the process probably allow to even go beyond
+  just synchronizing the abilities of the models. There may be some
+  additional complexification of quizzes that get accepted by chance.
+
+\item The current process to generate new quizzes, which simply sample
+  them at random is very rudimentary and probably not sufficient in a
+  real-data setup. It can probably be supplemented with a MCTS-type
+  search.
+
+\end{itemize}
+
 \section*{Appendix}
 
 The code is available at