projects
/
culture.git
/ commitdiff
commit
grep
author
committer
pickaxe
?
search:
re
summary
|
shortlog
|
log
|
commit
| commitdiff |
tree
raw
|
patch
|
inline
| side by side (parent:
7d0b423
)
Update.
author
François Fleuret
<francois@fleuret.org>
Tue, 25 Jun 2024 06:42:23 +0000
(08:42 +0200)
committer
François Fleuret
<francois@fleuret.org>
Tue, 25 Jun 2024 06:42:23 +0000
(08:42 +0200)
report/culture.tex
patch
|
blob
|
history
diff --git
a/report/culture.tex
b/report/culture.tex
index
7bf330e
..
43aaefe
100644
(file)
--- a/
report/culture.tex
+++ b/
report/culture.tex
@@
-198,7
+198,13
@@
present in the original quizzes:
quizzes in the representation landscape of GPTs. There probably is a
subtle relation between the temperature (mutation rate) and the
number of models used to validate with the ``all but one'' criterion
quizzes in the representation landscape of GPTs. There probably is a
subtle relation between the temperature (mutation rate) and the
number of models used to validate with the ``all but one'' criterion
- (survival).
+ (survival criterion).
+
+\item The ``all but one'' could be ``all but K'', and there may be
+ some information-theoretical thing, where the goal is to maximize
+ mutual information, with $K=N$ being total randomness, so high
+ entropy but no structure, and $K=0$ is total determinism, so no
+ information to share.
\item The setup does not push toward any specific invariance or
property in the generated quizzes, their consistency is entirely due
\item The setup does not push toward any specific invariance or
property in the generated quizzes, their consistency is entirely due
@@
-216,17
+222,21
@@
present in the original quizzes:
\item This overall process probably fight the ``simplicity bias'': If
a model is lacking a ``cue'' that the others have, there will
\item This overall process probably fight the ``simplicity bias'': If
a model is lacking a ``cue'' that the others have, there will
- rapidly be quizzes that require
s
this cue, they will be added to the
+ rapidly be quizzes that require this cue, they will be added to the
training data, and that model will catch up.
\item The randomness of the process probably allow to even go beyond
just synchronizing the abilities of the models. There may be some
additional complexification of quizzes that get accepted by chance.
training data, and that model will catch up.
\item The randomness of the process probably allow to even go beyond
just synchronizing the abilities of the models. There may be some
additional complexification of quizzes that get accepted by chance.
-\item The current process to generate new quizzes, which simply sample
- them at random is very rudimentary and probably not sufficient in a
- real-data setup. It can probably be supplemented with a MCTS-type
- search.
+\item The current process to generate new quizzes, which simply
+ samples them at random is very rudimentary and probably not
+ sufficient in a real-data setup. It can probably be supplemented
+ with a MCTS-type search.
+
+\item There may be already in the generated quizzes some structure
+ that \textemph{we} do not pick up (e.g. certain color or motion
+ patterns).
\end{itemize}
\end{itemize}