| … | … | @@ -3,7 +3,7 @@ Structure Learning |
| 3 | 3 | New set for notes concerning the learning of structure. Different |
| 4 | 4 | chapter than the earlier notes. |
| 5 | 5 | |
| 6 | | Start with a recap of the earlier proect for learning structure. |
| 6 | Start with a recap of the earlier project for learning structure. |
| 7 | 7 | |
| 8 | 8 | Recap of Previous Learn Project |
| 9 | 9 | ------------------------------- |
| … | … | @@ -81,3 +81,86 @@ Status: Punt. Item C) is the aspirational goal. The code located at the |
| 81 | 81 | https://github.com/opencog/atomese-simd directory is some pre-prototype |
| 82 | 82 | of A) and B) and I might yet tinker with it some more. I do not yet have |
| 83 | 83 | an firm road-map for getting to C). |
| 84 | |
| 85 | What is Structure? |
| 86 | ------------------ |
| 87 | I wish to define "structure" in some quasi-model-theoretic terms. |
| 88 | "Quasi" because the goal is not to be slavish to the formal textbook |
| 89 | definition, but to convert it into an artistic, sculptural impression. |
| 90 | |
| 91 | Structures consist of axioms, rules, n-ary function terms, relations. |
| 92 | Formally, they also include the bool logical operators, but I want to |
| 93 | convert those into yet more functional terms. Why? Lets take a detour |
| 94 | into proof theory. |
| 95 | |
| 96 | Proof theory works with axioms and inference rules. We have some |
| 97 | collection of axioms, asserted a priori, and some other collection of |
| 98 | inference rules, also asserted a priori. A theorem prover chains these, |
| 99 | forward or backward, to find "proofs" for "theorems". Instead, I want to |
| 100 | think of this process as being a kind of generation, where rules and |
| 101 | axioms are specific building blocks that are assembled. |
| 102 | |
| 103 | The rules and axioms are jigsaw pieces; theorem proving is the assembly |
| 104 | of these. If it is free an unconstrained, then it is generation; if its |
| 105 | constrained, then some sort of satisfiability solver is needed: the |
| 106 | jigsaw much assmble so as to build be bridge between constraints. The |
| 107 | satisfiability algorithm must be fast, but also the bridge that is build |
| 108 | mst be minimal or optimal in some way. |
| 109 | |
| 110 | The words "minimal" and "optimal" make me personally think of minimal |
| 111 | surfaces: soap bubbles, for instance. Geodesics in Rimannian geometry. |
| 112 | Assemblies of jigsaws do not have smooth differentiable structure, but |
| 113 | still, some of the ideas carry over analogously, because there are |
| 114 | several natural ideas of "minimal" for jigaw assemblies. For example: |
| 115 | the assembly that uses the minimal number of pieces. Or, if we weight |
| 116 | each piece with e.g. log probability (i.e. "entropy" or "information") |
| 117 | then minimize the sum of that. |
| 118 | |
| 119 | The general "feel" of continutity is taken up by the Scott topology, |
| 120 | where we can talk about the homotopic deformation of jigsaw aassemblies. |
| 121 | There is a very easy way of understanding this if you are a software |
| 122 | programmer: a "homotopic deformation" is a "refactoring of the |
| 123 | software". Any collection of small changes that can be made to a |
| 124 | software program that does not alter it's function. Move if-statements |
| 125 | out of loops. Unroll loops. Stuff like that. These are homotopic |
| 126 | transformations: they alter the structure without altering function, |
| 127 | and, when taken as small, tiny changes, can be thought of as providing |
| 128 | a kind of "continuous deformation". |
| 129 | |
| 130 | Consider an optimizer inside a compiler: it has some assembly snippet, |
| 131 | using some registers and some insns, and it can re-arrange some of the |
| 132 | insns, replacing them with others, choosing different register |
| 133 | assignments, thus creating more efficient, compact code without altering |
| 134 | its function. The `-O2` flag on a C compiler. The optimizer does not |
| 135 | work on bare assembly, but in abstract insn sets, the "intermediate |
| 136 | langauge" being a way-point. |
| 137 | |
| 138 | But this analogy can be pushed "all the way": A compiler is a homotopic |
| 139 | defomration of a C/C++ program into assembly code; an assembler is a |
| 140 | further deformation (and linking!) into machine binary. A bridge between |
| 141 | two different represenations that does not alter the function: Whatever |
| 142 | the programmer wrote the C/C++ code to do, the machine binary will do |
| 143 | the same thing. The transformation from one to the other was |
| 144 | accomplished in a very long sequence of small steps. |
| 145 | |
| 146 | What about differentiability? Even that has an analog: finite |
| 147 | differences. When collections of jigsaws can be assigned a number (e.g. |
| 148 | total number of pieces; total weight; length of shortest path; total |
| 149 | number of connections/connectors; width (fatness) of necks, or of |
| 150 | waists... etc.) These may vary be discrete amounts, but differences can |
| 151 | be taken, giving an analogy to differentiation. |
| 152 | |
| 153 | Can one write a finite-difference version of the Euler-Lagrange |
| 154 | equations (the solution of which defines geodesics)? I don't know. |
| 155 | Perhaps. People already run around claiming that one should maximize |
| 156 | free energy (Gibbs free energy) or perhaps the "integrated information"; |
| 157 | all of these ideas imply some kind of differentiable stucture minimized |
| 158 | or maximized via some Euler-Lagrange-style differentail or difference |
| 159 | eqns. I don't want to get hung up on the details, but rather the |
| 160 | artistic impression: when there is stuff to be minimized and maximzed, |
| 161 | and when one has paths that can take you from here to there in some |
| 162 | quasi-continuous fashion, then one has some local difference-type |
| 163 | gradient that helps figure out how to move. |
| 164 | |
| 165 | |
| 166 | pairs. |