| … | … | @@ -57,11 +57,11 @@ usually in cursive. Cursive writing is not standardized: there is a vast |
| 57 | 57 | variety of styles, especially problematic in 18th and 19th century |
| 58 | 58 | European records. There are errors from the transcription of these |
| 59 | 59 | texts. The spelling of names and surnames presents a challenge, |
| 60 | | espcially for European immigrants, where the spelling conventions clash |
| 60 | especially for European immigrants, where the spelling conventions clash |
| 61 | 61 | between the native language and English: many names are simplified and |
| 62 | 62 | Anglicized. Catholic Church parish records often convert native names to |
| 63 | 63 | Church Latin. Census records indicate that some people are either sloppy |
| 64 | | or are fudging the facts, or perhaps they really do not know thier |
| 64 | or are fudging the facts, or perhaps they really do not know their |
| 65 | 65 | birthdays or how they were spelling their surname a decade earlier. |
| 66 | 66 | Many surnames are extremely common, and there are many couples named |
| 67 | 67 | "John & Mary". |
| … | … | @@ -76,8 +76,8 @@ name, date, location, spelling, event. |
| 76 | 76 | The judgement call is often hierarchical in nature. For example: do the |
| 77 | 77 | father's and mother's names match? Well, perhaps they do, but there is |
| 78 | 78 | some alteration in spelling: Maria vs. Mary, Elizabeth vs. Betty. One |
| 79 | | can assign some probability or confidence in the beleif that these are |
| 80 | | the same parents. To butress that beleif, one can look at sibilings: if |
| 79 | can assign some probability or confidence in the belief that these are |
| 80 | the same parents. To butress that belief, one can look at siblings: if |
| 81 | 81 | two children are born three months apart to the same mother, then |
| 82 | 82 | this is a piece of negative evidence that absolutely demands a rejection |
| 83 | 83 | of a match. That is, of course, only if the recorded birthday is |
| … | … | @@ -96,11 +96,11 @@ aggregated and weighted in a large variety of ways (random forests, |
| 96 | 96 | collections of experts, membership clubs, etc.) |
| 97 | 97 | |
| 98 | 98 | A much simpler, and more easily automated example occurs in operating |
| 99 | | system storage integrity. Here, one is presented with mutiple files, |
| 99 | system storage integrity. Here, one is presented with multiple files, |
| 100 | 100 | often having the same name, but not always; often having the same |
| 101 | 101 | timestamp, but not always; often appearing in the same directory, but |
| 102 | 102 | not always; often having the same size, but not always; often having the |
| 103 | | same content, but sometimes haveing all-zeros where the data should have |
| 103 | same content, but sometimes having all-zeros where the data should have |
| 104 | 104 | been. Or perhaps MP3's with missng bytes, or images with steganographic |
| 105 | 105 | watermarks. The challenge is to find the copy of the file that is least |
| 106 | 106 | likely to be corrupted, and is most likely to be "correct". Despite the |
| … | … | @@ -124,7 +124,7 @@ sub-pieces also fit? This confidence may also be obtained by spreading |
| 124 | 124 | over the local neighborhood: is coherence maintained over not just |
| 125 | 125 | nearest neighbors, but also next-nearest-neighbors, etc.? In Link |
| 126 | 126 | Grammar, this is called a "linkage": all jigsaws must be linkable, from |
| 127 | | one end to the other; this is a global property infered from the local |
| 127 | one end to the other; this is a global property inferred from the local |
| 128 | 128 | linkage constraints. |
| 129 | 129 | |
| 130 | 130 | There do structural factoids come from? In natural language parsing, |
| … | … | @@ -161,19 +161,19 @@ but at a meta-level: i.e. after preliminary analysis has been done. |
| 161 | 161 | Thus, this is a recursive process. |
| 162 | 162 | |
| 163 | 163 | Figuring out what to perceive is also a learning process: examining and |
| 164 | | comparing input steams allows common patterns to be perceived in those |
| 164 | comparing input streams allows common patterns to be perceived in those |
| 165 | 165 | streams. The common patterns are "learned". The common patterns can then |
| 166 | 166 | be used to classify the input stream into buckets. Thus, classification |
| 167 | 167 | can also be thought of as a sensory organ. It's not instant-on: it takes |
| 168 | 168 | a while to come up to speed. Once running, it can provide descriptions |
| 169 | 169 | of data. Which perhaps sounds fine, but raises the problem of the |
| 170 | 170 | homunculus: who, exactly is perceiving the structural forms that the |
| 171 | | classification susbsystem is generating? |
| 171 | classification subsystem is generating? |
| 172 | 172 | |
| 173 | 173 | Clearly, there are a number of recursive issues. Transformers side-step |
| 174 | 174 | these issues into two ways. The low-level issue of perceiving structure |
| 175 | | is swept aside by only asking for the probablity P(X_n | Y_1, ..., Y_k) |
| 176 | | of some occurance of X_n conditioned in a set of (earlier) Y's. This |
| 175 | is swept aside by only asking for the probability P(X_n | Y_1, ..., Y_k) |
| 176 | of some occurrence of X_n conditioned in a set of (earlier) Y's. This |
| 177 | 177 | pushes all structural information into the weight matrix, in some opaque |
| 178 | 178 | and non-interpretable way. Yes, leaps and bounds in interpretability are |
| 179 | 179 | being made: the tens of thousands of employees of the |
| … | … | @@ -199,7 +199,7 @@ TensorFlow. These are structural, symbolic languages. Insofar as there |
| 199 | 199 | is progress at the meta-level, then it is in the use of vibe coding to |
| 200 | 200 | design transformers. Which is indeed an interesting approach. But the |
| 201 | 201 | goal here could be rephrased as saying "a system that can vibe-code |
| 202 | | itself". And the basic research to acheive this. |
| 202 | itself". And the basic research to achieve this. |
| 203 | 203 | |
| 204 | 204 | Methodology |
| 205 | 205 | ----------- |