| … | … | @@ -116,9 +116,9 @@ Introduction |
| 116 | 116 | \end_layout |
| 117 | 117 | |
| 118 | 118 | \begin_layout Standard |
| 119 | | What follows will be free-form unstructured writing about aggregate satisfiabili |
| 119 | What follows will be free–form unstructured writing about aggregate satisfiabili |
| 120 | 120 | ty. |
| 121 | | It is an attempt to reconcile a number of quite common and inter-related |
| 121 | It is an attempt to reconcile a number of quite common and inter–related |
| 122 | 122 | ideas, and to develop a mathematical frammework for them. |
| 123 | 123 | The key ingredients are: |
| 124 | 124 | \end_layout |
| … | … | @@ -129,7 +129,7 @@ The Ising model as a motivational example. |
| 129 | 129 | \end_layout |
| 130 | 130 | |
| 131 | 131 | \begin_layout Itemize |
| 132 | | Link Grammar parsing, in which syntactic jigdaw pieces are assembled to |
| 132 | Link Grammar parsing, in which syntactic jigsaw pieces are assembled to |
| 133 | 133 | create parses, called |
| 134 | 134 | \begin_inset Quotes eld |
| 135 | 135 | \end_inset |
| … | … | @@ -149,7 +149,7 @@ Aggregate satisfiability, in that satisfiability refers to discrete logical |
| 149 | 149 | \end_layout |
| 150 | 150 | |
| 151 | 151 | \begin_layout Standard |
| 152 | | There are also some other inter-related concepts, which may of may not the |
| 152 | There are also some other inter–related concepts, which may of may not the |
| 153 | 153 | ideas to be developed here. |
| 154 | 154 | These include ideas like: |
| 155 | 155 | \end_layout |
| … | … | @@ -169,7 +169,7 @@ Hopfield networks: these have commonality with regard to assorted physics |
| 169 | 169 | \begin_layout Itemize |
| 170 | 170 | Mean field theory. |
| 171 | 171 | Who knows. |
| 172 | | A knee-jerk reaction is that mean–field theory will describe the rsults. |
| 172 | A knee–jerk reaction is that mean–field theory will describe the rsults. |
| 173 | 173 | I suppose. |
| 174 | 174 | Seems plausible. |
| 175 | 175 | \end_layout |
| … | … | @@ -261,7 +261,7 @@ What are the |
| 261 | 261 | always, no matter what, and then the sum is simply a sum over the costs |
| 262 | 262 | of the disjuncts appearing in the sentence. |
| 263 | 263 | This is the conventional LG model. |
| 264 | | Another intrpretation is to write |
| 264 | Another interpretation is to write |
| 265 | 265 | \begin_inset Formula $w_{i}=+1$ |
| 266 | 266 | \end_inset |
| 267 | 267 | |
| … | … | @@ -270,7 +270,7 @@ What are the |
| 270 | 270 | \end_inset |
| 271 | 271 | |
| 272 | 272 | in the sentence, else it is zero. |
| 273 | | To arrive at this, we have to be more carefuly with the notation: Define |
| 273 | To arrive at this, we have to be more carefully with the notation: Define |
| 274 | 274 | \begin_inset Formula |
| 275 | 275 | \[ |
| 276 | 276 | \delta\left(w,v\right)=\begin{cases} |
| … | … | @@ -308,9 +308,9 @@ so that the subscript |
| 308 | 308 | \begin_layout Standard |
| 309 | 309 | This interpretation is unsatisfying, and wrong, the analogy doesn't work. |
| 310 | 310 | First of all, the spins in the Ising model can flip up and down, whereas |
| 311 | | the observed word-order in a given sentence is fixed, inviolable. |
| 311 | the observed word–order in a given sentence is fixed, inviolable. |
| 312 | 312 | The above picture could work for sentence generation, where we are considering |
| 313 | | how to generate some senttence, but that is not yet currently viable given |
| 313 | how to generate some sentence, but that is not yet currently viable given |
| 314 | 314 | what is written here, so far, |
| 315 | 315 | \end_layout |
| 316 | 316 | |
| … | … | @@ -319,9 +319,9 @@ The other issue is that, in classical LG, the disjunct costs |
| 319 | 319 | \begin_inset Formula $C_{uvw\cdots}$ |
| 320 | 320 | \end_inset |
| 321 | 321 | |
| 322 | | are hand-crafted by the academic scholar defining the lexis. |
| 322 | are hand–crafted by the academic scholar defining the lexis. |
| 323 | 323 | The learn project attempts to replace these costs by those learned from |
| 324 | | a training corpus, starting from a pair-wise frequenist counting scheme. |
| 324 | a training corpus, starting from a pair–wise frequentist counting scheme. |
| 325 | 325 | The |
| 326 | 326 | \begin_inset Formula $C_{uvw\cdots}$ |
| 327 | 327 | \end_inset |
| … | … | @@ -405,7 +405,7 @@ sentence |
| 405 | 405 | to be independent. |
| 406 | 406 | For now. |
| 407 | 407 | Of course, ten books written on the same topic are not really independent |
| 408 | | of one-another, but lets not get ahead of ourselves. |
| 408 | of one–another, but lets not get ahead of ourselves. |
| 409 | 409 | \end_layout |
| 410 | 410 | |
| 411 | 411 | \begin_layout Standard |
| … | … | @@ -448,7 +448,7 @@ where the sum ranges over all sentences in the corpus, and |
| 448 | 448 | |
| 449 | 449 | \begin_layout Standard |
| 450 | 450 | And so here we meet our first conceptual problem: the linkage is not unique: |
| 451 | | any given sentence may have many linkages, and, in fact, we have to consdier, |
| 451 | any given sentence may have many linkages, and, in fact, we have to consider, |
| 452 | 452 | in principle, all possible linkages. |
| 453 | 453 | \end_layout |
| 454 | 454 | |
| … | … | @@ -533,7 +533,7 @@ For each sentence |
| 533 | 533 | |
| 534 | 534 | . |
| 535 | 535 | As a set, the order no longer really matters. |
| 536 | | But it is a multi-set, so its OK to have a word show repeatedly. |
| 536 | But it is a multi–set, so its OK to have a word show repeatedly. |
| 537 | 537 | An edge is just a conventional graph edge: it is an ordered pair of vertexes. |
| 538 | 538 | Notation: |
| 539 | 539 | \begin_inset Formula $E_{ij}=\left(w_{i},w_{j}\right)$ |
| … | … | @@ -550,7 +550,7 @@ For each sentence |
| 550 | 550 | \end_inset |
| 551 | 551 | |
| 552 | 552 | the set of edges can be thought of that the adjacency matrix. |
| 553 | | This is all very super-basic stuff, but I am trying to be as clear as posible |
| 553 | This is all very super–basic stuff, but I am trying to be as clear as possible |
| 554 | 554 | here. |
| 555 | 555 | Sorry. |
| 556 | 556 | The adjacency matrix is usually take to be a matrix woth just ones and |
| … | … | @@ -563,11 +563,11 @@ For each sentence |
| 563 | 563 | A connected graph is one where every vertex is reachable by a path from |
| 564 | 564 | any other vertex. |
| 565 | 565 | We want to consider only connected graphs; the ability to disconnect will |
| 566 | | be handled via a different machanism. |
| 566 | be handled via a different mechanism. |
| 567 | 567 | \end_layout |
| 568 | 568 | |
| 569 | 569 | \begin_layout Standard |
| 570 | | For a fixed senttence |
| 570 | For a fixed sentence |
| 571 | 571 | \begin_inset Formula $S$ |
| 572 | 572 | \end_inset |
| 573 | 573 | |
| … | … | @@ -614,6 +614,59 @@ Z_{\beta}=\sum_{G\in\mathcal{G}}e^{-\beta h\left(G\right)} |
| 614 | 614 | \end_inset |
| 615 | 615 | |
| 616 | 616 | So, which is it? I'm confused, here. |
| 617 | \end_layout |
| 618 | |
| 619 | \begin_layout Standard |
| 620 | Lets go outside and touch grass. |
| 621 | In classical physics, in the classical Ising model, we say that the spins |
| 622 | at each lattice site have a distinct, explicit value. |
| 623 | Any given physical instance consists of only one configuration of those |
| 624 | spins. |
| 625 | The thermodynamic ensemble is the collection of all possible spin arrangements. |
| 626 | The partition function is a sum over the entire ensemble, giving a Boltzmann |
| 627 | weight to each particular spin arrangement. |
| 628 | \end_layout |
| 629 | |
| 630 | \begin_layout Standard |
| 631 | What, exactly am I doing here, if I am confronted by a sentence, a sequence |
| 632 | of words, a sequence of tokens? Should I say that there is only one acceptable |
| 633 | parse of that sentence? This is the classic Chomskian answer. |
| 634 | Do I need to acknowledge the possibility of multiple ambiguous parses? |
| 635 | Of course: |
| 636 | \begin_inset Quotes eld |
| 637 | \end_inset |
| 638 | |
| 639 | I saw the man with the telescope. |
| 640 | \begin_inset Quotes erd |
| 641 | \end_inset |
| 642 | |
| 643 | said the linguistics textbooks, back when I was in college. |
| 644 | Is my reality split, a superposition of two different possibilities? Well, |
| 645 | yes it is. |
| 646 | The quantum people say so. |
| 647 | The Bayesian people say so, and so do the neuroscientists. |
| 648 | The linguists merely state that both parses are possible, until I learn |
| 649 | some additional information: how has the telescope: me, or the man? Once |
| 650 | I learn this additional bit of information, the wave function collapses, |
| 651 | the sentence is no longer ambiguous, I know which parse to use, and how |
| 652 | to determine the meaning of that sentence. |
| 653 | \end_layout |
| 654 | |
| 655 | \begin_layout Standard |
| 656 | Several problems here. |
| 657 | First, the mistake of treating sentences as assertions of truth. |
| 658 | Peotry is ambiguous, and one of the points of poetry is that it never collapses. |
| 659 | You get to reinterpret poetry in various ways: its evocative, not assertive. |
| 660 | Next, parsing is a syntactic activity; we've not touched base with semantics. |
| 661 | \end_layout |
| 662 | |
| 663 | \begin_layout Standard |
| 664 | So, where does that leave us? For now, lets ditch semantics: the original |
| 665 | problem was to discover a collection of disjuncts that adequately describe |
| 666 | possible syntactic parses of sentences. |
| 667 | The goal was to do this with some sort of expression of aggregate satisfiabilit |
| 668 | y: some kind of function assigning weights to disjuncts, and maximizing |
| 669 | it. |
| 617 | 670 | |
| 618 | 671 | \end_layout |
| 619 | 672 | |