| … | … | @@ -553,7 +553,7 @@ For each sentence |
| 553 | 553 | This is all very super–basic stuff, but I am trying to be as clear as possible |
| 554 | 554 | here. |
| 555 | 555 | Sorry. |
| 556 | | The adjacency matrix is usually take to be a matrix woth just ones and |
| 556 | The adjacency matrix is usually take to be a matrix with just ones and |
| 557 | 557 | zeros in it, with one's taken to mean there is an edge, and zero meaning |
| 558 | 558 | there is not. |
| 559 | 559 | This can be generalized, but we're not doing that yet. |
| … | … | @@ -660,6 +660,11 @@ Several problems here. |
| 660 | 660 | Next, parsing is a syntactic activity; we've not touched base with semantics. |
| 661 | 661 | \end_layout |
| 662 | 662 | |
| 663 | \begin_layout Section* |
| 664 | Training vs. |
| 665 | Parsing |
| 666 | \end_layout |
| 667 | |
| 663 | 668 | \begin_layout Standard |
| 664 | 669 | So, where does that leave us? For now, lets ditch semantics: the original |
| 665 | 670 | problem was to discover a collection of disjuncts that adequately describe |
| … | … | @@ -667,7 +672,96 @@ So, where does that leave us? For now, lets ditch semantics: the original |
| 667 | 672 | The goal was to do this with some sort of expression of aggregate satisfiabilit |
| 668 | 673 | y: some kind of function assigning weights to disjuncts, and maximizing |
| 669 | 674 | it. |
| 670 | | |
| 675 | \end_layout |
| 676 | |
| 677 | \begin_layout Standard |
| 678 | That is, the corpus is fixed, immutable. |
| 679 | The words in the sentences are fixed, immutable. |
| 680 | There's a confusion between two different tasks. |
| 681 | In one task, the costs of the disjuncts are known, fixed, and the goal |
| 682 | of parsing is to find the collection of disjuncts that best fit the sentence. |
| 683 | The sum–total of these costs is the aggregate satisfiability. |
| 684 | In this mode, the disjuncts are analogous to the Ising spins, and we want |
| 685 | to find an arrangement of these spins/disjuncts that minimize the parsing |
| 686 | error: minimize the parse cost. |
| 687 | \end_layout |
| 688 | |
| 689 | \begin_layout Standard |
| 690 | Just like a ferromagnetic lattice, we acknowledge that there might be multiple |
| 691 | ambiguous parses, and that's OK, there is nothing wrong with that ambiguity. |
| 692 | For this task, it is entirely appropriate to use the partition function, |
| 693 | because the partition function captures the likelihood of any given parse |
| 694 | as being correct. |
| 695 | That is, |
| 696 | \begin_inset Formula $h\left(G\right)$ |
| 697 | \end_inset |
| 698 | |
| 699 | is the |
| 700 | \begin_inset Quotes eld |
| 701 | \end_inset |
| 702 | |
| 703 | energy |
| 704 | \begin_inset Quotes erd |
| 705 | \end_inset |
| 706 | |
| 707 | of a parse |
| 708 | \begin_inset Formula $G$ |
| 709 | \end_inset |
| 710 | |
| 711 | and the probability of that parse being reasonable is |
| 712 | \begin_inset Formula |
| 713 | \[ |
| 714 | P_{\beta}\left(G\right)=\frac{e^{-\beta h\left(G\right)}}{Z_{\beta}} |
| 715 | \] |
| 716 | |
| 717 | \end_inset |
| 718 | |
| 719 | This is the |
| 720 | \begin_inset Quotes eld |
| 721 | \end_inset |
| 722 | |
| 723 | configuration probability |
| 724 | \begin_inset Quotes erd |
| 725 | \end_inset |
| 726 | |
| 727 | of the parse |
| 728 | \begin_inset Formula $G$ |
| 729 | \end_inset |
| 730 | |
| 731 | , and there sentences like |
| 732 | \begin_inset Quotes eld |
| 733 | \end_inset |
| 734 | |
| 735 | I saw the man with the telescope |
| 736 | \begin_inset Quotes erd |
| 737 | \end_inset |
| 738 | |
| 739 | just says there are two parses |
| 740 | \begin_inset Formula $G$ |
| 741 | \end_inset |
| 742 | |
| 743 | and |
| 744 | \begin_inset Formula $G^{\prime}$ |
| 745 | \end_inset |
| 746 | |
| 747 | with |
| 748 | \begin_inset Formula $h\left(G\right)=h\left(G^{\prime}\right)$ |
| 749 | \end_inset |
| 750 | |
| 751 | . |
| 752 | \end_layout |
| 753 | |
| 754 | \begin_layout Standard |
| 755 | So that is one task: find the statistical distribution of parses that are |
| 756 | the most likely. |
| 757 | \end_layout |
| 758 | |
| 759 | \begin_layout Standard |
| 760 | The other task is to learn what the disjuncts are, to learn the cost of |
| 761 | each disjunct. |
| 762 | The costs are NOT fixed, we are instead exploring the space of possible |
| 763 | disjunct assignements. |
| 764 | For this we need to ... |
| 671 | 765 | \end_layout |
| 672 | 766 | |
| 673 | 767 | \begin_layout Section* |