| … | … | @@ -246,6 +246,136 @@ The analogy here is strained, and the notation is inadequate. |
| 246 | 246 | If the idea pans out. |
| 247 | 247 | \end_layout |
| 248 | 248 | |
| 249 | \begin_layout Standard |
| 250 | What are the |
| 251 | \begin_inset Formula $w_{i}$ |
| 252 | \end_inset |
| 253 | |
| 254 | really? One interpretation is to say |
| 255 | \begin_inset Formula $w_{i}=+1$ |
| 256 | \end_inset |
| 257 | |
| 258 | always, no matter what, and then the sum is simply a sum over the costs |
| 259 | of the disjuncts appearing in the sentence. |
| 260 | This is the conventional LG model. |
| 261 | Another intrpretation is to write |
| 262 | \begin_inset Formula $w_{i}=+1$ |
| 263 | \end_inset |
| 264 | |
| 265 | if that word appears in position |
| 266 | \begin_inset Formula $i$ |
| 267 | \end_inset |
| 268 | |
| 269 | in the sentence, else it is zero. |
| 270 | To arrive at this, we have to be more carefuly with the notation: Define |
| 271 | \begin_inset Formula |
| 272 | \[ |
| 273 | \delta\left(w,v\right)=\begin{cases} |
| 274 | 1 & \mbox{if }w=v\\ |
| 275 | 0 & \mbox{if }w\ne v |
| 276 | \end{cases} |
| 277 | \] |
| 278 | |
| 279 | \end_inset |
| 280 | |
| 281 | where |
| 282 | \begin_inset Formula $w,v$ |
| 283 | \end_inset |
| 284 | |
| 285 | are specific words in the lexis. |
| 286 | Then the Hamiltonian is |
| 287 | \begin_inset Formula |
| 288 | \[ |
| 289 | \mathcal{H}=-\sum_{uv\cdots}C_{uv\cdots}\delta\left(u,w_{i}\right)\delta\left(v,w_{j}\right)\delta\left(\cdots\right)\cdots |
| 290 | \] |
| 291 | |
| 292 | \end_inset |
| 293 | |
| 294 | so that the subscript |
| 295 | \begin_inset Formula $i$ |
| 296 | \end_inset |
| 297 | |
| 298 | encodes the position of a word in a sentence, and |
| 299 | \begin_inset Formula $w_{i}$ |
| 300 | \end_inset |
| 301 | |
| 302 | is the word in that position. |
| 303 | \end_layout |
| 304 | |
| 305 | \begin_layout Standard |
| 306 | This interpretation is unsatisfying, and wrong, the analogy doesn't work. |
| 307 | First of all, the spins in the Ising model can flip up and down, whereas |
| 308 | the observed word-order in a given sentence is fixed, inviolable. |
| 309 | The above picture could work for sentence generation, where we are considering |
| 310 | how to generate some senttence, but that is not yet currently viable given |
| 311 | what is written here, so far, |
| 312 | \end_layout |
| 313 | |
| 314 | \begin_layout Standard |
| 315 | The other issue is that, in classical LG, the disjunct costs |
| 316 | \begin_inset Formula $C_{uvw\cdots}$ |
| 317 | \end_inset |
| 318 | |
| 319 | are hand-crafted by the academic scholar defining the lexis. |
| 320 | The learn project attempts to replace these costs by those learned from |
| 321 | a training corpus, starting from a pair-wise frequenist counting scheme. |
| 322 | The |
| 323 | \begin_inset Formula $C_{uvw\cdots}$ |
| 324 | \end_inset |
| 325 | |
| 326 | are interpreted as mutual entropies, not energies. |
| 327 | There is a reasonable argument for why it should be entropy. |
| 328 | For the present case, we note that energies are also additive, so we can |
| 329 | keep our minds open about what is being additive, here: energy, information, |
| 330 | something else abelian, maybe. |
| 331 | \end_layout |
| 332 | |
| 333 | \begin_layout Standard |
| 334 | The symbolic learning project is to determine the lexis automatically. |
| 335 | To learn the correct numerical values of |
| 336 | \family roman |
| 337 | \series medium |
| 338 | \shape up |
| 339 | \size normal |
| 340 | \emph off |
| 341 | \bar no |
| 342 | \strikeout off |
| 343 | \xout off |
| 344 | \uuline off |
| 345 | \uwave off |
| 346 | \noun off |
| 347 | \color none |
| 348 | |
| 349 | \family default |
| 350 | \series default |
| 351 | \shape default |
| 352 | \size default |
| 353 | \emph default |
| 354 | \bar default |
| 355 | \strikeout default |
| 356 | \xout default |
| 357 | \uuline default |
| 358 | \uwave default |
| 359 | \noun default |
| 360 | \color inherit |
| 361 | |
| 362 | \begin_inset Formula $C_{uvw\cdots}$ |
| 363 | \end_inset |
| 364 | |
| 365 | . |
| 366 | And I want to do this by writing down some kind of Hamiltonian–like summation, |
| 367 | and attempting to minimize that. |
| 368 | But to do that, I need a dramatic change of notation. |
| 369 | Lets try it. |
| 370 | See what happens. |
| 371 | I have no idea if this is a good idea or a bad idea. |
| 372 | Or an old idea or a new idea. |
| 373 | \end_layout |
| 374 | |
| 375 | \begin_layout Standard |
| 376 | So, first, define a corpus: a collection of sentences. |
| 377 | \end_layout |
| 378 | |
| 249 | 379 | \begin_layout Section* |
| 250 | 380 | The End |
| 251 | 381 | \end_layout |