| … | … | @@ -5371,6 +5371,359 @@ WTF Am I doing? |
| 5371 | 5371 | that is what I am doing. |
| 5372 | 5372 | \end_layout |
| 5373 | 5373 | |
| 5374 | \begin_layout Subsection* |
| 5375 | 23 Jan 2026 6PM |
| 5376 | \end_layout |
| 5377 | |
| 5378 | \begin_layout Standard |
| 5379 | Have I been just plain wrong for the last 10–plus years? |
| 5380 | I've been searching for a way to learn by tokenization and counting – counting the co–occurrences of tokens. |
| 5381 | These naturally fall into a Gaussian distribution, |
| 5382 | indicating that pair–wise relationships between tokens are uniformly distributed on a very high–dimensional sphere. |
| 5383 | Since its a sphere, |
| 5384 | high–dimensional (sparse) vectors and vector products become the natural representational form for the data. |
| 5385 | The typical results similar to old vector embedding results on neural nets, |
| 5386 | followed. |
| 5387 | This implies that the results obtained from gradient descent on neural nets can also be obtained by counting statistics; |
| 5388 | that the structural vectorization of the data is independent of whether those vectors came from neural–net algorithms, |
| 5389 | or from counting algos. |
| 5390 | If the end result is the same – i.e. |
| 5391 | high–dimensional vectors, |
| 5392 | then the inquiry pivots to the performance of algorithms, |
| 5393 | and to second–order effects from the structuralist representation — |
| 5394 | the |
| 5395 | \begin_inset Quotes eld |
| 5396 | \end_inset |
| 5397 | |
| 5398 | structuralist representation |
| 5399 | \begin_inset Quotes erd |
| 5400 | \end_inset |
| 5401 | |
| 5402 | coming naturally from counting, |
| 5403 | and absent from weight matrices obtained from gradient descent. |
| 5404 | \end_layout |
| 5405 | |
| 5406 | \begin_layout Standard |
| 5407 | Before the advent of LLM coding assistants, |
| 5408 | the slog was long and slow. |
| 5409 | I wrote and rewrote MI computation code I don't recall how many times, |
| 5410 | and the last one, |
| 5411 | in opencog/matrix |
| 5412 | \begin_inset Quotes eld |
| 5413 | \end_inset |
| 5414 | |
| 5415 | worked |
| 5416 | \begin_inset Quotes erd |
| 5417 | \end_inset |
| 5418 | |
| 5419 | but was deficient in that it was (a) batching (b) not Atomese and thus not adaptable to real–time pipeline flows. |
| 5420 | So my attention shifted to working on real–time processing flows for sensory data. |
| 5421 | Which has proven to be quite the slog, |
| 5422 | itself, |
| 5423 | requiring lots of complex code and devoted attention to debugging. |
| 5424 | \end_layout |
| 5425 | |
| 5426 | \begin_layout Standard |
| 5427 | So I'm writing my bike today, |
| 5428 | and I realize, |
| 5429 | well, |
| 5430 | gee, |
| 5431 | I don't have to use tokenization and counting for the sensory processing, |
| 5432 | I could delegate it to DL–NN models. |
| 5433 | And this thought is what prompts this diary entry. |
| 5434 | \end_layout |
| 5435 | |
| 5436 | \begin_layout Standard |
| 5437 | And, |
| 5438 | like all my thoughts obtained during bike–riding, |
| 5439 | it contains a kernel of truth, |
| 5440 | but now, |
| 5441 | as I sit here, |
| 5442 | writing, |
| 5443 | I'm realizing that perhaps I wasn't wrong for the last ten–plus years. |
| 5444 | It's not at all clear–cut. |
| 5445 | \end_layout |
| 5446 | |
| 5447 | \begin_layout Standard |
| 5448 | The simplest example arises from LLMs. |
| 5449 | I can ask an LLM to read a large blob of text, |
| 5450 | and extract logical (factual) assertions from it. |
| 5451 | And, |
| 5452 | as is their nature, |
| 5453 | convert those assertions into any given symbolic representational form. |
| 5454 | LLM's excel at this. |
| 5455 | And ... |
| 5456 | and then what? |
| 5457 | Levi asked it to do just this, |
| 5458 | for a legal document, |
| 5459 | and asked it to convert the assertions in the legal text into deontic logic, |
| 5460 | expressed in prolog notation, |
| 5461 | and then jammed the the prolog into an ASP solver, |
| 5462 | turned the crank, |
| 5463 | and lo and behold, |
| 5464 | finds that that system is able to generate |
| 5465 | \begin_inset Quotes eld |
| 5466 | \end_inset |
| 5467 | |
| 5468 | correct |
| 5469 | \begin_inset Quotes erd |
| 5470 | \end_inset |
| 5471 | |
| 5472 | judgements, |
| 5473 | correct up to the point where any ambiguities in the source text were glossed over. |
| 5474 | \end_layout |
| 5475 | |
| 5476 | \begin_layout Standard |
| 5477 | So what is this? |
| 5478 | A stupid computer trick, |
| 5479 | I say. |
| 5480 | Surely someone has done this before, |
| 5481 | I imagine. |
| 5482 | Or maybe not. |
| 5483 | If not, |
| 5484 | its a great business opportunity. |
| 5485 | I don't know the legal profession, |
| 5486 | what they do, |
| 5487 | what they want, |
| 5488 | and have never been particularly drawn to it, |
| 5489 | beyond some superficial appreciation of it's importance to society. |
| 5490 | Getting a law degree takes years. |
| 5491 | I'm not going to put in that kind of time. |
| 5492 | So sure, |
| 5493 | the above architecture might be a great product, |
| 5494 | if some bigger corporate goon is not already creating just exactly that. |
| 5495 | But is it a good AI idea? |
| 5496 | That is the question. |
| 5497 | \end_layout |
| 5498 | |
| 5499 | \begin_layout Standard |
| 5500 | So again: |
| 5501 | my knee–jerk reaction is that it's just some stupid computer trick. |
| 5502 | Is it a worthy trick? |
| 5503 | The LLM is being used as a perceptual device, |
| 5504 | to perceive a certain kind of structure in it's input dataset. |
| 5505 | The human at the wheel told it exactly what to look for: |
| 5506 | laws and commandments, |
| 5507 | permissive and prohibitive expressions, |
| 5508 | written in English, |
| 5509 | and convert these to deontic logic. |
| 5510 | Why deontic logic? |
| 5511 | Because it is, |
| 5512 | duhh, |
| 5513 | |
| 5514 | \begin_inset Quotes eld |
| 5515 | \end_inset |
| 5516 | |
| 5517 | obviously |
| 5518 | \begin_inset Quotes erd |
| 5519 | \end_inset |
| 5520 | |
| 5521 | appropriate to the domain: |
| 5522 | scholars (and philosophers) have developed it precisely for this kind of application. |
| 5523 | So, |
| 5524 | there we have it: |
| 5525 | some software engineering, |
| 5526 | taken to the next step, |
| 5527 | thanks to programming assistants like ChatGPT and Claude, |
| 5528 | that can write code to perform this structural extraction and then apply a symbolic reasoning step to it. |
| 5529 | It's not a bad idea, |
| 5530 | and it plays to the strengths of both LLMs and of coding assistants. |
| 5531 | \end_layout |
| 5532 | |
| 5533 | \begin_layout Standard |
| 5534 | So why am I not impressed? |
| 5535 | Because it seems not to address any of the fundamental or important issues. |
| 5536 | Am I wrong? |
| 5537 | Am I missing something? |
| 5538 | I guess I should attempt to make a list of the fundamental, |
| 5539 | important issues. |
| 5540 | And then try to imagine how this kind of layering could represent some kind of important step. |
| 5541 | Lets try it. |
| 5542 | Some of these will be cheap shots, |
| 5543 | but I gotta start somewhere. |
| 5544 | \end_layout |
| 5545 | |
| 5546 | \begin_layout Itemize |
| 5547 | Asking the LLM to extract a certain kind of symbolic data from textual input is clearly a good engineering idea. |
| 5548 | \end_layout |
| 5549 | |
| 5550 | \begin_layout Itemize |
| 5551 | The human engineer has to select what it is that needs to be extracted. |
| 5552 | \end_layout |
| 5553 | |
| 5554 | \begin_layout Itemize |
| 5555 | Works well for scientific and technical texts, |
| 5556 | where factual statements are easy to come by. |
| 5557 | \end_layout |
| 5558 | |
| 5559 | \begin_layout Itemize |
| 5560 | Works poorly for philosophical texts, |
| 5561 | love poetry, |
| 5562 | artistic endeavors, |
| 5563 | mostly because I don't think a map of the human heart can be reduced to a small number of closed–form statements. |
| 5564 | \end_layout |
| 5565 | |
| 5566 | \begin_layout Standard |
| 5567 | Ohh!! |
| 5568 | A hah! |
| 5569 | That last bullet might be the thing! |
| 5570 | In science, |
| 5571 | math, |
| 5572 | law (biology, |
| 5573 | history ...) we work with a limited (small) number of factoids, |
| 5574 | limited by human abilities: |
| 5575 | 5-9 items in short–term memory, |
| 5576 | the speed of |
| 5577 | \begin_inset Quotes eld |
| 5578 | \end_inset |
| 5579 | |
| 5580 | System 2 |
| 5581 | \begin_inset Quotes erd |
| 5582 | \end_inset |
| 5583 | |
| 5584 | thinking applied to this collection of |
| 5585 | \begin_inset Quotes eld |
| 5586 | \end_inset |
| 5587 | |
| 5588 | crisp |
| 5589 | \begin_inset Quotes erd |
| 5590 | \end_inset |
| 5591 | |
| 5592 | relational assertions: |
| 5593 | if this, |
| 5594 | then that, |
| 5595 | If Czar Nicolas had not wasted his money sending a fleet to attack Japan, |
| 5596 | he could have instead built a railroad. |
| 5597 | A single–sentence if–then assertion that has a clear historical foundation from which factual assertions could be made. |
| 5598 | We humans can work with only a handful of these |
| 5599 | \begin_inset Quotes eld |
| 5600 | \end_inset |
| 5601 | |
| 5602 | in real time |
| 5603 | \begin_inset Quotes erd |
| 5604 | \end_inset |
| 5605 | |
| 5606 | , |
| 5607 | and, |
| 5608 | over a life–time accumulate tens of thousands of these factoids, |
| 5609 | that we maintain in long–term memory, |
| 5610 | and call upon, |
| 5611 | at will, |
| 5612 | when we cogitate. |
| 5613 | \end_layout |
| 5614 | |
| 5615 | \begin_layout Standard |
| 5616 | Insofar as text replaces long term memory, |
| 5617 | or, |
| 5618 | rather, |
| 5619 | is the mechanical implementation of long–term memory, |
| 5620 | and insofar as we deal with a domain that is filled with factual assertions, |
| 5621 | then yes, |
| 5622 | having the LLM do the reading, |
| 5623 | and then perform the causitive, |
| 5624 | logical extraction, |
| 5625 | which can then be piped into classical symbolic reasoning systems, |
| 5626 | this works. |
| 5627 | I will grant even that this will be (if it is not already) a boom industry for any and all domain–specific industrial applications. |
| 5628 | This is the de facto steam–engine of the modern era. |
| 5629 | This is the core, |
| 5630 | central engineering invention. |
| 5631 | This will drive yet another industrial revolution, |
| 5632 | and the investment of a trillion dollars into the effort seems not outlandish, |
| 5633 | given the nature of the beast. |
| 5634 | \end_layout |
| 5635 | |
| 5636 | \begin_layout Standard |
| 5637 | And yet, |
| 5638 | what do we do with love poetry and the human heart? |
| 5639 | This seem unscathed by this new invention. |
| 5640 | Will I still have sleepless nights and anxiety attacks? |
| 5641 | Of course. |
| 5642 | OK, |
| 5643 | disclaimer, |
| 5644 | I personally have these only somewhat rarely; |
| 5645 | I single them out as a prototypical human affliction, |
| 5646 | right up there with falling in love, |
| 5647 | having a mental disease, |
| 5648 | being angry, |
| 5649 | getting drunk, |
| 5650 | and being a snobby asshole seeking attention because whatever it is I have right now, |
| 5651 | its not good enough, |
| 5652 | and I need more, |
| 5653 | more ketchup, |
| 5654 | more pickles more than the ordinary grind. |
| 5655 | Can I ask an LLM to write love poetry? |
| 5656 | Of course, |
| 5657 | and it seems pretty damn good at it. |
| 5658 | Can I ask an LLM to extract logical assertions from it? |
| 5659 | No, |
| 5660 | because there is no particular logic to love, |
| 5661 | anger or being stoned. |
| 5662 | Can I ask the LLM to cure mental disease? |
| 5663 | No, |
| 5664 | or at least, |
| 5665 | not until NeuralLink or some other sci–fi technology can invade the synapses and perform something more subtle there than a lobotomy, |
| 5666 | quaaludes or lithium. |
| 5667 | \end_layout |
| 5668 | |
| 5669 | \begin_layout Standard |
| 5670 | So that's my criticism: |
| 5671 | a symbolic reasoner layered on an LLM is not AGI. |
| 5672 | Let me resume listing: |
| 5673 | \end_layout |
| 5674 | |
| 5675 | \begin_layout Itemize |
| 5676 | Lack of visual processing. |
| 5677 | Well, |
| 5678 | today. |
| 5679 | It should not be that hard to develop a visual subsystem that can identify a fast–moving object, |
| 5680 | and perform some motion prediction on it. |
| 5681 | Ballistic, |
| 5682 | rocket–power, |
| 5683 | a helicopter, |
| 5684 | a football, |
| 5685 | golf–ball, |
| 5686 | a ballerina or an ice–skater. |
| 5687 | Motion is clearly central to sports and the martial arts. |
| 5688 | I assume the military–industrial complex will continue to pour hudreds of billions of dollars into sensory systems. |
| 5689 | \end_layout |
| 5690 | |
| 5691 | \begin_layout Itemize |
| 5692 | It gets interesting, |
| 5693 | when applying AI to spy–agency stuff. |
| 5694 | Contact tracing, |
| 5695 | terrorist tracking, |
| 5696 | mole–hunting. |
| 5697 | And that looming one: |
| 5698 | disinformation. |
| 5699 | I have to put these on a back–burner. |
| 5700 | They deserve much more attention than I've given them in the past, |
| 5701 | but here and now is not the place to focus on these. |
| 5702 | \end_layout |
| 5703 | |
| 5704 | \begin_layout Standard |
| 5705 | But still, |
| 5706 | the above touches on the topic of perceiving abstract structure in data. |
| 5707 | How do I know that Paris is a capital? |
| 5708 | Because I *read about it* in a wikipedia article, |
| 5709 | and LLMs are good at reading. |
| 5710 | How is the ontological relationship grid created? |
| 5711 | Historically, |
| 5712 | it was done by humans: |
| 5713 | scholars and experts defined SUMO, |
| 5714 | an upper ontology, |
| 5715 | and developed FrameNet and whatever is the big one these days, |
| 5716 | whose name I can't remember. |
| 5717 | \end_layout |
| 5718 | |
| 5719 | \begin_layout Itemize |
| 5720 | Engineered: |
| 5721 | \end_layout |
| 5722 | |
| 5723 | \begin_layout Itemize |
| 5724 | axiomatic structure is predefined |
| 5725 | \end_layout |
| 5726 | |
| 5374 | 5727 | \begin_layout Section* |
| 5375 | 5728 | The End |
| 5376 | 5729 | \end_layout |