| … | … | @@ -5716,12 +5716,189 @@ But still, |
| 5716 | 5716 | whose name I can't remember. |
| 5717 | 5717 | \end_layout |
| 5718 | 5718 | |
| 5719 | | \begin_layout Itemize |
| 5720 | | Engineered: |
| 5719 | \begin_layout Standard |
| 5720 | So one of my daydreams, |
| 5721 | in pursuing the tokenized pair–correlation systems is that this is a way of obtaining symbolic ontological relationships between |
| 5722 | \begin_inset Quotes eld |
| 5723 | \end_inset |
| 5724 | |
| 5725 | things |
| 5726 | \begin_inset Quotes erd |
| 5727 | \end_inset |
| 5728 | |
| 5729 | . |
| 5730 | Early into this experiment, |
| 5731 | I discover that my par relationship are naturally these very high–dimensional vectors, |
| 5732 | and that this leads very naturally to a fuzzy categorization problem. |
| 5733 | I take my high–dimensional sphere, |
| 5734 | and slap a k–means or whatever clustering algo onto it. |
| 5735 | And if I squint, |
| 5736 | doing k–means by letting the points attract one–another |
| 5737 | \begin_inset Quotes eld |
| 5738 | \end_inset |
| 5739 | |
| 5740 | gravitationally |
| 5741 | \begin_inset Quotes erd |
| 5742 | \end_inset |
| 5743 | |
| 5744 | starts looking a lot like gradient descent. |
| 5745 | So, |
| 5746 | although I started with tokenized data, |
| 5747 | and hoped to discover a symbolic ontology out of it, |
| 5748 | what really happened was that a vector representation happened |
| 5749 | \begin_inset Quotes eld |
| 5750 | \end_inset |
| 5751 | |
| 5752 | automatically |
| 5753 | \begin_inset Quotes erd |
| 5754 | \end_inset |
| 5755 | |
| 5756 | , |
| 5757 | and the ontology emerges only from clustering. |
| 5721 | 5758 | \end_layout |
| 5722 | 5759 | |
| 5723 | | \begin_layout Itemize |
| 5724 | | axiomatic structure is predefined |
| 5760 | \begin_layout Standard |
| 5761 | And yet, |
| 5762 | these clusters have specific labels. |
| 5763 | And the relationships between clusters is also categorizable. |
| 5764 | So, |
| 5765 | in my daydream, |
| 5766 | I am discovering axioms and inference rules. |
| 5767 | Which is something that LLM's seem not capable of, |
| 5768 | at this time, |
| 5769 | or rather, |
| 5770 | (and this is important) ***not automatically***. |
| 5771 | \end_layout |
| 5772 | |
| 5773 | \begin_layout Standard |
| 5774 | That is, |
| 5775 | I can as an LLM to extract logical relationships between things, |
| 5776 | but as the human engineer, |
| 5777 | I have to have that logical system specified, |
| 5778 | a priori. |
| 5779 | Can I ask the LLM to extract axioms and inference rules and relationships, |
| 5780 | do novo? |
| 5781 | No, |
| 5782 | not at all, |
| 5783 | impossible. |
| 5784 | I already tried, |
| 5785 | been there, |
| 5786 | done that. |
| 5787 | Claude can talk a big game – its been trained on AI textbooks, |
| 5788 | – but it has no clue whatsoever what any of it means, |
| 5789 | so when you ask it to do something, |
| 5790 | its like some high–school sophomore handing in some mediocre essay that poorly regurgitates what the teacher had written on the blackboard only hours earlier. |
| 5791 | \end_layout |
| 5792 | |
| 5793 | \begin_layout Standard |
| 5794 | So, |
| 5795 | for me, |
| 5796 | its not AGI–ish until it can infer relations and inference rules from scratch, |
| 5797 | without human intervention. |
| 5798 | And so far, |
| 5799 | I believe that my frequentist counting approach, |
| 5800 | plus clustering, |
| 5801 | can achieve this, |
| 5802 | and have not been dissuaded from this vision. |
| 5803 | But its also a vision I have been utterly unable to communicate to anyone else; |
| 5804 | something about how I express it gets pooh–poohed. |
| 5805 | Oh well. |
| 5806 | As always, |
| 5807 | the question is how to find the time to build this system. |
| 5808 | \end_layout |
| 5809 | |
| 5810 | \begin_layout Standard |
| 5811 | And so, |
| 5812 | circling back to the original topic: |
| 5813 | can I build tools to build the tools? |
| 5814 | I'm trying to get Claude to write Atomese for me, |
| 5815 | and it sucks. |
| 5816 | I have various ideas on how to get Claude better at this. |
| 5817 | The bad news is Claude is proprietary, |
| 5818 | and I can be cut off from it from little notice, |
| 5819 | and replacing with a different LLM is ... |
| 5820 | uncertain. |
| 5821 | \end_layout |
| 5822 | |
| 5823 | \begin_layout Standard |
| 5824 | Above, |
| 5825 | I described the |
| 5826 | \begin_inset Quotes eld |
| 5827 | \end_inset |
| 5828 | |
| 5829 | steam engine for the modern era |
| 5830 | \begin_inset Quotes erd |
| 5831 | \end_inset |
| 5832 | |
| 5833 | ? |
| 5834 | Can I build a steam engine for myself? |
| 5835 | Err, |
| 5836 | well, |
| 5837 | what I need is not just domain–specific, |
| 5838 | but the domain is opaque: |
| 5839 | I'm running design experiments. |
| 5840 | I do know how to extract deontic logic from text, |
| 5841 | I don't know how to extract experimental software design logic from text. |
| 5842 | \end_layout |
| 5843 | |
| 5844 | \begin_layout Standard |
| 5845 | I did, |
| 5846 | at one point, |
| 5847 | try to build a memory prosthesis for Claude. |
| 5848 | I should get back to that project, |
| 5849 | someday. |
| 5850 | That is worthy. |
| 5851 | And it would be a prosthesis not just for Claude, |
| 5852 | but for something I could even run locally, |
| 5853 | on the GPU's right here. |
| 5854 | \end_layout |
| 5855 | |
| 5856 | \begin_layout Standard |
| 5857 | Well, |
| 5858 | OK, |
| 5859 | lets wrap it up. |
| 5860 | What have I learned? |
| 5861 | Yes, |
| 5862 | LLM's can be used as a perceptual system into text, |
| 5863 | and yes, |
| 5864 | asking the LLM to extract logical relationships from text, |
| 5865 | and attaching that logic to a symbolic reasoner is the paradigm that is driving, |
| 5866 | will drive the next number of decades of economic, |
| 5867 | industrial growth. |
| 5868 | No doubt about it. |
| 5869 | \end_layout |
| 5870 | |
| 5871 | \begin_layout Standard |
| 5872 | I guess I see this |
| 5873 | \begin_inset Quotes eld |
| 5874 | \end_inset |
| 5875 | |
| 5876 | early |
| 5877 | \begin_inset Quotes erd |
| 5878 | \end_inset |
| 5879 | |
| 5880 | , |
| 5881 | before 98% of the general population. |
| 5882 | But I see it |
| 5883 | \begin_inset Quotes eld |
| 5884 | \end_inset |
| 5885 | |
| 5886 | late |
| 5887 | \begin_inset Quotes erd |
| 5888 | \end_inset |
| 5889 | |
| 5890 | : |
| 5891 | the 2% of the industry insiders already know this, |
| 5892 | and are already building out the data centers, |
| 5893 | and sucking down the investment capital for it. |
| 5894 | So, |
| 5895 | ehh. |
| 5896 | \end_layout |
| 5897 | |
| 5898 | \begin_layout Standard |
| 5899 | Life goes on. |
| 5900 | Later, |
| 5901 | dude. |
| 5725 | 5902 | \end_layout |
| 5726 | 5903 | |
| 5727 | 5904 | \begin_layout Section* |