Fernando B. Avila-Rencoret, MD. Hamlyn Centre for Medical Robotics. Imperial College London, United Kingdom. fba13ATic.ac.uk, favilarATgmail.com. DOI: 10.6084/m9.figshare.22596856 LinkedIn Post on my profile: https://www.linkedin.com/posts/favi1ar_bridging-the-gap-a-self-supervised-hybrid-activity-7052073916348981248-tpc8
We create a program to understand language human-level by combining learning from lots of text and reasoning. The program learns from "moderators" helping make better connections between words and ideas.
We made a "baby" model to see how well it works. The baby model learns and gets better at understanding language. The program could help doctors, make computers talk better, and behave well.
We propose a hybrid model combining language models (LLMs) and human reasoning for human-level language understanding. Using moderator feedback, the self-supervised model optimizes parameters and connectivity in layers of semantic complexity, represented by layer graphs, semantic memory, combination rules, and transformation matrices.
The framework addresses neural network brittleness and symbolic AI challenges through iterative refinement with new inputs. The model initializes parameters randomly. Iterating with new inputs and feedback tunes the model. The "baby" model validates progress with new inputs. The "baby" model measures progress through accuracy, stability, and qualitative metrics. Following Constitutional AI, natural feedback cultivates understanding.
We propose a hybrid model combining human reasoning laws and language models (LLMs) for human-level language understanding. Building semantic complexity in layers, the model uses moderator feedback to optimize parameters. Progress is validated through a "baby" model.
While neural networks, like the Transformer (Vaswani et al., 2017) and BERT (Devlin et al., 2019), have achieved remarkable performance on NLP tasks, they lack the reasoning capabilities of symbolic AI (e.g. SHRDLU (Winograd, 1972), Cyc (Lenat, 1995)). However, symbolic approaches struggle with unstructured data. To address these limitations, we present a hybrid model that leverages neural networks for understanding language statistics and symbolic AI for cultivating reasoning. Our model achieves human-level language understanding through a self-supervised learning framework guided by moderator feedback.
The framework consists of several components:
-
Layer graphs represent the model's understanding of language at different levels of complexity. They are combined and transformed using semantic memory, combination rules, and transformation matrices. Semantic memory stores knowledge, while combination rules and transformation matrices control how the model combines and transforms linguistic structures.
-
The framework begins with layer graphs, semantic memory, combination rules and transformation matrices initialized randomly. It then iterates with new inputs and moderator feedback to refine the model. The validation model measures progress using new inputs.
-
Moderators evaluate interlayer connectivity and combination quality. Backpropagation optimizes semantic memory and access probabilities for better connectivity and combination. The validation model uses new inputs to measure progress.
-
This iterative process aligns with the concept of Constitutional AI, which proposes achieving AI safety through natural feedback (Waser, 2016). Like models for grounded language learning (Hermann et al., 2020), our model develops understanding through open-domain dialogue rather than strictly supervised data.
The framework iteratively refines the model's understanding of language using moderator feedback, optimizing its semantic memory and access probabilities for better connectivity and combination quality. The validation model is used to measure the model's progress empirically. This notation represents the main steps of the framework, which iteratively refines the model's understanding of language using moderator feedback and optimizes its semantic memory and access probabilities for better connectivity and combination quality. The validation model is used to measure the model's progress empirically. The full description is as follows:
-
Generate layer texts (lt) from layer LLMs.
$lt = GenerateText(l, llm)$
-
Extract linguistic primes (lp) using NLPK.
$lp = ExtractPrimes(ProcessTextNLPK(lt))$
-
Set initial weights (sm) and access probabilities (pt) for each layer.
$sm = RandomWeights(llm)$ $pt = EqualProbs(llm)$
-
Calculate connectivity tensors (C12, C23, C34).
$C_{12} = TensorProd(sm_1, pt_1, sm_2)$ $C_{23} = TensorProd(sm_2, pt_2, sm_3)$ $C_{34} = TensorProd(sm_3, pt_3, sm_4)$
-
Moderators (m) evaluate interlayer connectivity (lc) based on generated texts.
$lc = \langle \phi l_i | \phi l_{i+1} \rangle$
-
Backpropagate (∇C) to optimize semantic memory (sm) and access probabilities (pt).
$\nabla C = Backprop({C_{12}, C_{23}, C_{34}}, lc)$ $sm = sm + \nabla C_{sm} \cdot \alpha$ $pt = pt + \nabla C_{pt} \cdot \epsilon$
-
Combine nodes within layers to create layer graphs (lg) and compute their stability (st).
$lg = CombineNodes(l, sm, pt, cr, tm)$
-
Moderators rate the combination quality (crat).
$st = Mean(GraphStabilityMetrics(lg))$
-
Backpropagate to update semantic memory (∇SM), access probabilities (∇PT), combination rules (∇CR), and transformation matrices (∇TM).
$crat = RateCombinations(l)$
-
Define the validation model (vm) to evaluate the model's understanding using new primes (newP) and new rules (newR).
$sm = sm + Backprop(sm, crat) \cdot \alpha$ $pt = pt + Backprop(pt, crat) \cdot \epsilon$ $cr = cr + Backprop(cr, crat) \cdot \delta[]$ $tm = tm + Backprop(tm, crat) \cdot \delta[]$
- Compute validation results (vr), which include accuracy (acc) and stability (st).
-
$vm(newP, newR) = {vt, vs, acc}$ where $vt = GenerateText(llm)$ $vs = Summarize(vt, newP, newR)$ $acc = Mean(RateAccuracy(vs, vt))$
- Compute validation results:
$vr = {acc, st}$
The model starts with randomly initialized parameters. The iterative process tunes it using new inputs and moderator feedback. The "baby" model validates progress with new inputs.
This is implemented as wolfram language, not for execution, but for explicit implementation of the mathematical framework:
generateTexPLANtsC[l_, llm_, m_, lh_, cr_, tm_, α_, ε_, δ_] := Module[
{lt, lp, sm, pt, lg, lc, crat, ∇C, ∇SM, ∇PT, ∇CR, ∇TM, vr, vm},
lt = Table[Table[GenerateText[l[[i, llm[[i]]]]]], {i, lh}];
lp = AssociationMap[ExtractPrimes, Table[ProcessTextNLPK[#], {lt}]];
sm = Table[RandomWeights[llm[[i]]], {i, lh}];
pt = Table[EqualProbs[llm[[i]]], {i, lh}];
C12 = TensorProd[sm[[1]], pt[[1]], sm[[2]]];
C23 = TensorProd[sm[[2]], pt[[2]], sm[[3]]];
C34 = TensorProd[sm[[3]], pt[[3]], sm[[4]]];
lc = {m[[1]]〈φl[[1]]|φl[[2]]〉, m[[2]]〈φl[[2]]|φl[[3]]〉, m[[3]]〈φl[[3]]|φl[[4]]〉};
∇C = Backprop[{C12, C23, C34}, lc];
sm += ∇C[[1]] α;
pt += ∇C[[2]] ε;
lg = Table[CombineNodes[l[[i]], sm[[i]], pt[[i]], cr, tm], {i, lh}];
st = Mean[GraphStabilityMetrics[lg]];
crat = {m[[1]] RateCombinations[l[[1]]], m[[2]] RateCombinations[l[[2]]], m[[3]] RateCombinations[l[[3]]], m[[4]] RateCombinations[l[[4]]]};
∇SM = Backprop[sm, crat];
sm += ∇SM α;
∇PT = Backprop[pt, crat];
pt += ∇PT ε;
∇CR = Backprop[cr, crat];
cr += ∇CR δ[];
∇TM = Backprop[tm, crat];
tm += ∇TM δ[];
vm[newP_, newR_] := Module[{vt, vs, acc},
vt = Table[GenerateText[llm], {i, 10}];
vs = Table[Summarize[vt[[i]], newP, newR], {i, 10}];
acc = Mean[Table[m[[i]] RateAccuracy[vs[[i]], vt[[i]]], {i, 10}]]
];
vr = {acc, st};
]
The model should improve accuracy, stability, and qualitative language understanding. Scaling and computational complexity are key challenges, addressed through parallel processing, distributed computing, and optimizing algorithms.
This work addresses AI alignment and human-level understanding. Future work incorporates commonsense reasoning and social intelligence. Implications span AI safety, ethics and applications like healthcare where human judgment is critical.
Our model, implemented in Wolfram, integrates human feedback to optimize parameters and validate results. Combining neural networks and symbolic AI, it tackles fundamental questions in AI alignment and human-level language understanding.
Broader impacts include cultivating trustworthy AI and enabling real-world applications where human-level judgment is essential. This approach aligns AI development with human values through open-domain interactions and natural feedback.