- Joined
- Jan 20, 2026
- Messages
- 345
- Reaction score
- 2,366
A perfectly transparent algorithm is our secure future.
Artificial intelligence often works like a black box: there is an answer on the output, but the path to it remains a mystery even for developers. Scientists from the University of Loughborough in the UK suggest not to try to open a ready-made neural network after training, and from the very beginning to build a system so that learning, memory and solutions can be traced step by step.
The paper describes the mathematical basis for AI, which shows how he learns, what remembers and why it chooses a particular option. For current neural networks, this is a sore place: knowledge is distributed according to a huge number of internal parameters, so the final answer often has to be explained outside, through additional tools and approximate interpretations.
The team assembled a prototype with two related parts: a computational module that plays the role of a conditional brain, and a separate memory. The system can learn continuously and at the same time not destroy the old knowledge. For AI, this ability is important: many models after training cope worse with old tasks. This failure is called disastrous forgetting. A person can learn a new melody and not stop recognizing the old one, and a neural network without special defenses sometimes reorganizes internal connections so that the former skill just breaks down.
The authors are also trying to solve the problem of false memories. In AI, this is understood as a situation where the system confidently uses information that was not in its experience, or binds the data incorrectly. Generative models have a similar effect in plausible but erroneous responses. In the prototype from Loughborough, memory changes transparently: you can see what information has been preserved, which has intensified over time, what has weakened and where the specific state came from.
The first checks were simple, but the idea was well shown. The prototype without training with the teacher memorized musical notes and short musical phrases, and also recognized and stored colors in images, for example, in cartoon pictures. In all tasks, the system behaved predictably: researchers could trace how memory changed and how the model came to the result.
The approach is based on a mathematical construct, which the authors call a plastic vector field. The vector field can be represented as a map of directions: at each point, the system understands where and how to move on. The word plastic here means the ability to change under the influence of experience.
The main idea of the work is that the behavior, memory and physical structure of the system cannot be considered separately. In the physical intelligence, behavior is associated with brain activity, and brain activity depends on how the connections are arranged and how they change. Researchers are trying to transfer this logic to AI: not just teach the model to give the correct answer, but to set a clear mechanism that shows how the answer arose.
Separately, scientists criticize modern artificial neural networks. In their opinion, the problem of explanation is not limited to a lack of convenient analysis tools. The restriction is deeper, in the architecture itself. The neural network can work well in practice, but the developer does not control the full way as the model learns and where exactly stores information. Therefore, the explanation is often conjecture, not a direct observation.
The prototype is still far from the systems that can be immediately introduced into medicine, finance or automated control. The model needs to be scalable, checked on more complex data, and understand how it will behave beyond demonstration tasks. The team also wants to learn how such a principle can be transferred to new types of equipment, where memory and computing will be closely related than in conventional software neural networks. Well, it's still ahead...
Artificial intelligence often works like a black box: there is an answer on the output, but the path to it remains a mystery even for developers. Scientists from the University of Loughborough in the UK suggest not to try to open a ready-made neural network after training, and from the very beginning to build a system so that learning, memory and solutions can be traced step by step.
The paper describes the mathematical basis for AI, which shows how he learns, what remembers and why it chooses a particular option. For current neural networks, this is a sore place: knowledge is distributed according to a huge number of internal parameters, so the final answer often has to be explained outside, through additional tools and approximate interpretations.
The team assembled a prototype with two related parts: a computational module that plays the role of a conditional brain, and a separate memory. The system can learn continuously and at the same time not destroy the old knowledge. For AI, this ability is important: many models after training cope worse with old tasks. This failure is called disastrous forgetting. A person can learn a new melody and not stop recognizing the old one, and a neural network without special defenses sometimes reorganizes internal connections so that the former skill just breaks down.
The authors are also trying to solve the problem of false memories. In AI, this is understood as a situation where the system confidently uses information that was not in its experience, or binds the data incorrectly. Generative models have a similar effect in plausible but erroneous responses. In the prototype from Loughborough, memory changes transparently: you can see what information has been preserved, which has intensified over time, what has weakened and where the specific state came from.
The first checks were simple, but the idea was well shown. The prototype without training with the teacher memorized musical notes and short musical phrases, and also recognized and stored colors in images, for example, in cartoon pictures. In all tasks, the system behaved predictably: researchers could trace how memory changed and how the model came to the result.
The approach is based on a mathematical construct, which the authors call a plastic vector field. The vector field can be represented as a map of directions: at each point, the system understands where and how to move on. The word plastic here means the ability to change under the influence of experience.
The main idea of the work is that the behavior, memory and physical structure of the system cannot be considered separately. In the physical intelligence, behavior is associated with brain activity, and brain activity depends on how the connections are arranged and how they change. Researchers are trying to transfer this logic to AI: not just teach the model to give the correct answer, but to set a clear mechanism that shows how the answer arose.
Separately, scientists criticize modern artificial neural networks. In their opinion, the problem of explanation is not limited to a lack of convenient analysis tools. The restriction is deeper, in the architecture itself. The neural network can work well in practice, but the developer does not control the full way as the model learns and where exactly stores information. Therefore, the explanation is often conjecture, not a direct observation.
The prototype is still far from the systems that can be immediately introduced into medicine, finance or automated control. The model needs to be scalable, checked on more complex data, and understand how it will behave beyond demonstration tasks. The team also wants to learn how such a principle can be transferred to new types of equipment, where memory and computing will be closely related than in conventional software neural networks. Well, it's still ahead...
