Author: Xu iron - cruiser Technology, Text adapted from papers Professor Omri Barak
Neuroscience of machine learning against complex systems Machine learning and complex systems are transforming the traditional methods of industry and academia in many areas. One excellent example is the neuroscience itself, which gives great inspiration to machine learning. I hereby use a living example to demonstrate the development of God's method in this field.
At the beginning of the article, I would like to ask you a question. If you create a model for a cat, what is the best way? This issue is very profound. Everyone can read the answers while looking for answers.
Neuroscience is a branch of biology that is highly data-dependent, because you have to understand that the brain is the device that is used to calculate. To study it, you first study its input and output, and then associate it with the animal's corresponding behavior.
The first part of this story is about a complex system because the nervous system is essentially a kind of complex network . We hope to understand the connection between neural cell data and behavior by building a complex network similar to a brain neural network on a computer. This effort was very popular in the 1980s and 1990s. For example, the earliest hopefield network explained the memory phenomenon.
We give a typical example:
Here scientists discovered that monkeys can be trained to compare two sounds of different frequencies (with punishing errors). This is a typical experiment to measure short-term memory. If a monkey needs to compare two sounds, it is necessary to put the first one. The signal of the sound is placed in the head and then compared with the second voice. The ability to make decisions based on information at different times can be said to be critical to the survival of the organism, and this experiment is its foundation.
In the past, neuroscience research methods may simply describe this behavior, and then put ideas into electrodes in the brains of monkeys to measure what the corresponding neural signals are.
For example, we can see the figure above neuronal discharge signal is measured, the frequency of neurons in the sound increases as the frequency rises (red for high frequency sounds), so we can conjecture out a model based on this. The cells measured in the figure below are the opposite. Can you design a model based on these two figures to explain the monkey's behavior?
Really! We see that the simplest physical model can explain this phenomenon. In the image on the left, we describe the model (algorithm) with a small ball in the valley. This is to put the whole problem in a very low dimension for easy understanding. The entire external input (voice) can be seen as an external force on the monkey's nervous system. The external force can be seen as some kind of driving force to change the orientation of the monkey's neurons like a small ball (the state of the monkey's nervous system) tends to the bottom in a valley that is shaped by external forces (the state with the highest probability of appearance, corresponding to the energy in the physical The lowest point is that the bottom position is the record (memory) of the first signal. When the external force (f1) disappears, the terrain of the valley changes, but the external force does not change immediately. When the second signal (f2) arrives, the terrain changes again, this signal causes the valley to rise, causing the ball to roll to the new bottom (one each on the left and right), and the small ball eventually reaches the position this time. Not only related to this external force, but also highly relevant to the position caused by the previous external force (comprehensive historical information for decision-making), this virtually achieved the comparison of the two signals (here the simplest Turing machine), and Use a simple physical system. So how can we use the two neurons we just mentioned (proportional or inversely proportional to external signals) to achieve this functional loop? Please see the figure on the right. We can even draw a circuit diagram to explain this principle. The positive sign represents the positive neuron, the negative sign represents the inverse neuron, E represents a time-varying control signal, and S2 changes the circuit connection according to E. Then you can design a subtractor with a memory function to implement it.
However, this method is an analogy. Many real neuroscientists dubbed this method the toy model, and they ignored it. Because no matter how you say that the program in your computer behaves like a brain, it is far from reality. There are also many algorithms that can implement a certain function. Why do you say that the brain is working according to your imagination? Computational neuroscientists using this method often struggle with the question of how much simulation your own model requires. As some people say, you have to model a cat. The best is to find a cat.
Of course, the model is still much better than no model. After all, it gives us light up the connection between the biological world and the mathematical world. Makes a mathematical system that we may understand build on a complex and unwieldy system of living, allowing us to play with changing the parametric space.
However, the real moment when the model began to exert its power was the involvement of machine learning, and the traditional method of complex systems has since been innovated. why?
Because through machine learning, we have a powerful and accurate method for inferring models from data, rather than previous fuzzy analogies. The fundamental difference between machine learning and what I had before was that I didn't start immersed in designing the same model as the real system. Instead, I first learned the real input and output with a standardized model that was roughly similar to the real model principle. The thinking here is that you are not pursuing to draw a living cat. Instead, you must first do something that does not look like something. Let him learn the same behavior as a cat. When this behavior is truly indistinguishable, then you think it is Cats.
Once the machine learning begins to intervene, the model is given predictability, and the predictive model can be regarded as indistinguishable from the real system at the input and output level, thus solving the problem that the model complexity is not set well and the model is difficult to pass through the Austrian model. The defect of Kame's razor. The better point of the machine learning model than the previous model is the cross validation section. The model you get with real data must not only be known as a real cat, but also known to be input and output. The input of an unknown input is like a cat, which means that it really needs to be able to learn. Like a real cat, it can not only catch mice in the corner of the house, but also catch mice in the field.
With this method, our research framework has changed. We must first look for a set of specific input and output as a starting point for the study. For example, to study the visual area, you find a set of pictures of cats and dogs, and then the output needs to be the correct category. Then test again.
Different from the complex system model method, the machine learning method is a black box operation. We first make a standardized device, then send a lot of input to the standard device, and then let the standard system change the parameters to get a set of what we want. Output. The advantage of the standardized black box is that the method of training to get the parameters is known because it is often an extremely difficult task, and it is different from the requirement that the traditional complex system model only pursue qualitative description.
If the above problems by the method of machine learning to solve, it becomes:
The protagonist here is a standardized neural network RNN ​​(recurrent neural network). We give input-output mappings under various conditions, and then train the network under the framework of supervised learning-that is, adjust the network. The connection weights reach the process of simulating this input and output. At this time, we do not need to design. Two neurons that are in line with the experiment (proportional to the opposite, and vice versa) will spontaneously emerge. In other words, the cat that the machine learns fully possesses all the functions of the cat, and even if we don't over-examine it, it looks like a cat (functional and morphological correspondence?).
We now have a cat that will catch mice on various occasions, but we don’t need a robot cat after all, but we’re going to use cats to study the behavior of cats and how cats function through the underlying elements. After all, Electronic cats are much more obedient than real cats. They can also be used to dissect and change parameters. How to do it? Return to the dynamic analysis in the complex system again:
The new arrows added here are from high-dimensional neural networks to low-dimensional dynamic systems. Remember the toy model of the hillside ball that we said earlier? Its mechanism is perfect, but it is a pity that people are not convinced. Here we have the opportunity to do justice again. We take out the RNNs we train that are indistinguishable from the real system, and use it for reverse engineering (reverse "Engineering"" takes a low-dimensional system from it and sees if its working mechanism is in line with our conjecture. There should be applause here, because here we use an abstract method that uses real data to answer an assumption that is almost impossible to verify.
There is also an implicit belief here that trained black boxes are considered to learn the essence of biological systems. There is a difference between your RNN's ability to perform tasks and reality. However, here we have reached the limit that we can currently achieve. This involves a rather fundamental problem, that is, the core of machine learning lies in the mathematical representation of the real system through data learning, and how close to the real the representation is, it is still difficult to quantify and clarify.
By mapping this high-dimensional RNN to a low-dimensional dynamics system, we get a simpler schematic that explains the operation of the entire system. Through learning, a high-dimensional chaotic system begins to appear as a fix point, and the fixed point contains important information of the input and output of the system. Here is a simple binary classifier. The principle of this classifier working is a saddle point in dynamics. For f1>f2 and f1 This kind of we have seen machine learning and complex systems among you and I have lingering relationships with you. First, machine learning is an algorithm, and algorithms always have a physical implementation (complex system). We first have the representational world of data, the symbolic world of algorithms, and finally the abstract physical world that truly explains the mechanism of the world. These three kinds of representational worlds have entered the symbolic world through machine learning, and the symbolic world can ingeniously communicate with the physical world. Machine learning is a bridge, complex systems are souls, and without bridges, souls are empty. We can further ask why the machine learning method works. Why does the brain work? What exactly does this place have? Pay attention to the low-dimensional to high-dimensional, and high-dimensional to low-dimensional problems that we are repeatedly talking about. The problems in real life are often high-dimensional to low-dimensional mapping, such as the signal (high-dimensional)-decision (often two-dimensional) process. We can find low-dimensional embedding in high-dimensional space, and often represent a real-world model representing neural networks. Because of these models, we have the ability to "infer one thing in the same way" and "generalize." More profound, it is not my article can be involved. Welcome to pay attention to the follow-up machine learning on the cruiser against the complex system combat series. Lei Feng Network (search "Lei Feng Network" public concern) Note: This article was issued by Xu Tie - cruiser technology authorized Lei Feng network, if you need to reprint, please contact the original author WeChat 562763765, and indicate the author and source.
The Bluetooth headset is to apply Bluetooth technology to the hands-free headset, so that users can avoid the annoying wires and easily talk in various ways. Since the advent of the Bluetooth headset, it has been a good tool for mobile business people to improve efficiency.The electromagnetic wave of the Bluetooth headset is much lower than that of the mobile phone. When you talk on the phone, you only need to put the mobile phone in your briefcase or in your pocket, and put on the Mini Bluetooth Headset to talk easily. It is not necessary to raise your hand high, and it can effectively reduce the influence of electromagnetic waves on the human body.
Bluetooth Headphone,Customize Headphone,Call Center Headset,Mini Bluetooth Headset
Shenzhen Linx Technology Co., Ltd. , https://www.linxheadphone.com