Learning & Optimization

If I am asked to name two topics I find most fascinating at present then my answer will be learning and optimization. These two concepts make the backbone of modern intelligent systems. There are many definitions of learning depending on the context. In certain situations learning is considered a synonym of education also. In technical areas such as artificial intelligence, the definition from the Wikipedia is quite useful and it is short, crisp and represents everything what we expect learning should incorporate.

"Learning is the process of acquiring new, or modifying existing, knowledge, behaviors, skills, values, or preferences." - Wikipedia 

A typical database supports four operations - add, update, search and delete. Apart from one (delete) rest of all the three operations are applicable to human learning also. In the case of human learning it is hard to 'unlearn' what we have already learned. However, in most case we can add some new experience that can counter the old experience we want to get rid off. In a Bayesian learning model, we can keep updating the knowledge or information we have about certain thing, in the form of prior probability by multiplying that with the likelihood we get from the new data. 

Without optimization learning will be quite challenging. Optimization has to do with picking the best choice out of many that maximizes or minimizes certain outcome, generally modeled in the form of a cost function. We want to find out how much of a fixed amount of money we should invest in different financial instruments so that our return is maximum. This is a problem in decision science also and for it the experience we have from our past learning is quite valuable. 

Learning generally results in the form of data, information, knowledge or insight depending on the level of learning. For example the most effective learning will lead maximum insight that can be used to rule out many choices when we need to make a decision in a certain context. 

Learning and optimization are not something limited to the human activities only. Nature does the optimization all the time. We know that only certain type of configurations exists in nature. For example, most trees are right angle to the surface of the earth or velocities of molecules of gas inside a box, at some constant temperature have, follow a particular distribution. In natural systems nature tries to minimize certain physical quantities (such as potential energy) and that rules out many possibilities. 

Any system natural or intelligence must be capable of optimization and learning and both of these process involve manipulation of information. For the case of natural system the choice of the optimization method is decided by Physics and the choice of learning by the evolutionary process, which itself is a learning process. For the case of artificial systems we have much more flexibility when it comes to choices for optimization & learning. 

You can like & share this article if find useful. Your comments & feedback are most welcome.