## Link to the paper
The paper can be found here.
This paper explores an alternative paradigm for machine learning that more closely models the diversity, competence and cumulative nature of human learning (called never-ending learning). It compares, the current day machine learning systems which have a very narrow scope and learn only a single function from very specific and limited training examples in a particular format, to the broad learning that humans undergo. It also presents a case study of NELL(never-ending language learner) from CMU and discusses the acheivements and laggings in the system. The paper has little formalism and nearly no mathematical concreteness but explores a powerful machine learning paradigm, backed up with intuitive reasoning, that may see some light in the future.
For some concreteness we start by defining any general purpose agent (supervised learning) in machine learning characterised by <T,P,E> - learning task T, performance metric P and the past experiences E. The goal is to learn a function f:X->Y given a set of input/output pairs {(xi,yi)}. Other paradigms of unsupervised clustering and others also learn a single function or data model from a single dataset.
Contrary to above, Never-Ending learning focuses on:
* Learn from different types of knowledge different functions
* optimsed over years of diverese, self-supervised expericence in a circular fashion
* with self reflection coupled with the ability to formulate new representations and learning tasks that helps agent avoid plateus.
The never-ending learning problem faced by the agent consists of a collection of learning tasks, and constraints that couple their solutions. This type of agent is defined by an ordered 2-tuple L consisting of a pair consisting of:
1. a set L={Li} where each Li is a learning task as above.
2. C={(PHIk, Vk)} - a set of coupling constraints among solutions of various learning tasks that primarily helps the never ending learner to avoid plateus unlike the usually encountered ML agents.
Given a learning problem containing n learning tasks, a never-ending learning agent outputs a sequence of solutions (functions fi) to these and modifies the solutions over time to improve their quality (as measured by the Pi’s)
The paper then talks about the never-ending language learner which is a module based language learner with over 2500 learning tasks and more 80 billion of confidence weighted beliefs as of November 2014. When the agent started it had a bare minimum of a few ontolies and binary relation and about a dozen of labelled example from each ontology. It has developed such a vast graph of beliefs over months of exploring web and interacting with humans via http://rtw.ml.cmu.edu. It has been able to successfully formulate newer ontologies, as well as coupling constraints in the form of horn clauses that seek and try to maintain consistency with the internal knowledge base (KB) of the agent.The paper discusses the NELL system in a great detail describing the modular architecture and a few set of learning tasks and coupling constraints. This maybe of interest to the reader in case he wishes to work on similar sort of never-ending learning agent or wishes to draw a rough idea of how such an agent can be really made to function and such a reader is encouraged to go through the paper and explore it in greater detail.
The paper concludes with with a short discussion on achievements of such a never ending learner that has been able to consitently learn over 5 years now and still avoid the plateaus unlike the other older ML paradigms that saturate quickly. It also describes a few important design features that the author recommends for other interested in making one such never-ending learner. They are as follows:
* Couple training with many different learning tasks to achieve successful semi-supervised learning
* Allowing agent to learn additional coupling constraints is of critical importance since it adds a great deal to maintain consistency while learning
* Learning new representations and ontologies is important for the never-ending learning task to avoid reaching plateaus
* Organize the set of learning tasks into an easy-to- increasingly-difficult curriculum.
Limitations of NELL are then discussed in the paper which ofcourse depends on how the never-ending learner was implemented and formulated. Of greater importance though are the other two theoretical concerns that the author raises in context of such never-ending learners.
One issue is of the “correctness and consitency”. All that we learn may be consistent by not essentially correct. The same goes with the agent. The best an agent can do is maintain consistency in beliefs, but distinguishing whether the observed consistency is due to correct predictions or incorrect perceptions is difficult and he question remains open under what circumstances does increasing consistent learning also guarantee increasingly correct learning?
The second is of convergence guarantees. “What agent architecture is sufficient to guarantee that the agent can generate a sequence of self modifications that improve its performance increaingly while avoiding performance plateaus”
Overall the paper introduces to a very intuitively powerful idea. How much of this idea is worth in practice remains of question. I would be interested in reading on more mathematical formalism that builds upon the idea and proves its concreteness.