Wednesday, March 15, 2017

Entropic AI Research

A.D. Wissner-Gross a Physicist at MIT and C.E. Freer a mathematician who taught pure mathematics at MIT published in 2013 a paper about Causal Entropic Forces.

In the paper it talked about a potential deep connection between intelligence and entropy maximization.

Alex's website: http://www.alexwg.org/

Causal Entropic Forces: http://www.alexwg.org/publications/PhysRevLett_110-168702.pdf

I first heard about Alex's work 3 years ago in his TED talk "Equation For Intelligence". After watching that TED talk i was captivated by the idea that intelligence could actually be a real physical force in the universe. I was even more captivated by the idea that Alex had created an AI that was capable of making "Intelligent" decisions, such as balancing a pole on a cart, using a tool to retrieve an object stuck in a confined space, solving a cooperation puzzle, and even executing a buy low sell high strategy on a simulated stock. The AI did all of these things, Yet they had never given it a goal.



Humans, animals, insects, and even plants can create their own goals. Yet virtually every AI we have created has required someone to specifically specify it's goal. As such i think Alex's work is an important step and direction in AI development.

After watching that TED talk i started doing research and immediately began connecting these concepts to some of my philosophical ideas as shown in my post Objective Rationality. A few years later i started forming a new idea/theory around AI connecting these ideas of intelligence with Jeremy England's (a biophysicist at MIT) ideas regarding the origin of life and its connection with entropy, and Kevin Kelly's (a thought leader in technology) idea's regarding technology and entropy.

Recently I went to Exosphere to learn python programming, and while i was there i was able to do more research into this new field.

I ended up getting in contact with another mathematician and physicist in Spain who is doing research into this field as well. His name is Sergio Hernandez and he has a blog called entropicai, and with a bit of his help and help from my teacher Roberto Panai who is getting his phd in pure mathematics. I was able to make an AI based off of the principles of maximizing future freedom of action as stated in Alex's research paper.

The AI i made is pretty simple, it uses monte carlo to search "path space" looking for and choosing paths with the most amount of "freedom". More freedom is defined as a path having more "steps".

These were the first "feelers" i made, they moved in only a diagonal direction, and randomly changed between a 30 degree and 60 degree angle.

It was limited in "sight" due to only being able to search path space in a 45 degree angle.

In order to maximize it's ability to search path space i made it able to search in more directions by increasing its search area to 360 degrees and even change it's direction allowing it turn and travel in circles.

I then started to make obstacle logic so that when one of the paths enters a certain area, it stops generating steps. I also made a box to contain all the paths so they don't go off forever into the ether, saving computing power.

Ending up with this rather beautiful image.

As you can see whenever the paths reach either an obstacle or the square boundaries, it stops generating steps.

Now that i had made a way to search through path space, the next step was to make the algorithm that chooses the path with the most futures.

The algorithm essentially simulates the future by simulating a bunch of random paths from its starting position, it then chooses the path that managed to generate the most steps, then the algorithm takes an X number of steps along that chosen path, then it loops on it, simulating another set of random paths from its new position,

Here's a visual example of it simulating hundreds of futures from its initial starting position which is inside a ring with a small opening.


Here's a visual example of it simulating futures from its new starting position which is now outside the ring.

The algorithm as a result is able to find its way out of any maze no matter how difficult, given enough computing power.

Even though it can find it's way out of any maze, it is of course just a simple algorithm brute forcing a solution. Also in this situation i am giving my algorithm a goal by deciding what i mean by "most futures". Where as in the research paper, most futures is determined by something much deeper, far more general, and more complex.  Also the only thing my algorithm can really do is solve maze problems, it can't do much else. Where as the algorithm in the research paper can balance a pole on a cart, or use a tool to retrieve a trapped object, all without redefining "most futures".

There is however one very interesting thing that my algorithm does.

Without changing the algorithm, the very same algorithm that can escape mazes, also does something very interesting.

If it has no maze to solve, it will always gradually move to the center of its container and stay there.
This is interesting because not only did i not tell it do this, nor did i expect this to happen when first starting this project. But perhaps more importantly, it's interesting because it correlates very well with the first experiment Alex does as shown in his research paper, which is the particle in the box experiment. 
As said in the research paper. "As a first example of causal entropic forcing applied to a mechanical system, we considered a particle in a twodimensional box [13]. We found that applying forcing to both of the particle’s momentum degrees of freedom had the effect of pushing the particle toward the center of the box, as shown in Fig. 2(a)"
The reason my algorithm gradually moves to the center of the box, is the same reason why the algorithm in the research paper also moves the particle towards the center of the box. That reason is the fact that by moving towards the center, it maximizes its future freedom of action.

I unknowingly was able to replicate Alex's particle in the box experiment by making an algorithm that can solve any maze using the principle of trying to maximize future freedom.

I feel as if this hints towards some deeper truths regarding this idea of maximizing future freedom.

Even though the math behind the causal entropic forcing as shown in the research paper may be too difficult to apply to real world problems. It could be that simply focusing on trying to create better and better algorithms that maximize future freedom of action. Could naturally result in the emergence of more general artificial intelligent systems that increasingly do a wide variety of unexpected things. 

I think Deep Q Neural Networks share similarities with this concept of maximizing future freedom of options.