neataptic

<html> <head> <title>Neataptic.js - Target-seeking AI</title>  <meta charset="UTF-8"> <meta name="description" content="Neural agents learn to seek targets through neuro-evolution"> <meta name="keywords" content="target seeking, AI, genetic-algorithm, NEAT, Neataptic"> <meta name="author" content="Thomas Wagenaar">  <link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css"> <link rel="stylesheet" type="text/css" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css"> <link rel="stylesheet" type="text/css" href="../../custom.css">  <link rel="shortcut icon" href="http://i.imgur.com/rPZtAOc.png">  <link href="https://fonts.googleapis.com/css?family=Roboto:300,400,500,700" rel="stylesheet">  <script async> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-51480408-2', 'auto'); ga('send', 'pageview'); </script>  <script src="https://code.jquery.com/jquery-3.2.1.min.js"></script> <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ tex2jax: { inlineMath: [ ['$','$'], ["\$","\$"] ], processEscapes: true } }); </script> <script src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> </head> <body>  <nav class="navbar navbar-default navbar-fixed-top"> <div class="container"> <div class="navbar-header"> <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar"> <span class="sr-only">Toggle navigation</span> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <a class="navbar-brand" href="../.."><p class="brand">Neataptic</p></a> </div> <div id="navbar" class="navbar-collapse collapse"> <ul class="nav navbar-nav"> <li class="current active"><a href="../..">Home</a></li> <li class="current"><a href="../../docs/">Docs</a></li> <li class="current"><a href="../">Articles</a></li> </ul> <ul class="nav navbar-nav navbar-right"> <li><a href="https://github.com/wagenaartje/neataptic" target="blank"><i class="fa fa-github fa-1x"></i><b> GITHUB</b></a></li> </ul> </div> </div> </nav> <div class="container" style="margin-top: 60px"> <div class="row">  <div class="col-sm-2 col-md-2 col-lg-2"> <ul id="sidebar" class="nav"> <li class=""> <a href="../neuroevolution/">Neuroevolution</a> </li> <li class="active"> <a href="./">Target-seeking AI</a> </li> <li class=""> <a href="../agario/">Agar.io AI</a> </li> <li class=""> <a href="../classifycolors/">Classify colors</a> </li> <li class=""> <a href="../playground/">Playground</a> </li> </ul> </div>  <div class="col-sm-10 col-md-10 col-lg-10 content"> <h1 style="margin-bottom: 0px;">Target-seeking AI</h1> <p><a style="text-decoration: none" href="https://github.com/wagenaartje/neataptic/edit/master/mkdocs/templates/articles\targetseeking.md"><i class="fa fa-github fa-1x"></i> Edit on Github</a></p> <hr> <p>In the simulation below, neural networks that have been evolved through roughly 100 generations try to seek a target. Their goal is to stay as close to the target as possible at all times. If you want to see how one of these networks looks like, check out the <a href="https://wagenaartje.github.io/target-seeking-ai/">complete simulation</a>.</p> <div id="field" height="500px"> </div> <p><em>Click on the field to relocate the target! Source code <a href="https://wagenaartje.github.io/target-seeking-ai/">here</a>.</em></p> <p>The neural agents are actually performing really well. At least one agent will have 'solved the problem' after roughly 20 generations. That is because the base of the solution is quite easy: one of the inputs of the neural networks is the angle to the target, so all it has to do is output some value that is similar to this input value. This can easily be done through the identity activation function, but surprisingly, most agents in the simulation above tend to avoid this function.</p> <p>You can check out the topology of the networks <a href="https://wagenaartje.github.io/target-seeking-ai/">here</a>. If you manage to evolve the genomes quicker or better than this simulation with different settings, please perform a pull request on <a href="https://wagenaartje.github.io/target-seeking-ai/">this</a> repo.</p> <h3 id="the-making-of">The making of</h3> <p>In the previous article I have gone more into depth on the environment of the algorithm, but in this article I will focus more on the settings and inputs/outputs of the algorithm itself.</p> <p>If you have any questions about the code in the linked repo, please create an issue on <a href="https://github.com/wagenaartje/neataptic">this</a> repo.</p> <h3 id="the-agents">The agents</h3> <p>The agents' task is very simple. They have to get in the vicinity of the target which is set to about 100 pixels, once they are in that vicinity, each agents' score will be increased proportionally `(100 - dist)`` to the distance. There is one extra though: for every node in the agents' network, the score of the agent will be decreased. This has two reasons; 1. networks shouldn't overfit the solution and 2. having smaller networks reduces computation power.</p> <p>Agents have some kind of momentum. They don't have mass, but they do have acceleration, so it takes a small amount of time for a agent to reach the top speed in a certain direction.</p> <p><strong>Each agent has the following inputs</strong>:</p> <ul> <li>Its own speed in the x-axis</li> <li>Its own speed in the y-axis</li> <li>The targets' speed in the x-axis</li> <li>The targets' speed in the y-axis</li> <li>The angle towards the target</li> <li>The distance to the target</li> </ul> <p>The output of each agent is just the desired movement direction.</p> <p>There is no kind of collision, except for the walls of the fields. In the future, it might be interesting to add collisions between multiple agents and/or the target to reveal some new tactics. This would require the agent to know the location of surrounding agents.</p> <h3 id="the-target">The target</h3> <p>The target is fairly easy. It's programmed to switch direction every now and then by a random amount. There is one important thing however: <em>the target moves with half the speed of the agents</em>, this makes sure that agents always have the ability to catch up with the target. Apart from that, the physics for the target are similar to the agents' physics.</p> <h3 id="the-genetic-algorithm">The genetic algorithm</h3> <p>The genetic algorithm is the core of the AI. In the first frame, a certain amount of players are initialized with a neural network as brain. The brains represent the population of a generation. These brains are then evolved by putting the entire population in óne playing field and letting them compete against each other. The fittest brains are moved on the next generation, the less fit brains have a high chance of being removed.</p> <pre><code class="javascript">// Networks shouldn't get too big for(var genome in neat.population){ genome = neat.population[genome]; genome.score -= genome.nodes.length * SCORE_RADIUS / 10; } // Sort the population by score neat.sort(); // Draw the best genome drawGraph(neat.population[0].graph($('.best').width(), $('.best').height()), '.best', false); // Init new pop var newPopulation = []; // Elitism for(var i = 0; i < neat.elitism; i++){ newPopulation.push(neat.population[i]); } // Breed the next individuals for(var i = 0; i < neat.popsize - neat.elitism; i++){ newPopulation.push(neat.getOffspring()); } // Replace the old population with the new population neat.population = newPopulation; neat.mutate(); neat.generation++; startEvaluation(); </code></pre> <p>The above code shows the code run when the evaluation is finished. It is very similar to the built-in <code>evolve()</code> function of Neataptic, however adapted to avoid a fitness function as all genomes must be evaluated at the same time.</p> <p>The scoring of the genomes is quite easy: when a certain amount of iterations has been reached, each genome is ranked by their final score. Genomes with a higher score have a small amount of nodes and have been close to the target throughout the iteration.</p> <p><strong>Some (configurable) settings</strong>:</p> <ul> <li>An evaluation runs for 250 frames</li> <li>The mutation rate is 0.3</li> <li>The elitism is 10%</li> <li>Each genome starts with 0 hidden nodes</li> <li>All mutation methods are allowed</li> </ul> <h3 id="issuesfuture-improvements">Issues/future improvements</h3> <ul> <li>... none yet! <a href="https://github.com/wagenaartje/neataptic">Tell me your ideas!</a></li> </ul> <p><strong>Forks</strong></p> <ul> <li><a href="https://corpr8.github.io/neataptic-targetseeking-tron/">corpr8's fork</a> gives each neural agent its own acceleration, as well as letting each arrow remain in the same place after each generation. This creates a much more 'fluid' process.</li> </ul> </div> </div> </div>  <script src="../../js/articles/target-seekingai/import.js"></script> <script> $("table").each(function(){ $(this).addClass('table table-striped'); }); $(".current").each(function(){ var text = $(this).text().toLowerCase(); if(window.location.pathname.indexOf(text) >= 0){ $('.active').removeClass('active'); $(this).addClass('active'); } }) </script> </body>