Hi,

I have a bit of improvement point about the query_cppn function in hyperneat.py. In line 85-88, a value below the threshold is replaced with 0.0, so that range [-0.2, 0.2] of the value drop out in this implementation.

However, the original paper (http://axon.cs.byu.edu/Dan/778/papers/NeuroEvolution/stanley3**.pdf) says "The magnitude of weights above this threshold are scaled to be between zero and a maximum magnitude in the substrate." on page 8.

Thus, I suggest changing the query_cppn function like it returns a value of continuity range [-max_val, max_val].

 ****** Running generation 0 ******

Traceback (most recent call last):
  File "c:\Users\Silver\.vscode\extensions\ms-python.python-2020.2.64397\pythonFiles\ptvsd_launcher.py", line 48, in <module>
    main(ptvsdArgs)
  File "c:\Users\Silver\.vscode\extensions\ms-python.python-2020.2.64397\pythonFiles\lib\python\old_ptvsd\ptvsd\__main__.py", line 432, in main
    run()
  File "c:\Users\Silver\.vscode\extensions\ms-python.python-2020.2.64397\pythonFiles\lib\python\old_ptvsd\ptvsd\__main__.py", line 316, in run_file
    runpy.run_path(target, run_name='__main__')
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "g:\Emulators\ML AI open AI\env2.py", line 51, in <module>
    winner = run(200, env)[0]
  File "g:\Emulators\ML AI open AI\env2.py", line 37, in run
    winner, stats = run_es(gens, env, 200, config, params, sub, max_trials=200)
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\pureples\shared\gym_runner.py", line 50, in run_es
    pop.run(eval_fitness, gens)
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\neat\population.py", line 89, in run
    fitness_function(list(iteritems(self.population)), self.config)
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\pureples\shared\gym_runner.py", line 25, in eval_fitness
    net = network.create_phenotype_network()
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\pureples\es_hyperneat\es_hyperneat.py", line 46, in create_phenotype_network
    hidden_nodes, connections = self.es_hyperneat()
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\pureples\es_hyperneat\es_hyperneat.py", line 151, in es_hyperneat
    root = self.division_initialization((x, y), True)
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\pureples\es_hyperneat\es_hyperneat.py", line 110, in division_initialization
    c.w = query_cppn(coord, (c.x, c.y), outgoing, self.cppn, self.max_weight)
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\pureples\hyperneat\hyperneat.py", line 84, in query_cppn
    w = cppn.activate(i)[0]
  File "C:\Users\Silver\AppData\Local\Programs\Python\Python37\lib\site-packages\neat\nn\feed_forward.py", line 14, in activate
    raise RuntimeError("Expected {0:n} inputs, got {1:n}".format(len(self.input_nodes), len(inputs)))
RuntimeError: Expected 840 inputs, got 5

I ran this through the Debugger and found that at some point some random float values replace the existing number of inputs that initially gets set.

I could even see that at some point during execution the correct number of inputs was actually used.

I've been fighting to find the cause and I've come to the conclusion that something has to be wrong in the module.

for some context, I took one of the examples and attempted to configure it to run a gym retro env.

As you can see though the only thing stopping me is the inputs being messed up somehow.

If you need more information please let me know.

Hi!

Very cool project, thanks for making it available. I have a toy project I am working on with Gym for function approximation, and which is a discrete-valued observation space consisting of 12 integers; action space is also discrete-valued, three integers used to determine the correct agent action based on the sequence of 12 integers.

So does pureples support discrete observation and action spaces, and would the cartpole experiment make for a good starting point for this?

Thanks in advance!

Hi,

The following part seems to be different from the algorithm in https://eplex.cs.ucf.edu/papers/risi_alife12.pdf.

160 | for i in range(self.iteration_level):  # Explore from hidden.
161 |     for x, y in unexplored_hidden_nodes:
162 |         root = self.division_initialization((x, y), True)
163 |         self.pruning_extraction((x, y), root, True)
164 |         connections2 = connections2.union(self.connections)
165 |         for c in connections2:
166 |             hidden_nodes.add((c.x2, c.y2))
167 |         self.connections = set()
168 | 
169 | unexplored_hidden_nodes -= hidden_nodes

According to pseudocode on page 47, line 169 should be indented once again. Also, unexplored_hidden_nodes will always be the empty set if we remove hidden_nodes from unexplored_hidden_nodes (because hidden_nodes is always greater than unexplored_hidden_nodes). I think it needs to be corrected as follows.

160 | for i in range(self.iteration_level):  # Explore from hidden.
161 |     for x, y in unexplored_hidden_nodes:
162 |         root = self.division_initialization((x, y), True)
163 |         self.pruning_extraction((x, y), root, True)
164 |         connections2 = connections2.union(self.connections)
165 |         for c in connections2:
166 |             hidden_nodes.add((c.x2, c.y2))
167 |         self.connections = set()
168 | 
169 - unexplored_hidden_nodes -= hidden_nodes
    +     unexplored_hidden_nodes = hidden_nodes - unexplored_hidden_nodes

Hey,

First of all you did great work, easy to use and understand! What I am trying to do is, using ES-HyperNEAT to exploit the Geometrical Informations in the Picture's Pixels of an Atari Game. OpenAI Gym gives an observationspace of (210, 160, 3), i have downsized it to (84, 84, 1) without colours. These are 7056 input-Nodes, instead of 100800.

Now the Problem is that the outputs of the substrate's outputnodes are always Zero.

The Input Layout is:

for y in range(1,85):
	for x in range(1,85):
		input_coordinates.append((x , y))

Is there some configuration in the CPPN i should watch out for, is the substrate too large, or is there a max Range for the Node-Placment in the substrat (exp just between -1, 1)?

Thanks in advance!

Hi @ukuleleplayer,

I've been working on a PUREPLES-based project with your gym runner but I can't find any resources on inference with an evolved ANN? It looks like the phenotype gets pickled and model saved whenever the reward in +1., but what type of model format is that in and how to deploy for inference tasks?

What I want to do is implement an additional loop whenever a +1. reward is found, to test it n more times to see if it has generalized to other examples.

And does it make sense to restart an episode on each of those saved pickles for subsequent runs?

TIA!

Hi.

Connection's __eq__ is expected to return a boolean, but it returns a tuple (float, float, float, bool, float, float, float). However, the library seems to be working correctly at first glance.

Tentatively, I will create a PR.

Hi, I think in es_hyperneat.py on line 30/31 the ranges for the input- and output_nodes should be transformed to a list with list().

Otherwise return neat.nn.RecurrentNetwork(input_nodes, output_nodes, node_evals) throws an error: unsupported operand type(s) for +: 'range' and 'range'

Without that change skripts like es_hyperneat_xor_large.py do not work.

The same problem seems to appear in hyperneat.py

Could anyone please explain the following line of code in es_hyperneat.py?

        # Number of layers in the network.
        self.activations = 2 ** params["max_depth"] + 1

Thank you very much.

I have been trying to use ES-Hyperneat on a custom environment. The size of input to ES-Network is (49360,) and for output is (1024,). The "net = network.create_phenotype_network()" method is sometimes taking more than 30 minutes to execute for a single genome. Does it mean that the larger the size of input and output of network the more time it will take to create network?

Is there any solution for this?

Hi @ukuleleplayer

I've noticed that all of the examples use sigmoid activation functions for ES-HyperNEAT; is the use of multiple activation function at the per-neuron level possible with PUREPLES?

Or any activation function other than sigmoid for ES-HyperNEAT?

TIA

Hi, first of all thank you for your library, it's great! I am going through the code trying to understand what each step does, regarding the pole balancing environment. There is a point that really leaves me confused: in run_hyper(), it seems we create the population and test it for one trial, then again for 10 trials, and then for max_trials trials. Any reason to do that? Thanks