CAs have a lot of nice properties: translation invariance, locality, arbitrary population sizes, parallel computation, simplicity of implementation, and emergent complexity. Interesting to see where this line of work goes
Yeah, I was thinking about symmetries <=> conservation laws because of Noether's theorem. Think of regular NN training as not having any symmetries...since they aren't baked into the model. But we can let the model learn symmetries <=> conservation laws by adding the Euler-Lagrange constraint to the forward pass of a NN.
Chris, glad you like it. I can relate to that feeling, which I also had as an undergrad. It was a big motivation for this work -- and I think the analogy can go even further. For example, there's a really cool lecun paper about NN training dynamics actually being a lagrangian system: http://yann.lecun.com/exdb/publis/pdf/lecun-88.pdf. Another thing I want to try is literally write down a learning problem where S (the action) is the cost function and then optimize it...
That does seem interesting! After skimming that paper, I think I'm going to need to sit down with it in order to really parse through things, though. Some of the operator combinations seem to be things I haven't worked with jointly before. I'd definitely be interested to see the results of using the action as the cost function, though!
