Very cool idea! The current outputs have a relatively simple effect though, not so far from a simple overlay where the blocks are "active". And the blocks which are not active get restored to the base image again. Maybe one could do selective in painting instead. For example, inpaint the cells which turn non-active in a generation with one kind of prompt, and the ones that turn active with another prompt. So it would pull cells towards two different concepts, but remember a bit the history/state of a cell.

Since the area the generated image changes is known between frames, the animations could be made more consistent by only changing the relevant masked pixels instead of the full images.

Yeah, right now there's no conceptual link between the image generation and the pixels

Love this, absolutely pointless as far as I can tell and yet totally beautiful. This is pure Art. Just started using controlNet myself, wasn’t aware of using it for QR codes, thanks for the idea!

It'so cool, love it.

Controlnet is the bees knees!