That sounds cool, but this quickly gets complicated. Some aspects that need to be addressed:
- where does the automatically defined struct live? Data segment might work for static, but doesn't allow dynamic use. Stack will be garbage if closure outlives function context (ie. callback, future). Heap might work, but how do you prevent leaks without C++/Rust RAII?
- while a function pointer may be copied or moved, the state area probably cannot. It may contain pointers to stack object or point into itself (think Rust's pinning)
Yes that's what I'm thinking. Essentially a stateful function definition defines both a function, and a struct containing the state. I think there needs to be two ways of invoking a stateful function f: (1) if you invoke f within another stateful function g, each call site in g that calls f automatically gets a distinct state instance that becomes part of g's state, on the other hand, (2) if you want to invoke f in a regular (non-stateful) function, you need to manually manage the state and explicitly pass it in. That would be one purpose of the statetype(f) operator: to allow you to explicitly declare a state instance. Manual state management would also be used when you want to invoke f with the same state multiple times (e.g. from within a loop).
In C I don't think the copy/move thing is an issue. It has the same hazards as copying struct instances. And yes I am thinking of this as a C extension.
Another complication is that it would be beneficial to be able to optimize state storage in the same way that stack frame resources are optimized, including things like coalescing equal values in conceptually distinct state instances. This would (I think) preclude things like sizeof(statetype(f)) which you really want for certain types of manual memory management, or it would require multiple compiler passes.
- where does the automatically defined struct live? Data segment might work for static, but doesn't allow dynamic use. Stack will be garbage if closure outlives function context (ie. callback, future). Heap might work, but how do you prevent leaks without C++/Rust RAII?
- while a function pointer may be copied or moved, the state area probably cannot. It may contain pointers to stack object or point into itself (think Rust's pinning)
- you already mention recursion, compilation
- ...