> Was very glad to see them come to python and soon to mainstream javascript.
I really need to get around to writing a blog post to explain this in detail since this misapprehension is endemic. Python and JavaScript do not have coroutines, they have generators. Lua has actual coroutines.
The latter is dramatically more expressive than what you can do with what Python, JavaScript, and C# offer. This mistake drives me crazy because it means people don't know what they're missing.
Here's a quick example. Let's say we've got a little Python class for binary
trees:
class Tree:
def __init__(self, left, data, right):
self.left = left
self.data = data
self.right = right
We'll add a method to do an in-order traversal. It takes a callback and invokes the callback for every data value in the tree, like so:
def walk(self, callback):
"""Traverse the tree in order, invoking `callback` on each node."""
if self.left:
self.left.walk(callback)
callback(self.data)
if self.right:
self.right.walk(callback)
We can create a little tree and then print the data items in order like so:
(This works in Python 3, in Python 2, you'll have to make a little fn for print.) Swell, right?
Later, we decide we want to iterate over the items in a tree. Easy-peasy, Python has generators! We can just make a function that takes a Tree and returns a generator. We already have a method to walk the nodes, so we just need to call that and then yield the items, like so:
def iterateTree(tree):
def callback(data):
yield data
tree.walk(callback)
Then you can just use it like so:
for x in iterateTree(tree):
print(x)
Perfect, right?
Actually, no. This doesn't work at all. You can't yield from the callback passed to walk. That's because walk() itself doesn't know that the callback is a generator.
This is the problem with generators: they divide all functions into two categories: regular functions and generators. You run a regular function by calling it. You run a generator by iterating over it. The caller must use it in the correct way.
At its simplest level it means you have to be careful when refactoring. If you have a generator function that gets too big and you want to split it up, you have to remember that the functions you split out are also special generator functions if they contain a yield. You have to remember to flatten it when you "invoke it".
It's more than just annoying though: it means it's impossible to write code that works generically with both kinds of functions. In other words, all of your higher-order functions like map, filter, etc. now only work with some of your functions. (Or, I suppose, you could explicitly implement them to support both but that's more work and I don't think most languages do.)
In languages like Lua, the above code just works. You can yield from anywhere in the callstack and the entire stack is suspended. It's fantastic.
(If I can be forgiven a bit of self-promotion, I'll note that my programming language Wren[1] can not only express full coroutines like Lua, but also supports symmetric coroutines which can express some things Lua cannot. They are roughly like the equivalent of tail call elimination for coroutines.)
and the pattern library is built entirely around embedding in streams and yielding others streams. it uses this for very interesting numeric music patterns.
result = yield from future – suspends the coroutine until the future is done, then returns the future’s result, or raises an exception, which will be propagated. (If the future is cancelled, it will raise a CancelledError exception.) Note that tasks are futures, and everything said about futures also applies to tasks.
result = yield from coroutine – wait for another coroutine to produce a result (or raise an exception, which will be propagated). The coroutine expression must be a call to another coroutine.
Yield and generators (i.e. save stack; return value to caller; receive value from caller) are really a language feature for writing a runtime for a different language with coroutines. Or if you're willing to write your program in a way that looks like it was generated by a source-to-source transformation tool, you can write your coroutines on top of them. The most basic construct is something like this:
CurrentCoroutine = None
def run(main, arg):
global CurrentCoroutine
CurrentCoroutine = main
while CurrentCoroutine is not None
CurrentCoroutine, arg = CurrentCoroutine.send(arg)
def corodecorator(coro):
@functools.wraps(coro):
def init():
c = coro()
c.next()
return c
return init
And this is pretty much it. A simple example for two coroutines that pass control to each other would be:
As far as I can tell, that's still just a little local syntactic sugar for making composing generators nicer. There's a still a fundamental distinction between generators and other functions.
Think of it like this. Let's say you have a chunk of code that you want to refactor out into a separate method. The usual way to do that is to pull it out into a separate method and then call it from the place where the code used to be inline.
In languages with coroutines, you can just do that, regardless of what's in that chunk of code. In Python, you have to think, "Oh, does this chunk of code contain a yield?" If so, you need to do a "yield from" the function you pulled out instead of a regular call.
It forces you to constantly be cognizant of and design around the split between normal code and generators.
I really need to get around to writing a blog post to explain this in detail since this misapprehension is endemic. Python and JavaScript do not have coroutines, they have generators. Lua has actual coroutines.
The latter is dramatically more expressive than what you can do with what Python, JavaScript, and C# offer. This mistake drives me crazy because it means people don't know what they're missing.
Here's a quick example. Let's say we've got a little Python class for binary trees:
We'll add a method to do an in-order traversal. It takes a callback and invokes the callback for every data value in the tree, like so: We can create a little tree and then print the data items in order like so: (This works in Python 3, in Python 2, you'll have to make a little fn for print.) Swell, right?Later, we decide we want to iterate over the items in a tree. Easy-peasy, Python has generators! We can just make a function that takes a Tree and returns a generator. We already have a method to walk the nodes, so we just need to call that and then yield the items, like so:
Then you can just use it like so: Perfect, right?Actually, no. This doesn't work at all. You can't yield from the callback passed to walk. That's because walk() itself doesn't know that the callback is a generator.
This is the problem with generators: they divide all functions into two categories: regular functions and generators. You run a regular function by calling it. You run a generator by iterating over it. The caller must use it in the correct way.
At its simplest level it means you have to be careful when refactoring. If you have a generator function that gets too big and you want to split it up, you have to remember that the functions you split out are also special generator functions if they contain a yield. You have to remember to flatten it when you "invoke it".
It's more than just annoying though: it means it's impossible to write code that works generically with both kinds of functions. In other words, all of your higher-order functions like map, filter, etc. now only work with some of your functions. (Or, I suppose, you could explicitly implement them to support both but that's more work and I don't think most languages do.)
In languages like Lua, the above code just works. You can yield from anywhere in the callstack and the entire stack is suspended. It's fantastic.
(If I can be forgiven a bit of self-promotion, I'll note that my programming language Wren[1] can not only express full coroutines like Lua, but also supports symmetric coroutines which can express some things Lua cannot. They are roughly like the equivalent of tail call elimination for coroutines.)
[1]: https://github.com/munificent/wren