yield next vs. yield* next January 9, 2014

One question a couple of people have asked is "what is the difference between yield next and yield* next? Why yield* next?" We intentionally do not use yield* next in examples to avoid new users from asking this question, but this question will inevitably be asked. Unfortunately, there isn't any very good explanations on these "delegating yields" as generators are relatively new. Although we Koa uses it internally for "free" performance, we don't advocate it to avoid confusion.

For specifications, view the harmony proposal.

What does delegating yield do?

Suppose you have two generators:

function* outer() {
  yield 'open'
  yield inner()
  yield 'close'
}

function* inner() {
  yield 'hello!'
}

If you iterate through outer(), what values will that yield?

var gen = outer()
gen.next() // -> 'open'
gen.next() // -> a generator
gen.next() // -> 'close'

But what if you yield* inner()?

var gen = outer()
gen.next() // -> 'open'
gen.next() // -> 'hello!'
gen.next() // -> 'close'

In fact, the following two functions are essentially equivalent:

function* outer() {
  yield 'open'
  yield* inner()
  yield 'close'
}

function* outer() {
  yield 'open'
  yield 'hello!'
  yield 'close'
}

Essentially, delegated generators replace the yield*!

What does this have to do with co or koa?

Generators are confusing already as it is. It doesn't help that koa's generators use co to handle control flow. Many people are and will be confused between native generator features and co features.

So suppose you have the following generators:

function* outer() {
  this.body = yield inner
}

function* inner() {
  yield setImmediate
  return 1
}

What is essentially happening here is:

function* outer() {
  this.body = yield co(function inner() {
    yield setImmediate
    return 1
  })
}

There's an extra co call here. But if we use delegation, we can skip the extra co call:

function* outer() {
  this.body = yield* inner()
}

Essentially becomes:

function* outer() {
  yield setImmediate
  this.body = 1
}

Each co call creates a few closures, so there's going to be a tiny bit of overhead. However, this isn't much overhead to worry about, but with one *, you can avoid this overhead and use native language features instead of this third party library called co.

How much faster is this?

Here's a link to a discussion we had a while ago about this topic: https://github.com/koajs/compose/issues/2. You won't see much performance difference (at least in our opinion), especially since your actual application code will slow down these benchmarks significantly. Thus, it isn't worth advocating it, but it's worth using it internally.

What's interesting is that with yield* next, Koa is currently faster than Express in these "silly benchmarks": https://gist.github.com/jonathanong/8065724. Koa doesn't use a dispatcher, unlike Express who uses multiple (one from connect, one for the router).

With delegating yield, Koa essentially "unwraps" this:

app.use(function* responseTime(next) {
  var start = Date.getTime()
  yield* next
  this.set('X-Response-Time', Date.getTime() - start)
})

app.use(function* poweredBy(next) {
  this.set('X-Powered-By', 'koa')
  yield* next
})

app.use(function* pageNotFound(next) {
  yield* next
  if (!this.status) {
    this.status = 404
    this.body = 'Page Not Found'
  }
})

app.use(function* (next) {
  if (this.path === '/204')
    this.status = 204
})

Into this:

co(function* () {
  var start = Date.getTime()
  this.set('X-Powered-By', 'koa')
  if (this.path === '/204')
    this.status = 204
  if (!this.status) {
    this.status = 404
    this.body = 'Page Not Found'
  }
  this.set('X-Response-Time', Date.getTime() - start)
}).call(new Context(req, res))

Which is ideally how a web application should look if we weren't so lazy. The only overhead is the initiation of a single co instance and our own Context constructor that wraps node's req and res objects for convenience.

Using it for type checking

If you yield* something that isn't a generator, you'll get an error like the following:

TypeError: Object function noop(done) {
  done();
} has no method 'next'

This is because essentially anything with a next method is considered a generator!

For me, I like this because I, by default, assume that I'm yield* gen(). I've rewritten a lot of my code to use generators. If I see something that isn't written as a generator, I'll think to myself, "can I make this simpler by converting it to a generator?".

Of course, this may not be applicable to everyone. You may find other reasons you would want to type check.

Contexts

co calls all continuables or yieldables with the same context. This particulary becomes annoying when you yield a function that needs a different context. For example, constructors!

function Thing() {
  this.name = 'thing'
}

Thing.prototype.print = function (done) {
  var self = this
  setImmediate(function () {
    console.log(self.name)
  })
}

// in koa
app.use(function* () {
  var thing = new Thing()
  this.body = yield thing.print
})

What you'll find is that this.body is undefined! This is because co is essentially doing this:

app.use(function* () {
  var thing = new Thing()
  this.body = yield function (done) {
    thing.print.call(this, done)
  }
})

and thus, this refers to the Koa context, not thing.

This is where yield* comes in! When context is important, you should be doing one of two things:

yield* context.generatorFunction()
yield context.function.bind(context)

By using this strategy, you'll avoid 99% of generator-based context issues. So avoid doing yield context.method!