I/O is slow

Numbers by Jeff Dean (abridged)

L2 cache reference 7 ns
Main memory reference 100 ns
Send 2K bytes over 1 Gbps network 20,000 ns
Disk seek 10,000,000 ns
Send packet CA->Netherlands->CA 150,000,000 ns

Synchronous I/O


import socket
sock = socket.socket()

...

def do_important_work():
    sock.recv(1024)
    logger.info('Did 1024 bytes of I/O')

    sock.send('hello')

    sock.recv(512)
    logger.info('Did 512 bytes of I/O')

    ...
                

Async with callbacks


import socket
sock = socket.socket()

...

def do_important_work():
    register_callback(sock.recv, 1024, first_done)

def first_done(event):
    logger.info('Did 1024 bytes of I/O')
    register_callback(sock.recv, 512, second_done)

def second_done(event):
    logger.info('Did 512 bytes of I/O')
    register_callback(...)
                

Event loop


while True:
    events = wait_for_an_IO_device_to_be_ready()
    for event in events:
        callback, args = dispatch_table[event]
        callback(event, *args)
                

gevent

"gevent... uses greenlet to provide
a high-level synchronous API
on top of the libevent event loop."

gevent style


from gevent import socket
sock = socket.socket()

...

def do_important_work():
    sock.recv(1024)
    logger.info('Did 1024 bytes of I/O')

    sock.send('hello')

    sock.recv(512)
    logger.info('Did 512 bytes of I/O')

    ...
                

How?

gevent

gevent... uses greenlet to provide
a high-level synchronous API
on top of the libevent event loop."

greenlet

"Lightweight in-process concurrent programming"
... you mean threads?

Threads

  • More overhead
  • Locking / race conditions
  • Scheduled by the OS

Greenlets

  • Super lightweight
  • Explicit switching
  • Serialized execution

greenlet example


from greenlet import greenlet

def hi_bye():
    print 'hi'
    g2.switch()
    print 'bye'

def middle():
    print '...'
    g1.switch()

g1 = greenlet(hi_bye)
g2 = greenlet(middle)
g1.switch()
                

greenlets in gevent

  1. Greenlet performs "blocking" I/O call
  2. Call yields control back to "hub" greenlet
  3. Hub greenlet swiches to next runnable greenlet

gevent style


from gevent import socket
sock = socket.socket()

...

def do_important_work():
    sock.recv(1024)
    logger.info('Did 1024 bytes of I/O')

    sock.send('hello')

    sock.recv(512)
    logger.info('Did 512 bytes of I/O')

g1 = gevent.spawn(do_important_work)
g2 = gevent.spawn(do_important_work)
gevent.joinall([g1, g2])
                

Co-operation

In order to co-operate, a greenlet must eventually yield by performing a "greened" blocking operation. So what if a library isn't co-operative?

Monkey patching (is evil)


from gevent import monkey
monkey.patch_socket()

import socket
sock = socket.socket()
...
                

Why is it evil?

  • Don't mutate global state
  • Modules: the ultimate global state
  • Only use to make other packages cooperative
  • Will not affect blocking C extensions!
    • Look for "green" versions of your favorite packages

Summary

  • Asynchronous code in a synchronous style
  • Lightweight, cooperatively scheduled threads
  • Requires other modules to be cooperative too
  • Monkey patch or use "green" versions

Links

End