The Golden Age of Distributed Computing

I have been invited for the first time as keynote speaker. I'm honored and thankful to the organizers of the Python San Sebastián for the opportunity, and for getting me the biggest impostor syndrome shot of my entire life. The other two keynote speakers are Jessica McKellar, Director of Engineering at Dropbox, and Peter Wang, co-founder and CTO of Continuum Analytics. My profile is dwarfed by theirs.

Here's my abstract:

The Golden Age of Distributed Computing

I built my first computer cluster during the summer of 2004 using two old PCs, a laptop and a cheap fast Ethernet switch. My plan was to do some parallel number crunching with Octave, looking for something better than C and Fortran for large-scale numerical computing, or using a somewhat trendy terminology, Big Data. That was the during bronze age of distributed computing.

Now we can run complex distributed computing pipelines using task graphs connected to the event loop within a microservice acting as a broker. Beyond all these buzzwords, it's obvious that distributed computing has made a giant leap in a little more than a decade in terms of capability and simplicity. I think that we are at the gates of a golden age, where distributed computing will be used just as a component of larger applications. This talk is a review of the key concepts and technologies that have made this leap possible, and some hypotheses about what the future may bring.