Simulation in a Big Data World
An invited talk at London Infinitecon 2017 conference on Big Data and Fast Data, at the wonderful SkillsMater venue.
Introduction:
Many organisations are currently making a large investment in “big data” technology and training. In this talk I show how “simulation” workloads can also effectively re-use this technology and expertise so that they can be run at low cost and controled through same easy-to-use interfaces as the big data queries. By “simulation” I mean workloads which take few data (parameters, initial conditions) but consume significant CPU resources.
I demonstrate this approach with the example I of valuation of priceses of financial derivatives on Amazon AWS Elastic Map-Reduce clusters.
Slides:
