James Stanier - Probabilistic Data Structures

James Stanier introduces probabilistic data structures like Bloom filters and HyperLogLog, explaining how they trade accuracy for extreme scale and speed.

This talk was recorded at our May 2016 talks night. For more information on our meetups and to join us at our next live event visit meetup.com/Brighton-Java ----------- Acceptably inaccurate: probabilistic data structures - James Stanier Writing software for the Internet puts engineers face to face with traffic that makes doing even simple things difficult. Seemingly straightforward operations such as counting, set membership and set cardinality become either extremely slow or prohibitively expensive to do using traditional methods. Help is at hand, however: probabilistic techniques are both fascinating and extremely useful. By sacrificing a predictable amount of accuracy, we can perform operations at scales we never thought possible, and fast! We'll introduce approaches such as Bloom filters for set membership, count-min sketch for frequency in streams, and HyperLogLog for cardinality. No maths PhD is required. We'll look at the before and after effects of using these techniques in real-world scenarios, and present the libraries that you can go away and play with right now.