Skip to content
Hallway
0:00 / 0:00

#quickcheck#property-basedtesting#clojure

John Hughes - Testing the Hard Stuff and Staying Sane

ClojureTV

111K subscribers • 136 videos on Hallway

YouTube
  • Recommended by 7 people: 4 on Hacker News, 2 on Lobsters and 1 curated list.
  • A classic. People kept recommending it from 2016 to 2026.
  • Presented by John Hughes, co-creator of QuickCheck

Intermediate. Assumes working knowledge.

39K views12 years ago7 recommendations

Tools and practiceSystems programmingTestingFunctional programming

John Hughes explains how property-based testing with QuickCheck can find complex, low-level bugs in critical systems like the CAN bus standard.

The uploader added no description.

Why it's here

7 people recommend this

What people say. Highly recommended as an eye-opening demonstration of the power of property-based testing on real-world production systems.

Naomarikon Hacker News • Recommends

On “QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs (2000) [pdf]”

I'd recommend watching this link to this video to see how powerful this can be. I've just discovered generative testing due to clojure.spec. Describing specs for your data works not only to help validate data structures but can be plugged in as properties for generative testing. After the "aha" moment it was followed by a feeling that I've wasted so much time writing specs just to validate data structures and writing specs just to see if my functions are working as they should. This is one of the things I wish was obvious in my learning track right after vim/emacs will help you code faster.
View on Hacker News

hwayneon Lobsters • Recommends

Testing the Hard Stuff and Staying Sane
6View on Lobsters

programming-talkscurated list on GitHubRecommends

Under “Testing”

Testing the Hard Stuff and Staying Sane 00:47:16 by **John Hughes** (2014)
View the list

dllthomason Hacker News • Recommends

On “Show HN: IHP v1.0 – Batteries-included web framework built on Haskell and Nix”

While of course Haskell has more normal testing infrastructure available (eg. link ), my favorite bit of Haskell testing is QuickCheck, which IIUC started life in Haskell and has been reimplemented in other languages with various degrees of effectiveness and various degrees of connection to the original project. John Hughes (not the filmmaker) gives a great talk about it: link to this video
View on Hacker News

robertfwon Hacker News • Recommends

On “Hypothesis: Property-Based Testing for Python”

I feel like this talk by John Hughes showed that there is real value in this approach with production systems of varying levels of complexity, with two different examples of using the approach to find very low level bugs that you'd never think to test for in traditional approaches. link to this video
View on Hacker News

devinon Hacker News • Recommends

On “Mysteries of Dropbox: Property-Based Testing of a Distributed Sync Service [pdf]”

One of the authors, John Hughes did a talk on property-based testing at Clojure West some number of years back. Worth a watch if you're interested: link to this video
View on Hacker News

asthasron Lobsters • Recommends

Testing the Hard Stuff and Staying Sane (2014)
1View on Lobsters

u/rpglover64in r/haskell • Mentions

On “Porting Haskell ideas across languages”

No. Fuzzing is the idea of throwing adversarial inputs at your executable and seeing if it breaks. It's usually more of a black-box test and focused on detecting undefined and exploitable behavior. Property testing is about defining invariants/specifications/properties of sections of your code (functions, interfaces, etc.) and checking them, in a mechanized, but not necessarily in a randomized fashion. A good experience report is this paper (I haven't watched the associated link to this video).
9View on Reddit

u/stevanain r/haskell • Mentions

On “John Hughes - Building on developers' intuitions (...) | Lambda Days 19”

Do you perhaps mean the circular buffer example written in C from "Testing the Hard Stuff and Staying Sane" paper and talk? If so, we've reproduced that example in `quickcheck-state-machine` (QuickCheck + state machine modelling a la Erlang QuickCheck) here.
5View on Reddit

u/rpglover64in r/haskell • Mentions

On “How to use QuickCheck for stateful testing?”

You may like to read this experience report (talk here). Additionally, this blog post might be relevant. I can summarize as follows: Make a model, then test that your model accurately represents your system by generating a trace of operations and comparing the real system with the model.
4View on Reddit

illiarianon Hacker News • Mentions

On “Car Thieves Using Tech Disguised Inside Old Nokia Phones and Bluetooth Speakers”

> CAN bus is a brilliant thing for car diagnostics and modification but with every good system normally has a sword of damocles. There are over 100 issues (aka bugs) in the spec though. Uncovered by these guys: link to this video (starting at 19:10, includes examples)
View on Hacker News

troupoon Hacker News • Mentions

On “Voters Overwhelmingly Pass Car Right to Repair Law in Maine”

Possibly due to amount of data coming from various sensors and systems. BTW, CAN standard is riddled with bugs: link to this video at 26:18
View on Hacker News

u/jysticin r/haskell • Mentions

On “State machine testing with Hedgehog”

Fwiw, the gap in complexity between the model and the real implementation starts to get pretty wide once you have web servers and databases involved. If you want some more details on how these models pan out, I highly recommend: * John Hughes - Testing the Hard Stuff and Staying Sane (link to this video) * John Hughes - The Mysteries of Dropbox (link to this video)
2View on Reddit