Dylan Beattie explores the hidden complexities, history, and assumptions behind what we call plain text files.
Software is complicated. Machine learning, microservice architectures, message queues... every few months there's another revolutionary idea to consider, another framework to learn. And underneath so many of these amazing ideas and abstractions is text. When you work in software, you spend your life working with text. Some of those text files are source code, some are configuration files, some of them are documentation. Editors, revision control systems, programming languages - everything from C# and HTML to Git and VS Code is based on the idea of "plain text files". But... what if I told you there's no such thing?
When we say something is a "plain text file", we're relying on a huge number of assumptions - about operating systems, editors, file formats, language, culture, history... and, most of the time, that's OK. But when it goes wrong, "plain text" can lead to some of the weirdest bugs you've ever seen... why is there Chinese in the event logs? Why is the city of Aarhus in the wrong place? And why does Magnus Mårtensson always have trouble getting into the USA? Join Dylan Beattie for a fascinating look into the hidden world of text files - from the history of mechanical teletypes to encodings, collations and code pages. We'll look at some memorable bugs, some golden rules for working with plain text - and we'll even find out the story behind the mysterious phrase "pike matchbox" and what it has do with driving in Belarus.
Check out more of our featured speakers and talks at
ndcconferences.com/ndcoslo.com/
Why it's here
2 people recommend this
What people say. Recommended for its engaging explanation of character encodings, ASCII, and UTF-8.
Moments people linked to
w
u/wosmoin r/C_Programming • Recommends
On “Weird and kinda long question about ASCII and UFT-8 I came up with after trying to recreate some basic c functions”
Be very careful treating extended ascii as a standard - it's not, it's a set of standards called codepages. So positions 0-127 are standardised, but 128-255 depend on the codepage in use.
This is what python is protecting you from in your example - ö is not in the lower 128, so it's not standard ascii. It's position 0xf6 in the ISO-8859-1 codepage (and in utf8, which adopted ISO-8851-1 as the Latin1 set) - but it does not exist in standard ascii.
>>> 'ö'.isascii()
False
>>> bytes('ö','ascii')
UnicodeEncodeError: 'ascii' codec can't encode character '\xf6' in position 0: ordinal not in range(128)
>>> bytes('ö','utf8')
b'\xc3\xb6'
So when you ask python to render ö…
On “What's your most painful experience of debugging an issue that only reproduced in production?”
I recently attended a talk by Dylan Beattie, and he has a wonderfull story about a production issue. It's in his talk 'Plain Text' (which is certainly worth watching in full).
Part mentioned starts here:
[video
