Skip to content
Hallway
0:00 / 0:00

#tree-sitter#parsing#glr

"Tree-sitter - a new parsing system for programming tools" by Max Brunsfeld

Strange Loop Conference

88K subscribers • 107 videos on Hallway • Conference channel

YouTube
  • Recommended by 5 people: 4 on Hacker News and 1 curated list.
  • A classic. People kept recommending it from 2018 to 2024.
  • Presented by Max Brunsfeld

Intermediate. Assumes working knowledge.

63K views8 years ago5 recommendations

Systems programmingTools and practiceCompilers and interpretersVim, Emacs and editors

Max Brunsfeld introduces Tree-sitter, an incremental parsing system designed for real-time code analysis in text editors and developer tools.

Developer tools that support multiple programming languages generally have very limited, regex-based code-analysis capabilities. Tree-sitter is a new parsing system that aims to change this paradigm. It provides a uniform C API for parsing an ever-growing set of languages. It features high-performance incremental parsing and robust error recovery, which allow it to be used to parse code in real-time in a text editor. There are bindings for using Tree-sitter from Node.js, Haskell, Ruby and Rust. We're in the process of integrating Tree-sitter into both GitHub.com and the Atom text editor, which will allow us to analyze code accurately and efficiently, paving the way for better syntax highlighting, code navigation, and refactoring. We'll demo some new features that Tree-sitter has enabled in GitHub.com and Atom, discuss some the interesting algorithms that it uses, and share thoughts on some potential future applications. Speaker: Max Brunsfeld

Why it's here

5 people recommend this

What people say. Recommended as an excellent explanation of the design choices, incremental parsing, and error recovery mechanisms behind Tree-sitter.

ievanson Hacker News • Recommends

On “Difftastic, a structural diff tool that understands syntax”

…tooling was often the best approach because it is guaranteed to keep up with the latest syntax. IMO the genius of tree-sitter is that it's made it way easier than with traditional grammars to keep the language parsers updated. Highly recommend Max Brunsfield's strange loop talk if you want to learn more about the design choices behind tree-sitter: link to this video And this has resulted in a bunch of new tools built off on tree-sitter, off the top of my head in addition to difftastic: neovim, Zed, Semgrep, and Github code search!…
View on Hacker News

programming-talkscurated list on GitHubRecommends

Under “Data Structures & Algorithms”

Tree-sitter - a new parsing system for programming tools 00:38:37 by **Max Brunsfeld** (2018)
View the list

lemmingon Hacker News • Recommends

On “Owl: Parser generator for visibly pushdown languages”

For a really nice solution to the error message problem, see this recent strangeloop talk: link to this video Basically it uses the parse tree disambiguation from the GLR parser to look for the most likely mistake the user made - it's very clever.
View on Hacker News

rtsaoon Hacker News • Recommends

On “Introducing Semgrep and r2c”

…and OniVim 2. Hopefully regex-based syntax highlighting is a thing of the past soon. If you haven't seen the Strange Loop conference talk on tree-sitter 2 yet, it's worth a watch. I think a Prettier-like code formatter using tree-sitter would be cool, both in terms of potentially broader language support and native performance. 1: link 2: link to this video…
View on Hacker News

AlexErranton Hacker News • Recommends

On “Lessons from building GitHub code search [video]”

I also enjoyed the Treesitter talk from 5 years ago by Max Brunsfeld link to this video I'm currently building a query language whose grammar is very much inspired by Github's search syntax. I'm using Lezer, which is a GLR like Treesitter, so this talk learned me some parser generators (I've no formal CS education). Here's my grammar, a playground, and an example search query if anyone wants to play with it link link -(a) spider-man -a b -c -"(quote\"d) str" OR "l o l" OR a b c ((a "c") b) tag:what -deck:"x y" I just converted the syntax tree to an abstract syntax tree so I can do De Morgan's law transformations. Literally the only time in my professional life where I feel like I'm solving a leetco…
View on Hacker News

u/shackra'(secretaria.el font-man-mode flymake-golangci tortuga)in r/emacs • Mentions

A few comments on tree-sitter for Emacs via rust bindings for Emacs lisp Hello! I would like to talk a bit about tree-sitter which is a new parsing system that aims to change this paradigm of limited regexp based code-analysis. It provides a uniform C API for parsing an ever-growing set of languages. It features high-performance incremental parsing and robust error recovery, which allow it to be used to parse code in real-time in a text editor and which have bindings for using Tree-sitter from Node.js, Haskell, Ruby and Rust (see this keynote from the creator of Tree Sitter to know more). emacs-tree-sitter is a project that bases its bindings for Emacs lisp on the Rust bindings of Tree S…
47View on Reddit

u/theindigamerin r/ProgrammingLanguages • Mentions

On “Tree-sitter - a new parsing system for programming tools”

Link to talk: link to this video
3View on Reddit

u/raymyersin r/ProgrammingLanguages • Mentions

On “Writing a language Server”

This HN post and Strange Loop talk by Max Brunsfeld discuss design tradeoffs of tree-sitter and in particular why he sees LSP as unsuitable for syntax highlighting. Maybe OP will find useful.
3View on Reddit

throwaway_pdp09on Hacker News • Mentions

On “Advanced Compilers: Self-Guided Online Course”

Parsing (use of rather than theory) matters as it affects my work. So I followed up. See link to this video Tree sitter is based on LR parsing (see 23:30 in above video) extended to GLR parsing (see 38:30). I've had enough of fools on HN posting unverified crap to make themselves feel cool and knowledgeable (and don't kid yourself that you helped me find the right answer by posting the wrong one). Time I withdrew. Goodbye HN.
View on Hacker News

u/therealjohnfreemanin r/ProgrammingLanguages • Mentions

On “Good design patterns when writing "forgiving parsers"?”

Take a look at Tree-sitter and see if it meets your needs. There's a talk if you don't want to dig through docs.
2View on Reddit