Parser and Lexer — How to Create a Compiler part 1/5 — Converting text into an Abstract Syntax Tree

Bisqwit demonstrates how to build a lexer and parser in C++ using re2c and GNU Bison to convert source code into an AST.

In this tool-assisted education video I create a parser in C++ for a B-like programming language using GNU Bison. For the lexicographical analysis, a lexer is generated using re2c. This is part of a multi-episode series. In the next video, we will focus on optimization. Become a member: youtube.com/Bisqwit/join Downloads: — github.com/bisqwit/compiler_series/tree/master/ep1 All the material associated with this episode can be downloaded here. Acknowledgements: — Picture: Processors :: Jason Rogers — Music¹: Aryol :: The Strategy Continues :: Kyohei Sada (converted into MIDI and played through OPL3 emulation through homebrew software) — Music²: Star Ocean :: Past Days :: Motoi Sakuraba (SPC-OPL3 conversion) — Music³: Rockman & Forte :: Museum :: Kirikiri-Chan and others (SPC-OPL3 conversion) — Music⁴: Famicom Tantei Club Part II: Ushiro ni Tatsu Shōjo :: Dean’s Room :: Kenji Yamamoto (SPC-OPL3 conversion), original composition: Bach's Invention № 15 — Music⁵: Aryol :: Arrest :: Kyohei Sada (SPC-OPL3 conversion) — Music⁶: Ren & Stimpy Show : Fire Dogs :: Main Theme :: Martin Gwynn Jones and others (SPC-OPL3 conversion) — Music⁷: Aryol :: Warmup :: Kyohei Sada (SPC-OPL3 conversion) — Music⁸: Energy Breaker :: Golden-Colored Wind :: Yukio Nakajima (SPC-OPL3 conversion) — Music⁹: Wonder Project J :: House :: Akihiko Mori (SPC-OPL3 conversion) — SFX: Mostly from YouTube Audio Library. Some are recorded from video games like The Guardian Legend, Lunar Ball, and Super Mario All-Stars. ¹ 00:37, ² 02:46 & 39:26, ³ 10:10, ⁴ 16:06, ⁵ 27:18, ⁶ 37:20, ⁷ 38:58 & 45:58, ⁸ 49:00, ⁹ 50:40 My links: Twitter: twitter.com/RealBisqwit Liberapay: liberapay.com/Bisqwit Steady: steadyhq.com/en/bisqwit Patreon: patreon.com/Bisqwit (Other options at bisqwit.iki.fi/donate.html) Twitch: twitch.tv/RealBisqwit Homepage: iki.fi/bisqwit/ You can contribute subtitles: youtube.com/timedtext_video?ref=share&v=eF9qWbuQLuw or to any of my videos: youtube.com/timedtext_cs_panel?tab=2&c=UCKTehwyGCKF-b2wo0RKw… ---Rant--- [9:35 PM] Bisqwit: Now uploading to YouTube. Within about 24 hours I will know if the rogue AI at YouTube slams the “limited or no advertising" stamp into it, or not. Actually, I only know if it does so *when* it does it. Then, I need to wait an additional 25 hours for YouTube staff to manually review it and clear the flag. If the flag does not appear, then it is possible that the bot just has not scanned it yet and I need to wait longer. Premature publication could mean that the bot will mark it after it has already been published, and then I will not receive any revenue for the first spike of views. It used to be 18 hours (since uploading that the bot does its evil deeds), but nowadays YT recommends waiting just 3 hours. We will see, we will see. #Bisqwit #Compiler #Tutorial