Ohhhhh shit. I’m going to build a mini lisp!. It’s about time for me to take a break from building user-facing products and throw myself deeper into that non-gui-programming/the-developer-is-the-user deep-end.

I’ve maintained a healthy envy of people writing their own languages, compilers, interpreters, etc. The closest I come to that in the Front End World is hearing people argue about Babel. So, yeah compilers. They have an intimidating aura about them.

After paying attention to the Lisp world for about a year, I’ve come across enough comment-nests and conversations of people praising Lisp, saying: It’s just so simple! It understands itself. Metaprogramming! Eval! Homoiconicity!

But, I’ve never written a macro for a production code base. I’ve read the definition of Homoiconicity, and I’ve heard people say Code is Data plenty of times. I don’t think these values have really sunk in with me yet – and why would it? Previously, I’ve been developing front-end interfaces with ClojureScript, not writing parsers or planting Abstract Syntax Trees.

I think (trying) to see how things under the hood work (as in the case of BYOL) might not be the best approach to better understanding the unique features of Lisp, but I think it’s a start. Also, it seems like a great chance to get a better grasp of the more challenging parts of writing C, as well as the C ecosystem.

As with most things I learn in programming, I don’t imagine it will make sense right away. At the very least, BYOL is a sweet resource.

I’ll be digging into the content a bit more thoroughly so here are a few initial impressions/surprises from reading the first 6 chapters.


Impressions

This book starts out with the right kind of programming-kindness-love that I adore. I found this book when I was pretty burnt out programming-wise and it helped light the tiniest of binaric fires back into my cold heart. Check it out:

  1. Regarding Friendly Quotes

    I’ve tried to make this book as friendly as possible to beginners. I welcome beginners the most because they have so much to discover! But beginners may also find this book challenging. We will be covering many new concepts, and essentially learning two new programming languages at once.

    If you look for help you may find people are not patient with you. You may find that, rather than help, they take the time to express how much they know about the subject. Experienced programmers might tell you that you are wrong. The subtext to their tone might be that you should stop now, rather than inflict your bad code on the world.

    After a couple of engagements like this you may decide that you are not a programmer, or don’t really like programming, or that you just don’t get it. You may have thought that you once enjoyed the idea of building your own programming language, but now you have realized that it is too abstract and you don’t care anymore. You are now concerned with your other passions, and any insight that may have been playful, joyful or interesting will now have become an obstacle.

    For this I can only apologize. Programmers can be hostile, macho, arrogant, insecure, and aggressive. There is no excuse for this behaviour. Know that I am on your side. No one gets it at first. Everyone struggles and doubts their abilities. Please don’t give up or let the joy be sucked out of the creative experience. Be proud of what you create no matter what it is. People like me don’t want you to stop programming. We want to hear your voice, and what you have to say.

    Isn’t that beautiful!

  2. Regarding easy beginnings.

    The book starts out very friendly all the way up until around chapter 5 (IIRC) when the external MPC parser is introduced. I read some comments online bemoaning the fact that the book has the reader using an external library for the parser when “writing one is just so simple!“.

    Admittedly, going from some basics of C to including a black box library for parsing your Lisp-To-Be is an intimidating chunk, but I found that once I started just using the library as an API I could make some educated guesses on what was happening.


Surprises / Realizations

A few short ones:

Some longer ones:

I like this definition of what a parser is:

mpc is a Parser Combinator library I have written. This means it is a library that allows you to build programs that understand and process particular languages. These are known as parsers. There are many different ways of building parsers, but the cool thing about using a Parser Combinator library is that it lets you build parsers easily, just by specifying the grammar … sort of.

Many Parser Combinator libraries actually work by letting you write normal code that looks a bit like a grammar, not by actually specifying a grammar directly. In many situations this is fine, but sometimes it can get clunky and complicated. Luckily for us, mpc allows us to write normal code that just looks like a grammar, or we can use special notation to write a grammar directly!

Seeing the grammar directly written is pretty cool – it’s nitty-gritty. It’s manual labour. It’s hammering and screwing together a language out of blocky wood blocks.


Next up - the Abstract Syntax tree. I’ve heard this term thrown around a lot but haven’t dug into it. I’m starting to think that maybe things aren’t as complicated as I made them out to be (a Good Thing) and that, maybe what I’m trying to do is not much more complicated than this (looking forward to pleasantly being proved wrong!):

  1. Build a set of rules (grammar) for identifying a big ol’ string of chars.
  2. Build a tool (function) for iterating recursively through that big ol’ string, breaking it into pieces depending on how it fits into a grammar.
  3. Applying other functions, these logic-based, based on the broken down input.
  4. Do this recursively through a tree structure.

This was made pretty clear by chapter 7 when the first eval function is written. And woah, hey there, I guess I just built a repl? How did that happen? Where did my hamburger go? (I ate it). Sure, it only does basic arithmetic, but it’s pretty cool to see that all that is happening is:

I haven’t done enough work with AST’s to feel super comfortable with them; tree data structures haven’t really shown up that much in most Front End work so I’m not really sure what will get me more acquainted with the concepts.

Onwards

The content is fun and engaging and well put together, and oh-so-free-online.

Aside: Turns out there are a couple of similar resources for building a lisp in haskell and a whole repository of lisp’s built with all different languages. Amazing.

Keep your (parens))))) tuned into this url for another post.