A new language for every new problem

I got excited with the previous issue and I wanted to talk more about programming languages and Lisp.

Hello, Hypers!

What if I tell you that you can always have the language that better fits your problem?

We are always implementing design patterns and abstractions to adapt our programming language to the domain of the problem we are solving. We create the classes Store, Product, and Customer. We create functions to represent the operations of selling, buying, updating the inventory, etc. But all those building blocks are always expressed in terms of a general-purpose language that was not thought to fit our very specific necessities.

In this article, I'll talk about the paradigm that is used in the Lisp programming language. With Lisp, we not only create a program but also create a specific language at the same time. Thus, we end up with a Domain-Specific Language (DSL) and a short, elegant, and clean program.

Understanding the point

We always create new entities and operators when programming. We implement functions with meaningful names that express part of the domain of our problem. But those functions and classes are created in the same way, with a rigid syntactical structure. No matter how elaborated your abstractions are, you always build them in the same way. For example, creating a class in Python will always be like this:

Maybe you can imagine another more expressive way to create classes in the domain you are working in, but there is no way to change that syntax into a kind of shortcut to create those domain-specific classes. You can solve a lot of problems with inheritance and other mechanisms, but a function always needs to be built with the same syntactical structure, and the same happens with classes.

You can redefine operators like +, and ==. But you can't modify what the class reserved word does. You can't add a new operator to create classes either.

All the programs you create in Python are written with the same vocabulary. Your coding expertise can make the code to be expressive, but no matter the expertise, that expressiveness will always have a limit. A limit imposed by the syntax of the programming language and its reserved words.

Most of the developer community is so used to these limitations that not even question them. We learn to code in a specific language, we learn some algorithms and the way we can translate them into that programming language. We get used to building programs in that way.

But there's another way. I just want to make a little detour right now and then dive into the main topic.

Limitations of C-like macros

If you have programmed in C/C++, Assembly, or some other language that supports macros (but not the Lisp-like macros), you could think that the previous section problem does not apply to that language.

There is a big difference between C-like macros and what we're going to talk about in the next section.

With C-like macros, we can define a pattern and then substitute any occurrence of that pattern in our program for a piece of code, in compilation time. For example, we can do:

Before actually compiling the program, the C compiler substitutes any occurrence of MAX for 1000000. Extending that functionality, we could modify the syntax for for loops and if statements if we wanted. But there's still an important limitation.

We don't have all the power of the language to define those macros. We just can define some mapping between string patterns and pieces of code. And macros can't make any computation in compilation time. They're just a text substitution, nothing more.

Let's see the real power of Lisp.

Creating a new language every time

As I said before, when you code in Lisp (in the right way), you are doing two things. Firstly, create a new language that is specific to your domain, and secondly, code a program in that expressive and very convenient language you've just created.

Actually, there's no order. You create both the language and the program simultaneously. Both of them evolve during the development process and the final product is a highly readable and clean program.

You can use another brand-new operator to create classes that represent stores for example:

That could be equivalent to building a class named my-store, which might implicitly inherit from a store parent class. We are passing a capacity, a budget, and a description. Then, in compilation time, the define-store operator can do all the things you can imagine, maybe it is not even a class definition. The thing is, we are declaring an entity in a very clear and domain-specific way.

It might be a toy example and it is totally out of context. But the cool thing is that you can imagine any sort of complicated computations running under the hood, and that will be a possible scenario. With enough expertise, you can transform Lisp into the language you need to make the shortest and cleanest program.

In the above example, the operator define-store is a macro. A macro in Lisp is like a special function. They can receive parameters and they are executed in compilation time, in a step called macroexpansion. Then, the macro call is substituted by its own result in compilation time. For example, we can define the following macro:

We defined a macro called hundred that receives no parameters. It returns the result of multiplying 2 and 50.

Notice that we write the arithmetic operator first and the operands later. This is the default way of doing arithmetic operations in Lisp, but of course, you can use some advanced Lisp magic to change this syntax ;-)

Now, what happens if we write

The first step is macroexpansion. The compiler substitutes the macro call for the result of that call and the code becomes:

Then, the program is executed and we obtain the result 130.

Notice the difference with C-like macros. There was more than just text substitution, there was computation and then, code substitution.

If we change the macro definition by adding just one more character (notice the quote):

Then the macroexpansion changes a little bit too:

But the output remains the same. Now the macro returns the code as-is, and there are no further computations in compilation time.

Beyond macros

Talking deeper about Lisp macros requires having a basic knowledge of Lisp. In this article, I don't assume any prior knowledge about Lisp. So, the examples I can use are very limited. It wouldn't make any sense to include a complex macro definition. But my point is not to teach you how to define some macros in Lisp but to show you another way of thinking about programming.

The idea of building your program and the language in which your program is written at the same time is, at least, beautiful. The elegance of the process can only be compared with that of the final result.

We end up with two products: the desired program and a language that we can use totally or partially in our next journey.

Just knowing about this philosophy is totally worth it.

Conclusions

In this article, I have talked about Lisp's macros.

They are a powerful feature that allows us to define new Domain-Specific Languages for every problem we face. With them, we can create the cleanest programs and get the language in which that program is written at the same time.

That immense power makes us approach programming in a very different way. Trust me, there are a lot of more cool things to talk about. Let me know if you like this content and want me to keep writing about it. Leave your impression and/or your comment, and don’t forget to subscribe and share Under the Hype with your peers.

See you next week!