Introduction to Makefiles

I often use Makefiles in some of my projects.
I really like the flexibility it gives, and I often find myself writing a Makefile instead of a simple shell script to automatize tasks.

So here's a little crash course.
I'll obviously only cover the basics, but I hope this will give you a good idea on how you could improve your workflows using Makefiles.

About Make

Make was developed in 1976 mainly as a build automation tool, to produce executable files or libraries from source code.

While it excels as a build system, it can also be used for a lot of different things.
If you do write shell scripts to automatize certain tasks, you'll be able to use Makefiles instead.
As we're going to see, a Makefile can have several advantages over a regular shell script.

This tutorial will only be focused on the GNU version of Make, as it's the most widely used and the most powerful.

Basics

First of all, when you invoke the make command, it will look for a file named Makefile in the current working directory.
This is the default, but note that a specific Makefile can be used with the -f flag, followed by the file name or path.

If such a file is found, it will by default execute the all target.

Make is target-based system.
Your Makefile can specify multiple targets, and targets may be executed individually when invoking make. But more on this later.

For now, we'll just start by creating a basic hello world example.

hello, world

In some directory, create a file called Makefile with the following content:

all:
    
    echo "hello, world"

all is the target name. Target definitions are followed by a colon sign.
As mentioned earlier, make will by default look for a target called all. So this is our main entry point.

Inside the target, you'll simply execute shell commands. Here, we print the hello, world string, using the shell's builtin echo command.

Note that target commands need to be indented with at least a single tab.
While spaces can be used elsewhere for indentation, tabulation is mandatory inside a target.

Now from a command prompt, cd to that directory and type make.

make will read the Makefile, and execute the all target, giving the following output:

echo "hello, world"
hello, world

As you can see, make will first print the full command, before printing any output.
This can be disabled by using an @ sign before the command:

all:
    
    @echo "hello, world"

Now the output is simply:

hello, world

Additional targets

You can define as many targets as you want.
For instance:

all:
    
    @echo "hello, world"

foo:
    
    @echo "hello, foo"

bar:
    
    @echo "hello, bar"

While invoking make will still only execute the all target, the foo or bar targets can be executed individually by specifying their names:

$ make
hello, world

$ make foo
hello, foo

$ make bar
hello, bar

Target dependencies

A target may depend on another target, or on multiple other targets.
This is called a prerequisite.

Prerequisites follows the target name:

foo: bar
    
    @echo "hello, foo"

Here, the foo target depends on bar. This means that when foo is about to be executed, bar will be executed first.

Multiple prerequisites are simply separated by a space:

foo: bar all
    
    @echo "hello, foo"

Here, upon executing foo, make will start by executing bar, then all, and finally foo.

And obviously, chaining works too:

all: foo
    
    @echo "hello, world"

foo: bar
    
    @echo "hello, foo"

bar:
    
    @echo "hello, bar"

all depends on foo, which depends on bar. So when invoking make, you'll get the following output:

hello, bar
hello, foo
hello, world

And you can also manually execute foo by typing make foo, which will give:

hello, bar
hello, foo

Error handling

A very nice thing about make is that it does error handling for you.
If a command returns a non-zero exit status, make will report the error and abort execution.

This means that if a command fails inside some target (which may be a prerequisite of another target), the whole execution will stop.
So you don't have to do any manual error checking, as you would/should do with a shell script.

For instance:

all: foo
    
    @echo "hello, world"

foo: bar
    
    @echo "hello, foo"

bar:
    
    @echo "Executing false"
    @false
    @echo "hello, bar"

Note that in the bar target, we execute the shell's false command, which always returns a non-zero exit status.
Now if we invoke main, we'll get the following output:

Executing false
make: *** [bar] Error 1

make will execute all, which needs to execute foo, which needs to execute bar. bar will print the first message, and then execute the false command.

As it returns a non-zero exit status, this is detected as an error, and execution is stopped.
The remaining message in bar will not be printed, and the foo and all targets won't be executed.

Debugging

Also note that you can obtain detailed informations about how make reads your Makefile using the --debug flag.
With the previous example:

$ make --debug
Reading makefiles...
Updating goal targets....
    File `all' does not exist.
        File `foo' does not exist.
            File `bar' does not exist.
        Must remake target `bar'.
Executing false
make: *** [bar] Error 1

Variables

You can also define variables inside your Makefile.
Variables are defined outside targets, and can be referred to with a $ sign and parenthesis:

HELLO := hello, world

all:

    @echo "$(HELLO)"

Variables may also be overridden when invoking make, giving extra flexibility.
For instance, with the example above:

$ make HELLO="This is a test"
This is a test

We'll cover more about variables later.

Real life example - Build system

Now that we have covered the basics, let's take a more useful example.

We'll create a simple build system for the C programming language.
The goal is to compile C source files, and to produce an executable.

We'll start by a very simple build system, and work on it step by step to achieve a more generic one.

Project structure

Here's the basic project structure:

• build (directory)
• Makefile
• source (directory)
  • main.c

We have a build directory for the final executable and temporary files, the Makefile, and a source directory with a single main.c file.

The main.c file is a basic hello world program:

#include <stdio.h>

int main( void )
{
    printf( "hello, world\n" );
    
    return 0;
}

Producing a simple executable

We'll start with a very simple Makefile that invokes the clang C compiler.
You can obviously replace it with gcc if you want:

all:
    
    @clang -Wall -Werror source/main.c -o build/main

When invoking make, it will compile the source/main.c and produce an executable in build/main.
Dead simple.

Compiling multiple files

Now let's say we want to compile multiple C files to produce the executable.

We'll first create a function named hello in the source/hello.c file:

#include <stdio.h>
#include "hello.h"

void hello( void )
{
    printf( "hello, world\n" );
}

And we'll also add the corresponding header in source/hello.h with the function prototype:

#ifndef HELLO_H
#define HELLO_H

void hello( void );

#endif

Our main.c file will then call the hello function:

#include "hello.h"

int main( void )
{
    hello();
    
    return 0;
}

Now the Makefile could simply be:

all:

    @clang -Wall -Werror source/hello.c source/main.c -o build/main

However, this is not really flexible, and this is usually not how individual files are compiled.
Instead, we'll produce an object file for each C source file, and link them together to produce the final executable:

all:

    @clang -Wall -Werror -c source/hello.c -o build/hello.o
    @clang -Wall -Werror -c source/main.c -o build/main.o
    @clang -Wall -Werror build/hello.o build/main.o -o build/main

Note the additional -c flag, needed to tell the compiler to produce an unlinked object file, instead of an executable.

But we obviously want the compilation to happen in separate targets, so we'll create a specific target for each C source file.
The all target will depend on these, and be responsible for linking the executable:

all: main hello
    
    @clang -Wall -Werror build/hello.o build/main.o -o build/main
    
main:
    
    @clang -Wall -Werror -c source/main.c -o build/main.o
    
hello:
    
    @clang -Wall -Werror -c source/hello.c -o build/hello.o

Also notice that the compiler flags (-Wall -Werror) are now repeated in each target.
Time to create a variable:

CFLAGS := -Wall -Werror

all: main hello
    
    @clang $(CFLAGS) build/hello.o build/main.o -o build/main
    
main:
    
    @clang $(CFLAGS) -c source/main.c -o build/main.o
    
hello:
    
    @clang $(CFLAGS) -c source/hello.c -o build/hello.o

This is obviously better, and it also mean we can now override the compiler flags when invoking make:

$ make CFLAGS=-Weverything

It might also be a good idea to create a variable for the compiler itself:

CC     := clang
CFLAGS := -Wall -Werror

all: main hello
    
    @$(CC) $(CFLAGS) build/hello.o build/main.o -o build/main
    
main:
    
    @$(CC) $(CFLAGS) -c source/main.c -o build/main.o
    
hello:
    
    @$(CC) $(CFLAGS) -c source/hello.c -o build/hello.o

So if you want to use gcc instead of clang, you can simply use:

$ make CC=gcc

And we should also add some output:

    
CC     := clang
CFLAGS := -Wall -Werror

all: main hello
    
    @echo "Linking executable"
    @$(CC) $(CFLAGS) build/hello.o build/main.o -o build/main
    
main:
    
    @echo "Compiling main.c"
    @$(CC) $(CFLAGS) -c source/main.c -o build/main.o
    
hello:
    
    @echo "Compiling hello.c"
    @$(CC) $(CFLAGS) -c source/hello.c -o build/hello.o

File dependencies

When invoking make, we'll now get:

Compiling main.c
Compiling hello.c
Linking executable

But there's an issue here. If we run make again, all files will be recompiled.
Ideally, we want to compile the files only if it's necessary.

Fortunately, make makes this very easy, as it supports targets that are based on real files.
If the name of a target specifies a file name, the target will only be executed if the file does not already exist.

This is what we need.
We only want to compile the C files if the object files don't already exist in the build directory.

So we can change our Makefile the following way:

CC     := clang
CFLAGS := -Wall -Werror

all: build/main.o build/hello.o
    
    @echo "Linking executable"
    @$(CC) $(CFLAGS) build/hello.o build/main.o -o build/main
    
build/main.o:
    
    @echo "Compiling main.c"
    @$(CC) $(CFLAGS) -c source/main.c -o build/main.o
    
build/hello.o:
    
    @echo "Compiling hello.c"
    @$(CC) $(CFLAGS) -c source/hello.c -o build/hello.o

Notice how we replaced the main and foo target names with the expected produced files.

Now if we run make again, the build/main.o and build/hello.o targets won't be executed, because these files already exist.
For make, it means that the prerequisites are already satisfied.

And obviously we can do the same with the executable, to avoid linking it every time:

CC     := clang
CFLAGS := -Wall -Werror

all: build/main
    
    @echo "Build successful"

build/main: build/main.o build/hello.o
    
    @echo "Linking executable"
    @$(CC) $(CFLAGS) build/hello.o build/main.o -o build/main

build/main.o:
    
    @echo "Compiling main.c"
    @$(CC) $(CFLAGS) -c source/main.c -o build/main.o

build/hello.o:
    
    @echo "Compiling hello.c"
    @$(CC) $(CFLAGS) -c source/hello.c -o build/hello.o

Here we created an additional build/main target for the executable, on which all now depends.

As always, we can see what's going on with the --debug flag:

$ make
Reading makefiles...
Updating goal targets....
    File `all' does not exist.
        File `build/main' does not exist.
            File `build/main.o' does not exist.
        Must remake target `build/main.o'.
Compiling main.c
        Successfully remade target file `build/main.o'.
            File `build/hello.o' does not exist.
        Must remake target `build/hello.o'.
Compiling hello.c
        Successfully remade target file `build/hello.o'.
    Must remake target `build/main'.
Linking executable
    Successfully remade target file `build/main'.
Must remake target `all'.
Build successful
Successfully remade target file `all'.

Common useful targets

This is fine, but what if we want to force a full compilation again?

It is common practice to define a clean target that will remove temporary build files.
Nothing difficult here, we'll just remove the files from the build directory:

clean:
    
    @echo "Removing build files"
    @rm -rf build/*

We can also add a test target that will run the executable:

test: all
    
    @./build/main

Generic targets

Our Makefile is looking good so far.
But we can already see an upcoming issue.

If we want to add more C files, we'll have to add additional targets, which is not convenient at all.

Fortunately, make supports targets that match a specific pattern. You can think of it as a kind of wildcard.

The generic part is denoted with the % character in the target name.
It is called the stem.

This means we can create a single target named build/%.o, that will match every .o file in the build directory:

build/%.o:
    
    @echo "Compiling ???"
    @$(CC) $(CFLAGS) -c ??? -o ???

But we also need to retrieve the actual file name, so we can replace the ??? in the example above with the correct values.

For this purpose, make has predefined variables, such as:

• $@ The full name of the target, with the stem (%) expanded.
• $* The value of the stem (%).

Using these variables, we can create a generic target that will compile C files from the source directory into the build directory:

CC     := clang
CFLAGS := -Wall -Werror

all: build/main
    
    @echo "Build successful"
    
clean:
    
    @echo "Removing build files"
    @rm -rf build/*
    
test: all
    
    @./build/main

build/main: build/main.o build/hello.o
    
    @echo "Linking executable"
    @$(CC) $(CFLAGS) build/hello.o build/main.o -o build/main

build/%.o:
    
    @echo "Compiling $*.c"
    @$(CC) $(CFLAGS) -c source/$*.c -o $@

Now every time we execute a target with a build/ prefix and a .o suffix, such as build/main.o, it will compile the corresponding C file ($*) from the source directory into the destination file, which is the target name ($@).

And we can do the same for the executable target:

CC     := clang
CFLAGS := -Wall -Werror

all: build/main
    
    @echo "Build successful"
    
clean:
    
    @echo "Removing build files"
    @rm -rf build/*
    
test: all
    
    @./build/main

build/%.o:
    
    @echo "Compiling $*.c"
    @$(CC) $(CFLAGS) -c source/$*.c -o $@

build/%: build/main.o build/hello.o
    
    @echo "Linking executable"
    @$(CC) $(CFLAGS) $^ -o $@

Notice that we also moved the build/% target after build/%.o. The target order is important in the way make consider targets.

As build/% can match any file in the build directory, we want to give a higher priority to the build/%.o target, so its considered first.

We also introduced a new variable: $^
This contains the full list of the target's prerequisites.

Precious targets

When invoking main with the last example, we can notice a small difference:

$ make
Compiling main.c
Compiling hello.c
Linking executable
Build successful
rm build/main.o build/hello.o

Notice the last line. make is now automatically removing the .o files from the build directory after a successful build.

Why this sudden change?
It's because the build/%.o target is now called from a target that also contains a stem (%) - build/%.

make considers that files produced by a target with a stem called from a target with a stem are temporary.
And its default behavior is to remove them upon completion.

We can instruct make to keep these files by declaring the target as precious:

CC     := clang
CFLAGS := -Wall -Werror

.PRECIOUS: build/%.o

all: build/main
    
    @echo "Build successful"
    
clean:
    
    @echo "Removing build files"
    @rm -rf build/*
    
test: all
    
    @./build/main

build/%.o:
    
    @echo "Compiling $*.c"
    @$(CC) $(CFLAGS) -c source/$*.c -o $@

build/%: build/main.o build/hello.o
    
    @echo "Linking executable"
    @$(CC) $(CFLAGS) $^ -o $@

The .o files will no longer be deleted.

Detecting changes

Now what if we make changes to a C file and run make again?

$ make clean && make
Compiling main.c
Compiling hello.c
Linking executable
Build successful

$ touch source/hello.c
$ make
Build successful

This doesn't work.
We can obviously manually run make clean before, but this will recompile everything.
Ideally, we want to recompile only the changed files.

With make, this is really easy to achieve.
As we saw earlier, targets may represent existing files.
This also means we can use files as target prerequisites:

build/%.o: source/%.c
    
    @echo "Compiling $*.c"
    @$(CC) $(CFLAGS) -c $< -o $@

Here we added source/%.c as a prerequisite, which means that .o files in the build directory now depends on their corresponding C file in the source directory.

And as the C file is now a prerequisite, we no longer need to specify it manually in the clang invocation.
Instead, we use the $< variable, which contains the first prerequisite of the target.

Let's try that again:

$ make clean && make
Compiling main.c
Compiling hello.c
Linking executable
Build successful

$ touch source/hello.c
$ make
Compiling hello.c
Linking executable
Build successful

We can see that hello.c was recompiled, and that the executable was also relinked.
The build/% target was automatically executed, because one of its prerequisite needed to be executed again.
This is exactly what we want!

Variables manipulation

Our build system is quite nice so far, but we still have to manually specify the files we want to build:

    build/%: build/main.o build/hello.o

Let's start by making a variable for this; it will already be a little more convenient:

CC      := clang
CFLAGS  := -Wall -Werror
FILES_O := build/main.o build/hello.o

.PRECIOUS: build/%.o

all: build/main
    
    @echo "Build successful"
    
clean:
    
    @echo "Removing build files"
    @rm -rf build/*
    
test: all
    
    @./build/main

build/%.o: source/%.c
    
    @echo "Compiling $*.c"
    @$(CC) $(CFLAGS) -c $< -o $@

build/%: $(FILES_O)
    
    @echo "Linking executable"
    @$(CC) $(CFLAGS) $^ -o $@

We defined FILES_O, and we use it now as a prerequisite.

But we can make it better.
make provide several functions that can process text.

addprefix

For instance, we have a function called addprefix that adds a prefix to a variable.
As an example:

TEXT  := world
HELLO := $(addprefix hello ,$(TEXT))

Here we add the hello prefix to the TEXT variable.
The HELLO variable now contains hello world.

And obviously, there is also a function called addsuffix.

subst

make can also do text replacement with the subst function:

TEXT  := hello world
HELLO := $(subst world,universe,$(TEXT))

Here we replace the world string by universe in the TEXT variable.

patsubst

make also provides a replacement function that works with patterns on whitespace separated strings: patsubst

FILES_C := hello.c main.c
FILES_O := $(patsubst %.c,%.o,$(FILES_C))

Here FILES_O now contains hello.o main.o. For every string in FILES_C, we replaced the .c extension with .o.

And we can also combine with addprefix to add the build directory on each file, since addprefix also works on string lists:

FILES_C := hello.c main.c
FILES_O := $(addprefix build/,$(patsubst %.c,%.o,$(FILES_C)))

This would be a nice enhancement, be we can do even better.

Listing files

make is also able to get a file list from a directory, with the wildcard function:

$(wildcard source/*.c)

This will get every .c file in the source directory.

If we have multiple directories, we can use the wildcard function as above in a foreach:

    
DIR_SRC := source other
FILES_C := $(foreach dir,$(DIR_SRC),$(wildcard $(dir)/*.c))

Here, for each string in DIR_SRC, the foreach function will declare a variable called dir and pass it to its last argument, $(wildcard $(dir)/*.c), which gets all C file in the directory.

This way, we no longer have to specify the files we want to compile.
We can just add them to the source directory, and they will be compiled.

Wrapping up

Our final example is:

CC        := clang
CFLAGS    := -Wall -Werror
EXEC      := main
DIR_SRC   := source
DIR_BUILD := build

FILES_C   := $(foreach dir,$(DIR_SRC),$(wildcard $(dir)/*.c))
FILES_O   := $(addprefix $(DIR_BUILD)/,$(patsubst $(DIR_SRC)/%.c,%.o,$(FILES_C)))

.PRECIOUS: $(DIR_BUILD)/%.o

all: $(addprefix $(DIR_BUILD)/,$(EXEC))

    @echo "Build successful"

clean:

    @echo "Removing build files"
    @rm -rf $(DIR_BUILD)/*

test: all

    @./$(DIR_BUILD)/$(EXEC)

build/%.o: $(DIR_SRC)/%.c

    @echo "Compiling $*.c"
    @$(CC) $(CFLAGS) -c $< -o $@

build/%: $(FILES_O)

    @echo "Linking executable"
    @$(CC) $(CFLAGS) $^ -o $@

We also added a variable for the build directory and for the executable name.
We now have a pretty generic build system.

Conclusion

That's all I have for today. I really hope you found this article useful.

As you can see, Makefiles can be quite simple and very powerful.
While this article was focused on a build system, you can probably see how it can be applied to other tasks and conveniently replace shell scripts for day to day automation.

Finally, the last example is also available on my GitHub.
Feel free to use and adapt it as you want.