* Makefiles

# simple program
$ gcc -o myprog main.c
$ ./myprog

# most programs need to compile many source files

# compile all 4 src files at once
$ gcc -o myprog main.c net.c debug.c user.c

# assume you edited just debug.c
$ gcc -o myprog main.c net.c debug.c user.c
# problem: we're recompiling 3 files that haven't changed

# solution: compile each source file individually, then link them together
# recall 4 stages of C compiler: cpp, ccom, as, ld
# -c option runs a .c file through first 3 stages: cpp, ccom, and as
$ gcc -c main.c
$ gcc -c net.c
$ gcc -c debug.c
$ gcc -c user.c
# link all 4 .o object files into final executable (just run ld)
$ gcc -o myprog main.o net.o debug.o user.o

# now if you only changed debug.c
$ gcc -c debug.c
$ gcc -o myprog main.o net.o debug.o user.o

# main observation: building s/w converts a source S to destination D using
# some process P.  Example if S is foo.c, D is foo.o, and P is "gcc -c".

# how can you tell if debug.c changed? check stat(2) on debug.c, compare it
# to debug.o, and compare the last modification time of both files (e.g.,
# mtime).

Important that all computers that are used to build s/w w/ Makefiles have a
synchronized clock, else builds can fail, not compile stuff that needs to be
rebuilt, or rebuild stuff unnecessarily.  Often using synchronized
network-time servers (e.g., NTPD).

# stat(2) and file times in POSIX:
# 1. modification time (mtime): last time file's data was changed/written
# 2. access time (atime): last time file's data was read
# 3. inode change time (ctime): last time file's meta-data was changed (uid
# owner, gid, file permission modes, etc.)
# "ctime" is NOT the file's original creation time!

# Many users wish to have an actual, immutable, file creation time.  Not in
# POSIX.  But some OS's have added it to their kernel and stat(2)
# implementation.  For example, Mac OS X has added another timestamp to the
# inode called the "birth time" of the file.

Makefiles help you figure out what needs to be compiled, checking
timestamps, etc. and will issue only the minimum set of commands needed.

Makefiles also determine DEPENDENCIES and will issue commands in dependency
order.

Syntax of Makefiles:

# Makefiles are simple text based files, in a file called "Makefile" by
# default, but can be any name.  Historically, also "makefile" works.

# hash signs are comment delimiters

A makefile is a sequence of RULES, each rule looks like this:

target: dependency1 dep2 dep3 ...
	command1 args...
	command2 args..
	...

Target: the "thing" we want to satisfy (or build)

Dependencies: 0 (zero) or more rules that must be satisfied before this
target can be satisfied.

Commands: 0 (zero) or more commands and their options that if executed in
order, will satisfy this target.  Each command must be preceded by exactly
one TAB character (not 8 spaces)

Example:

main.o: main.c
	gcc -c main.c

Meaning: main.o depends on main.c.  If main.o is older than main.c, or
main.o doesn't even exist, then execute the command "gcc -c main.c"

# add rules for remaining files, eg

debug.o: debug.c
	gcc -c debug.c

etc.

# need rule to link all 4 .o files

myprog: main.o net.o debug.o user.o
	gcc -o myprog main.o net.o debug.o user.o

# meaning: if ANY of the 4 .o files are newer than myprog, or myprog doesn't
# exist, then run the "gcc -o ..." cmd.  But first, Makefile will check that
# each of the 4 dependencies THEMSELVES are satisfied, and if need be, will
# first execute any of the individual "gcc -c" rules.

# sometimes useful to list .h files as a dependency, esp. if a .h file is
# #include'd in a .c file

foo.o: foo.c common.h
	gcc -c foo.c

# the order of the rules in a Makefile does not matter!
# however, the first rule in a Makefile, is the "default" one that is often
# run, if nothing else specified.

all: myprog someotherprog

# other rules for individual programs and .o files, etc. in any order.

# variables can be set in Makefiles: customary for vars to be all uppercase
# there are some "common" variables that Makefiles use

#CC=llvm
CC=gcc
CFLAGS=-Wall -Werror -O2
LDFLAGS=-lssl

foo.o: foo.c
	$(CC) $(CFLAGS) -c foo.c

# Using Variables in Makefiles must use () or {}:
# e.g., $(CC) or ${CFLAGS} (NOT $CC or $CFLAGS, which will be just ${C})
# confusing: in shell scripts, $MYVAR is same as ${MYVAR}

# how Makefiles are executed:
# the program that processes Makefiles is called make(1)
# there are all sorts of variants of make: BSDmake, GNUmake (gmake), etc.

# execute default (first) rule
# will look for "Makefile" in current dir, else "makefile"
$ make

# tell make to use a different makefile
$ make -f myMakefile

# execute some other rule
$ make foo.o

# execute several other rules
$ make myprog install

# check what will be done w/o doing it
$ make -n

# turn on debugging (maybe -D)
$ make -d

# By default, make executes all commands in a single rule in order, then
# checks the exit code from the command.  A '0' means the program succeeded,
# and a non-zero exit code means the program failed.  If any command failed,
# make aborts running any more commands and exits itself with a non-zero
# status code.

# If any cmd failed to run (exited with a non-zero), don't abort running
# subsequent rules, but keep going.  Make will abort on the first error it
# gets, which it knows from the exit(3) code of the program

$ make -k

* Makefiles, cont.

1. command prefixes: @ and -

Prefix with a '@' to tell make to run cmd, but NOT show the cmd to execute
before running it.  By default, make will first show you the cmd it's about
to run, and then run it.

Prefix with a '-' to tell make to ignore errors from a single command.

foo: bar
	@echo I am running and installing program foo
	$(CC) ...
	-mkdir bindir
	cp ...

In above example "mkdir" will fail second time; just as "/bin/rm" will fail
if trying to delete a file that doesn't exist.  These commands by default
are not idempotent.  But with switches, you can turn these commands from
non-idempotent to idempotent ones:

$ mkdir -p bindir	# create dir if not exists, else don't complain
$ rm -f dummy.txt	# delete file if it exists, don't complain if not

2. implicit % rules, wildcard rules

many rules look like

foo.o: foo.c
	$(CC) $(CFLAGS) -c foo.c

cumbersome to repeat above rules many times.  Use implicit rules:

%: a wildcard that stands for any part of a file listed in the target or
 dependency (but not in the cmds to execute).  Useful in target or
 dependency.  A '%' in Makefiles is similar to a '*' in shell scripts.

Use built in variables in any one of the commands (and only in commands):

$@: variable stand-in for target
$<: first dependency
$?: all new deps
$^: all deps

# many make programs hard-code implicit rules for various programming langs
# Note: whatever matched '%' in the target, will be used as the dependency
# for that rule's evaluation.
%.o: %.c
	$(CC) -c $^

# a rule to compile a .c into a .o, but also depend on a common header file
#MYDEBUG=-DENABLE_EXTRA_TESTS
%.o: %.c common.h
	$(CC) $(CFLAGS) $(MYDEBUG) -c $<

myprog: foo.o main.o bar.o depend.o
	$(CC) $(LDFLAGS) -o $@ $^

Common Makefile variables

# name of C compiler
CC=gcc
# default compile flags (for rules that convert .c to .o)
CFLAGS=-O2 -Wall
# if debugging with GDB, turn off all -O<n> flags and enable -g (debug
# symbols)
CFLAGS=-g -Wall -Werror
# default linker flags (for rules that link several .o's into one binary)
LDFLAGS=-lwrap -lssl

# additional useful rules

# declare "clean" rule as a .PHONY,
# so you can run "make clean" to remove any regenerable file
.PHONEY: clean
clean:
	rm -f *.o myprog
	rm -f core \#* *~ *.bkp

or, in the old days, we create a "FORCE" rule

clean: FRC
	rm -f *.o myprog
	rm -f core \#* *~ *.bkp

FRC:

# rule to install binaries to "users" to use
# see special program called install(1)
install: myprog1 myprog2
	mkdir -p /usr/local/bin
	cp myprog1 /usr/local/bin
	chmod 755 /usr/local/bin/myprog1
	cp myprog2 /usr/local/bin
	chmod 755 /usr/local/bin/myprog2

# run tests
PROGS=myprog1 myprog2
TESTS=test1.sh test2.sh test3.py test4.sh
tests: $(PROGS) $(TESTS)
	sh test1.sh -x $(PROGS)
	sh test2.sh
	python test3.py
	sh test4.sh

tests2: $(PROGS) $(TESTS)
	for i in test*.sh; do sh $$i; done

# in modern systems, with large s/w bases, we rarely write Makefiles
# directly, instead we use higher level tools like GNU automake, cmake, etc.

# in large software systems, there's many folders, each with some source
# files.  Each folder has a Makefile to build stuff inside that folder, then
# there's a parent Makefile, and that Makefile's parent, etc. up to the
# top-level Makefile.  The top level will call make on sub-makefiles, going
# down the source tree.

# assume sources are broken into folders debug, tests, docs, and main
subdirs:
	(cd debug; make)
	(cd tests; make)
	(cd docs; make)
	(cd main; make)

# more efficient
SUBDIRS=debug tests docs main
subdirs:
	for d in $(SUBDIRS); do (cd $$d; make);done

You can even pass default make flags from the cmd line down to individual
invocations of make in a subdir.  Say you ran

# pass -n and define DEBUG=yes variable as if it was in the Makefile
$ make -n DEBUG=yes

Then "-n DEBUG=yes" becomes $(MAKEFLAGS) or $(MFLAGS).  Then the subdir rule
becomes:

SUBDIRS=debug tests docs main
subdirs:
	for d in $(SUBDIRS); do (cd $$d; make $(MAKEFLAGS));done

# anything you can run on the cmd line, you can run in a makefile command

foo: bar
	sh foo.sh
	python bar.py
	bash mytest.py

# parallel builds using -j (GNU make).  If you have many rules to execute,
# you can tell make to run them in parallel, up to some number of parallel
# threads, using -j N.  Make will parse your makefile, figure out which
# parts of the dependency tree can be executed in parallel, and issue them
# in separate threads.  Useful on multi-core systems and large programs.

# rule of thumb: use N parallel threads for make, equal to the number of
# cores (or hyper-threading cores) in your processor that are free.

$ make -j 4

# warning: this'll fork as many threads as make can parallelize, regardless
# how many cores your system has
$ make -j