I'm trying to adjust my Makefile for my GoLang project. I have several rules which should:
youtube-dlffmpegPreviously I was doing this via shellscript and checking for each file manually, but in the act of converting my script to a Makefile I seem to be missing something.
The only rule that doesn't re-run is the pre-hook, but I think that's because I'm not using a variable for my target/rule name?
.default: install
.phony: install generate clean
export bin_directory = bin
export asset_directory = data/assets
export song_url = https://www.youtube.com/watch?v=z8cgNLGnnK4
export song_file = ${bin_directory}/nsp-you-spin-me-cover.mp3
export loop_file = ${asset_directory}/spin-loop.mp3
install:
go install .
generate: $(loop_file)
go generate ./data
$(loop_file): $(song_file)
mkdir -p "${asset_directory}"
ffmpeg -i "${song_file}" -ss 00:01:13.30 -to 00:01:30.38 -c copy "${loop_file}"
$(song_file): .git/hooks/pre-commit
mkdir -p "${bin_directory}"
youtube-dl "${song_url}" --extract-audio --audio-format mp3 --exec "mv {} ${song_file}"
.git/hooks/pre-commit:
cp ./pre-commit .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
clean:
git clean -xdf
UPDATE: I've found this works correctly if I lump all the dependencies onto the generate rule like so (which seems like the wrong thing to do)
.default: install
.phony: install generate clean
bin_directory:=bin
asset_directory:=data/assets
song_url:=https://www.youtube.com/watch?v=z8cgNLGnnK4
song_file:=$(bin_directory)/nsp-you-spin-me-cover.mp3
loop_file:=$(asset_directory)/spin-loop.mp3
install:
go install .
.git/hooks/pre-commit:
cp ./pre-commit .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit
$(song_file):
mkdir -p "${bin_directory}"
youtube-dl "${song_url}" --extract-audio --audio-format mp3 --exec "mv {} ${song_file}"
$(loop_file):
mkdir -p "${asset_directory}"
ffmpeg -i "${song_file}" -ss 00:01:13.30 -to 00:01:30.38 -c copy "${loop_file}"
generate: .git/hooks/pre-commit $(song_file) $(loop_file)
go generate ./data
clean:
git clean -xdf
You don't specify but I assume you're running make generate when you see this behavior. It's always best when asking questions if you show the command you typed, the output you got (cut and paste and formatted properly) or at least sufficient of it to see the problem, and point out exactly which part of the output is unexpected AND what you expected to happen instead.
Assuming your environment and makefile are accurately described above, then the most obvious cause of the behavior you're seeing is that this command:
youtube-dl "${song_url}" --extract-audio --audio-format mp3 --exec "mv {} ${song_file}"
is not updating the timestamp on the output file $(song_file), so it always looks older than the targets that depend on it, so make always rebuilds it.
After you run your makefile, use ls -l on all the expected output files and see if their modification times are updated. If not you may need to add touch $@ to your rule after the youtube-dl command, to make sure it was updated:
$(song_file): .git/hooks/pre-commit
mkdir -p "${@D}"
youtube-dl "${song_url}" --extract-audio --audio-format mp3 --exec "mv {} $@"
touch $@
Also as mentioned in the comments above, and as described in the GNU make documentation, the special targets are .DEFAULT (uppercase) and .PHONY (uppercase), not .default and .phony. Those latter are simply targets you've defined, just like foo or whatever; they have no special meaning to GNU make. makefiles, like all UNIX/POSIX tools and languages, are case-sensitive so be careful to use the correct case for keywords as described in the GNU make manual (and for the targets you define in your makefile)
Removing the export won't change anything: GNU make allows a variable to be assigned and exported on the same line.
Both of your makefiles start with the line
.default: install
The intent seems to be to set the default target to install. That doesn't work because the target names are case-sensitive -- you wanted .DEFAULT, and also .PHONY. But by happenstance, because of the location of that line in the file, you get a similar effect: the default target is .default, and it has install as a prerequisite. The install target will therefore be checked on every run, and if no such file in fact exists (remember that it's not actually phony, either) then its recipe will run.
So if you run make without specifying a target then make will probably run the install recipe (and otherwise, it won't do anything visible to you). I can't find any reason to think that go install would run make recursively, so if in fact you see other rules being run then I conclude that you must be naming a target, and if the rule for generate is one of those being run then either that or .phony must be the target you're naming, because none of the others have generate for a prerequisite.
So think about what happens if you run make generate with the original makefile.
The generate target has $(loop_file) as a prerequisite, so whatever that expands to is checked.
Whatever it expands to, there is a rule for $(loop_target) with $(song_target) as a prerequisite, so whatever that expands to is checked.
There is a rule for $(song_target) with .git/hooks/pre-commit as a prerequisite, so it is checked, and finally
the rule for .git/hooks/pre-commit has no prerequisite, so no further target is checked. The recipe for .git/hooks/pre-commit will be run if and only if that file does not already exist. The rule was not running for you, so I guess the file existed.
The prerequisite for $(song_target) not being found out of date, the only reason for $(song_target) to be rebuilt is that it did not already exist. It appears that in that case your rule will in fact generate it, so well and good. But then,
its prerequisite having been found initially missing or out of date, the recipe for $(loop_file) will run, regardless of whether that target itself already exists. And that appears to be sensible, because the recipe uses $(song_file).
Finally, its prerequisite having been (re)built, the recipe for generate will run. This is sensible if go generate ./data will make use of $(loop_file) or $(song_file), but otherwise those should not be (ordinary) prerequisites of that target.
Thus, $(song_file) needing to be built causes all other targets directly or indirectly depending on it to be rebuilt, too, and that's the right thing to do.
Now let's think about your second makefile. I presume that you again run make generate, rather than just make without specifying a target. In this case,
The generate target has .git/hooks/pre-commit, $(song_file), and $(loop_file) as prerequisites, so all of these are checked. And note well that there is no guarantee about the order in which they are checked or updated. It is incorrect to rely on a particular order, and doing so will bite you in a tender place -- maybe not today, and maybe not tomorrow, but if you count on a specific order then your day will come.
$(loop_file) -- even if it already exists, it ought to be rebuilt if $(song_file) is.if any of its prerequisites was built, then generate will be, too. That doesn't appear quite right: I don't see why it needs to be rebuilt on account of .git/hooks/pre-commit, and if it does not depend directly on $(song_file) then it's messy and superfluous to name it as a prerequisite.
It's hard to be sure what hidden dependencies there may be, but it looks like your original makefile is pretty reasonable, and the behavior you describe is what you should expect. If you can, though, I'd look at pulling the git hook out from the bottom of the dependency stack, because it doesn't look like anything else actually depends on it.
But if you want different behavior, such as considering a target's prerequisites only if the target does not already exist, then make's model isn't really what you're after. You can still handle that particular desire by engaging make recursively, but in that case I don't see what's gained over your original shell script.