hpr-knowledge-base/hpr_transcripts/hpr4657.txt

Episode: 4657
Title: UNIX Curio #8 - Comparing Files
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4657/hpr4657.mp3
Transcribed: 2026-07-31 16:16:06 (official HPR transcript)

---

This is Hacker Public Radio Episode 4657, for 2026-06-09
Today's show is entitled, "UNIX Curio #8 - Comparing Files"
The host is Vance and the duration is 00:14:20
The flag is Clean, and the license is CC-BY-SA
The summary is "Finding out what's the diff, without using "diff""
Hi there, I'm Vance and welcome to Unix Curio number 8.
This series is dedicated to exploring little-known and occasionally useful, trinkets lurking
in the dusty corners of Unix-like operating systems.
Before getting into today's topic, let me start with an apology and corrections to previous
episodes.
In a recent conversation, a friend pointed out to me that the historical BSD versions were
not necessarily developed in numerical order.
Instead, the 2.x BSD versions were created to run on the PDP-11 while 3 BSD and 4.x
BSD ran on the Vax, at least until support was added to 4.x for other processors.
For a while, they were developed alongside each other.
So 2.9 BSD actually appeared after 4.2 BSD.
In addition, the net 2 release came after 4.3 BSD, but before 2.11 BSD.
Probably others like me have fallen into the trap of assuming larger version numbers
came later, but I should have paid closer attention to dates while researching.
In Unix-Curio 3, I implied that the base name utility first appeared in BSD as part
of version 2.9 in 1983.
While it actually was in 3 BSD from 1979, shortly after Bell Laboratories 7th edition
was in the net 2 release from 1991, not 4.4 BSD in 1993.
In Unix-Curio 7, about compression, I got the date right for the compact utility.
It was first developed for BSD in 1979.
However, the link in the show notes points to the source code file in 2.9 BSD when the
program appeared first in 3 BSD.
I will try to be more careful in the future about identifying where and when certain commands
originated.
Sorry again if you were misled.
Getting back to today's episode, it came about when I was clearing out some open browser
tabs and stumbled upon a topic I had forgotten investigating.
Most users of Unix-Light systems are probably familiar with the diff utility.
It is widely used with source code to compare two files and see what the differences are
between them.
Non-programmers, like me, also use it to examine what has changed in different versions
of scripts or configuration files.
Quite a few pieces of newer software can compare different versions of data and express changes
in a format either identical to or similar to diff output.
However, there are two other long-standing tools for this purpose that are far less known
and deserve in my view to be termed Unix-Curios.
The first of these is CMP in the phonetic alphabet spelled Charlie Mike Papa.
While diff is primarily intended to be used on text files and compares them line by line,
CMP compares files byte by byte.
In my experience, its main use is to see whether two binary files are in fact identical.
If they are, CMP outputs nothing and returns an exit status of zero.
Back when methods of transferring files were not as reliable as they are today, this was
the tool I would reach for sometimes.
For example, you could use it to confirm that the data on a CD-ROM you burned was the same
as the original.
If there is a difference between the files, CMP will return an exit status of one.
By default, it will also print the location, the byte and line number of the first differing
byte.
When used with the hyphen L option, that's Lima in the phonetic alphabet, it will print
the location and value of every byte that differs.
There is one exception to these.
If the files are the same, except that one is shorter than the other, it will print on
message to that effect.
The exit status will still be one in that case.
Using the hyphen S option, that's Sierra in the phonetic alphabet, with CMP, will cause
it to be totally silent and output nothing.
The exit status will indicate whether the files are the same, different, or if the exit
status is greater than one, that an error occurred.
This makes it useful for scripting.
For example, in case you wanted to confirm, that a file copied to another location arrived
fully intact.
It is worth noting that diff is also capable of comparing binary files.
However, it is not required by posics to report what is actually different or where the difference
is occur.
The same exit status as in CMP is returned.
Zero if the files are the same, one if they are different, or greater than one if an error
occurred.
While many implementations offer an option to suppress the output, this is not in the standard.
The most portable method would be to instead redirect output to slash dev slash null.
On my system, the diff utility is three times the size of CMP, so if you don't need
its extra capabilities, it is a less efficient way of doing the job.
The other Unix Curio for today is calm, spelled COMM, or in the phonetic alphabet, Charlie
Oscar Mike Mike, and this utility is also intended to compare two files.
This time to see what is common between them.
Ben Fallen briefly talked about it a few years ago in hacker public radio episode 3889.
Compared to the others, it has a much more specific use case.
The two files are expected to be text files that are already sorted.
What COM will do is print a tab separated list of all the lines appearing in either or both
files.
Lines only in the first file will appear in the first column.
Lines only in the second file will be in the second column, and lines in both files will
be in the third column.
Any combination of the options, hyphen 1, hyphen 2, and hyphen 3 can be used with COM to
suppress printing of the first, second, or third column respectively.
By the way, these options are the numerals, one, two, and three.
Using all three options at the same time is supported, but it results in no output, so that isn't
very useful.
Unlike the other utilities, the exit status of COM doesn't tell you anything about the
two files.
It will be zero if the program ran successfully, and greater than zero if it didn't.
I'm not sure if I have ever actually used COM for anything practical.
I find it's default output to be a bit difficult to meaningfully interpret, plus you need
to ensure the two files are already sorted.
It seems to be best suited to comparing lists, and one use case that Kenfall and mentioned
would be comparing two lists of files to see if any are missing.
The command COM, hyphen 3, list A, list B, would print files that only appear in list
A in the first column, and those only in list B in the second column.
This would let you ignore all the file names that appear in both, and focus on those
that were absent from one or the other.
If, on the other hand, you only wanted to see the file names that are on both lists,
COM, hyphen 1, 2, list A, list B, would give you that.
Some more frivolous potential uses also come to mind.
If for some reason the cat utility is broken on your system, you could use COM, list
A, slash dev, slash null, to print the file list A instead.
If you want to insert tab characters before every line of a file, but have an a version
to using said or all, then COM, slash dev, slash null, list A would output list A with
one tab before each line, and COM, list A, list A would insert two tabs.
A bit silly, but it would work.
The GNU implementation of COM even lets you choose something other than a tab to separate
the columns, so you could go wild with that.
According to the POSIX specifications for CMP and COM, one of the two file names given as
arguments, but not both can be a hyphen.
In which case, standard input will be used for that, quote, file, unquote, in the comparison.
So the results are undefined, if both arguments are the same, phyphospecial, character
special, or block special file.
Some implementations might not have these limitations, but you shouldn't rely on that
everywhere.
All three of these were developed quite early.
The CMP utility appeared in 1971's first edition Unix, while COM and Diff seemed to have
made their debut in fourth edition Unix from 1973.
I'm not 100% sure about that.
Diff might have come a little bit later, but still in the early 1970's.
The original versions might not have behaved exactly like their modern counterparts, and
newer implementations, especially of the Diff utility, have acquired additional options
and capabilities, but the basic operation of each has stayed the same.
The next time you need to compare files against each other, consider whether CMP or COM might
be appropriate before automatically reaching for Diff.
They all have their uses in different situations.
Thanks for listening.
I haven't yet written the next Unix Curio, but there are a couple topics I'm working on.
Keep an eye out here on Hacker Public Radio for them to appear.
If you have an idea for a topic, or want to ask a question, please comment on this episode
on the HPR website.
You have been listening to the Hacker Public Radio podcast, at hackerpublicradio.org.
Today's show was contributed by a HPR listener like yourself.
If you ever thought of recording a podcast, then visit the HPR site to find out how easy it really is.
Hosting for HPR has been kindly provided by anhonesthost.com, the Internet Archive, rsync.net, and the HPR Community Content Delivery Network.
Unless otherwise stated, today's show is released under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.