hpr-knowledge-base/hpr_transcripts/hpr4587.txt

Episode: 4587
Title: UNIX Curio #1 - Shell Archives
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4587/hpr4587.mp3
Transcribed: 2026-07-31 16:14:33 (official HPR transcript)

---

This is Hacker Public Radio Episode 4587, for 2026-03-03
Today's show is entitled, "UNIX Curio #1 - Shell Archives"
The host is Vance and the duration is 00:12:08
The flag is Clean, and the license is CC-BY-SA
The summary is "Putting an archive into a self-extracting shell script"
Hi there, I'm Vance and welcome to Unix Curio No. 1.
This is the first entry in a series dedicated to exploring little known and occasionally
useful, trinkets lurking in the dusty corners of Unix Lake operating systems.
In case you want some background on this series, entry No. 0 gives a little information
on what you're supposed to be about.
You can check it out at Hacker Public Radio Episode 4-5-7-4.
To make the easier to find, all of these episodes will have the tag Unix Curio.
You could also pull up my host page for a listing.
Today's episode was inspired by an article I read on the Linux Journal website back
in 2009.
It described a custom-built bash shell script that would contain a tar archive.
When run, the script would extract the contents of the archive onto the user's system.
This to that article and other relevant references appear in the show notes for this episode.
When I encountered this article, I felt a wave of nostalgia and my thoughts immediately went
back to how I used it and email used to work in the olden days of the 1980s and early
1990s.
Back before 1993, there was not yet a mind-standard to govern how to attach files to messages,
so you had to put everything into the body of your message.
It was pretty easy to send a single text file as the message body, though it would be a
good idea to inform the recipient and advance that it was coming so that they knew what
to do with it.
But what if you wanted to send a bunch of files to someone and keep them all together?
Maybe they're all part of a single project.
Of course, tar existed, but the file format could differ between systems, so the recipient
might not be able to unpack the archive.
You could send each file in a separate email message, but then you had the risk of one or
more messages not being delivered, and even if they all made it, they wouldn't be bundled
together in one place.
Another common approach I remember seeing at the time was to put all the files into one
message with delimiting lines, often saying cut here, to designate where one file ends
and another one begins.
But this required the recipient to untangle things manually.
A task which could be tedious if you have several files.
The solution developed back then was a shell archive created by the char program.
That spelled SHAR or in the phonetic alphabet, Sierra Hotel Alfa Romeo.
This program wraps all your files in a neat shell script that the recipient can just run
and have the files magically pop out.
All they would need to have is the born shell and the said utility, both of which are
standard on any unix-like system.
The way it works is to run the SHAR command followed by the names of the files you want
to send.
If some were all of the files are to be contained in a sub-directory, you need to name the
directory first, then the files.
So the generated script will create the necessary directory before extracting the files to
be put inside it.
Typically the output of the SHAR command would be piped into a program for sending email
or posting use net articles, though it could be redirected into a file.
See the show notes for some usage examples.
For a recipient of your message, can then pipe the message body into the born shell with
the hyphen S as in Sierra option to save the files on their system with the original names
intact and the directory structure if any.
Different implementations of SHAR have varying capabilities.
For example, the BSD and Mac OS editions can only really manage plain text ASCII files.
Email back then and even today treats handling 8-bit data as an extension to the core
mail transfer protocol.
So trying to send binary files takes some additional work.
The BSD and Mac OS versions also require, as described earlier, that you name a directory
before naming any files inside it.
One typical way is to let the find command do the work for you.
It produces a list in the right order.
The GNU implementation is more flexible and can take just a directory name automatically
including everything underneath.
It can also handle binary files by using UU and code.
UU and code is a method that was developed around 1980 for encoding 8-bit data as ASCII text.
It takes arbitrary 8-bit binary data in chunks of three bytes and turns it into a form
consisting only of printable ASCII characters representing each chunk with four characters.
When the current base 64, mind-standard was developed and became widely supported, it effectively
made UU and code obsolete.
The GNU-shar rather nicely auto-detects whether the input file is text or binary and
acts accordingly and it can even compress files if asked.
In addition, instead of using UU and code for binary data, it can base 64 and code the data
as an alternative.
However, unpacking UU encoded or compressed files from such an archive requires the recipient
to have the corresponding utility to decode and or uncompressed the files.
While these are easily available on systems from the last 20 years or so, on older ones
that wasn't guaranteed, so the documentation has lots of warnings about using UU and code
or compression.
I should point out that my previous mention of BSD refers to the historical BSD 4.4 version
from the University of California.
Both free BSD and net BSD now have a library called Lib Archive that is used for reading
from and writing to various archive formats.
One of these formats is Sharr and the TAR command can be told to create a Sharr archive
on these systems.
This newer method is able to UU encode binary files and it can also have the resulting
shell script attempt to send the owner permissions and flags on the extracted files.
Lib Archive and TAR on these systems are not able to read Sharr files, but that's not
really necessary because they are designed to be interpreted by a shell anyway.
Open BSD does not seem to use this same library and instead it relies on the historical
BSD implementation of Sharr.
Looking at commercially unique systems, the version on Hewlett Packard's HP UX can also
UU encode binary files.
In addition, as a special bonus, it adds code to the script that will compile and use
a simple UU decode tool written in C so the recipient doesn't need to already have one,
but they would need to have a C compiler.
It will even handle device files and put the corresponding mate node commands into the script,
probably making it the most full-featured implementation of all.
IBM's AIX doesn't appear to come with Sharr.
Neither do Son OS and Solaris, both from Sun Microsystems, which seems quite odd since
original development of the program is credited to James Gosling.
If that name doesn't sound familiar to you, he had a long career at Sun during which he
did a few different things such as creating the Java programming language, no big deal
there.
Now that we are well into the 21st century, the standardization of archive file formats and
methods for handling email file attachments means that Sharr doesn't really have that much
of a practical use any longer.
However, the author of the Linux Journal article apparently had such a need and created his
own solution, probably unaware that Sharr already existed.
Next time you're considering rolling your own script for a particular purpose, consider
searching or asking around to see if someone else has tackled the same problem.
Even if it's not a perfect match to your needs, they might have learned lessons that
you can apply to your own approach.
I mentioned archive file formats being standardized.
A future Unix Curio will cover that subject, but it won't be coming until a few more
entries down the line.
There are a couple of other topics to deal with first.
So for now, let's say goodbye to Sharr.
Thanks for listening and keep an eye out for the next year in Xcurio, which will be about
F-Grip here on Hacker Public Radio.
If you have a question or a suggestion for a future topic, please comment on this episode
on the HPR website.
You have been listening to the Hacker Public Radio podcast, at hackerpublicradio.org.
Today's show was contributed by a HPR listener like yourself.
If you ever thought of recording a podcast, then visit the HPR site to find out how easy it really is.
Hosting for HPR has been kindly provided by anhonesthost.com, the Internet Archive, rsync.net, and the HPR Community Content Delivery Network.
Unless otherwise stated, today's show is released under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.