lee/hpr-knowledge-base
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update metadata and transcripts through end of July 2026

Browse Source 
Refreshed episodes/hosts/comments/series from hpr.sql, and added
official HPR transcripts for the 180 episodes aired since the last
sync (hpr4516-hpr4695).
This commit is contained in:
Lee Hanken
2026-07-31 16:18:57 +01:00
parent 37e8274bff
commit 087f80c44d
184 changed files with 61169 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
hpr_transcripts/hpr4617.txt
+159
View File
@@ -0,0 +1,159 @@
Episode: 4617
Title: UNIX Curio #4 - Archiving Files
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4617/hpr4617.mp3
Transcribed: 2026-07-31 16:15:13 (official HPR transcript)
---
This is Hacker Public Radio Episode 4617, for 2026-04-14
Today's show is entitled, "UNIX Curio #4 - Archiving Files"
The host is Vance and the duration is 00:15:57
The flag is Clean, and the license is CC-BY-SA
The summary is "Utilities and file formats, especially "pax""
Hi there, I'm Vance and welcome to Unix Curio number four.
This series is dedicated to exploring little known and occasionally useful, trinkets
lurking in the dusty corners of unix-like operating systems.
When you think about creating and managing archives on a unix system, tar is probably
the utility that comes to mind.
But this was not the first archiving program, R spelled AR, or in the phonetic alphabet,
Alfa Romeo, was in first edition Unix and CPIO also predates it sort of.
Coming to the NetBSD manual page, CPIO was developed within AT&T before tar, but did
not get widely released until system 3 Unix after tar was already well known from the
earlier release of 7th edition Unix, also known as version 7.
You might think that AR and CPIO are old and irrelevant these days, but these formats
do live on.
Each deviant package file is an AR archive which in turn contains two tar files.
On Red Hat, Fedora, Suza, and some other distributions, each RPM package file contains
a CPIO payload.
So these may very well be an use on your modern Linux system.
But let's get back to the subject of what you might want to use to create archives today.
The tar utility has persisted in its popularity over the decades and you most probably
have a version installed on your Unix-like systems.
One of the problems with tar, however, is that it has not kept a consistent file format.
Also differing implementations have used differing syntax at times.
There are excellent reasons for the file format changing.
The names people give files have gotten longer over time and the original 7th edition
tar format could only handle a total path name length of 100 bytes for each archive member.
In addition, file names were in ASCII format and Miron file systems now accommodate richer
encodings with characters that aren't in ASCII.
The size of each archive member was limited to 8 gigabytes.
Unthinkably large back then, but not so big these days.
User and group ownership could only be specified by numeric ID, which can vary from one
system to another.
Many other types of information simply couldn't be stored at all.
Lock and character device nodes, phyphos, sockets, extended attributes, access control lists,
and SELINIX context.
As a result, the tar format had to evolve over the years.
One important version was the U-Star format, spelled U-S-T-A-R, created for the 1988
physics standard.
The POSIX committee decided to try standardizing both the file format and syntax for
the tar command.
While the U-Star format addressed some shortcomings, progress marched on.
File systems started allowing file names in different character sets and let more types
of information be attached to files.
So for the 2001 revision of POSIX, they gave up on standardizing the tar utility and
came up with a new format and utility, which is our actual Unix Curio for this episode.
That utility is called PACS, spelled P-A-X, or in the phonetic alphabet, Papa Alpha X-ray.
Since the PACS program didn't have historical baggage, they could specify its options,
behavior, and file format, and be sure everyone's implementation would match.
Developers of different tar implementations had been reluctant to change away from their
historical options syntax to the standard.
The PACS utility was also an attempt to avoid taking sides between those who advocated for
tar and fans of CPIO.
The PACS file format was an extension of U-Star with the ability to add arbitrary new attributes
tied to each archive member as UTF8 Unicode.
Some of these attribute names were standardized, but implementors could also define their
own, making the format more future proof.
Older versions of tar that could handle the U-Star format should still be able to process
PACS archives, but might not know what to do with the extra attributes.
Genu tar developed its current archive format alongside the standardization of the U-Star
format.
The Genu format was based on an early draft, which later went incompatible changes, so
the two, unfortunately, are not interchangeable.
Unlike U-Star, the Genu format has no limits on the size of files or the length of their
names.
In addition to its own format, Genu tar is able to detect and correctly process both
U-Star and PACS archives.
In situations where its native format can't store necessary information about a file, such
as PACS' access control lists or extended attributes, Genu tar will automatically output
the PACS format instead.
In the documentation, this is called PACS written in lowercase.
However, it still uses the Genu format by default, though the documentation has been
threatening to move to the PACS format for at least 20 years.
The good news is that the U-Star PACS, Genu tar, and 7th edition tar formats are well documented,
and utilities across many Unix-like systems are able to handle these, depending on which
formats existed when the utility was developed.
While your system may not have PACS itself installed, there are other archiving utilities
that can read the file format, including Genu tar.
What amusingly, Debian and some other free software operating systems package a PACS utility
developed by MirBateBSD, which largely follows the POS-Expecified Interface, but it doesn't
support reading or writing archives in PACS format.
We should look at the manual page for the tar or CPIO utilities on your system to see if
they can handle PACS archives.
Perhaps one aspect that is worked in favor of tar and other Unix archive formats is that
they only concern themselves with storing files and make no attempts at compression.
Instead, it is common for a complete archive file to be compressed after a creation.
Many utilities can be told to do this step for you, but it is not typically the default
behavior.
Therefore, if a better compression method comes along, the archive format doesn't need
to change.
If you do use compression, be careful to choose a method that is available on the destination
system.
Compressing files is a big enough subject to deserve its own episode, so we won't talk
any more about it here.
So which format should you use when creating an archive?
Unfortunately, there is no single answer that applies in all circumstances.
The PACS format is supported among modern Unix-like systems and can represent all types
of files and metadata.
While other systems, their file systems and archive utilities might not be able to properly
make use of all of the metadata, they should at least be able to extract the data contained
in files.
And if Unicode is supported, give them appropriate file names.
If you intend to unpack the archive on an older system, more research might be needed to
figure out what formats it is able to handle.
The seventh edition tar format, often called V7, is widely supported, including by older
systems, but has limitations in what it can contain as described earlier.
Moving beyond the Unix world, things get a bit more complicated.
Apple's Mac OS with its free-BSD underpinnings easily handles tar files, however, when it
comes to MSDOS and Windows, things are a bit different.
There a multitude of archiving programs and formats arose, usually combining archiving with
compression.
PKZIP was probably the most popular of these, and its zip format became common in many
places, helped by the fact that PKWAR openly published the specification.
While there is only a single zip format, it has many options, some proprietary, and different
implementations have diverged in the way some aspects are handled, or not handled.
An ISO IEC standard for zip was published in 2015, giving a baseline profile, and sticking
to it produces files that can be widely extracted successfully.
Other format, file formats, like Open Document, use the zip format, and typically
hue to the standardized profile.
Windows File Explorer, starting with Windows XP, can natively extract zip files.
The Infosip program is a free software implementation for a wide variety of systems, even
rather obscure ones.
While it might not be installed on yours, if you are copying the archive file over,
you can probably copy over its unzip utility at the same time to unpack it.
So zip probably has the broadest support, although it might not already be present on
every system.
However, as Clientube points out, in Hacker Public Radio episode 4557, zip files and
applications handling them aren't always great at maintaining metadata about files.
The zip format doesn't seem to have any way to represent unique file permissions, and
user and group ownership can only be included as numeric IDs.
Other types of metadata on UnixLike systems are not saved at all.
This is probably not a problem in some cases, such as with a collection of photos, but for
others it might be a concern.
While Pax as a utility does not seem to have gained much popularity or support, expect
on commercial e-nix systems, where including it was required to conform to the POSIC standard,
its file format has persisted.
Free software systems have generally avoided the Pax interface, preferring to stick with the
tar utility on the command line, but usually have good support for archive files in the
Pax format.
Outside of UnixLike systems, zip seems to have become the most common file format, and
support for it is also good in the Unix world, though it might not be built in.
Thanks for listening, and keep an eye out for the next Unix Curio, which will be about
some things the Kill Command can do here on Hacker Public Radio.
If you have a question or a suggestion for a future topic, please comment on this episode
on the HPR website.
You have been listening to the Hacker Public Radio podcast, at hackerpublicradio.org.
Today's show was contributed by a HPR listener like yourself.
If you ever thought of recording a podcast, then visit the HPR site to find out how easy it really is.
Hosting for HPR has been kindly provided by anhonesthost.com, the Internet Archive, rsync.net, and the HPR Community Content Delivery Network.
Unless otherwise stated, today's show is released under a Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.