Update metadata and transcripts through end of July 2026
Refreshed episodes/hosts/comments/series from hpr.sql, and added official HPR transcripts for the 180 episodes aired since the last sync (hpr4516-hpr4695).
This commit is contained in:
Lee Hanken
@@ -0,0 +1,151 @@
|
||||
Episode: 4557
|
||||
Title: Why I prefer tar to zip
|
||||
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4557/hpr4557.mp3
|
||||
Transcribed: 2026-07-31 16:13:53 (official HPR transcript)
|
||||
|
||||
---
|
||||
|
||||
This is Hacker Public Radio Episode 4557, for 2026-01-20
|
||||
Today's show is entitled, "Why I prefer tar to zip"
|
||||
The host is Klaatu and the duration is 00:32:52
|
||||
The flag is Clean, and the license is CC-BY-SA
|
||||
The summary is "The ways tar is better than zip as a command and archive format"
|
||||
Hey, everybody, this is Clat 2. I'm gonna talk about archiving specifically with tar and even more specifically why I prefer tar over zip.
|
||||
I think it's fair to say at this point that especially in the world of open source, we are spoiled for choice when it comes to archiving files.
|
||||
On my Slackware system alone, I've got tar, zip, gzip, bzip to x, z, a, r, zoo, packs, and 7 zip, p7 zip, whatever it's called, and probably more that I've forgotten about.
|
||||
Just as obvious as that statement has been that we're spoiled for choice, I think it's also pretty obvious that out there in the real world, zip has gained ubiquity.
|
||||
Everyone's got zip, it's the one that you can count on to archive and extract data on nearly any system.
|
||||
I'm talking Linux, Unix, Freedos, Android, Windows, Mac OS, it's everywhere.
|
||||
For a very long time now, I have defaulted it to tar over zip and there are several reasons for that.
|
||||
One is just sort of habit. I mean, tar is, when you're learning Linux, tar is one of those commands that you learn pretty early on. I don't exactly know why, but at least when I was learning Linux 20 years ago now, if you can believe that, it was, it was one of those introductory lessons that you just for some reason got taught real of frequently and early.
|
||||
So I've been using tar for a long time, and I do believe it's the superior choice out of all of these other choices, and I'm going to tell you why that is in this episode, but first let's just kind of set the, set the foundation.
|
||||
So first of all, each archiving format has an associated command. So there's the tar command for a tar archive, there's zip for a zip, there's G zip for a dot G z file and gunsip to un to un to extract the data from that there's x z for and dot x z file and so on that's just a note because it's kind of, it's important to remember that the command and the format.
|
||||
are two different things. It's a little bit of an esoteric differentiation, but it does matter a little bit, and you might see why later as we talk about what commands do and what actual file formats are able to do.
|
||||
So anyway, in terms of compression, they all tend to be honestly in my tests, and I did some pretty rigorous tests on my old podcast, the getting world order.
|
||||
I did actually do some pretty rigorous testing of compression algorithms, and in terms of compression, I found they basically broadly speaking, they're basically all the same.
|
||||
Now, you might save a few kilobytes or megabytes even with one compression algorithm given a specific combination of file types, but it's fair to say that very broadly, the the end result is basically the same.
|
||||
You're going to get a file object that contains lots of files and the size is going to be smaller than what you put into it, usually again, there are a lot of variables here.
|
||||
So if you're if you're compressing and already compressed file, then you're not going to see a lot of compression because there's just not that much there that's to be compressed.
|
||||
Whereas if you do if you compress something that has not been compressed, then you tend to see larger larger differences in file size, but between two algorithms, like the G-Zip and A-Zip, you're going to look at the if you do an L-S-L and you look at the factual file size of the end of the zipped object.
|
||||
You're going to be within you know kilobytes or within megabytes depending on what file sort of category we're in here, but you're not going to it's going to be very rare, I think, for you to try a tar G-Z and discover that you've you've gotten something down to let's say 10 megabytes and then you use zip and it's like 50 megabytes like that kind of order of magnitude delta is not something that I tend to see.
|
||||
So colloquially, casually, we could simply say it doesn't really matter what compression algorithm you use, they're they're all basically that they're all modern algorithms, they've been updated, they're they're doing the best that they can and they're basically all the same, but now let's talk about how they're not the same because in fact, the commands are quite different, so the end result might be basically the same casually speaking the same.
|
||||
But the commands that you use to get to the end result are drastically different, but wait, I hear you say I've used both tar and zip and they're basically the same as well and you know what you're right at first glance tar and zip are pretty darn similar in capability, so by default the tar command generates an archive that's not compressed it's just a single file object that contains smaller file objects within it.
|
||||
The result, the the tar file archive dot tar or whatever is going to be the same as the sum of its parts and you can try this out with both tar and zip it's an interesting experiment not because it's just super obvious and true, but but there is an edge case here that that I might as well mention so let's do a dd BS equals one, so this is bite size of one count equals let's do one at 1024
|
||||
in file i f equals slash dev slash u random out file o f equals file dot txt okay, so we've just created a one kiloby file and if you do an l l you see file dot txt exists and it's reading is one dot zero k that's it's perfect or I guess that was an l l h was er yeah l s dash l h that's what we've done and now let's create
|
||||
a larger file to the tune of one megabyte, so we'll do a dd BS equals one count equals one m in file equals dev u random out file equals let's do dot slash file one dot txt and we'll do an l s dash l h and there's file one dot txt is one megabyte and one file I guess I could should have called it file zero dot txt is one dot zero k let's tar that up tar
|
||||
dash dash create dash dash file archive dot tar let's call it and we'll include file dot txt and file one dot txt in our in our object in our archive okay and
|
||||
let's do the same thing really quick well no let's let's stick with tar for now so l s dash l h one dot one megabyte is what archive dot tar is so I mean
|
||||
really almost sort of exactly what you would have expected one megabyte one kilobite put them together add some some some kilobites of header in you know file header information and that's
|
||||
sort of thing and you get an archive dot tar that is one one plus one one point one megabytes pretty much what you'd expect because there was no compression involved in
|
||||
fact if anything where we're probably adding some bytes to these objects by combining them in some format that then needs magic cookie and kind of
|
||||
information about where to find the next file and so on now you can do basically the same thing with zip and zip to do a zero compression zip container you use the dash zero option so that's
|
||||
zip space dash zero and then the name of the file that you want to create which we'll say in this case is archive dot zip and then the names of the files that you want to include so again in this case we'll do file one dot txt and file dot txt and it tells me that it's
|
||||
included those two files with zero percent compression if I do an LS dash LH I see that archive dot zip does in fact exist and do exactly the same size as the archive dot txt one dot one megabyte again this is the kind of a
|
||||
demonstration of my of my initial point which is the end result is basically the same that that's the commands are really similar on at first glance you mean they're practically the same
|
||||
target requires a few more options zip you just do zip and maybe an option of compression level and then the name of the archive and then the files but I mean they're really really similar and the
|
||||
end result is really really similar the end result is a little bit deceptive the way that I'm presenting it I'm doing an LS dash L for human readable so it's kind of rounding up around and down whatever and and so that's that that's
|
||||
giving us a very clean one dot one megabyte result for this demonstration if you do look a little bit closer LS dash L without the H you do get numbers that are a little bit different so you get one million 54,720 bytes for archive dot tar and one million 49,912 bytes for archive dot zip so
|
||||
tar is a little bit bigger than the zip in this case because of the header information I'm assuming so that there is a slight difference but as I've said
|
||||
basically casually we can say they're the same result and in both cases casual they're not it's it's that they're really really similar like both of them
|
||||
tar no compression one file object containing lots of little files zip zero compression can contain lots of files one one file with lots of little files in it but the most common use case of each command
|
||||
definitely includes compression so let's talk a little bit about compression and how that how each command handles that because that's where there what that's one place where the difference really comes into play
|
||||
so the balance in choosing either an algorithm a compression algorithm in the case of tar or a compression level in the case of zip is a trade off between compression speed and the end result the size of your file object in the end in theory
|
||||
the slower you let the command compress the smaller your resulting archive the faster the compression the bigger the archive so in
|
||||
other words the more work it has to do the slower it runs or you could say at the other way the faster it goes the less work it's going to do both commands
|
||||
strive to provide you with some control over this and we've already kind of mentioned the the zip control which is a dash zero for zero compression and you can go all the way up to dash nine for nine compressions or for you know maximum compression really more accurately it's not really nine compressions it's it's it's lots of compression like do lots of compression don't worry about speed just make it really good
|
||||
default level is dash six according to the man page so whatever you use zip just kind of on its own without specifying anything you're getting a dash six which you know if you think about it it's
|
||||
it's basically middle of the range but leaning toward being slower and smaller than faster and bigger which seems like a reasonable default now to add compression to the
|
||||
tar command you have a couple of options and it's to me this is already one way that tar is superior to zip for me because with tar you do have this choice zip you're doing you're doing exactly the same thing with this command every time you're you're creating a singular object a file object and applying compression to it with tar you do have a choice you can either
|
||||
use a separate command entirely to compress the resulting tar file so you can do the tar command and then take your tar file and run it through some compression or you can
|
||||
you can use one of several options to choose a compression algorithm to integrate into the tar command while you are creating your tar file so dash z or dash dash g zip uses g zip dash j or dash dash b zip to uses b zip to dash capital j or dash dash x z filters your archive through x z dash slash l zip does that dash capital z or dash dash compress
|
||||
filters it through the compress command dash dash z std filters the target through the your archive through z std and dash dash no dash auto dash compress prevents tar from using the archive suffix to determine the compression program so you can specify one or not yours yourself so you've got access to one two three four five six seven at least seven that I that I can think of seven compression algorithms
|
||||
as part of the tar command so that's pretty convenient I like that level of choice and I mean heck you could even not use one of those options create your tar file and then run that through zip you can zip a tar file like
|
||||
you you can have you can literally have the best of both worlds if you want I see this as a real bonus like the decoupling of
|
||||
the process of archiving from compression I think that's a real power feature that tar kind of just casually gives you without making a big deal about it but if you think about it having that that option
|
||||
like literally those options of seven options and the other that extra option of not using option and just using some you know just taking your tar file and running it through a compression
|
||||
program of your choice that gives you a lot of flexibility so in some cases you might make the determination of which algorithm what compression to use based on maybe the type of data you're compressing
|
||||
maybe you've done some tests and you found that this algorithm does text better and this one does I don't know music
|
||||
better or something like that I for the record I've not done those kinds of tests or I haven't made note of the results of those kinds of tests
|
||||
I think I did actually have a bunch of different file types in my tests but I never I wasn't monitoring like oh what how did how did this compression algorithm treat
|
||||
this specific file type so I don't know if that's even a thing but it might be you might try it or you might be limited by capabilities of your target system
|
||||
you maybe you know that your target system doesn't have XZ so you could choose GZIP instead or something like that or you might just want to test you know a really cool new algorithm
|
||||
compression algorithm that you've heard about or that you didn't need to try you can try you know you can you can do that all with tar because tar gives you the option to either opt into a specific compression algorithm or
|
||||
forgo it entirely and I guess technically in a weird way zip dash zero kind of gives you the flexibility to opt out
|
||||
but it doesn't it absolutely does not give you the flexibility of changing what compression algorithm you're using
|
||||
okay so I want to do another demo but I'm going to have to use different files for this because trying to compress stuff from you random is not really practical
|
||||
so I'm going to get rid of file zero no file one and file dot TXT and I'm going to generate well you know what I'm just going to search my logs
|
||||
for something that's suitably, suitably large yeah here's a 44 megabyte log so I'm going to I'm going to cat that log file into a new file called
|
||||
log dot TXT I guess so now I've got a 44 megabyte file here in my directory and I'm going to do a zip dash nine so that's maximum compression I'm going to call it archive dot zip and I'm just going to include log dot TXT and that took I mean that took noticeably longer to compress I mean it took like a moment but I mean I saw the moment okay so gotten that down to four dot four megabytes down from 44 megabytes
|
||||
it's four dot four megabytes so let's take a look at what TAR can do and again I've already said they're all basically the same right that's what I'm saying I might my premise and my my belief is
|
||||
that like my sincere belief is that casually speaking they're all the same so this is not like a let's see how much we can compress things compared to other things this is just a demonstration of flexibility
|
||||
so don't worry so much about the numbers worry more about the the way we got there so zip that's the one way we can get there right zip dash nine full compression of this file we've gotten it down to four point four megabytes
|
||||
which is really really good 44 megabytes to four dot four megabytes that's a huge improvement you could email one and you could not email the other I mean you know on most emails systems
|
||||
okay so let's do a tar dash dash creates dash dash jz dash dash file archive dot tar dot jz and will include obviously the log file because that's what we're
|
||||
compressing all right now if we do an LS dash LH on this folder we got the archive dot zip at four dot four megabytes and the archive dot tar dot jz at four dot six megabytes so I know I said they're basically the same
|
||||
right 4.6 megabytes 4.4 megabytes that's basically the same but what if what if there was a flex what if you had some flexibility here and we said well
|
||||
jz is okay but it's kind of old I've heard a lot of great stuff about this new x z compression and when I say new it's like you know 10 years old now but anyway
|
||||
let's try the x set one so we go tar dash dash create dash dash x z dash file archive dot x z and then log dot tx t you know that's taking a little bit longer and there it goes archive dot tar dot x z
|
||||
LS dash LH we've got our archive dot zip at four point four megabytes we've got archive dot tar dot jz at four dot six megabytes and look at this one archive dot tar dot x z three point three megabytes again
|
||||
basically the same it's only a difference of like a megabyte well one dot one megabyte compared to zip but I mean and that's that's not going to make her break something but the cool thing about what I just did really
|
||||
is that it's the same command I just used tar and a different option and got a very sort of a a noticeably improved result now obviously that's not going to always be the case we can't predict what everything that we try to compress is going to
|
||||
is going to be and and how well it's going to compare to some other algorithm but I think that's kind of my point is that with tar you have a bunch of different choices going in
|
||||
you can choose the algorithm that you want whether it's for the end result file size or whether it's because of compatibility with other systems or whatever it might be you do have that option whereas with zip you're always going to be using the zip compression that that's your one choice with zip
|
||||
there's no other choice there is no other algorithm to try or to fall back to or to opt into it's always zip all right let's talk about output manipulation because this is probably even a bigger deal for me so when you extract data from a tar or zip archive you can choose to either extract specific files one by one or to extract everything all at once and I think I will say casually it's most common to extract everything I mean that is certainly the default behavior on the major
|
||||
desk tops like genome I think macOS probably still does that I think maybe one does I'm not sure but it usually if you right click on a file and you tell it to extract or if you double click on it in some the desk tops the default behavior is just to extract the whole thing that that's a very very common kind of thing now with both the tar and unzip commands even when you choose to extract everything all at once you still have a choice of where to put the files you've extracted and that is the that's the
|
||||
default behavior on some desktop as well to prompt you where do you want to put the things that I'm about to to extract for you but with the commands you definitely have that option so by default both
|
||||
tar and unzip extract all files into the current directory if the archive itself contains a directory then that directory serves as kind of a container for the
|
||||
extracted files otherwise the files appear in your current directory this can get messy and it is a common sort of frustration we we've probably if you've I think
|
||||
probably everyone is at some point opened a an archive and ended up with a directory full of files that you didn't expect it to be full of um that is like I mean not not a directory
|
||||
but like you know in your current place you've suddenly all all the contents of that archive just go everywhere it's called a tar bomb
|
||||
I mean maybe a zip bomb if it's a zip file I don't know it's it's not uncommon it does happen it is frustrating and it's so frustrating to the point
|
||||
that the plasma desktop at least has an option when you extract an archive you can right click and choose to extract and auto
|
||||
auto auto detect the subjectry and if there's no subjectry if there's no directory sort of sheltering your files from just going everywhere it creates a
|
||||
directory with the name with the same name as the archive had before you extracted it so it's quite a useful tool I use it all the time when when doing this in the GUI
|
||||
but the command of course you you have you have control to to decide these things before before you actually do something
|
||||
the important thing here to note is that a tar bomb or probably a zip bomb it's not inherently bad it is a valid use case when you want to
|
||||
essentially I think of it as an overlay you're kind of overlaying maybe updated or additional files into an existing file system for example
|
||||
suppose you have a website consisting of a bunch of PHP files within lots of different directories you know you've got a config directory and a
|
||||
data directory and lots of different directories and and there's PHP files all over the place you could take a copy of your site
|
||||
maybe take it off the server copy it from the server put it on your development machine make some updates and then you create an archive of the files
|
||||
you've updated so now what you've got essentially is a you've got a patch you've got an archive that contains all of the things that need to change
|
||||
or that need to be changed on the server so if you extract you put in that archive back up to your web server you extract that archive in your web directory and each
|
||||
new version of any file that you've changed is extracted exactly where it originated from because both tar and unzip or rather tar and zip retain the file systems structure.
|
||||
I use this feature all the time when doing like a dot release update of a couple of different content management systems I mean I I administer on servers and it's a great trick.
|
||||
It's like it makes it updates really really easy both unzip and tar the commands provide an option to change directory before extraction.
|
||||
So you can store an archive in one directory but send to the extracted files to a different location here's how you do that you use the dash dash directory option with the tar command.
|
||||
So let's say you do you've decided that you want to send all the files into a folder called my tar.
|
||||
So you make directory my tar tar dash dash extract dash file archive dot tar dot xz dash dash directory path to my tar you do that and then you do you do a list of your path to my tar and sure enough there's file dot txt and pick dot jpeg and file dot php and whatever else you've got in there.
|
||||
Okay, so for unzip you do the same thing basically except it's just dash d there's no dash dash directory it's just shortcut dash d so you make sure my zip somewhere on your system you unzip archive dot zip dash d path to my zip and then all of the files that was that was in the zip archive file dot txt pick dot jpeg file dot php appear in my zip now the feature unzip does not have.
|
||||
is the ability to drop directories from the archive before extraction like excluding a directory clipping a directory off of the archive so for example suppose you want to extract files directly into my zip but you've been given an archive containing a leading directory called.
|
||||
Chaff so you could do unzip archive dot zip dash d to my zip and then you list my zip and there's a directory called chaff do a list of my zip slash chaff and there's your file dot txt and your pick dot jpeg and your file dot php that's not what you wanted right you don't want chaff that's not the director you need.
|
||||
Chaff was the directory that whoever sent you these files had to roll everything into in order to sort of cleanly zip them up.
|
||||
Well there's no option in unzip to skip it and the worst thing about this conundrum is that the unzip command is essentially encouraging this anti pattern.
|
||||
In order to avoid delivering a zip bomb to someone because you you don't want to do that you know how annoying is it bomb is so you very thoughtfully nest your files in a useless folder call it chaff but by nesting everything in this useless folder you've also prevented your user from extracting only the files they require to the location they want to extract to.
|
||||
The tar command solves this problem I think pretty elegantly you can protect your users from a tar bomb by nesting your files in a useless directory we can call it chaff because tar allows any user to skip over any number of leading directories.
|
||||
For instance tar dash dash extract dash dash strip dash components equals one dash dash file archive dot tar dot xz dash directory path to my tar this strips the components the first component of of whatever gets extracted it strips that off so while tar is extracting it it detects that there's this folder called chaff but you have passed the option dash dash strip dash components equals one.
|
||||
So it it drops chaff as if though it just didn't even exist and everything in a side of chaff gets extracted to your target directory.
|
||||
If if if chaff isn't the only useless directory maybe there's wheat slash chaff slash archive or no slash file dot txt log dot txt then you could dash dash strip dash components equals two and then you would be dropping the wheat you would be dropping the chaff and you would only be getting log dot txt to your target directory.
|
||||
Which is exactly what you want so that level of control for output is I mean, it's for me is priceless. I mean, that's the kind of thing where you can do a tar.
|
||||
Tvf whatever that stands for what is that something like list probably tar let's do man tar what is dash t list yeah dash dash list so tar dash dash list dash dash file archive dot tar that gives you the whole that give you the structure of everything inside of that tar that tar ball and if you see directories leading you know leading directories that you don't need then you can strip those components and extract the files where you want them to be.
|
||||
Right final final thing is permission and ownership this is another kind of big deal the zip file doesn't preserve file ownership the tar file format does it's kind of as simple as that I'll elaborate a little bit but I mean it is what it is you might not notice this when you use zip or tar just like sort of on your own personal systems because once a file is extracted from an archive you own the file you've just created that file object on your system you it just inherits the normal.
|
||||
ownership sort of policy that you have to find on your system but when using tar as a super user you might notice or with the option dash dash same dash owner.
|
||||
Then when you extract each file it gets the same ownership it had when archived assuming that the same user and group is available on the on the system where you're you're extracting it to so by default when you extract a file from a tar archive.
|
||||
It reassigned it assigns those files to you, but if you do if you extract something with from a tar archive as with pseudo or as root or with the option dash dash same dash owner then the original ownership policy the the the ownership of that file that you're extracting gets carried over onto the target system there is just no option for that with the unzip command because the zip file format itself.
|
||||
does not track ownership it just drops that information the zip command can preserve file permissions but again tar offers a lot more flexibility you have options like dash dash same dash permissions dash dash no dash same permissions dash dash mode where you can change things so you've got a lot more control.
|
||||
with for over permissions in the with the tar command and the tar file format I guess well no really the tar file the the command because the file formats both of them preserve file permissions the command the tar command gives you the ability to alter them so those are my sort of three big reasons why I keep.
|
||||
I it's not that I just default the tar or it's not that I just prefer it on principle there's it's just it's feature for feature there are some really good features in the the tar command and as it turns out within the tar file format that zip the the command zip unzip and the zip file format simply don't have zip won't the command will not let you change.
|
||||
your compression algorithm which could mean that you don't get as as good results or maybe that you don't have the same kind of flexibility depending on what system you're trying to target although let's be honest it's zip it's probably going to work on any system so.
|
||||
it's mostly I think about flexibility in terms of of the end result for that one unzip doesn't give you any control over the permission file permissions and it does not give you any permission over any flexibility or control over.
|
||||
I'm leading directories sort of the the way that you strip components you can't do that the tar command gives you that ability and then finally ownership zip doesn't do ownership tar does those are really good reasons to use tar.
|
||||
I think tar is clearly the the technically superior choice here whether it is the quote unquote correct choice for your use case depends I think entirely on your target audience.
|
||||
There's really no doubt that zip has greater support I mean like I said at the top zip is basically everywhere a lot of people if you told them that you were going to send them a tar file they wouldn't know what you were talking about and then once they got it they wouldn't know what to do with it.
|
||||
I mean trust me a lot of people are not going to know what to do with a tar file so zip is obviously the superior sort of like choice for you I guess universal compatibility but tar the tar format and the tar command are definitely the compression algorithm and archiving tool.
|
||||
of choice for me thank you very much for listening I'll talk to you next time.
|
||||
You have been listening to Hacker Public Radio at Hacker Public Radio.org.
|
||||
Today's show was contributed by a HPR listener like yourself.
|
||||
If you ever thought of recording podcast, click on our upload link
|
||||
to find out how easy it is.
|
||||
Hosting for HPR has been kindly provided by an AnHonestHost.com, the Internet Archive,
|
||||
rsync.net, and our mirror network.
|
||||
Unless otherwise stated, today's show is released under a Creative Commons
|
||||
Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.
|
||||
Reference in New Issue
Block a user