hpr-knowledge-base/hpr_transcripts/hpr3811.txt

Episode: 3811
Title: HPR3811: mkfifo and named pipes
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr3811/hpr3811.mp3
Transcribed: 2025-10-25 05:47:30

---

This is Hacker Public Radio Episode 3,811 from Monday the 13th of March 2023.
Today's show is entitled, Make Fifo and Named Pipes.
It is part of the series' bash scripting.
It is hosted by Klaatu and is about 11 minutes long.
It carries a clean flag.
The summary is, have you ever named a pipe?
If not, this is the episode you've been waiting for.
Hey everybody, this is Klaatu on Hacker Public Radio.
Today I wanted to talk about Make Fifo.
That is MkFiFo.
But in order to understand Make Fifo, you have to understand what a FIFO is or a FIFO.
Let's say, if I do a Man FIFO, first in, first out, special file, commonly called a Named Pipes.
You may have heard of a Named Pipes.
It usually appears in a bullet list of file types on Linux or something like that and
it's always sort of varied at the bottom because a lot of people making these lists don't
actually use the thing that they're listing.
So they're just like, you know, there's a file and like a directory.
And then this Named Pipe, yeah, that thing.
And they kind of leave it at that.
And to be honest, that's where it kind of kept, that's where it stayed for me for a long
time.
I didn't really understand what a Named Pip was.
Didn't know when I could use a Named Pip or why I would use it.
Still, I guess, kind of don't.
And I think if Named Pipes were taken away from me forever, I think I would survive.
I don't think I would complain that much.
But there are some minor conveniences that they do provide.
I guess I'll talk about some of those here.
But if you have ways that you use Named Pipes that you rely on or that you think are really
cool, I'd love to hear about it in comments because yeah, I'm still looking for like that
sort of definitive use case for a Named Pipe.
Well, before talking about what you can do with a Named Pip and how to do it, I should
talk really just quickly and briefly about what a pipe is.
So if you don't know what a pipe is, then for instance on Linux or in a Unix terminal,
you should be able, you can, you could say something like Echo Hello World.
And that returns the words Hello World to your terminal.
Well, one of the cool things about the terminal is that you can pipe the output of something,
like in this case, the phrase Hello World, you can pipe that as it were into another
command.
And so for instance, if I typed Rev Hello World, hit Return, that doesn't work.
Didn't know that didn't work.
How about if I do a, oh, none of those things work without a pipe?
Okay, well, what a great way to use a pipe in an example, then.
So Echo Hello World, and then the pipe symbol, which is like that vertical bar, and then
the word Rev, for instance, that echoes the word Hello World, the phrase Hello World.
Before it gets to you, before it gets to your terminal output, it goes through this imaginary pipe,
this vertical bar on your keyboard, and it gets processed by this other command Rev, which
reverses whatever it gets as its input.
So Echo Hello World sends that output through the pipe into the input of Rev, and then that,
the natural flow of that is to spit out output into your terminal, just like Echo would have,
if we hadn't put a pipe in front of it.
And so we get the word, we get the phrase Hello World except spelt backwards, D-L-R-O-W-O-L-L-E-H.
That's a very simple example, and there are lots of really cool examples of what you can do
with pipes, because again, its superpower is that it intercepts the output of one command,
and sends it to another command.
Not all commands know to look for input from a pipe, so you might have to do
some special arrangement, or some special syntax, often with a dash symbol to tell that command.
Hey, don't look, don't look in a file for your input, look from to standard in for your input.
So that's something that happens, but essentially a pipe takes something from standard out and
pipes it, or sends it to standard in.
That's what that does.
So knowing that, and that's an unnamed pipe, that's just a normal everyday pipe.
It has a symbol on the keyboard, it's usually, at least on the US keyboard, it's above the
backslash key, or it is on the backslash key, shift backslash gives you a pipe.
All right, now that we all understand what a pipe is, what's a named pipe?
Named pipe is similar, but it is a file object to your computer, but it essentially does the same
thing. It takes input from one, rather output from one place, and sends it, or kind of puts it
into a holding pattern, and then sends it to the input of something else on demand.
So for instance, you could do echo quote hello world, redirect, so that's the greater than symbol,
or the right, was it right? Yeah, right, angle bracket, the one that points to the right.
And then, oh wait, we don't have a named pipe yet, we first need to make a named pipe.
That's Mcfee though, mkfifo. We'll call it my pipe. Okay, so now if I do an ls in my current directory,
I see that there's a special file here, or a file here, that if you have, if you do an ls dash,
what is it, capital F, I just have it turned on all the time. I think it's capital F for classify.
It'll show you that this file is not like the other files. It's a file name, in this case,
my pipe, and then the pipe symbol at the end of it. So it's denoting that this is a special
file is specifically, well, you can find out what specifically it is. You do file my pipe.
It is a FIFO, FIFO named pipe. So if I, for instance, cat my pipe, nothing happens. In fact,
it steals my prompt. I can't get out of it, so control C, control C. All right, we're back.
So what can we do with this? Well, you can do an echo hello world redirect. That's the right
angle bracket into my pipe. And once again, it kind of steals your prompt. It just kind of hangs
there. Okay, well, that's fine. Open up another terminal window and do a cat of my pipe, wherever
my pipe happens to be. And obviously you need to know the location of that, which I was honestly
sure that I did know. There it is. Okay. So cat till the path to my pipe, I get hello world in this
new terminal, spit out into the output. I go back to the original terminal and I'm just back at a
prompt. So I have managed to echo hello world into another terminal on my system. That shouldn't
be possible. Echo hello world just sends a thing to standard output in, you know, in this terminal.
It shouldn't be able to appear in another terminal. And yet that's what my pipe has made possible.
So that's what a named pipe does. It takes your your data from standard in and holds it. And it
kind of just holds it there until something comes by and opens up that release valve and gets the
output at whatever location they're in. It's kind of cool, kind of fun. I mean, like I say,
it is a little bit weird to a little bit spectacular to be able to dump output into a file in one
location. So here's an LS BLK redirect my pipe. And then again, cat path to my pipe in a different
terminal. And I get the output of LS BLK in this other terminal. So that's kind of a neat trick
on its own. The other use case that I could think of was sort of as a replacement of temporary
files. Granted, you're still making a file, but you're only making potentially one file. So for
instance, let's let's imagine a really test a really simple test script here. So hash bang slash
bin slash SH echo quote hash tag or whatever they're called space hello world clothes quote
right angle bracket for redirection my pipe. And then the ampersand symbol so that we get we don't
get caught in in our pipe. And then let's do a pan doc space dash dash from markdown space to
dash dash to HTML space dash dash output test dot HTML left angle bracket my pipe. Okay, so we're
putting the word we're putting the phrase hello world pre-pended with a octo-thorpe into my pipe.
So now it's it's in my pipe. It's in a holding pattern. We use the ampersand symbol to sort of
get out of the pipe. And then we use pan doc, but we for the input of of pan doc, we are redirecting
the contents of my pipe into pan doc. So if I run that script SH space dot slash test dot SH
and then cat test dot HTML, I get h one ID equals hello dash world closed quote hello world
closed tag. So pan doc has successfully run and converted the text hello world to a heading
in in HTML, but there has not been what where did that HTML come from? There was never a there
was never a markdown file for pan doc to process. Well, there was there was the named pipe and it
contained all that data, the one line of data. And then pan doc was able to act on those contents.
That's a simple example in any one off example isn't going to seem probably all that useful,
but I could imagine a named pipe being a useful thing to use during a longer process. So for instance,
maybe you have, you know, some some parsing that you do every day or multiple times a day.
Well, rather than creating a temporary file every single time that cron job kicks off,
just make one named pipe and use that location as sort of your temporary data storage for the
process that runs all the time anyway. That's not to say that my pipe is super intelligent. Like
if you try to do something like echo hello world redirect into my pipe, I guess we'll do an
ampersand just to get our prompt back and then do for instance a cat of my pipe, which normally would
print hello world. So then I'll pipe that to rev that works as expected. So now I'm going to echo
hello world into my pipe again. And now I'm going to do a cat of my pipe pipe that into rev and then
redirect the output of that back into my pipe. And now I'm I'm trapped. I don't know where I am.
Well, I'm out of prompt, but I am not able to like I'm not getting anything out of my pipe
where I am, but I'm getting nothing out of my pipe. So don't try to take something out of
out of a named pipe and then put it right back into it that it doesn't work that way. I mean,
you can reuse the the named pipe, but you just you cannot you can't literally like cat the
contents of my pipe to your to to output, which you then pipe back into the input of my pipe that
it does not do what you would think it would do. Okay, I think that's everything I have to say
about named pipes. If you have something that you do with named pipes, record an episode about it.
I'd love to hear about what you're doing with them. And that's it. Thanks for listening.
You have been listening to Hacker Public Radio as Hacker Public Radio does work. Today's show was
contributed by a HBR listener like yourself. If you ever thought of recording podcast,
you click on our contribute link to find out how easy it really is. Hosting for HBR has been
kindly provided by an honesthost.com, the internet archive and our sings.net. On the Sadois
status, today's show is released on their creative comments, attribution, 4.0 International