hpr-knowledge-base/hpr_transcripts/hpr2090.txt

Episode: 2090
Title: HPR2090: A Docker Dialog
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr2090/hpr2090.mp3
Transcribed: 2025-10-18 14:12:32

---

This is HPR episode 2009 to entitled A Knocker Nialog.
It is hosted by Taj Sara and in about 33 minutes long.
The summary is Taj and I'll X1101 have a discussion about knocker and its use.
This episode of HPR is brought to you by AnanasThost.com.
Get 15% discount on all shared hosting with the offer code HPR15, that's HPR15.
Better web hosting that's honest and fair at AnanasThost.com.
Let's get back to our public radio.
This is Taj and I'm here with the new episode and I'm actually joined by my buddy and cohort
from the Uranda podcast X1101, better known as Lyle.
Howdy folks.
And we're going to kind of do two episodes together but this is going to be the first one.
I got a hold of X1101 because I wanted to talk about knocker and I know that that's
a huge thing that he deals with with this work now.
And so I was just looking for some answers and so I guess the best place to start would
be to go back and to the beginning and talk about why I wanted to talk about knocker
and just my new mission is regarding the whole thing.
So about three years ago, I think it was three years ago, the Google reader shut down
happened and that was the moment when I realized that I was really needed to be all in on this
free and open source train because I was being things were going the way I wanted them
to because the company owned it and I had no control over what was going on.
And the problem was when I looked at alternatives, all the alternatives I was capable of using
was a service that I had to go to somebody else to use and or pay for or just give somebody
else my data and the same thing could just happen again and that bothered me.
So I decided I'm going to learn how to self host things now prior to this.
I am a user very much everything's gooey just using the computer.
I was using Linux.
I was comfortable there.
I could use the command line, but I knew nothing about networking really didn't understand
how servers worked.
I was just like they're cool and I'm going to start.
So I did that I learned from basically reading, you know, wikis and websites on how to
build a server from scratch and what I did was I eventually put up several applications
on my server, got it to where I could get to wherever I wanted and it was wonderful.
Everything I wanted was exactly where it needed to be about a year ago.
I wanted to start playing with new self hosted things and I really didn't know how to add
any more to the server I already had without, you know, doing a patchy and engine ex at
the same time and it was just going to get to be a nightmare and then I started reading
about Docker containers.
And I guess for people who don't know what a Docker container is, a Docker container
is just kind of like a smaller little computer that you put in your big computer and it
does everything that the smaller the bigger computer would do, but it's self contained
and it's portable and it's really just a little ball of awesome.
And so I learned how to start a Docker container and write up on them and watch the videos
which are very helpful and learned about it.
What happened this past week, I went to update said server which had been running great
for three years because I was really behind.
It was on a Ubuntu base which is Ubuntu and I went to upgrade it and it totally destroyed
everything.
So my server basically fell to pieces.
I'm a big rolling release fan, especially on my normal computer.
So I decided that I was going to do rolling release on my server and to do this, I was
going to run rolling release and then do all my applications as Docker container.
And for the most part, I understood the basics of how to get these Docker containers running,
but some of the nuances for some of the applications that I wanted to work with.
I really didn't know what I was doing.
And so I've been on this path to try to figure that out and that's why I brought X1 101
here and we're going to talk about that tonight.
So what do you think about that?
Well, there is one thing you said that I want to kind of put a elaborate on because you
said something that's not wrong, but it's not all the way right.
The way you talked about Docker is you were talking about it the way as if it were virtualization,
but it runs on top of a virtualization stack, but that's not what it is.
You know, smaller computers inside your computer, that's standard virtualization.
We've had that for a decade and a half or something.
The Mware, Zen, Hyper-V, KVM, you name it and a virtual box, all of them.
And those are part of the Docker infrastructure, but not environment, but not really the
magic.
And then you really hit on the second part is they're self-contained, they're transportable,
they're kind of atomic, and one of the other nice things is they're layered.
So when you build a Docker machine, that machine most likely has dependencies way up the
stack until you get to the Linux base that it's on, and that actually has a bunch of dependencies.
So it's this layered, unison file system, where each layer contributes a little bit.
So you say you want the PHP Docker container, well that contains the Apache container,
which contains the Ubuntu container, which contains all this other Linux file system
containers, and a change in one of those, you rebuild and it propagates all the way down
stream without you having to do anything, and it's really cool.
But it's not virtualization so much as it's kind of this in-between space, between virtualization
and the classic container concept like charutes.
Yeah, and that was one of the big stumbling blocks for me is one of the first things that,
when I started learning about Docker, I went to what is essentially our local lug.
It's not really a lug, it's, they call it an open source society, or I forget exactly
how it's, it's how they termed themselves, but basically it's a lug.
Most of the people there are familiar with Linux, and they use it, a lot of them resisted
men's or developers.
The one thing that I got from talking to them, which was really at the beginning of Docker,
Docker, I kind of came in, I guess right at the ground floor when it was starting, and
the one thing I heard them say is, standard practices, you never keep your data inside
a container.
That has always scared me, and that was one of the things that I was super worried about
when I started doing this, because I know nothing about databases.
I know, I know how to put up a lamp stack, and I know that my SQL is in there, and I
can install it, but what it does from there, and how that interacts with different containers,
I had no idea.
That was the first place that I kind of got stuck, and I know that that is sort of changed
a little bit, or at least it seems that way, it seems like it's okay to keep your data
in a container, but not the same container as the container you're doing your stuff with,
or something like that.
Well, see, you're conflating your data with a database, a database is just a way you
access your data.
When they're talking about not keeping your data in the same plate in a container, what
they mean is the actual files that back it up.
You may have a MySQL Docker container, but the files that back up that MySQL database,
the NODB files, or I don't know how it's stored for the IASM databases, those are linked
into the container.
They are not stored in the container.
If you were to bundle up your app and ship it to me, you would have to either send me
those files separately, or I don't have your data.
It's really, and this is how I use it for work, too, is you're breaking the connection
between your application, your code, and your data.
Your Docker machines are a way to access your data.
They do not contain your data.
Now, as you said, that is changing where they'll have specific Docker machines as part of
a Docker cluster that those exist to store data.
Most of those are just connecting to a backend data source to prevent and not prevent, present
a more convenient interface.
Yeah, and the reason that I kind of got the impression that that was changing is because
a lot of what, when I started doing this, you would do, there's a dash v for volumes command,
which I was like, voodoo to me, I had no idea how it worked, and I was totally guilty
of the cardinal sin of, here's some code copy and paste it in and see what happens to
get some things running before, but it seems like that has kind of been replaced in most
of the containers that I saw.
And when I'm looking at containers, I'm not building these containers, these are containers
that I'm going to like Docker hub and pulling down, and I usually go with the trusted ones
that are made by the company, so like one of the things that we've put on my server that
we use is MatterMost, and so I go with MatterMost's official Docker container.
And one of the thing, the thing that keeps popping up is where, as far as the databases,
they are having you start a container and then using the link command to link it to a database
container.
And I don't know because what you were saying earlier is like the, the layers on top
of each other inside of container, it seems like it's almost like they're taking some
of the layers and moving them to their own containers, is that?
No, possibly.
You could run all of that in one as if you were running an entire LAMP stack in one container,
but when you get to the microservices world, that's kind of the wrong way to do it.
You want to put your web server, and if you really if your web server is different than
your backend application server, you separate those and you separate your database server,
you separate all of them into different Docker images.
And then you use, and this is where there's a perfect place to bring up Docker Compose.
And so you're talking about using, you know, the volume command and the link command, all
in this one nasty looking Docker command line.
And that's one way to do it.
But the more modern way and that Docker group is, I don't know how old it is, it's most
of the time that I've been working with Docker, which has really only been a few months,
but it's kind of every day thing.
So a few months adds up quickly, it's Docker Compose.
And so you put, you basically use this and it's all YAML, which is a, it's yet another
markup language, as its name suggests, it's literally what it stands for.
Yes, yes it is.
And it's all tools like Docker and Ansible and other Python adjacent tools use this kind
of same language.
But so you define a YAML file that says, okay, I want to start this Docker container and
I want to connect them in this way and I want to put these volumes on this one and I want
to set these environment variables.
So it's a way to take a standard container, adjust it to your needs and build an entire
application stack out of containers, all from a single, easily pass a rep passable file.
Now is a Docker Compose file, sort of analogous to, because the way you're explaining it and
then I'll be honest, you brought up Docker Compose and I went and did a little research
on it and it kind of was kind of a little bit over my head.
And so I, I was just like, I kind of need to get things running.
So I just went with the way I understood how to do it.
But I kind of do because Docker Compose seems to be the new hotness, I do want to get
there.
Yeah.
It is very much the hotness.
To me, that sounds like a Docker file, but no string a bunch of them together.
It references Docker files.
So this is, okay, think about a Docker file defines an individual server for lack of
the better word or service is really what we're talking about, it defines a service.
The Docker Compose file defines all of the services you need to bundle an application
and how they're connected and what goes in with them.
So you define your Apache container and the volumes it needs and the ports it needs and
the environment variables it needs and you define your database container and all of
those same things with it and you talk about how they're linked because you could actually
have a Docker Compose file with, you know, a hundred Docker machines in it, but if you
don't tell them to connect to each other, you've got a hundred different machines, not
a hundred machines that are interconnected.
So you said this a couple times that you would use like Docker Compose to create an application.
Did you use Docker Compose to do multiple applications that share the same kind of structure
or like connected in some way?
Let's stick with your matter most example and talk through it a little bit.
So let me get in my head exactly how the matter most stack works is that it's got, is
it servia Apache or engine X or something or is it a standalone application?
It's as far as I know it's standalone and matter most is actually one of the interesting
ones, even though I used to just my example for the link command.
It's actually one that doesn't have the link command.
It has everything in itself, like you just run the matter most machine or the container
and it just everything spins up.
So I'm assuming inside that container, there is either Apache or engine X, you know,
and what I do is I just use the dash P flag and assign that to whatever port I wanted
to go to and in this case, matter most has a standard port and so that was open.
So I just use that one.
But as far as what's in that container without going into the container, I actually don't
know.
Okay.
So if you were doing that with Docker compose, what you would do is you would have your,
well, assume that it's using either Apache or engine X, some kind of web server.
So you'd have your web server container and in that container, it would have the configuration
to install and stand up matter most.
That would all be in one container and then it would have your database container, not
your file system stuff, but your actual database in another container and it would link those
two containers together.
And then probably your database container, if you wanted to be able to persist it between
times bringing it up, which you kind of do, that would be where you would link in storage
so that your MySQL files are written to a persistent storage on disk and connected to the database
whenever you start those containers.
So that would be how you'd re-architect the existing thing to use compose.
So when I'm pulling down like a prepackaged container, there's a pretty good possibility
that that's what they've done on their end to create it.
No, when you're pulling down a container, it is a container because you're pulling down
a Docker file, not a Docker compose file.
So okay.
So the difference between those two things is a container is literally what it is.
It's a self-contained one thing, spin that up, everything runs.
It may have multiple things installed within inside that container, but it is just one container
to where Docker compose is, I'm going to string these containers together to create something
bigger.
Is that Am I on the right track?
Yes.
And so let me give an example of the way, if we use it, I'm a DevOps engineer at a development
shop.
The way we use it is, so we've got four, five, seven developers working on one project.
And that project needs a web server and a database and a search engine server and like
a utility server.
So that's four machines.
So what you do is you'll determine, so we'll go get the Apache Docker file and the database
Docker file and the search engine Docker file and then kind of like a generic Ubuntu
Docker file.
And then we'll write a Docker compose file that connects all of them together and we'll
put all of that into a Git repo and pass it around.
So you run Docker compose up and it spins, it pulls down your Apache server and your
database server and your search server and your utility server builds those individual containers
and then links them together in the way that you've told them to build an entire application,
a delivered application, not a Apache is an application, but I'm talking about an entire
completed application ready for someone to use.
Yeah, I am kind of jealous whenever I see whenever I was researching Docker compose and
it was just like, you know, Docker compose up was the command that they were using.
I was when I look at my commands to start my stuff, it's like a mile long and I was
very jealous.
Well, mostly what you've done is you've taken that command that looks like it's trying
to summon a great old one, put it into putting it into a YAML file and then just starting
the YAML file.
So and this is just me not knowing what is the benefit of doing that as opposed to running
it all in one container.
Well, it's it's more closely aligned to the microservices concept where you build an
individual machine for a purpose so that your web server and your database server are distinct.
That allows you to and when you get into larger deployments, you're thinking about things
like, well, I need four web servers and three database servers and two search servers.
And it gets harder to scale if you've got all of that bundled to one machine versus if
you got each service spun out on its own machine.
Okay, that makes sense because like the first half of that explanation, I heard, oh, the
Linux philosophy, one tool for one job and then just string them together with what to
do what you want to do.
And then the second half makes sense, it's it's it may be a scale thing.
If I'm trying to do something much larger, it would be impractical to do it in one container
that.
Okay, that makes sense.
So the TLDR, the Unix philosophy because scale, dig it.
So have I kind of given you answers to your questions and pointed you in some better directions?
Yeah, I mean, I definitely, one of the things that that just boggled my mind when I started
doing this is a lot and even there's tons of really cool stuff that I guess whoever
is in charge of Docker, I think you said maybe the Docker foundation or whoever that is,
they put out tons of tutorials.
There's there's actually a live site where you can go in and start playing with it where
you're like, okay, this is how you spin up a container and it shows you and I learned
everything pretty much from that and from their YouTube videos.
But the biggest problem I had is I'm more than a noob but less than a cis admin and the
noob stuff they've got covered really well and the cis admin stuff they've got covered
really well.
It's that weird in between part where it's a lot of it's terminology, even in some of
the tutorials, they're using cis admin terminology that I just don't understand.
I'm just like, I don't, I don't understand what you're saying and I'll go look it up and
I'm like, it's using more cis admin terminology that I don't know.
And so it's just sort of the snowball where the concepts kind of get too big to rip your
head around.
And so yeah, like this conversation has helped a lot because like we were talking about,
you know, the biggest thing in the world that threw me was that dash or the the flag for
volumes and like, what, what hell does that mean?
I understood that I could reach out into the base operating system and pull files and
stuff.
But the way you've explained it kind of helps me a little bit more with that and why I
would use that.
And the fact that actually now I'm not using that at all in any of the containers that
I did because I could take the data from the thing and export it out and it's basically
a backup for me.
Well, and I don't know specifically with the volumes command, but I know another option
for storing your data and you would set this up in your containers would be to actually
back end everything through a service that you don't host yourself or when you do.
In my case, we're very much into the AWS world.
And so if we want to store files somewhere to persist them and not persist them on the
server so that we can scale better, you throw them in an S3 bucket.
Yeah.
And I can see that.
And I think a lot of to me, I get in this loop where I'm worried that I know enough
to get myself in trouble and to do something really stupid.
And to me, with all the stuff running everything in one container, kind of scared me because
of all the warnings I heard about the data and you know, separating out the idea of the
data and the databases important, I mean, I didn't until tonight, I didn't really understand
that difference and that's good.
But just remember the Unix philosophy and then everything is a file.
So you may have a database, but that is just software to organize and access your data.
The data still has to be written on disk somewhere, which means it has to be in a file.
Right.
And that makes sense because I'm looking at, I just installed Next Cloud.
That was, I had own cloud and I love own cloud, but Next Cloud apparently is the new
hotness, just like Tucker compose.
And so I loaded Next Cloud and it has a database that it hasn't internally, but the data
it's pulling is off of the rotating drive and I have to set that up.
And that's part of the commands in that big long string that you were saying it's like
it's looking the old ones and that data is still on the disk and so that I didn't understand
that, but now I do.
Like that makes sense that I'm doing that.
And I just want to put a disclaimer out there that I, as I said, I have not been doing
that this long.
It's just, you know, looking at it every day.
So if somebody thinks they know better than me, either do a follow up show and yell at
me or, you know, I'm on IRC most days, you can find me lurking around.
Come yell at me.
Tell me I'm wrong.
I'm happy to have a discussion with somebody.
I think one of the things that has become clear to me is that Docker is kind of different
things from different people where some of the context you're using it in are different.
I'm kind of using it, I mean, boiling down to brass tags, I'm using it as an application
store and I'm just downloading applications and kind of using it as like a universal application
installer and just being able to do that that has way more options than a normal app install
would.
And that's a little different than building an entire application from the bottom up.
But yet this is capable of doing both and that's interesting.
And I think what got me off track is maybe I was listening to people in sort of a different
context and trying, they were trying to show me things and even like Docker compose,
that may not be something that I'm using yet, but it's probably something I will once
I understand where I'm going.
Like my next thing I want to learn how to do a Docker is let's, I want to start learning
how to build these containers myself instead of relying on somebody else to do it because
then I understand what's going on.
One of the problems I don't like about it is it's kind of a black box right now for me.
Well, and you know, even if you're building it yourself, you would only build the top
layer.
You know, this is kind of the perfect invocation of the standing on the shoulders of
the giant's analogy because you literally every layer is built on the layer above it
or below it, whichever direction you want, and so you keep building.
So you only customize the last layer.
You say, I need, so you're building a web server.
You would say, I need the Apache Docker file.
And then manually put in, okay, and then I need PHP and I need SSL and whatever.
And that's why it works both as an application store because, you know, you've got your
application next cloud, matter most, whatever.
You can say, I need this thing and I need that thing and I need this other thing to run
and it pulls all those in.
And then all you have to do is put your secret sauce on the top or your, your sauce on
the top layer.
And then I as, you know, someone who is building those applications can do the same thing
and then put my sauce on the top layer.
So we're leveraging the same framework to achieve different kinds of goals.
Yeah, it's just, it's one of those tools that to me seems amazingly powerful.
And I guess some people could argue that really if you were a package maintainer, you're
kind of doing the same thing, even a traditional application thing.
But the fact that I can do all this and hand it, you know, I can do it on arch and I can
hand it to somebody who runs a boom to and they can run it just right off the top.
That's to me, that's super powerful.
Well, and you can put your work in a get repo and publish it on GitHub.
And all I have to do is get clone your repo and rock and roll because you're not putting
the data there.
Most of that data is either in a public or private Docker hub.
And it's funny.
You say that because the man or most container that I pulled down, which is the official
man or most container, there was a problem and they knew there was a problem.
And so I went to the GitHub and I put in an issue and literally the guy was back with
me in a couple hours and he was like, do a get pull on this and run it and see if it
in a fixed it.
And so you get that benefit of having sort of that open source ethos going on.
And if you come at it again from the larger scale application development framework, if
you're just using Docker, you can move the same Docker instance from one developer to
10 developers to a staging environment to a production environment in one of several
clouds with the latest releases of Docker.
It's still in beta right now.
You can now run native Docker without kind of the need of a virtualization shim in the
middle on OS X and Windows as well as on Linux.
So that means, you know, you could run Docker machines, AWS Google Cloud, RAC space, Azure.
Pretty much this at this point.
If they have a public cloud, you can probably run your Docker stuff in it.
So it gives you some portability there that's really cool.
So as cool as that is, what instantly just went through my mind when you said I could
do this on Windows was that terrible Windows computer that I have to use at work.
You mean I could pull down an arch Docker container and basically have art on my Windows
machine.
That sounds awesome.
You could already do that with virtual box.
What I'm telling you is you could pull down matter most or next cloud and run a local
next cloud server from your Windows machine.
And you could do that before because I've done it and it's ugly and it's terrible, but
you can do that.
You could do that before.
They've just made it succ less.
So they're going, it's completely cross platform as they can.
That's got to be crazy on the back end to be able to pull that off.
Well, OSX and Windows both have native virtualization.
So I think it's still just virtualizing Linux.
It just before it kind of depended on virtual box.
So what you would do is you would install the official Docker for Windows, really installed
Docker virtual box and get.
And this just rips the virtual box port part out and uses the native virtualization.
So I think it still virtualizes a Linux machine underneath.
Yeah, all this stuff is so cool to me.
I mean, this like I said, this is my goal is to get better at this because I think
it's especially from the server perspective, one of the things that I liked about the
Docker containers more than the traditional kind of install your apps, install your
labs lamp stack and then install your apps was when my operating system got completely
messed up, I had would have to go through and set all these things back up where now the
way it is.
And since I'm on a rolling release, I'm on an arch derivative.
If my operating system dies, I learned about a command, just it's called the export command
where you can just export your Docker containers.
And my goal is to get to set up this week is to basically set up a script that is a
cron job that just fires off this export for all my containers.
And it basically takes whatever state that container is in and puts it in a tarball and
just dump that down to my backup.
And then worst case scenario, I reload the operating system on my server, which would
take, you know, all of maybe an hour worst case scenario and then just fire off a script
and everything that I had is backup right where it was before that that kind of portability
is amazing.
And to have a little more insight on how that works and how to do that with things that
aren't already done is definitely where I want to go with it.
So I guess is there any other words of wisdom you wish to impart on me before I go on
my journey of Docker craziness?
It's cool.
Keep going.
That's what I feel.
And I will be like most things that I've in the free and open source realm, everyone
I've encountered and talked to and I've reached out to several people involved this week
just because I was I was in a little over my head and I was trying to do this because
I do the idea is something I wanted to do, but the knowledge to do that idea was a little
lacking.
Everybody was super friendly.
I never once did I have an encounter where somebody wasn't willing to bend over backwards
to help me out.
So I feel like it's a pretty good community too.
And so I'm looking forward to it, man, hopefully this will spur some people into trying out
some Docker containers because you can I play with them on my laptop all the time.
And like you said, I have just my own little instance of something running on my laptop.
So it's it's definitely something I want to play with and I think other people should
do too.
I want to take a minute and plug our other project.
I'm actually going to be one of my goals.
I don't know if it'll be this month next month, sometime this summer, I hope is taking
all of the stuff needed to build and host the you random site and get it in a couple of
Docker containers.
I can't wait for that.
It's going to be so awesome.
Here's again, the code lives and get then then the only thing we're pulling in is all
of the tools to build it in one machine and all the tools to host it in another and then
a pipeline to kind of move code between them.
I mean, just looking at the code that's already there, I'm not a coder.
I can read a little bit, but I mean, I can follow along enough to see what is happening,
especially with the site that's at Nikola, the engine, yeah, Nikola is freaking awesome.
And just seeing how everything is separated out and how very, how very unixie it is, I
think, how everything's just broken apart.
I think it's going to be cool when it's a Docker container because then I could literally
just spin up the whole flow on my laptop if I wanted to just to see what happens.
And that's the point and the idea would be, you know, you'd run it on your laptop.
I run it on my desktop, we'd run it on the server and it's the same stuff everywhere.
It makes redundancy really, really, really easy.
So that's all I've got.
That's all I've got on Docker.
All right.
Cool.
So like he was saying, if you're so inclined, you can check out myself and Lyle and
Pokey on our podcast, which is called you random, you random-podcast.info, correct?
Correct.
It's just three dudes getting together, talking about random stuff as you would probably
assume from the title.
It's a lot of fun.
So you should come check it out and everybody within the sound of my voice should do an
episode on hacker public radio because they need episodes and obviously all it takes
is to get to guys that kind of know some stuff to get on mumble and hit the record button
and publish.
It's not hard.
So you should do it.
Indeed.
And with that, I will say later, peace out.
You've been listening to hacker public radio at hackerpublicradio.org.
We are a community podcast network that releases shows every weekday, Monday through Friday.
Today's show, like all our shows, was contributed by an HBR listener like yourself.
If you ever thought of recording a podcast and click on our contributing to find out
how easy it really is, hacker public radio was founded by the digital dog pound and the
infonomicom computer club and is part of the binary revolution at binrev.com.
If you have comments on today's show, please email the host directly, leave a comment on
the website or record a follow-up episode yourself.
Unless otherwise stated, today's show is released on the creative comments, attribution,
share a like, 3.0 license.