hpr-knowledge-base/hpr_transcripts/hpr1413.txt

Episode: 1413
Title: HPR1413: ohmroep hpr live 3, 01-08-2013, (Power)DNS
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr1413/hpr1413.mp3
Transcribed: 2025-10-18 01:07:17

---

music
Hello, welcome to Hacker Public Radio on Own Group on 104.7 FM.
It is 3 o'clock and with me today is my good friend and co-host, Ken Felon, and today we don't
have five guests but we have one but certainly not less than the last time. Our guest today
is Bert Hubert from PowerDNS.
Hello.
Hello, Bert, nice to meet you. Today I wanted to talk to you about DNS and for the people
who have no idea what I mean by the term. Could you explain for them what it is we're
going to talk about?
Sure. DNS is like it's been called the phone book of the internet. So every time you visit
the domain name, you visit the website, you enter a name because people like entering names
and don't like entering numbers or IP addresses. So DNS is the phone book that actually translates
these names into the IP addresses which saves a lot of remembering. If people ask do I actually
need DNS but you could not use the internet without it. So at most internet service providers
when they say our DNS is down for the average user that means the whole internet is down.
So DNS is, I think, but I would say that of course, I think it's one of the most important
parts of the internet. And what I try to, what I like to tell people is that we like
the sewer of the internet. As long as it works, you don't know it's there. And when it stops
working, you're in deep shit.
Okay. So the idea behind DNS, it's supported by all operating systems, even your mobile
phone or whatever. And you want to go, can you talk us through, I want to go to www.hacker
dust org. What happens on your computer?
Actually, an tremendous, a tremendous number of things happen. And when you hear it, it's
like, oh, it's amazing that it works at all. Because the internet, there is no central
place of the internet. Well, maybe, maybe it's here this week. But on the other side,
there is no central place of the internet. So there's no, not one big list that says, well,
here are all the domain names of the internet and here are all the IP addresses. That would
be a very interesting place to be if you had that list. So DNS is the domain name system,
but the D might as well stand for distributed. That means that it's all over the world. So
if you ask, let's say you want to go to www.om2013.org, your phone, for example, will ask
your internet service provider, do you have the IP address for me? And typically it will
say, yeah, I do hear this. But let's say it didn't already know that IP address. It would
have to start asking around the world to say, well, and let's start at a sort of central
place. It's called the root servers, the root of DNS. There are now hundreds of these root
servers and some of them in the Netherlands, some of them in the US, they're everywhere,
which is also interesting because if you own the root servers, then you own the internet.
But we'll probably get to that. And then your ISP will ask that root server, do you have
the IP address of Om2013.org? And the root server will say, no, I don't have that list.
But here is the org servers. Here are the IP addresses of the org. And you can ask over
there and they will actually go there and say, well, do you know the IP address of Om2013.org?
And the org servers will, again, say, no, we really don't know that. But we do know the IP
addresses of the Om name servers. Go ask there. And finally, the question gets repeated.
That's the IP address of www.Om2013.org. And you will finally get back the answer.
This can take 25 packets for one question. And that's the amazing part that it mostly
works. So on your own PC, you will get the DNS server that you talk to is the DNS server
normally supplied by your router, which is normally configured by your ISP. So it will
know your ISP is the DNS server. Yeah, there's a thing called a step resolver. And that's
a part of all operating systems. So it's also in printers and in phones and in webcams
and whatever. And this step resolver is a little library that doesn't really know a lot
about DNS. It just knows enough to ask a question of your internet service provider.
And internet service providers, they run this service for you and they call it a caching
resolver. So the story is your browser wants to know an IP address. It asks a step resolver.
And the step resolver asks your ISP. And the ISP starts talking to all the servers around
the world to actually figure it out. So it's the first server that you ask is responsible
for doing all the asking to the other servers. Yeah. And typically your own device, your
own phone will only be talking to your internet service provider. And the internet service provider
will on your behalf go ask around the world. So it will first presume it knows the root
servers because somebody's typed that in or that came with the operating system.
And that's that's called the hints. Yeah. And there used to be 12 root servers. And the
hints file was then used to ask any of these 12 do you know the current IP addresses of
the root servers. And they're responsible for all the top level domains. Yeah.
Just the dot com dot com dot orgs everything. So the root is really where everything starts.
So if you let's say you take dot and L out of the root, then dot and L disappears. Cases
to exist. Yeah. Okay. Interesting. Yeah. It's a powerful. I mean, the people that operate
these machines and they're actually not overly impressive machines. Like one unit high.
And you can just touch them. And it's just like feeling the sort of the core of the internet.
And then again, it's just just one one unit high machine. They both do them typically. Yeah. Okay.
Okay. Well, you mentioned earlier that in a in a bad case scenario, you can have like 25
packets going to and for from before you finally reach the answer. But for but for the regular
people, how much time is that in seconds if you're lucky? Yeah. Well, that's actually a very good
question. It turns out that that you can have 100 megabit or 100 gigabit internet connection. But
if your DNS is slow, your internet will feel slow. So it's it's it's really important how fast
your DNS is. And people find that when they change to a better name server. And we might get to
that because it takes quite a lot of work to build a really high performance good name server.
And I don't mean the software, but I just mean the taking care of it. But it typically a a
hit a cache hit is in a millisecond or something like that. It's just like ping. So you ask the
question, you get the answer. But if there is if the answer isn't there, it can can easily take 100,
200, 300 milliseconds. And during those 300 milliseconds, nothing else happens on your computer
because everyone is just waiting for that IP address. Oh, it really has a big, I think DNS right
now is because everyone has so much bandwidth. DNS is probably more the limiting factor in your
browsing speed than the actual bandwidth is. Yeah. But people tend to forget is you go to a page
like CNN.com and there are hundreds of lockups because the ads, there's redirections, just pictures
from somewhere else. Yeah. All these need to be resolved. Yeah. And actually the way it has worked
is people outsource part of their hosting to like Akamai, for example. Yeah. And during that
outsourcing, they also put a pointer in DNS that says, well, if you need this, I
yeah, we're still alive. If you if you need, if you want to show this ad, then also go through
the Akamai name servers, which also further delays things. So this is called a chain. Yeah.
A chain of lookups, see name chain. And some of these chains are now five steps deep. Yeah.
And also the I suppose we're hitting jump on the gun. The time to live on these will be relatively
short because you're you put into a name service and you want the lookups to be. So let's let's take
a brief step back because a DNS can be then take 200 300 milliseconds. We like to have caches.
We like to have them around. Your operating system has a cache. Your ISP has a cache. Your browser
typically also has its own cache. And these these things are used to to speed up the lookups.
But if you know that you have a rapidly changing IP address, then you don't want it to be cached
for too long. That's called the time to live. And in the time to live, you can say, well,
this answer is valid for a day, which means that that after one lookup, the next lookup
from that same server should come along only after 24 hours. But for some reason, people like to
set the time to live at five seconds, which would make sense if you change your IP address every three
seconds or something like that. But it's if you look it up, it's an it's an amazing amount of work.
Somebody actually sets us at five seconds. Oh, they do all the time. And and this is sometimes
it just a matter of not thinking. But there are also people that that by DNS based load balancers and
those load balancers, they just default to something. And that's stick with the default. So we had a
very big Dutch web shop, Wacomp. And Wacomp had their TTL at zero seconds. And you wonder why? But
people do. Oh, OK, OK. Sorry, I'm shocked and stunned here, shocked and stunned.
Well, we already went a bit into what's inside the DNS record. And it's the DNS system as
whole is something like a phone book. But what are the other other attributes you can derive from
a DNS entry? What's the information that's usually accompanied with it?
Well, I'll first give the usual answer. The usual answer is the other big thing that lives in
domain name system is where email should be delivered. So if you have an own 2013.org email address,
then there will be names. There will be a mail server associated with that domain. And that
then an MX record lookup in DNS. And that will tell the internet if you want to send email to these
folks, then you need to use this mail server. That's the other really big thing that's in there.
Another big thing that there is spam filtering information. So people can publish
or IP addresses like this. IP address has been known to send spam. And you can look that up. And
that also stored in DNS. So these are things that lots of people know about that are stored in DNS.
What is relatively little known is that any time you use your cell phone to sign on to the 3G
or a GSM or 4G or LTE network or whatever, there's a thing called an APN which says,
well, those are the details for your internet server provider. And that's on your SIM card.
And actually that APN sometimes looks a little bit like a domain name. And that's because it is.
So next to the internet use of DNS, all mobile phones in the world, whenever you sign them on
deep from down in the firmware, it does a DNS lookup to figure out the details on how it should connect
to the, well, to the left in the infrastructure. And these are like very tiny DNS, very small DNS
servers that each cell phone operator has to run one. And every time you turn on your phone,
there's a little DNS query to that. And you're talking just going to talk to us about power DNS.
Well, I could talk about anything. So the history is with my own history with DNS is power DNS.
We started out in 1998, I think, when there was really there were only two name servers in the world.
Buying, being one. Buying, and the other was the tiny DNS, DJB DNS. And at the time we wanted to
have a load balancing name server. Or you could say, well, we want to have the visitors from Europe,
go to a European IP address, we want to have the visitors from the US, go to US IP address,
and we want to solve that problem from DNS. And no one did that at the time. So then we,
and this is where the dot com days is when we were where you could start the company with a bad idea.
And you would still get money. And actually, our deal was not, not, when it was a bad idea,
because the world was not ready for commercial DNS, because we started out as a real dot com that
was not open source. And at the time, no one wanted, everyone used only open source DNS. So no
one was ready for that. But originally we started as a close source software company with the goal
of making this DNS based load balancer. And the interesting thing we never got around to actually
writing the DNS based load balancer. Sometimes you forget why you started something. And the DNS
based load balancing only arrived when the Wikipedia contributed that contributed that code to power
DNS and moved to it. So for a very long time, the Wikipedia used power DNS to distribute queries
around the world. And I think they still do. Okay. So power DNS is an open source name server.
And it can run. And that's that's the unique thing about it. I think that it can get data from
a database. Okay. And most name servers, if you have to edit the zone, you edit the text file.
And I personally, I like editing text files. I have no issues with that. But if you have a million
domain names, it becomes a bit unwieldy, a million text files. So we decided early on that we would
be doing DNS directly from mySQL, Postgresql or Cybase or whatever. And to this day, that's really
what we are best known for for doing that. And yeah, that's power DNS.
What when did it become open source somewhat licenses that under? We did that I think in 2001.
And we then went for the GPL version 2. Okay. And for reason, I don't quite remember at the time,
we decided to stick with version 2. Okay. And I sometimes feel a little bit bad about that because
GPL 3 probably has some advantages for us. But at the time, we thought, well, the standards
licensing theme that is in software programs is that this software is licensed under the GPL
or any new versions that the free software foundation releases of the GPL, which effectively
hands a blank check to the free software foundation to come up with anything we can call the
general public license. Yeah. So at the time, we thought, well, this Richard Stallman, well,
and then I admire him very much as a sort of a saint. But he's also a difficult person.
And everyone knows that. And so at the time, I was like, well, let's not hand him that blank
check to be able to change my licensing terms without me knowing it. Yeah. And Linus had also
met the same decision as well. Yeah. But I think, but afterwards, but I cannot quite recall why I
didn't like GPL 3. But the decision was taken before the GPL 3 was even on the horizon. Okay.
And who started the project? Who? That was actually that I was the technical guy. Yeah. And we had
an originals of launching customer, which is called V3 V3 redirection services. And they used to
be known for having these these nice URLs to go to and come to and browse to and surf to.
And people could use that because if they got some some web space from their service provider,
it would be would have a really nasty name. Yeah. And registering domain names was tricky. So they
you could with them, you could register come to and it would function like tiny URL.com
those today and redirect for you. And those were the people that actually wanted to have a
a load balancer solution because they were acquired by an American company. And so that that's
how we yeah, that was a joint joint effort of V3. And actually the original V3 guys are still
shareholders in power DNS. But they've been there. They're not they're not involved in writing
any actual code. So there's a company behind this as well. Yeah. There's a company called it's
called power DNS. Oh, that's very easy. So so well fully it's it's known as power DNS.com
beefy because that was the time when people when they registered their company, they actually called
it.com. Yeah. Okay. And and that company consists of very limited number of people and we're all
here. Okay. The entire company is that. So if an asteroid hits we're all screwed. Yeah. And
actually, we're switching back to there. There is a sort of issue with that that that we we
actually sell 24 seven high end support with 15 minutes response times, which means that we cannot
get drunk together. Okay. Or at least not so much that we cannot longer provide support. And
don't worry folks. I've all around the campsite. So fiber optic cables. So they're probably closer
to the backbone of the internet. No, no, we we yeah, we we are able to support people from here.
So we only had one support issue come up so far. Okay. This week. No, that's all. Yeah,
this week's forever. No, no, no, no, no, we actually get questions. And probably an interesting
question for people is you started as a total source company. You opened up the keys to the kingdom
and yet you can afford to live in lavish lifestyle with your Ferrari outside and stuff. How did
you make money from all of this stuff? There's not a Ferrari. It's a pushback. No, no, no, but
I think there's a very serious point in I think that the best open source is open source that's
able to fund itself so that the developers have lots of time to work on the actual project.
If you look at the statistics for the Linux kernel, for example, 80% of the Linux kernel is
written by by people that actually have a job, which means that they have to program on Linux.
Yeah. And I strongly feel that it's very difficult challenging to write open source in your
high quality open source in your spare time. Yeah. Because people, they have a job during
your day. They get home. They're tired. They've worked already. And then maybe they can squeeze
in two hours of development, which is only even only possible if they don't have a family or
other stuff that they want to talk to them. So I think that it's one of the, that's also my talk
for a Friday. We succeeded in living the open source dream. We are an open source company and
we live from our open source project and when we live quite well. And I think that that is a
model that other open source project should seriously look at because it allows you to devote
serious time to your open source project. Yeah. Okay. So what the code is open? What's
where does the cache come from? I want to thank Microsoft for this. Now actually, it works like this.
There was ingrained in people's memories, a model that software cost serious money. And then
on top of that serious money, there was a 15% maintenance fee to be paid. Yeah. That was the
industry standard. Yeah. Everyone knew that. And then in the open source world, software had always
been free. So the software was free. And then suddenly Microsoft started to give away
rather large pieces of software together with Windows. And they were able to, for example,
they at one point, they started giving away a media servers to companies doing serious broadcasting
on Windows 2008, I think. And okay, free digital broadcasting infrastructure came along with that.
But this is the interesting thing. The broadcasting organizations that worked with that said,
well, thank you that it's free. But frankly, we don't care that it's free because we will be basing
our business on it. We want you to guarantee that it works. We want to be able to talk to you.
And if there's an issue, we want you to feel responsible. So what happened? It is no longer
weird in the industry that software might be free. That doesn't change anything. If you look at the
big telecommunications service providers like Verizon and those kinds of places, they don't care
that the software is free. That doesn't matter at all. But they still want to pay for the guarantee
that someone will pick up the phone if it breaks. And it used to be weird for people to be paying
support for free software. But in very big companies, it's very normal these days.
Yeah, okay, cool. And you have, do you have any big companies that you can mention here that
might be using the power DNS or? No, well, let me call them. I'll give a list of five companies
and three of them are using PowerDee. Okay, so I can always deny that anyone saw that we were
French Telecom, Deutsche Telecom, British Telecom, Verizon and T-Mobile. Okay. And some of those are
actually using PowerDee. Okay, very good. And some of them are actually also paying for it.
That's that's also quite good. I myself worked for a company who used PowerDNS. And it actually
got a bad rap from a lot of people. And both when we when you investigate the reason why it was more
to do with the choice of database and my SQL database on the and replication issues with my
SQL database on the on the backend. And this was a long time ago, of course. What would you recommend
for somebody starting out what sort of database would be your personal favorite to use?
Yeah, I used to answer my SQL that question. And to get back to your story, if the whole,
if the PowerDNS installation has an issue, then then there is an issue. And we could of course
say, well, it's my SQL. But we need a database to function. So we do and we do know that my SQL
used to have issues with the replication. These days, I would always I would I think probably
recommend PostgreSQL for almost everything. But replication, I don't think anyone is really happy
with their replication setup. Why not? Well, if you ask people, you say, well, have you ever had
your database get out of sync? And everyone will say, yeah, yeah, that happened to me. And I had
a recent and it took ages. And so that's actually the so, for example, one setup iron is rather
unconventional. I use SQLite and I distribute my updates using R sync. Well, you use SQLite?
Yeah. And turns out SQLite is really fast. Yeah. And it's rock solid. And because it's just one
flat file, I can R sync it across. But is it not single thread that you're only allowed one
connection? No, no, no, no, no, you have to be sure you have to you have to think hard about
get not getting locked in your own. Yeah, that's what I mean. Yeah, yeah. But as long as you make
care, make sure that you don't lock yourself out of the house. Yeah, that's how to speak. Then
then there is no really no issue with it. So I actually, I actually, yeah, I like that one. But we
support my SQL, post SQL, site base, MongoDB and a whole array of other things, LDAP and stuff.
So I, we sort of as power DNS, we try to say, well, we support your database. And we don't,
we don't say, well, this is the main one we support. What sort of operating system is there?
Well, the usual one. So Linux, freebies, the openbies, the netbies, the Mac most of the time.
No windows. Now, we, we actually, that, that is a very interesting, I think, if it's a very
interesting story, every once in a while, we get the itch and we compile power DNS for Windows.
And it's about a week of work. So it's not, it's doable. Windows has grown enough
unique features that it's no longer that hard to do. And when we do that, an executable comes out
and we test it a little bit and we ask it three questions and it worked and we put it online.
It was around eight years ago. And there have been, I think, a hundred thousand downloads
of the Windows version, far more than ever have been of the Linux version.
And even though it doesn't actually work that executable, because it leaks memory like a CIF,
so you have to restart it every six hours or something like that, there are people having,
using it in full production. Because I think on Windows, people are used to software that sucks.
For that reason, and then we removed that executable, because people said, well, yeah,
you have it online, so you must be supporting it. We took away the executable and there is
now a script out there that tries to download the executable every 15 minutes just to see if it's
coming back. And that never seems so. At one point, I just briefly put the executable back to
get that script off of my server. But after a few weeks, it came back again.
The Windows version actually, no one really knows one does DNS on Windows.
No one that even really, really super duper Microsoft shops. And I'm not an anti-Microsoft guy,
but it's actually the case that even Microsoft doesn't do DNS on Windows.
So it's not something that we really work hard at.
But do you have any requirements underlying operating system requirements?
Yeah, we have a weird one. So power DNS is actually two products, at least. So it's the power
DNS authoritative server that you use to publish the phone book. And there's the power DNS
recursor, which is the one that that consults the phone book for you. So an ISP would run the
recursor for its customers. And the recursor made a choice to use some rather obscure
unique system calls, which almost no one uses. No one knows. They're called the set context,
get context, swap context, and make context. And they work very well these calls. But because no one
knows them, sometimes people forget to implement them. So for example, on the ARM CPU,
platform, the Linux people decide not to implement get context, which means that for a very
long time, it was not possible to run the power DNS recursor on ARM. And after now, I think
after more than a decade of this, someone added the required function call to libc. So it is coming
out. So that's the only weird requirement we have. Otherwise, it's a very standard piece of software.
Okay. Would you recommend is there any use for a home user to have a DNS server and
run their own DNS? Yeah. Is that even allowed from ISPs? I know there are a few really shity ISPs
that don't allow it, but they usually are truly shity. So our esteemed audience will not be
customers from such ISPs. And they disallow it because they mess with your DNS to inject
advertisements kind of stuff. So it's not a hacker friendly setup. There is actually a pretty
good reason to run your own name server. And that's what the privacy and security story.
Name servers are widely used to inject advertising and other stuff or hijack domains that don't
work. And you may want to be in control of your own DNS. So that's one good reason.
And the other good reason is that many service providers that are not really paying attention
run pretty shity name servers. Overloaded name servers. So it is quite quickly possible to run your
own name server and have better performance than your ISPs name server. Then all the requests
we go now is coming back for you. But you probably after one hour of browsing everything
you ever visited is in the cache. But it's true. The internet service providers
resolver should be faster. Yes, in principle. But quite often that's not the case.
The current very best name server out there is actually Google DNS 8.8.8.8.
Yeah, for no other reason than the replies to the thing.
Yeah, but it's actually there the best. But then the issue is then you hand all your DNS.
So then you hand the remaining bits of your DNS to internet traffic to Google. The few pieces
that they didn't see already. That's the downside of Google public DNS.
Do you support secure DNS at all?
Yeah, well, we do very much on the authoritative site.
For more than a decade we told everyone that DNS was too complex and over engineered
and hard to implement. And people confused that for that maybe we wouldn't be able to do it.
But we honestly thought that DNS was too complicated and I actually I still believe that.
But then after 10 years it became obvious that DNS would become mandatory in any circles.
So then after saying that it was shit for 10 years then we decided okay, but then if we have to
do it we're going to do the best and the easiest implementation we can think of.
So in power DNS authoritative we now have fire up and go DNS. So it's literally one line
and then it does DNS for you for your domain. And we worked very hard with our biggest users
to make it friendly to them. And what I find interesting result of that is that even after
saying it for a decade that we found DNS to be too complicated we now have a 97% market share
of DNS. Pretty impressive. Yeah, and it is and we have had a lot of help from friends to make
that happen. But we also listened really well to our the users that wanted to do DNS. So we
worked really hard to make it incredibly easy for them. And in the end what helped is that some
of the registries which are the national operators of DNS infrastructure they decided to hand out money
if you deploy DNS. Okay. And that convinced many of the biggest users. As an anecdote one of the
big power DNS users in the Netherlands was able to buy a new brand new motorcycle from deploying
DNS. And that was why he was quite motivated to make that happen. So that's how you get a 97%
market share. Okay. Oh, fair enough. Okay. Before we continue on the question could you go into
what DNS act is and how it works? That's very good. So in the most basic sense DNS is the
phone book of the internet and DNS act is sort of the phone book of the internet with an official
stamp that the answer you got was the right answer. DNS by itself does not have any cryptography on
it. So you get back an answer that says well this is the IP address of home 2013.org and good luck.
And anyone could change that packets midway and you never know that it has been changed. It's a plain
text an authenticated protocol and that didn't sit easy with people because they said well but it's
so important. It's so it's at the very root at the core of the internet. So we also want crypto
on that. So with and then they said something else they said but we love DNS so much that we don't
want to change it. And that's where it went wrong. The DNS is incredibly flexible. You can you
can put anything in DNS and people have put anything in DNS. And so they said well we can just
add the crypto to it. We'll just add a record type and we call it key and a record type and we
call it signature and a bomb and we're done. And then it turned out that this so that everyone
was in love with that idea. Everyone's in love that wow we built a protocol that's so flexible
that we can just add the encryption to it later. And then they discovered that that wasn't actually
true. That DNS didn't work like that because it would for example drop the keys because the
cache expired first and that kind of stuff. So it didn't we it was not possible to retrofit
DNS sec on DNS. And that's the point where people would have should have said well let's work on
DNS too. Let's just start from scratch. Let's buy let's build something better than DNS and make
it secure from the ground up. But that's not what they did what they did. They sort of squeezed
the crypto into DNS and had to change the rules had to modify how DNS actually works and out came
this monstrosity called DNS sec. And as an example of how monstrous that is there are now queries
you can ask on the internet. A very simple query that says tell me everything you know about
ikan.org. And I think right now you get back a four kilobytes packet answer with four
kilobytes of encryption encryption on it. So to boil it down it's a way to make DNS secure. But
while it made it secure the very security made these packets so enormous that they now are in
themselves a security risk. That's correct. I've had a complaint from somebody regarding my own
name server regarding a request done towards the site of that very packet. Yeah. Why is this?
But the problem let's let's let's make an example. I have a city let's say I would have a
city internet connection at home. Yeah. And and and I would send out a few few packets per
second to Nido's name server. And I generate maybe one megabit of traffic. And now Nido's name
server sends back answers that are 40 times bigger than my questions. So suddenly I turn my one
megabit per second of queries into a 40 megabit per second stream of answers. In itself that's not
a problem because the answers are all coming back to me. So I just did a denial of service attack
on myself. That's not that much of a problem. Now I fake my IP address. And I take I assume someone
else's IP address. And and I send questions from someone else's IP address and then at my one
megabit per query rate comes back as a 40 megabit per second denial of services attack. And this
is currently I was this morning I was approached by a big name server operator a big DNS operator
and he told me they now have 50 gigabits per second pipes and they get filled with 50 gigabits per
second of the denial of service attacks. Okay. How how is it possible to spoof the IP address?
Was it not t-speak connection? No, DNS is UDP. Oh. And that's that's the part where where I
think that when we when I said we should have just done DNS too. And that's where we would
sort of change that and and have some kind of two-way handshake there in there. But UDP it's
it's currently fully fully connectionless. And actually the the solutions that are being
pondered right now are just saying well you should be doing TCP IP for DNS. And for a long time
everyone said yeah but that's that's too slow because UDP is so much faster. And the reason
people said that was because there were no tools to benchmark that there was no DNS TCP benchmarking
tool. So everyone had just assumed that it was slow. And then we actually wrote that tool to try
it out. It turned out that this DNS of TCP is not slow. And why is that case everyone has been
optimizing their servers for HTTP for the past decade. And and the result of that is that in some
cases TCP is now actually faster than UDP because it's so it's so highly optimized. For example Linux
if you send the UDP packet from Linux it will send it sort of synchronously. Just drops the
packet in the queue of the Ethernet device and then it returns control to user space.
If you do the same thing for TCP it just copies the the data you want to send to the kernel and
then your user space can get on with whatever it was doing. And in the background the kernel will
actually take care of the transmission for you. So it turns out that if you do DNS over TCP you get
loads more packets. That is true. You get like six times more packets. But your operating system is
so well tuned for that that you can actually still achieve a very high query rate. Okay. And that
would solve the little service. That would at least improve it massively because that would
disallow the blind spoofing going on right now. And I think that within this year, within next 12
months we will see a movement towards making people actually mandate that they support DNS over
TCP. Okay. Because one of the denial of service mitigation techniques you can use is if you get a
lot of queries over UDP you send back one answer that says well I you the next question I need to
see is over TCP. And if someone then doesn't come back over TCP then you know that you are looking at
spoofed traffic. Okay. But as it stands, people routinely deny port 53 on TCP for DNS because there
are still the old guidelines out there. Let's say that it's good for security to block DNS
over TCP. A lot of people were using it as a transport to do tunneling over. Yeah. Oh yeah. That's
also true. Yeah. So that's kind of how that got locked. Yeah. Whatever we're going to lose in
some way. Yeah. But there you go. Do you have any idea where this recommendation came from to
block TCP DNS traffic? I think I think I do. DNS is the phone book of the internet and nice
and like a phone book you can make a copy of it. And this is called a zone transfer. And if you
do a zone transfer of any interesting recently I came across a zone for my rather big international
corporation. And if you see that zone then suddenly you see lots of domain names that you didn't
know exist. So this manufacturer of printers turned out to have FTP servers all over the place.
And you would not have known. But if you had downloaded the entire zone you suddenly saw FTP.South
Africa dot blah blah dot com. And who knew? So if you're able to transfer a zone you get a sort of
map of IP addresses which you can attack. And the easiest way to block zone transfers is to block
TCP because zone transfers always happen over TCP. Yeah. And I think that's the origin of the
recommendation to block TCP port 53. A lot of companies blocked as well because in the zone
transfers they've got new projects. Exactly. Products. But the thing is you can also block the
zone transfer just from the software. Yeah. You don't need to block it on the firewall. Yeah,
but true enough. Yeah. But any any auditor I mean it's probably I it's still being copy-pasted
in manuals that we also had that issue with Cisco default configurations. I think for
a half a decade everyone told Cisco update your default configuration because it blocks DNS
it blocks secure DNS. And but Cisco is apparently big enough that even if lots of people within Cisco
are trying to change the manual the manual does not change. And finally it took some some senior
vice president there to just put his foot down. And now finally if you take a default configuration
it will no longer block DNS sec but it might still block port 53 for I know. Okay.
Okay. Well let's see can we talk a bit more about the databases and how power DNS makes use of
those. Yeah. That's a good one. DNS a typical the kind of query rate that people care about in
DNS is in the order of 10 or 20,000 queries per second. Big operators like to be able to do that
kind of DNS query rates. You can go far higher but I think most people feel comfortable with 10 or
20,000 queries per second. Each DNS query can translate into three or four or five database
backend queries that has to do with looking up wildcards and c-names and other things. So that means
that that that's that's usual factor of four that you're quickly looking at 60 70 or 80,000
database queries per second. Most databases get very unhappy if you do that. It's not what they like.
So because of that we have our own internal name server three database caching engine. So we try
to make sure that we don't we don't keep asking the name server the same database the same question
all the time. And using this mechanism which is database independent so we do it for all our
backends. We are able to break down those five queries to maybe one on average. And this
allows us to achieve serious query rates on top of a database that itself would not be wanting to
supply that. And the most important things we block are queries that actually have no answer
because if you ask if you ask a database for a record that doesn't exist that is almost by
definition the hardest amount of work that has to do because that actually has to look everywhere
to see if the data actually really doesn't exist. So we block both the present first or we catch
both the presence of data and the absence of data. Okay. Let's see. And that will, sorry, that
will be obviously to be one look up where it says it's not there and then you then you maintain
that for how long? Well, that's configurable. Yeah. And the question of course for example,
we have one very big user that is on the denial of service attack a lot and they have set their
cache TTL at one day. Okay. Yeah. And but the problem of course with that is that you get very
little traffic to your database. But if you make any changes to the database, no one would know about
them. However, we have a very powerful infrastructure where you can do selective cache invalidations
within PowerDNS. And you can do that remotely. So that means that those folks have hooked up triggers
to their web interface that when someone makes a change to a domain name, the PowerDNS instances
get a message that says remove this stuff from your cache because the database changed. So they're
able to run at 24 hour cache lifetime, but still serve fresh answers. Very nice. And one of the things
that kind of annoyed me about buying was having to restart every time you met a change
year zone. Yeah. That's the thing we explicitly decided to avoid. We never wanted to force and
you want to do a restart for except when you really change the configuration. But if we but the other
thing is if you do a restart of PowerDNS, it takes one second because we don't have to read through
all the zones on startup. And you make a mistake and then you have to restore the zone. Yeah. Yeah,
exactly. Because we have a sort of philosophy that comes from RFC 1925, which is the 12 fundamental
networking crews, which is the best RFC ever. And everyone should read it, print it out RFC 1925.
And it has a set of rules for network infrastructure. And I think rule number two is it has to work.
And it's interesting that it needs to be set. And then explicitly that RFC 1925 does not say it has
to be right. It says it has to work. And we are very we have a lot of solidarity with the people
that are on call for 24 hours a day. And we want the software to just work and be up. And if you make
a mistake somewhere. So we have a limited number of mistakes, where the name server will just not
launch. But you really have to mess it up to make that happen. But as long as we know that we are
able to serve data correctly, then we will do it. And that means indeed that if you make one mistake
in a zone file that or in the, yeah, there is no centralized big variation files. It's tricky to
the the classic situation when the name serves going down is that someone put a semicolon
in the wrong place. And that's the beauty of DNS is that so resilient that quite often people
only notice that something is wrong when the last of the three name servers is down. And then when
people look up, it turns out the other two had been down for months already, but no one knew.
I would also like to comment a little bit on bind and other name servers. We get along pretty
well with everyone else. So it's actually so we drink beer with the people from from bind and
the people from an Lnet labs and other stuff. And we have our differences in our philosophy on how
we we write the software. But people often assume that we would be enemies with with the internet
software corporation. And that is so incredibly not the case that we then we also try not to make
fun of each other's securities mistakes because bind recently had a very big one. And we make no
fun of that because we know that one day we will have a very big security mistake. And it happens
to everybody. Yes, there was further. Okay. For regular people who are listening to this and
think, hey, maybe it's a good idea to deploy some DNS for myself because I don't really trust my
ISP or I think they're slow or whatever. Is there any recommendation you can do for them?
Well, so we have very big users and we have very small users. One of the things I like for the
more regular user. And you still have to be a Unix geek. I mean, if you're not a Unix geek,
then it's hard to take control of your own infrastructure.
But I think one of the nice small features we have in power DNS, as if you run it as your
recursor, we have a feature called export ETC hosts, which means that with no real work,
you can, if you have a printer with an IP address and you put a printer in slash ETC slash host,
and you configure power DNS with export ETC hosts, it will share that with your home network.
So that means if you have a few IP addresses in your home that you just want to work,
you don't need to set up an entire zone file, we will just work for you.
So that's, yeah. And then if you will do that, then at least you're no longer giving away
the few percent of traffic that Google was not seeing yet. And you're also probably getting
better performance. So the receiver will be one that you'd run locally. It's like the cache.
Yeah, it's a cache. Okay. So now it runs on an ARM processor. So technically,
if we're running a Raspberry Pi, that's also the main reason I think it's getting fixed.
We, because that's that we would, yeah, we would love to be able to do that.
We just have a little name server brick that just does its thing.
And hypothetically, you were saying that you run it on a DNS server, again, sorry, my brain is
fried in this case. You use SQL Lite. Yeah. That runs very nicely on the Raspberry Pi.
Yeah, yeah. So you could have this little name server brick and everything on there.
Could you run your own like dot hacker domain outside of the whole infrastructure?
And then, yeah, you can. And people do that kind of stuff a lot.
But the interesting thing is so people do that. Yeah, you can. And it's nice and it's fun to do
it. But here's the interesting thing. If you look at the root name servers, which are at the core
of the DNS of the phone book, 95% of the queries they get are for domains like dot hacker and
dot printer and dot corp and dot office and whatever. Yeah. And these are these sort of internal
queries that leak out to the internet. And they're leaking out at such a rate that now 95%
of the traffic at the DNS servers has nothing to do with the internet.
Okay, well, and those queries would have been resolved by everybody else.
They should have been. They should have been. Yeah, they should have been. They should never
have left the office. Yeah, exactly. But then when they do get out, they have to keep going up
to the very root because nobody else knows. And then the root says, well, I don't know about
dot corp and dot printer. And that's actually a very ongoing discussion right now because
there are these top level domains like dot com and dot net and dot com. And those,
there are now thousands of new ones being pondered. And they will be handed out
maybe this year, maybe next year. I don't know. But some of them that are being discussed for
handing out are ones that people might have been using internally. Yeah. So all of a sudden you
were leaking queries for dot corp or whatever. And suddenly dot corp starts to exist. And maybe
your printer that has been trying to report its status to a central server dot corp will suddenly
start reporting its status to a server in Russia or whatever. So yeah, you can run your own domain
names, but they always tend to leak and get out. So we can't over that. But these days I would
actually recommend just get a real domain name and just use that one because that will at least
always end up at your servers. Yeah. Well, I'm also thinking about you wanted to build your own
network. And you wanted, you know, for privacy issues, you wanted to keep your own network running
and a federation of friends, for example, you just brought up your own DNS.
It is done. People do it. And these are called sort of alternate routes. People say, well,
we have our own DNS infrastructure. The thing, the issue with that is that it's incredibly
it's very exclusionary when you do that. DNS is really you either on board with the alternate
route or your on board with the normal route. Yeah. And it's there. These organizations do
exist that have defined their own internet, but they're very lonely people. Okay. Because once you
get on board with their vision on the internet, suddenly the other parts of the internet disappear.
And or you have websites that work for you and for no one else. So it typically is tremendously
hard to have distributed fully decentralized naming. Naming is really implicitly a hierarchical thing.
I'm just wondering, you know, back in the day, there were zone files being passed, you know, host
files being passed around and then they become zone files and then DNS developed, you know,
with the event of BitTorrent or something like that, could you send like a terabyte file with all
the DNS records from everywhere in it? People are have been trying real hard to apply crypto principles
to enable fully distributed and totally uncontrolled decentralized naming. And the most advanced one
that is name coin, which is related to Bitcoin technologies. And from a philosophical standpoint,
I think they've come the furthest in because and I'm not a name coin expert, to be honest,
but one thing they got right is if you let's start naively with that. So let's say we
we three we band together and we start a list of all domain names. Yeah. And then we put on
there that home 2013.org is hosted by the NSA. And then someone else says, no, no, no, no,
that's not that's not true. It's hosted by ifcat and whatever. And and then as the casual
internet user, you wouldn't know who to trust. You never know if the IP address you got is from one
political faction or for the other. Yeah. What the name coin people did, the name coin people made
it sure that you need to invest in your name. You need to spend serious clock cycles on defending
your name. And that means that and every time people use it, it becomes sort of more embedded. So
they they solve the decentralized federated naming issue with proof of work statements. And that
means that that the one that has been claiming that home 2013.org is is his domain for the longest
time because they have the oldest proof of work. They can say, well, I have a valid claim to it.
But it's still incredibly messy. And like I said, the DNS is like the sewer of the internet.
You wanted to work or you're covered in shit. And that's the same stuff with the with the federated
naming. People have very little patience for DNS that is more secure, but doesn't actually work.
Yeah. And so I would typically recommend to get your security at a higher level than than DNS.
Okay. If you were doing DNS too, well, what would you do properly?
Oh, car plans. I would do money.
The weird, the one of the the oldest things about DNS is that DNS has no way of saying there is
no answer to your question. So, so if you say, give me the IP address for www.home 2014.org.
The answer to that is empty. There is no answer. It gives you an empty, empty answer.
That means there are no answers. When you try to do crypto based on that, you need to sign this
empty answer. The problem is all the empty answers are identical. Yes. So it's that's why the DNS
specification, the DNS specification is now bigger than the DNS specification. So we have an
enhancement that is bigger than the original to work around these issues. The first thing I would do
was would be to have a DNS that can just say, I now affordatively state that this domain name
does not exist. And then you could sign that one. Yeah. For example, the other big thing,
I would just put in that would help me troubleshooting DNS issues. Yeah.
Because then you know, yeah, you would actually say, oh, I hear it says here, because right now we
hacked that in. Actually, originally DNS really didn't have a way of saying it really doesn't exist.
Yeah. And then it would hack in that there's a little trick you do that in the DNS answer,
you put another record and then you derive from the presence of that other record that really is
no such domain. Okay. But it was trick. And everyone knows it's a trick. And so that's the first
thing I would definitely change. The interesting thing is that I met with the original inventor of
DNS Paul Moca-Peteris. And he worked in France these days. He's a really interesting guy. And
he explained how the internet was really designed and how it worked. And he told me that when
they came up with DNS, he got a question from John Postel. I'm sorry to have to interrupt here,
but we have a problem with the tent. We have to get everybody out. So we'll be on music for a while
while the problems with the tent are getting sorted. So sorry about that. Yeah. Okay. Well,
we're going to evacuate and maybe we'll finish this later.
You have been listening to Hacker Public Radio. It's Hacker Public Radio. Those are
We are a community podcast network that releases shows every weekday Monday through Friday.
Today's show, like all our shows, was contributed by a HPR listener like yourself.
If you ever consider recording a podcast, then visit our website to find out how easy it really is.
Hacker Public Radio was founded by the digital dot pound and the economical and computer cloud.
HPR is funded by the binary revolution at binref.com. All binref projects are crowd-responsive by
linear pages. From shared hosting to custom private clouds, go to lunarpages.com for all your hosting
needs. Unless otherwise stasis, today's show is released under a creative comments,
attribution, share a life, lead us our lives please.