Episode: 4379
Title: HPR4379: Mapping Municipalities' Digital Dependencies
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr4379/hpr4379.mp3
Transcribed: 2025-10-26 00:00:33

---

This is Hacker Public Radio episode 4379 for Thursday the 15th of May 2025.
Today's show is entitled, Mapping Municipalities Digital Dependencies.
It is hosted by Troller Coaster and is about 13 minutes long.
It carries a clean flag.
The summary is, asking your help in mapping public services of governmental websites.
Hi, Troller Coaster here.
And this time, I'm not going to rant, maybe a little bit, sorry, can't help it.
But I have a question for you, for the Hacker community all around the world.
I need your knowledge, I need our hive mind to think together.
So I'm working on a project.
I'm trying to map the dependency of IT services, of municipalities, of governments, on
third parties, online services that could go down and then break the whole system.
The two biggest ones, Microsoft and Google, are my main targets, but just because they're
the biggest ones.
I won't discuss the quality of these products, I'm pretty sure they are very decent products,
but I mean, that's the least you can expect if you have tens of thousands of employees.
So that's not the topic of my discussion or my research.
So I'm trying to figure out how big this problem is.
And if you think that's not relevant, I mean, the odds that these big services go down
might be small, but still, the consequences are really grave.
I mean, that's the reason why half the companies all around the world do fire evacuation,
drills, fire drills, that's why we do, in some countries, we do like emergency training
for disasters, because the odds are small, but the consequences can be very, very big.
If you're half a decent manager, you have sight on which risks are the biggest ones
for your organization, your company, your country, that's just good leadership.
Anyway, if you think the problem is far fetched, just go back to July of 2024 and look
up crowd strike.
All around the world, airplanes got put to the ground because somewhere in a company,
the developer made this stupid mistake, things happen, not blaming the developer.
It's just a stupid mistake.
There were errors in the process of that company too.
But the consequence was all around the world, planes were set on the floor because some
piece of software crashed, except in airports where they had a contingency plan.
In Brussels, it's maybe funny, but they had a pencil and paper and they had a complete
workflow where they could take off with airplanes just using pencil and paper.
I mean, they had a plan, it was there, they were prepared for power outages, they were
prepared for systems that went down, they just had a good approach, kudos to Brussels,
Brussels Airport for that.
Anyway, here's what I was doing, so I just created like a huge database of all the cities
of Europe.
And then I started guessing or looking up, using wikimpedia, open street map and then some
just random guessing, city name.top level domain name, for example.
And just looking up cities and looking up their DNS records, and in there, there's an
MX record and this MX record, it's like the postal office, it tells your email client
to which postal office to which mail server, a mail has to be sent and then there it will
be this patch to the right person.
So the MX record is public record, everybody can see it, has to see it because otherwise
you couldn't send emails.
And looking up these records, you can see if a domain is, for example, something.outlook.com
or something.google.com, and then you can safely assume that these two, they're the biggest
ones, are run by Microsoft or Google, yeah.
So that's basically what I tried to do.
Now for countries like Belgium, Finland, Netherlands, where this teaches us that over 70% yes, three
out of four roughly of the municipalities use Microsoft for their email as a public mail
server.
We should trigger warnings because just imagine Microsoft going down, I mean, the police
can't, the police is losing a lot of their stuff.
The fire departments are using it, the hospitals are using it, the doctors still have access
to medical records, if somebody has a newborn baby, can they declare birth at the city hall?
All these questions from the life, life-threatening ones to the very mundane ones, will the trash
car drive out, I mean, maybe the planning is an outlook or in some cloud service.
Anyway, you should map what is impacted if this service goes down.
And so for countries like Belgium, Finland, Netherlands, as I said, it's clearly a problem
and probably even a bigger one, I'll come back to that later because, but for other countries
like Germany and Hungary, where less than 5% of the municipalities show an MX record
at points to Microsoft or Google and tend to have a very big domestic presence of servers.
This could lead to some complacency, but maybe it's a false complacency because I've been
learning that there are two things that this technique of mine doesn't show.
First of all, there's the legacy, there's like people running an exchange server on their
own system, on their own network and these locally hosted servers obviously also have a domain
that is on their locally hosted, so it's their own domain and these won't show up, these
won't use something.outlook.com or the likes.
And then there's also the good practice, I must applaud that, of people using mail proxy
services to preemptively filter out spam or phishing attempts and keep internal mail inside
the network.
So in these cases, the public post office will be the spam filter and behind the spam
filter, this will forward stuff to the real mail server, but because it's not a public post
office, a public DNS record or MX record, we can't see it from the outside.
So my best guess is that from all the non third parties, cities, I think roughly half
to two thirds of them are actually also using Microsoft or Google behind the scenes.
But that's just a fat finger guess, so I can't make any claims based on that.
But from what I understand in Belgium, in the Netherlands, we're somewhere between 90
and 95% of the cities and public governmental services who actually use Microsoft and I think
it will even go more up.
And I think this is a problem.
So I want to have hard numbers.
I want to make this more specific, especially for countries with low, low, low numbers.
I mean, Germany can take pride that it only has like four or five percent singles for
Hungary.
And this can have different reasons.
Of course, one of the reasons can be that they are actually using a self-hosted service
server and I'm not saying that it's better because I mean, if some office clerk is running
a mail server on a Raspberry Pi under his kitchen sink, I don't think that's the best practice.
Then I'd rather have somebody using Microsoft or Google.
But here the question is how dependent on or beyond third parties because we can't fix
it ourselves.
We can't send a technician out.
And I'd like to ask your help to give me pointers how could I figure out if a domain
is using specifically Microsoft or Google behind the scenes.
If you can't determine it from the MX records or other services, maybe if you find them relevant.
Anyway, this is what I already tried.
So I had a look at the SPF records in DNS.
These tell something about mail security.
And often these also have a reference to, for example, Microsoft or Google.
And that gives a reasonable certainty that they are using this service.
But no guarantee because it could be an old artifact.
Same for some DNS records having some TXT records showing some kind of subscription key
or a DKIM indication that they are using a certain service.
But again, these could also be historical artifacts because if it's there and it's no longer
used, it still works, which is not the case for the MX records.
But they are still strong pointers in my opinion, but no guarantees.
So I'm a bit more hesitant to rely too much on these.
Anyway, then I also had to have tried to tell net into the mail service to do like an
illorequester or do some specific, ask some specific instructions that could be typical
for an exchange server.
And this actually I'm happy to notice that people have done their homework and they have
changed the default headers.
They have configured their mail servers so they don't click information about this kind
of stuff.
A good thing, but it makes it harder on me to figure stuff out.
What I'm planning to do, because this is slightly a different scope also, is looking
up the air records, so the DNS records to see in what IP range these servers are hosted
if they are running on an Azure or an Amazon or a Google Cloud Platform system, because
this is also a dependency.
I mean, if Google goes down or Microsoft goes down, for example, a trade embargo, for example,
there is of 600% and a GDPR violation, just a stupid screw up by the developer.
Things happen.
I mean, it's just human, very all humans.
Anyway, IP addresses are something that are on my to do list, but I still don't have
a good way to determine if a server is actually running Microsoft or Google services as
a two biggest one.
If you find things to fingerprint if somebody is using Next Cloud, that would also be great.
So let me have it.
I think those are the three biggest ones.
If you can help me with that, you would do me a really big favor.
There's like an intermittent map that you can find on my personal blog.
I'll put a link in the show notes.
It's my first name, not my last name, slash map.html.
There are two somewhere, but you'll find it in the show notes.
Anyway, I say, let's as hacker community help our governments, our regions to be conscious
of these risks and point them into the right direction to actually consider safer options
if they are needed, even if it means using pencil and paper.
I mean, I don't say everybody has to go to open source.
I don't say everybody has to abandon Microsoft right away.
I mean, if it's interwoven in all your systems, you can't fix it in two days.
This is like a 10-year plan.
But if you would want to leave it, but despite my own opinions, I understand that some
people want or need to stay with a company like that.
And why I think it's not a better ID is maybe a topic of a whole other podcast, but I'm
getting, I'm rambling too much already.
So please help me out, help me find ways to determine which services, which cloud services
ideally, which AI services may be, or being used by a certain domain.
And this would be of great help to me.
Thanks a lot.
And hear you around.
Bye-bye.
You have been listening to Hacker Public Radio at Hacker Public Radio does work.
Today's show was contributed by a HBR listener like yourself.
If you ever thought of recording a podcast, then click on our contribute link to find out
how easy it really is.
Hosting for HBR has been kindly provided by an honesthost.com, the Internet Archive
and our sync.net.
On the Saldois status, today's show is released on our Creative Commons, Attribution 4.0
International License.