hpr-knowledge-base/hpr_transcripts/hpr0298.txt

Episode: 298
Title: HPR0298: AutoNessus
Source: https://hub.hackerpublicradio.org/ccdn.php?filename=/eps/hpr0298/hpr0298.mp3
Transcribed: 2025-10-07 15:51:40

---

Is this the music hole?
Uh.
Hello ladies and gentlemen and welcome to today's episode of Hacker Public Radio. My name
is Ken Farron and today we're going to be talking about autoneses. Autoneses is the tool
that we use where I work and while I was reading up on the documentation of it I came across
a few interesting facts. First of all it's released under the GPL version 3. It's written actually
colleague here at work and thirdly there was going to be a demo on it last Saturday which I
attended. I found it very interesting and the topic that we would probably be interested here
on the Hacker Public Radio network. So I managed to corner the author Frank Bradenreich. I'm
going to say that right? Right enough. How about you introduce yourself? I'm Frank Bradenreich. I work
as a security engineer at your work fellows here. Okay very good. I don't know what's the best way
to begin? Do we start explaining what autoneses is? Or can you give us a rundown first of all
probably a plot of necessities? Well I can give you a rundown for those that are not familiar with it.
First of all I suggest they go and check it out themselves by downloading it from nesses.com website.
But basically nesses is a network vulnerability scanner. It scans from one host. It can scan
a number of other hosts or an entire network and search for vulnerabilities. So it starts with
a port scan. Looks for a port to open. It connects to those ports. It tries to guess what version of
the software it is. Sometimes it knows. Sometimes it needs to guess. And from that it tries to find
vulnerabilities in that software. This is a fairly well used tool. I hear the guys at pull.com
security we usually use it quite a bit. Yeah. The SecTools.org which is run by Fyodor, the author
of a NMAP. And he did a survey in 2005 and 2006 where this tool came as the top value tool for
security professionals and security interested people. It's not actually released anymore under
an open source of license. No, it's somewhere down the line and I think it was version 2.1,
the tenable, the company that builds and maintains nesses decided to switch from free as in
speech to free as in beer if I may put it that way. And they close the sources of nesses.
They put in a license model where additionally you could still use it for free and get the plugins
but get the registered plugins a week later than when they were released. And at the moment you
have to license it when you use it commercially and can get a if you can get a free license if you
use it privately. My experience with nesses has been very good but I've noticed that it produces
a lot of documentation. Yeah, especially in my previous job where I did security consultancy,
I worked with nesses quite a lot. Yeah, it produces an awful amount of output even if you condense it.
One of my reports that a client was at some point called the Dicke van Bredeck, just
had a fat book of Bredeck, basically because of the sheer volume of information that was in it.
And it's one of the things that you just can't help but noticing when you use the tool, especially
when you use it regularly. That said, it's very impressive when you come in with a
ring or a four-paper and you don't put on somebody's desk, kind of. Well, they feel like
their money is worth it for their consultant. However, if you've got 100 servers and you've got four
pages per server with the same things over and over again, it's difficult for a admin to kind of
get their head around that. Yeah, and that's nice with a one-off scan, but if you want to do
a more regular scan, say you want to scan your infrastructure every month or every quarter,
that was one of the things I noticed when I got working at Schubert Phyllis and I scanned the same
infrastructure more than one time. That's really an awful lot of duplication. There's not
a lot of difference between scanning an infrastructure now and then getting the information
a month later. And yeah, you're virtually reading the same report again.
You have me. It's like we've tried to find the 12 differences in two reproductions of the
the Mona Lisa. Well, that's fairly small. Let's say Rembrandt's Nachtwacht, which is an enormous
enormous painting and you need to try and find those five differences because that's basically
the differences you're going to see in your entry report as well. And essentially this is what
you're trying to achieve with the with the program also necessary. Yeah, it's an idea that
crept up on me. It's it's funny thing is I didn't actually that wasn't actually the goal I set out
when I rode autonomous for the first time. Initially, we just looked for a way to automate the
scans and kick them off at a reliable time. But once I saw how nest is actually stores its findings
in a backhand, it's like, wait a minute, that compare I'm doing every month is easily to automate.
Yeah. And and that's what yeah, that's what auto nest is about at this moment really the
Delta engine. I mean, it's not rocket science to to script something so that you can run it from
a command line and run it from a Chrome job. But the the backhand is still not rocket science,
but it's it's handy to have that Delta comparison in there. Okay, can you tell the listeners what
all of us is? Well, let me let me try and set it out. Obviously, when I did the demo, I had to
had the stuff there, which which makes it easier to demonstrate. Also, nest is a wrapper around
nestes. So the first thing you need is a host that can run your nestes demon. Yeah, that can be
a Linux host can be a Windows host if you want to. Yeah, you need a host that runs the Linux
and you need another or the same host that has the the nestest client on it. There's a bunch of
scripts around it that you can schedule firecrime job or can run manually to actually do a scan and
a command is handling called do dash scan. Yeah. So you start off, you've installed it. Well,
you start off by downloading it of my website, but you install it specifically written with with a
Linux Unix system in mind. So I haven't actually tried or tested the scanner on any Windows
platform. Yeah, you run it and then you need to set up a scan and you add a config file which
specifies where your nestes demon is running. You fill up a host file which defines what host you
are scanning and you run your first scan. You specify if you want a save mode scan or a complete
scan and then it starts to scan to do a nestest scan. A few points that we probably want to mention
here. You should not scan any network that you do not have explicit written permission in order
to do that. Yeah. This is a sort of default warning, so just be careful about that. Ideally, you
could scan your own home network, for instance, or if you're in a university or something
you're in a min for a network, you could scan that with permission from your employer.
There's a golden rule when you're doing Pantas commercially and that is get your get out of
jail free car first before you actually do anything. Having said that, so taking you have your
permission to scan something, you put that target in a host file and you run the scan.
What it does, it does the network scan, just as a normal nestest scan will do, but it
immediately converts the results to an HTML file, an XML file, and a NBE file and the NBE file is
nested back end. Then the part that I call the Delta engine kicks in and initially it will mark
all your findings as new. So you then can go to the website of Autonesses and with website,
I mean the local website, because Autonesses has a web GUI, so into Apache. Yeah, it plugs
into Apache. It's actually a classic CGI. What's written in Perl primarily with some bits of
shell script where I needed interaction with the shell, but it's a classic CGI, so there's no
reason why it shouldn't plug into another website, but the web server, but I haven't tried that.
So what it does is when you have the findings, it tries to categorize them for you, and the first
categorization is new, meaning this is a finding I've seen for the first time. So basically when you
run your first scan, everything is going to be new. Yeah, when you run your first scan, everything is
going to be new. Now, when I say finding what I mean is that each host has a number of open port,
each open port can have one or more plugins fired against it, and that plugin has an output.
So the combination of host, port, plugin, and the output of the plugin is what we've defined as a
finding. For instance, if you have a web server that's running both on standard HTTP and HTTP
s, it can fire the NICTO plugin against that. NICTO is another security scanner. So that can fire
both against port 80 and against port 443, and both the outputs will be in auto nest. So that's
two findings. That's two findings. Okay, so say, for example, we have run it against a workstation
or something, and you're finding that there's a port open on 80, which you think is okay, and you've
got a port open on, while you got a port open on 84, 443. Yeah, one port 80 isn't okay, and 443,
you're thinking is okay. What do you do? Yeah, basically what you go do is look at each of the
findings, and the GUI lets you sort out and filter the findings by host, by plugin ID. So I could
decide to look at all the NICTO outputs in one go, and then look at all the ports scanner outputs in
one go, and look at all the other plugins that could fire, which is sort of the way I prefer it,
but that's my personal preference, or you could go and look by port, so you could say, okay, I'm
first going to look at the findings from port 80, then look at the findings for port 443. So in this
case, if you looked at the findings for port 80, we found them all okay. There's basically an option
in the GUI, which has bulk update, and I can then, as a user, choose for a new finding, I can choose
between two statuses to assign to a finding, so either the status, no issue, which means I'm okay,
I'm happy with that plugin firing. You expected that, you expected the result that you found,
or I didn't expect it, but it's clean. Okay, yeah, it's basically whether or not there's
risk associated with that finding, not if I expected or not. Yeah, okay. And then if there is a risk
associated to it, I can mark it as being open, so this is something that I need to deal with.
Very good. So in your case, what we just said, the two, let's just talk about the two NICTL
reports that we got, one for port 80, one for port 443, so the one for 80 was okay, I've marked
that there's no issue. The one for port 443 had a risk with it, I marked it as open, and then
I'm going to do something that will make me fix those findings. For instance, I'm going to get a
phone-erable CGI script of that port 443. So in terms of the amount of work on the first scan,
it's not any less than it would be if you just did an all-in-ass scan or an SS scan yourself and
went through the findings. Yeah, it's not less work. What I found myself doing lately though is
when I ran a NICTL scan on a separate host, I actually took the NBE file and put it in alternatives
because I like the GUI and I like what the GUI does. So I can basically take off the findings that
I've done and it helps me organize my analysis process of the findings. But in terms of the number
of findings you have to look at, it's exactly the same. So it's still, for the first time, it's still
a useful tool to go through to make sense of that huge amount of documentation. To help you digest
that huge amount of findings that you will get if you scan a sizeable network. Okay, so that's
interesting enough, but where it has a lot of value is when you go back and you do the second scan.
Yeah, so you've produced a listed report for the system administrator, which may be yourself,
and you say, okay, this needs to be done, this needs to be done, this needs to be done, and you go
back two weeks later and then you run another scan. Then what happens? So we will regenerate that
amount of findings, but then it will do a Delta and comparison. We've made some changes to the system,
hopefully to address what we found the first time you run the scan again, and it comes with
basically the same or a slightly different amount of finding. That's a point where the Delta
engine will kick in, and the Delta engine will try to determine a new status for the findings
based upon what's the finding previously there? Is there something that we've now seen, for instance,
has a new open port emerged? Has a port been closed? It will look at that, and also it will look at
the output of the plugins. So for instance, in the case of the NICTO plugins, we just talked about
say port 443 would now be closed. That specific finding NICTO and port 443 on this host would be
marked as gone by the Delta engine. On the other hand, if it wasn't just gone, but you had deleted,
for instance, a vulnerable CGI from that website, it would be marked as changed, because the output
of the NICTO scan would change. So it would change, say, hey, unchanged, and then the plugin output
would change. So you need to look at it again. That's basically what change means. And by the same
mechanism, say that you have done this initial amount of work getting your system to a state where
you're happy with, you're on the scans, and you don't see any Delta's, say, then you'll update
NICTO and its plugins, and it detects another vulnerability. Yeah, that will be marked as new,
because that's a new plugin, it's a new combination of host port plugin, so it will be marked new.
So the whole idea behind the Delta engine was, instead of looking at two big paintings and
trying to find the differences, let's stop looking at the two paintings, let's try and find the
differences automatically and just look at the differences. So filter out the noise.
Filter out the noise, dampen the noise, and try to get the signal, basically get the signal out
of the noise, so that you just have to look at your signal. And that works pretty well in the
demo. I go from an initial 40 plus findings, and when I do the second scan, there's only something
like 20 findings that we have to look at. When we do the third scan, we just make my new changes
to the network. There's only four findings that we need to look at. And that's the experience,
I have in my daily work as well. We actually use it here. We use it here. In January, we scanned
over 4,000 IP addresses with it. They produced a total of 8,777 nestless findings in the
month of January. It's important to say that that's not all issues. No, no, it's not all issues.
I mean, nestless is also very good at reporting. You have a web server here, and this plugin
thinks everything is all right, which is kind of what we're doing. You have a web server running,
it reports that it's Apache. That's how you want to configure it. Thank you.
Which is good to know if you're running for the first time, but sort of becomes a very boring
message after the 24th or, you know, probably a little bit earlier.
And the danger is, of course, if you're going to these findings yourself, that you're scanning
down through this, and you go, okay, I've seen that before, seen that before, seen that before,
and you missed the one that's in between. That's where auto nestless really shines through.
As far as the automation goes, there's automation in running the nestless scans, but then there's
also so automation in running the reports and emailing. Can you tell us a little bit about that?
When it finishes comparing to scans, it will send out a condensed report out about the number of
open findings it finds, open new and changed findings. It will also output some of the
delta as it's seen so that you can make, yeah, a quick judgment. Is this something I need to look at
now or at my leisure, but the report that is sent out is intentionally kept fake because I don't
trust the transport mechanism of email. I mean, emails in the clear store and forward,
means if I send out an email, there's always a copy somewhere. That's why I wasn't always those
little notes at the bottom of an email, make me laugh, because you should make a copy. Well,
I need to deliver this for you. Sorry, side snapping. If you really want to get the depth and the
detail, you need to go look at the web web GUI. At the moment, there's still a manual processing
behind the findings. When we talk about the road map, I'll address it a little bit more because
that's what I want to talk to us about the road map. Currently, it just concentrates around
nestless findings. So it will go into yes, this plug-in fire, that's the output. And when I report
back to my colleagues, who need to fix the findings, there's an aggregation level above that.
Yeah. And yeah, if you want to call that issues or incidents or whatever name you want to give
that, and I haven't decided on the proper name yet. Observation? Yeah, observation. I think I'll
stick probably stick to issue. Okay. Yeah. Basically an issue of a porch being open or a vulnerable
version of software being run can be expressed in one or more plug-ins being fired. But also,
a single plug-in that fires and gives you some output of an HTTP demon can tell you that A,
it's an old HTTP demon, B, it reveals that via the headers, it reveals its internal IP address.
So a single plug-in, a single nestless finding, can actually have multiple issues in it.
So the idea at the moment is to extend the Autoneses GUI to also include that part of reporting.
Because what I noticed is that it's still hard to keep the consistency at them,
in telling the system administrators what's wrong and the nestless findings in the GUI.
That's still a, yeah, a bit of a, it's still a chore. So that's really where I want to
extend and bring Autoneses to do more and more of that vulnerability management to get a business
term in there. Okay. Fair enough. Anything else on the room up? Clearly there's open VAS
integration. We talked about nestless going close source. When nestless went close source,
the open VAS project checked out the latest open source version from Nestes. Basically they
did a search and replace for nestless dash open VAS and the open VAS project continued on that
road map. It's also now starting to get further and further away for nestless because it's going
its own way. But I've had several requests in for open VAS compatibility. So that's, that's
high on my road map. Also currently the quote unquote database I use is a, is a flat file system.
So if you install Autoneses and you go look into the far directory, you'll see a directory per
configured scan. In that directory, there's directory findings. In the findings,
directory is a directory per host. In there, there's the directory per port. And in there,
there's the directory per plug-in. And below that are our single files for each and every time
the scan was fired. And that actually works pretty well until the number of findings goes up. And
yeah, perglobbing tends to get slow when you have thousands and thousands of files to look at.
There's some optimization that can be done there. Also putting DBI support in there and thus
linking it up with a relational database, something like mySQL or Postgres. Yeah.
Something like Postgres will actually a help scalability and be help with extendability.
So we can extend the schema and put more vulnerability management capabilities in there.
Yeah, with the request for open VAS, with the request from VOTOR to put NMAP in there,
with a request from the guys who maintain NICTO to actually put, to see actually if I could put
standalone NICTO scans in there. If I'm going to normalize things and do multiple support,
maybe multiple scanner support is something that I want to add in the future as well. But that's
the far distance that the multi scanner support. So if you do this in your own time or if you do
it here on work or a bit of both, to be honest, I mean the boundaries between my private and my work
time tend to be not as sharp and clear as one might think. It is a work sponsored project
in the sense that me working on alternatives is part of my job. Then I can, I'm inclined to often
work on alternatives during my free time as well because it's, yeah, it is my baby if you want to
put it that way. So you're right, but super philosophical on the copyright. Well, first there's
the legal part of that basically Dutch law states that if you do anything with resources from your
work, so if you use your laptop or your boss's time to make to make something, it is copyright
of the company. On the other hand, I did write it in a slow summer where I set several days,
slash weeks of work time into it. And I think it's only fair that Schubert Phillips has a copy
right to it as well. But that copyright, they actually have released onto the GPL version three.
Yeah, that was a logic behind that. Logic behind that was that we actually use quite a lot of
open-source software here at Schubert Phillips, primarily one being, well, Sea of Engine,
obviously, but also Nadios Ranzet, MRTG, RRD tool, the Lambe architecture. It was about time that
we did something back. To be honest, I wouldn't have gotten this far with the program if I kept it
in my own circle. Whenever I release strangers on the system, I found that I tend to do things that
I don't expect because I had a certain mindset when I developed this. And I don't break free of that
mindset easily, whereas somebody else who comes in blank, all of a sudden finds and starts setting
illogic settings. Because it's logical to them and then I found that I need to adapt my program to
give a little bit more guidance to them. Let's put it that way. So how can any of the listeners
help out? First of all, I love feedback. So just using the product and let me know about it.
Report bugs too. And help me report bugs. There's an autonomous website, autonomous.com,
but the bug tracking system and the release system are on source forage as is the
source repository. Yeah, that's where you go to get it and file autonomous.com. You can contact me
as well. Positive and negative feedback. Highly appreciate it. Constructive. Constructive feedback
even higher. Highly appreciate it. Yeah, that's where where the best help can come out. If there's
somebody who's willing to look into the open fast compatibility, it's been on my list, it's high
on my list. But yeah, I've got nesses and open fast on it. So if somebody is interested in using
it straight with open fast, then that might accelerate getting that feature out as well.
Okay, I think we'll wrap it up there. There's just one thing I want to warn the listeners is that
if you do run nesses, there are some dangerous plugins that can bring down production systems.
So be careful. Do you ship with recommended? There is a safe configuration, but safe should
always be taken with a grain of salt. I'm not sure if that's a valid English expression.
Yeah, it's a relative safe setting that's in there. There's nobody who does any
pen test who will guarantee that any system will survive their pen test. It's always something
that they will exclude and that's also something that we have to, yeah, don't try this at home.
We're highly trained professionals, etc. Well, actually do try that home. Don't try it on your
neighbors. Okay, Frank, thank you very much. And if anybody has any feedback, you can send it to me
at can.fanon.gmail.com. Frank, you're working on an English version of the demo. We'll try to get
that up as soon as possible. Yeah, there will be a screen recording of the English version of the
demo, which will probably be posted on the Autoneses website. Okay, and your website is
www.autoneses.com. Okay, thank you very much ladies and gentlemen. And tune in tomorrow for another
exciting episode on Hacker Public Radio. Thank you for listening to Hacker Public Radio.
HPR is sponsored by Carol.net, so head on over to C-A-R-O dot N-E-C for all of us in the