Friday, 27 March 2015

Penetration Testing: You’re Doing it Wrong (?) – Part One

Sexual innuendos aside, I've wanted to write an article about the unspoken thoughts of penetration testers (at least my own and the great testers I've been lucky enough to work with) for quite some time, but 100 hour weeks and international travel for work tend to get in the way! The main focus of this post is to describe the typical approaches of both the security assessment services industry and organisations that consume them. I have given particular focus to new attempts to make testing more realistic and how this compares to what I term as ‘Traditional Penetration Testing’. I also touch on actionable OSINT and Threat Intelligence, as well as Threat and Risk modelling as a function of assessments strategies. The context of this post assumes a medium to large sized organisation and at least a moderate level of maturity such as maintaining an accreditation framework such as PCI DSS or ISO27001.

What Do I Define as Traditional Penetration Testing?

I describe a traditional penetration test, in much the same way as industry and accreditation bodies do, which is in terms of application and infrastructure testing. Typically, a large organisation will have two main streams of testing that pinpoint the new and the old. Business as usual (BAU) testing, includes existing infrastructure and applications that are tested on an annual (or periodic) basis, which includes compliance driven assessments. Project-based testing often comprises wholly new applications and infrastructure that need to be assessed before going ‘live’. The two most common (in my experience) approaches to servicing this requirement are a penetration testing framework that involves multiple suppliers (often using a round-robin approach to avoid claims of inequality) and a single supplier approach. Additionally, it’s worth nothing that SMEs will typically use a single or multiple suppliers to service ad hoc requirements and will often create RFPs for each requirement.

So, Why is This Wrong?

One of the key criticisms of this approach is that it’s unrealistic. For example, performing an eight-day black box Web application penetration test in your staging environment, is the equivalent of building a temporary six-foot wall around your brand new house and attesting the security of the en-suite bathroom by getting a fat kid to lean against the front gate for a couple of hours and play knock-down-ginger with their bespectacled best friend. This criticism is something that I agree with, as tests often lack context, realism and their approach is based on improper calculation of the likelihood of attack and compromise. Often, this type of approach is justified due to the low business impact or revenue generation of the asset or cries of ‘industry best practice’. The truth is that baked-in ‘security as a feature’ - with its genesis in SDLC, secure architecture and user education – consistently provides better return on investment (not to be mistaken for cheapness) and more robust attack surfaces than atomic assessment of network sub-sets and standalone applications.

Moreover, many key elements of how threat actors will approach an attack are left out, as vectors such as (D)DoS attacks and social engineering are ubiquitously omitted from scopes. There are obviously ‘good’ reasons why this may be the case, such as cost, perceived ROI, perceived risk and legal implications. Fundamentally, most of these assumptions are wrong. There are ways of reducing these types of risks to acceptable levels for almost all scenarios - so why don’t people do it? I think the most correct answer is that the decision / policy makers don’t know how, and more importantly they don’t want to admit they don’t know how. It’s a very safe option to subscribe to conventional (circa 1995) wisdom and take an additive approach to anything new, giving a hat-tip and a wink to the industry and your peers that you’re at the forefront. This often involves tacking ‘bleeding edge’ services or appliances on to end-of-year budget surplus rather than questioning the value of what’s being done and going down the rabbit hole bottom-up (ooo err), armed with the detail of new approaches.

Another key cogitation, is the quality of testing and the ultimate understanding that’s passed on to those who design and create IT systems and infrastructures. As traditional penetration testing services are largely undifferentiated and a commodity (in the UK market), it’s difficult to know what ‘good’ is, even within the scope of a concept that is arguably broken. The industry is largely underpinned by the CESG CHECK scheme as a baseline for individual skill, with CTM and CTL status being used as metrics. In reality, the testing quality you get depends on the individual doing the testing, and not so much the company you hire (although this affects the overall experience as a consumer). As a minimum, most pen testing service providers will normally quote a framework or a baseline that they align to (such as OSTMM, PTES or NIST 800-155), have a defined methodology, and in lots of cases a checklist they follow when testing. A good example of a baseline is the OWASP top 10, used as a measure for assessing web applications. The OWASP top 10 is simply a list of the top 10 most common vulnerability categories discovered on the Internet by OWASP. The issue with these types of measures, is that your top 10, may not be the OWASP top 10 and you may be missing key tests due to lack of context and a one-size-fits all approach. From experience of testing frameworks, if a client gives a supplier 100 web applications to test over the course of the year it’s likely that most of the vulnerabilities discovered will be repeated again and again due to reuse of badly written libraries or learned mistakes during the development process and server build / configuration. How many paid-for hours could be negated by trending analysis, code review of re-used libraries, documentation of secure builds and hardening rather than testing what’s essentially the same application over and over again and finding the same issues. It’s not uncommon to see this carry over year on year, with a new testing firms being brought in to validate findings.

Conclusions

To conclude, I believe that a lot of the shortfalls in basic security hygiene come from the people running the show (read CISO, CTO, InfoSec Manager). There is a simple lack of understanding of Cyber threat / risk and how to quantify, prioritise and remediate. It’s a lot easier to not rock the boat, make a metaphorical pinkie swear with China and North Korea to the effect of ‘don’t ask don’t tell’, than to admit you don’t understand your attack surface or how to manage it.

And then?


I think that I've carried on quite enough for a single post about the issues, so I’ll be continuing in a new post on how I believe these issues can be remediated and a detailed discussion of: simulated attacks, CREST STAR and CBEST, the pitfalls of changing tact, and the risk of doing nothing.

Monday, 26 May 2014

What You Need To Know to Become a Penetration Tester

It really has been a long time since I last posted. This post is more of an essay, so it may be a TL;DR for some, but hopefully a there is some good information for those who wish to break into Penetration testing or at the very least something I can point people to next time I'm, asked.

As I’m sure is the experience of other Penetration Testers, I’m often asked (or see slapped across LinkedIn Forums) by a whole range of people “How do I break into Penetration testing?” or the like. The prospect of becoming a ‘professional hacker’ is all too enticing for graduates, IT professionals and even Information Security bods in other functional areas alike. Having answered this question and posed many a question in rebuttal, I decided to formalise my experiences, musings and advice into a single blog post. I hope it helps.

What is Penetration Testing?

To begin with, it’s always worth checking your understanding of what a Penetration tester actually is, does and some of the attributes belonging to the ‘good and great ones’. Being a Penetration tester is not the same as being (what the media often portray as) a hacker. To dispel a couple of myths about Penetration testing, it’s worth describing some of the aspects of the job in bullet points for brevity.
  • For most of the industry, a Penetration test does not involve development of 0-day exploits.
  • Normally, you get a very limited amount of time (days) to compromise systems that hackers would have months to do.
  • You always have to document your findings in a written report at the end of the test. Often, you will be asked to explain in detail some or all the findings to technical and non-technical audiences.
  • In a lot of cases, clients aren’t looking for a single route to domain admin. They’re interested in their broader attack surface and coverage across their whole estate. 
  • Business considerations, such as loss of earnings due to downtime and cost of engagement are much more important than you may initially think (or probably not consider at all before testing).

What makes a good tester?

I believe that there are quite a few considerations beyond raw technical knowledge that make a good tester. It’s key to remember again that Penetration testing is not ‘hacking’ and although there is a place for the borderline-autistic who hacks the Pentagon on their neighbours’ wireless, it’s more likely they’ll be suited to research or the Russian mafia. Again, I’ve added a bullet pointed list to describe some of what I consider key attributes of good and great testers that I consider when I hire non-experienced testers.

  • The right attitude – This is much more important than someone who’s done a course (CEH or the like). The truth is, I don’t believe that you can be a good Penetration tester if you’re not passionate about IT Security and moreover technology. It does need to be more than a job, as the up-skilling and constant personal development is not possible to maintain if your heart’s not in it. Also, you need to be fairly autodidactic (into self-learning) as even now there are few good core texts on most topics and they become outdated quickly. Most experienced tester will have gone through a huge amount of self-learning and will be resentful if you expect it on a plate (others just plain don’t want to share).
  • Good fundamentals – The best testers know a lot about a few things, but something about everything else. Despite some academiphobia (is that a word?) within the industry, I think that an IT related degree provides a great grounding in computer science and will stand you in good stead. Similarly, experienced sysadmins, network architects and developers should have good foundations on which to build. It’s common to find testers with big knowledge gaps, it’s really important to understand all areas of enterprise infrastructure to some degree in order to progress through the ranks.
  • Technical Prowess – At its core, Penetration testing is an extremely technical discipline. Not only do you need to understand how things work at a low level, you need to subvert controls in a repeatable way and learn constantly as new versions of software/hardware are released. The ability to code or script is always an advantage, even if you’re limited to simple bash scripting. However, some of the best testers I know can’t write code but can read and break very well indeed. That said, it will limit your vision and scope ultimately if you can’t. More on this later!
  • Soft and Written skills – Often overlooked, these skills are what separates Penetration testers from hackers and script kiddies. Ask yourself, would you as a client accept the work of someone who cannot write a coherent sentence, who cannot express simple issues in plain English and pay over £1000 per day for the pleasure? The key deliverable for the client is the written Report. Penetration testing companies require consultants who can read, write and speak English well. Unless you’re a total genius who’s finding 0-days nobody else can, they’re unlikely to overlook total ineptitude in this area.

Where to Start

In general, I would say that it’s almost impossible to get into a Penetration testing job at any level if you lack any exposure to computer science or hacking. It’s not like sales where you can start at the bottom with nothing and there are thousands of jobs.  
From my experience, there are four common starting points that lead people to the Penetration testing path. Apologies if this doesn’t fit your circumstances or I’ve left out anything that’s obvious.
  1. A school or college leaver who hacks as a hobby.
  2. An existing IT professional (e.g. Sys admin, Developer, Network Engineer)
  3. A recent graduate of Computer Science, Cyber Security or Ethical Hacking
  4. A graduate or experienced professional in another field
Scenario 1

In Penetration testing, having a lack of schooling and higher / further education isn’t necessarily a hindrance if you have some skill. However, you will likely need to prove this to a greater extent than perhaps a graduate would. Bug bounties are a great way to prove your prowess, with sites such as Bugcrowd paying sizeable winnings to the best bug hunters. If this is beyond your current skill level, it’s worth playing with some of the teaching frameworks such as Metasploit Unleashed or DVWA to hone your skill. It’s worth remembering that companies pay large amounts of money to recruiters, so if you approach them directly with a well-written CV, they’re likely to appreciate your enterprising spirit as well as the potential cost savings! If your CV is short or particularly unimpressive, you should pay particular attention to writing a covering letter or email. This should be well constructed and demonstrate your keenness to work for that company specifically (everyone likes a bit of mild flattery) and a bit about your background and aspirations. It’s worth noting that most (me included) organisations will throw out emails or CVs that are poorly written or full of mistakes at first sift. If you’re not grammatically gifted, ask someone who is to assist.

Scenario 2

Existing IT professionals already have quite a bit of skill (potentially) in a useful area. This is often quite attractive for Penetration testing companies as they understand general working practices and professionalism (one would hope!) and are often very keen! My advice would be to avoid expensive courses and retraining in order to land a job, but to focus on moving into a role as quickly as possible. There are plenty of cheap or free resources online to gain some basic knowledge around testing to move up to a level whereby you can hold a decent conversation in an interview or demonstrate working knowledge of Metasploit or Burpsuite on a rig. I always believe that attitude is more important than current skill-level with inexperienced hires, and spending £3k on a SANS course won’t make you start any higher in an organisation without testing experience. Great resources for this type of up-skill can be found on sites like SecurityTube, Udemy, OWASP (or just YouTube) and courses like Metasploit Unleashed and Mutilidae will help you reach initial goals.

Scenario 3

As a recent graduate, you have likely been exposed to good basics across a wide breadth of IT; some of it you may even remember. For those of you who have done a Computer Science degree and particularly enjoyed a security module or completed a Security or hacking related dissertation, you may feel the next step is to apply for a Penetration testing job. In this scenario, my advice would be to make totally sure it’s what you want to do and commit. As an interviewer, I’m always looking for a passion for security and some evidence of learning outside of University in this field, or as a minimum, a good security-focused FYP. If you’re lacking this, it will likely seem as though you don’t know what you want to do and you saw a couple of jobs being advertised for senior Penetration testers for up to £100k. For the people who’ve done a Security-related degree (or Masters) at Universities such as Abertay, Glamorgan, Kingston, Royal Holloway etc. you’re likely to get to the interview stage based on the specificity of your experience (as long as you got a reasonable grade).

Scenario 4

If you’ve no experience or qualifications in the field, then it’s likely to be a struggle to get an interview on the strength of your CV alone. I would advise the same approach as a School or College leaver, get ethically hacking and learning! An impassioned covering letter and interest in being an unpaid intern will often turn the heads of a hiring manager.

Industry Accreditation

This is probably the most frequent area of questioning I get when people ask me about moving into the industry. As the industry is so heavily focused on the CESG CHECK scheme, it’s best to separate the discussion into two parts, the CHECK scheme and other qualifications.
The CHECK scheme was initially set-up in response to the increased demand for skilled Penetration testing to be performed against HMG (Her Majesty’s Government) assets and CNI (Critical National Infrastructure) in the form of ITHC (IT Health Checks). In short, the Government didn’t have the funding to hire enough heads internally to complete all the work at the right level (or the salaries to tempt good testers away from industry). Therefore, the CHECK scheme was created as a framework to manage the subcontracting of this extra effort. A large amount of Penetration testing firms rely heavily on work from the CHECK scheme and it remains a huge source of income. Both companies and individuals can accredit to the scheme, with approved organisations deemed as ‘Green light’ should they satisfy the required criteria. 

The accreditation of individuals is quite complex, and even experienced testers still sometimes get confused by the different options. There are essentially two grades of Penetration tester within the scheme, CTL (CHECK team leader) and CTM (CHECK team member) with the former being the top tier. In order to achieve either of these levels the individual must pass an exam, hold SC clearance and work for a CHECK green light company. SC clearance can be attained by UK and Foreign nationals’ resident within the UK. However, if you’re not a British citizen you will often be caveated with ‘UK eyes only’ on your clearance, which will stop you from doing certain types of work. At the time of writing, the following table shows the possible examination paths to accreditation.

CHECK Team Leader
CHECK Team Leader (Infrastructure)
CREST Infrastructure Certification Examination (www.crest-approved.org)
Tiger Scheme Senior Security Tester (www.tigerscheme.org)
Cyber Scheme Team Leader Examination (www.thecyberscheme.co.uk)
CHECK Team Leader (Web applications)
CREST Certified Web Application Tester (www.crest-approved.org)
Tiger Scheme Web Application Tester (www.tigerscheme.org)
CHECK Team Member
CHECK Team Member
CREST Registered Tester Examination (www.crest-approved.org)
Tiger Scheme Qualified Security Tester Examination (www.tigerscheme.org)
Cyber Scheme Team Member Examination (www.thecyberscheme.co.uk)
Fig.1 - Taken from CESG website

The main difference between a CTL and a CTM (apart from the difficulty of the exams) is that a CTL must author the report and lead the testing. A CTM cannot work solo on a CHECK test. Therefore, a CTL is more valuable to a company that performs CHECK work as they can work alone, which makes scheduling more easy and it’s likely the tester is more highly utilised. The CTL grade subdivides into Web Applications and Infrastructure, however, there is currently no restriction on what type of work a CTL can perform either on their own or with other testers (i.e. a Web Application CTL can perform Infrastructure work solo).

Most testers will aspire to pass these exams, certainly when working within a CHECK Green light organisation, as their employer will either mandate or encourage it. The CTL exams are all-day and quite challenging, even for experienced testers. More information on the specifics of the exams can be found on the sites listed in the table above.

Other Industry Accreditations

There are loads of courses and accreditations out there, I have just listed and given a few lines about the courses and my opinions on each in order to give a flavour of the common ones.

CEH – The general consensus of testers in my experience is that CEH lacks technical detail and isn’t really a very good qualification for a Penetration Tester. However, I have not completed this exam myself and base my opinions on the experiences of trusted colleagues and the course syllabus and examination guides. I wouldn’t recommend this certification if you wish to be a Penetration tester.

OSCP / PWK by Offensive Security – This is a pretty technical course, which many of my team and colleagues have done. The course uses Kali Linux as a base operating system and runs you through a challenging series of labs exploring scripting in bash and python, the basics of exploit development and loads more. It’s online and give you VPN access to the various labs required for the course. In order to gain the OSCP qualification, you are required to submit coursework from the labs and complete a challenging exam over the course of 24hrs. Feedback on this has always been that it’s awesome and very challenging.

SANS Courses – Again, I have never been on a SANS course, but I have seen a lot of the material (having put some of my team through these), know a few instructors and have heard good things across the board. My one criticism is that most of the exams are multiple choice and not practical-based, which is a bit disappointing. That said, the instructors of the on-site courses are always excellent and if you can keep up you’ll learn a lot. I believe that it is more of a standard in the US to go after the GWAPT or GPEN exams.

BEWARE recruiters

I decided not to do a section on CVs, as I felt it was a bit patronising. Suffice to say, don’t send your CV across with any mistakes (spelling, grammar or otherwise), don’t leave any unexplained gaps and don’t in clude huge sections on your non-security related hobbies.
I supposed that finding a job often starts with recruiters, or at least LinkedIn. I have a hard time working out which is which nowadays, but it’s where I shall begin. For a lot of companies, recruiters are a necessary evil, but it’s best to attempt to avoid going through one unless it’s a necessity (some larger companies may mandate this route).

I have literally hundreds of horror stories about recruiters in Penetration testing, which I won’t bore you with, but be wary of sending them your CV (and moreover them getting hold of your CV, sending it to loads of companies then ringing you to see if you’re interested in any that bite). I would advise that you take their advice with a pinch of salt as most don’t understand the market. You can waste a lot of time being sent to far-flung places for inappropriate jobs that a recruiter persuaded you was a good idea (in order to make their quote of candidates to interviews). If you’re unsure, don’t be afraid to ask to speak to the company on the phone first to clarify the role is suitable.

I would advise that you approach reputable companies directly, you can find a list of CHECK Green light companies here for reference. 

If you’re following my advice and taking a direct approach, I would always advise that you check the company website first to see if they mention hiring on there. As I mentioned earlier, most organisations will see it as a positive thing if you track down the right person (or at least an appropriate one) on LinkedIn, send them a bespoke email introducing yourself and asking them about hiring and if they’d consider you for an interview. This saves them money, and shows a bit of initiative on your part.

The Interview Process

Most interviews I’ve done (as an interviewee and interviewer) for testing roles have all followed a fairly formulaic approach. They usually follow a two or three stage process with a mix of the following aspects:
·         
  • Phone-based – Some simple technical questions (Nmap flags, Port numbers, What’s a XSS? etc.) and to check you speak good English and have a brain.
  • Face-to-face (Technical) – This normally is a series of technical questions and a technical assessment on a rig of some kind, often with you explaining as you go or the interviewer(s) watching on another screen what you do. This can be very intimidating as they tend to bring in very senior technical people to watch. Normally this will be pretty basic like popping a box with MSF (using MS08-067) or basic XSS.
  • Face-to-face (Presenting / meet-the-director) – You may be asked to present a Penetration test report or prepare something to assess your interpersonal skills. Depending on the company, you may need to meet a higher level business representative such as a director or partner.
  • Face-to-face (General) – A general interview will have a mix of discovery questions, often using HR methodologies such as the STAR model. This type of interview will seek to find out a bit more about you as a person and your aspirations to check for a cultural fit.

Before you go to the interview, it’s always worth asking what will be required and if you need to prepare anything. Interviews are an opportunity to sell yourself, mention anything related to the field of penetration testing that you have done. I recall one specific interview with a very shy gentleman, he really struggled explaining to me some really basic concepts such as what port scanning was and had never heard of the OWASP top 10; things were taking a nose-dive. Suffice to say, he hadn’t done well and didn’t appear to have a clue about anything in security, after having quite a promising CV. At least until I asked him the question ‘what do you do in your spare time?’, to which he answered “oh, I like exploit development and malware reversing normally, you know that bug that was found in IE last week, that was me, I got £10k from Microsoft for that”. Totally mental that he didn’t mention that, even after I asked him the ubiquitous ‘tell me about yourself?’ at the start of the interview.


I hope that my post has provided a useful amount of information to aspiring Penetration testers, if you have any comments, criticisms or more questions please leave them below.

Monday, 18 February 2013

De-duping multiple interface nessus results with sed.

A bit of a mouthful and not that useful for most, but this is saving me headaches left, right and centre at the moment (and is dead simple).

It's always an issue when testing a network that you can run into the same box multiple times with different addresses, this became all too apparent to me recently when I was testing 4 boxes with over 20 interfaces between them serving up different services. Now when it comes to reporting the customer isn't going to want to know about the same issues on the same ports on the same box multiple times, but  manually separating this lot out of Nessus is a nightmare.... sed to the rescue.

Lets assume that you have your Nessus output and have it it some useful parse-able format. (xmlstarlet anyone?)

lets also assume that you have a list of ips that match up to each hostname. First things first, create a ip2host.sed file and fill it with your replace statements, e.g.

s/192.168.0.1/host1/g
s/192.168.0.2/host1/g
s/192.168.0.3/host1/g
s/192.168.0.4/host1/g
s/192.168.0.5/host2/g
s/192.168.0.6/host2/g
s/192.168.0.7/host3/g
s/192.168.0.8/host3/g

Next step is nice and simple, either:

sed -f ip2host.sed << EOF | sort | uniq

and copy and paste your results into the terminal, ending with an EOF or...

sed -f ip2host.sed < fileofservices.txt | sort | uniq

if you've already saved the file. This will take:

192.168.0.1:443
192.168.0.2:443
192.168.0.3:443
192.168.0.5:25
etc

and convert it to:

host1:443
host2:25
etc.

Not a complicated one today, but always a handy one to remember.

Ben





Thursday, 15 November 2012

Proxying 3G iPhone Data



Hey! It's been a while, I promised you guys that I'd do this more often and I've failed you and for that I am sorry (well sort of). So today I'm taking a break from automation to talk to you lovely folks about something I've been working on lately, proxying. Not just proxying, but proxying iPhone apps. No wait, not just proxying iPhone apps, but proxying iPhone apps traffic over 3G. Is there a setting for that? No! (At least there isn't in iOS 5, 6 does, but only through the configuration utility)

It was a pain in the ass, but it is possible with caveats. Firstly, the iPhone has to be jailbroken, secondly, you need to edit some config files. If you're cool with that read on.

Step 1

Jailbreak your phone.

Step 2

Edit the /private/var/preferences/SystemConfiguration/preferences.plist file.

Locate the "ip1" section:

<dict> 
<key>Interface</key> 
<dict> 
<key>DeviceName</key> 
<string>ip1</string> 
<key>Hardware</key> 
<string>com.apple.CommCenter</string>
 <key>Type</key>
 <string>com.apple.CommCenter</string>
 </dict> 

Then add the following section afterwards:

<key>Proxies</key> 
<dict>
 <key>ProxyAutoConfigEnable</key>
 <integer>1</integer>
 <key>ProxyAutoConfigURLString</key>
 <string>file:///private/var/preferences/proxy.pac</string> 
</dict>

Step 3

Create the following file: /private/var/preferences/proxy.pac

and add the following.

function FindProxyForURL(url, host)
{
return "PROXY YOUR_EXTERNAL_IP:8080";
}

Note: As this is over the 3G network, your proxy needs to be available on the internet, if you're planning on using burp I'd probably use a netcat tunnel to use your proxy on a box you have on EC2, alternatively just open up a port on your home router and use that.

Step 4

Fire up your proxy and restart your phone, it doesn't get much simpler than that.

Step X

Something I've been doing to make app testing a bit easier is use Veency, it's a VNC tool (available on Cydia) for your iPhone allowing you to interact with it via your PC, it makes life a lot easier when you have full use of your keyboard and mouse on your phone.

Proxying 3G traffic actually yielded some interesting results, certain apps that weren't even active authenticated (over plain text) with their servers on phone boot. I won't give away who here, but they have been notified that this is bad.

Hope that was somewhat useful, it was for me anyway, until next time, come say hello @bdpuk.

Byeeeeeeeeeeee.

Tuesday, 17 July 2012

PaulDotCom Interview

A big thanks to Paul and Mike and Larry (and Carlos) for having us on the show, we really enjoyed it. Apologies for being a bit up-tight in places, but we're British, it's what we do. And, for the record, I like Nessus really (Printers don't) and SANS rock (apart from their examination style). You can check out the video of us chatting shite here: http://pauldotcom.com/2012/07/pentesticles-penetration-testi.html I'll be emailing Paul with some better 'your Network's shite' comments in the London vernacular! PENTESTICLES AWAY!!!!

Thursday, 12 July 2012

HackArmoury.com - A Pentesticles Project!


Recently, we at Pentesticles took over the ownership and full development of HackArmoury.com. So, I thought it was time to write a blog post about it and speak a bit about what it does, how to use it and what we’re planning for it in the future. We'll be talking a bit about this tonight (12th July 2012) at 11pm UK time (6pm ET) on pauldotcom also, make sure you don't miss it!.

HackArmoury is something I’ve personally been involved in since its creation (by me ol’ mucka @nopslider) and has proven to be a useful resource for the Penetration Testing community. Ben and I are now putting a bit of focus on it and continuing its development and maintenance. I've also skinned the site since the change over, I'm still not sure about the Tango-orange colour. It's not a dig at gingers, honest.

So, what is HackArmoury? For those who haven’t used it, it’s essentially a tool repository for Ethical Hacking and Penetration testing. The key advantage is that HackArmoury can be accessed over loads of popular protocols including SVN, TFTP, HTTP, IPv6 and Samba (see below for the full list and instructions) and older versions of tools are maintained. This means that if there are network restrictions on where you’re trying to update from, you have the best chance of being able to connect and get your tools.

Another key feature of the site is that the entire repository of tools is packed into a single ISO, which can be downloaded directly. Each time a new tool is added, the ISO is updated and re-packed meaning that it’s always up-to-date.

Our next addition with be GIT, as this is an obvious hole. Once we sort the technical aspects and work out the security implications, we'll be ready to go! 

We're always looking for trustworthy contributors, so if you fancy helping us tool-up, please drop me an email at lawrence[at]hackarmoury.com or through the comments on this blog. In the meantime, I hope you enjoy using the site and it proves useful.

How can I connect? There are lots of ways to connect up, you can do this via the following methods:


IPv6 is now supported by HackArmoury (2a02:af8:1000:8c::2f98:4ed7). If you want to access us directly over IPv6, and you can't remember a 128bit address, use the hostname ipv6.hackarmoury.com. All of our common protocols will be supported.


You can access all your tools straight over Samba using \\hackarmoury.com\tools\. No authentication required, just start->run->\\hackarmoury.com\tools\ and you're away.

For example, to run nc.exe, simple type \\hackarmoury.com\tools\all_binaries\nc.exe. If running on a Windows host with executable black listing or whitelisting, it's always worth testing over Samba too. In many cases this execution method is permitted without consideration for the consequences.

HTTP

Everything in the toolkit is browseable over HTTP and HTTPS. Navigate directly to http://hackarmoury.com/tools  and you're away.


To minimise download bandwidth, you can keep up-to-date with our tool set over Rsync. Use the following command to download after reading our licensing terms here:

rsync -avz rsync://hackarmoury.com/tools /ha

As with all other protocols, no authentication is required to download.


You can keep an offline copy of the armoury simply by doing a subversion checkout. If you're regularly running the tools, it makes much more sense to keep an offline copy for speed and portability. It’s a much more efficient way of keeping up-to-date with new tools, as you don't need to be scouting around the site or downloading large ISO images.

Simply type:

svn co svn://hackarmoury.com/live /ha

To update, navigate to your local directory and perform:

svn update


Executable files only are available over TFTP due to the inability of having a directory structure, and you must know the name of the file in advance.

You can download files like this:

tftp -i hackarmoury.com get nc.exe

You may find this useful in some poorly implemented egress filtering scenarios.

Tuesday, 29 May 2012

We Have the Port Scans, what now?


It's been a while, I hope you're good. I'm fine thanks, busy as sin but isn't that always the way? So where did we leave off? From reading back through my previous post, we'd scanned our little guts out and pulled a list of all ports that were open and all the services that can be interacted with. Boy haven't we been busy! 

It just so happens that now is when the real fun begins. Have a bit of a perouse through the results, not that easy to read aye? Sure we can quickly find some 135, 445 and have a quick fiddle through the lovely lovely file shares, but where's the automation? This post should cover some basics about gathering even more data from the services we've identified using our ever faithful set of tools such as nikto, gnome-web-photo, curl et al and keeping the data useable.

First things first, let's bring all of our results together in a more machine readable way. From the previous post we've grabbed all of our nmap output into the three decent formats, plain, greppable and xml. For the purposes of this post we'll be using the xml format and parsing it using xmlstarlet (for those of you that aren't already using starlet, grab a copy, it's a brilliant little command line parser that I can't live without, nessus, nmap, surecheck, anything that dumps xml suddenly becomes friendly to use again!)

The little gem i've been using for a while now is:

cat port_scans/hot-targets.tcp.services | xmlstarlet sel -T -t -m "//state[@state='open']" -m ../../.. -v address/@addr -m hostnames/hostname -i @name -o '  (' -v @name -o ')' -b -b -b -o "," -m .. -v @portid -o ',' -v @protocol -o "," -m service -v @name -i "@tunnel='ssl'" -o 's' -b -o "," -v @product -o ' ' -v @version -v @extrainfo -b -n -| sed 's_^\([^\t ]*\)\( ([^)]*)\)\?\t\([^\t ]*\)_\1.\3\2_' | sort -n -t.

This is a slightly bastardised version of this oneliner brought to you from the lovely folks at redspin. It takes an nmap xml output file (singular in this case) and creates output like this:

10.13.37.10,22,tcp,ssh,OpenSSH 4.3protocol 2.0
10.13.37.10,2301,tcp,http,CompaqHTTPServer *** httpd
10.13.37.10,2381,tcp,http,Apache httpd SSL-only mode
10.13.37.10,3260,tcp,iscsi,
10.13.37.10,5988,tcp,http,Web-Based *** httpd
10.13.37.10,5989,tcp,https,Web-Based *** httpd
10.13.37.11,427,tcp,svrloc,
10.13.37.11,443,tcp,https,VMware ESXi Server httpd
10.13.37.11,5989,tcp,tcpwrapped,
10.13.37.11,8000,tcp,http-alt,
10.13.37.11,8042,tcp,fs-agent,
10.13.37.11,8045,tcp,unknown,
10.13.37.11,80,tcp,http,
10.13.37.11,8100,tcp,tcpwrapped,
10.13.37.11,902,tcp,vmware-auths,VMware Authentication Daemon 1.10

Isn't this a lot more greppable than -oG ? Having the data dumped out to csv allows us to rapidly move through and select the exact services we want to interogate. An example:

root@bt:~# cat output.csv | grep http
10.13.37.10,2301,tcp,http,CompaqHTTPServer *** httpd
10.13.37.10,2381,tcp,http,Apache httpd SSL-only mode
10.13.37.10,5988,tcp,http,Web-Based *** httpd
10.13.37.10,5989,tcp,https,Web-Based *** httpd
10.13.37.11,443,tcp,https,VMware ESXi Server httpd
10.13.37.11,8000,tcp,http-alt,
10.13.37.11,80,tcp,http, 

An even better example:

root@bt:~# cat output.csv | grep http | cut -f 1,2 -d "," | tr "," ":"
10.13.37.10:2301
10.13.37.10:2381
10.13.37.10:5988
10.13.37.10:5989
10.13.37.11:443
10.13.37.11:8000
10.13.37.11:80

An even better example still:

root@bt:~# cat output.csv | grep http | cut -f 1,2 -d "," | tr "," ":" | while read line; do /pentest/web/nikto/nikto.pl -config /pentest/web/nikto/nikto.conf -h $line -output $line.txt; done
....snip....

You get the idea.

While we're on the subject, a nice precursor to nikto is a bit of web scouring. We've all been in the situation before where we've been on a internal test with limited time only to discover 100 webservers spread across the network, it's always a case of best efforts and being left wondering if the ones we missed were the ones that would've bent over. This is where webscour comes in. It's a little script from Geoff over at Cyberis (Available here) that given a list of addresses grabs screenshots (using gnome-web-photo) and header information from each webserver and produces a handy html file to view them all from. Suddenly all the default content and HP OpenViews can be found quickly and we can move straight onto the Accounting App running Classic ASP on IIS4 that hasn't been in use for 5 years.

Incidently this can be run in a similar fashion to the one liner above:

root@bt:~# cat output.csv | grep http | cut -f 1,2 -d "," | tr "," ":" | ./webscour.pl webservers.htm

As you can imagine there is a lot more we can do with webservers, dirbuster, skipfish the sitekiller and any other forcedbrowse/fuzzer can usually be used in this way, and as always the more data the better.

Next steps as far as web servers are concerned usually involve getting this information into Burp, that way we can play with it properly. Buby is a sensible choice and further down the line we'll look into automated spidering and active scanning from the terminal, replaying nikto/dirbuster output directly into Burp and also utilising the FuzzDB to profile out any CMSs we come across. But that unfortunately will have to wait for another post.

As usual any thoughts, critiques or straight forward calling out either head down to the comments or hit me up on twitter. Hwyl am Nawr!

Final Thoughts

It wouldn't be fair if I weren't to go off topic at least once in a post so here you go....

cat output.csv | grep 161,udp | cut -f 1 -d "," | /pentest/enumeration/snmp/onesixtyone/onesixtyone -c /pentest/enumeration/snmp/onesixtyone/dict.txt -o onesixtyone.out -i -

Have some snmp fun ya'll!