By Dan Cornell
There was a question on an application security mailing list I am on about the current landscape of application security tools - what were they good for, is static analysis better than dynamic analysis, etc. Here is my unedited reply:
We get asked that question quite often and here is my (short)
answer:
Application security tools are really good at what they do and
they are getting better. Also they are pretty much useless for the things
that they don’t do and that isn’t likely to change any time soon.
Tools are great for finding technical flaws in applications –
the types of flaws that deal with configuration, input handling and output
formatting issues. Things like SQL injection and cross site scripting.
Tools are typically really bad at finding logical flaws in applications – these
are the issues that deal with business logic, authorization and trust
issues. Examples would be insecure direct object references or passing
the price to an e-commerce application in a hidden variable. Tools have
no way to gauge intent so there are flaws that slip by.
Another good thing about tools is that they don’t get tired or
sloppy – they can chew through large applications and large code bases and
apply rules on a consistent basis. Organizations would do well to use
tools as an initial step in assessing the security of an application but they
MUST realize that no tool is going to be able to demonstrate that an
application is secure. To get to any reasonable level of assurance, tools
must be augmented by manual testing and inspection.
It has also been interesting to watch the tools popular in the
industry evolve over time. The early tools were black box scanners
(Watchfire, SPI Dynamics). These look and act pretty much like Nessus or
ISS but for web applications. This makes sense because the first folks to
really get concerned about application security were the information security
folks in an organization and these people tended to have network and sysadmin backgrounds.
They are used to scanning an operating environment to find flaws.
As the “problem” of software security has started to be adopted
by the software development groups in an organization we have seen an upsurge
in code-level analysis tools (Fortify, OunceLabs). That is because
software development folks have software development backgrounds (naturally)
and they are comfortable dealing with code and compilers.
As for what tools an organization should use – it is important
to look at what group is going to be running the tools. If the security
group is full of folks with network and system admin backgrounds it is unlikely
that they are going to become proficient enough .NET or JEE coders to get a lot
of utility out of a source code analysis tool. Also it might be hard for
them to get the software development groups to turn over source code (for some
reasons most enterprises we work with are a little sensitive about shipping
that all over the place…) For groups like that the dynamic analysis/scanner
tools allow them to get a solid analysis of the security state of the tools in
their portfolio.
If the application development group is going to be running the
tool, using tools that can take advantage of having full access to the source
code makes a lot of sense. The scanner tools have to make a guess as to
whether or not the request/response pattern they have seen is an actual
vulnerability. The source code analysis tools have a much more clear view
of what the software is actually going to do. For groups like these the
source code analysis tools make a lot of sense. At the end of the day for
large organizations it often makes sense to have both types of tools in use to
get a “second opinion” of the security state of their applications.
Obviously this answer is full of simplifications and
generalizations and solutions for one organization won’t necessarily work for
others. But I did say this was my short answer ;)
I forgot a couple of points so I had to post a follow-up:
Couple of things I forgot to mention:
The tools don’t run themselves. If you buy a tool you should
know how you are going to modify your processes to make use of it. Then
you need to make sure the folks who will be using the tool know how it actually
works and can operate it. And it pays to check back with those people
after a while and make sure that the tool is actually being used and is being
used as it should. Otherwise you can spend as much money as you like but
your applications will not get any more secure. You might fool a couple
of auditors – but you won’t fool the attackers ;)
I suppose I could shrink that down to an even shorter answer:
Use application security tools to apply consistent checks for technical application flaws. Do not expect to be finished after you run one. Be sure that you know how you are going to use the tool before you buy it and train the people who are going to use it.
Perhaps a post at a later date will examine why I seem to end all of my emails with the silly winky emoticon ;)
--Dan
dan _at_ denimgroup.com
