Posts tagged consulting

Technical and Political Boundaries of Security Assessments


Agreeing on the scope of a security assessment, such as a penetration test, is easier said than done. Define the scope too narrowly, and you will miss the vulnerabilities another attacker could have exploited. On the other hand, an overly-wide or ambiguous scope can unnecessarily increase the cost of the assessment and put at risk careers of people involved with the project.

Jake William’s recent blog posting Penetration Testing Scope - Murky Waters Ahead! highlights the challenges associated with determining what may be examined by the tester. For instance, if the assessment is supposed to exclude web applications, should a PHP vulnerability that allows remote code execution be excluded as well?

Defining the scope is as much of a technical as it is a political issue. Many assessments focus exclusively on “web applications” or only on “network services,” instead of combining the two efforts, because these resources are usually maintained by different teams. As the result, the funding for these assessments comes from different sources and different people will be blamed for security problems.

Unfortunately, this means that many penetration tests fail to adequately mimic a typical attacker’s actions, because the scope of the assessment is artificially constrained.

One way to determine whether a particular vulnerability is in scope is to ask the client whether they are the ones responsible for patching that vulnerability. That’s not satisfying , I know. The good news is that the assessor can still provide value by asking questions about the vulnerability, explaining its significance and highlighting the need for someone within the client’s organization should take a look at it.

A well thought-out penetration test will define the rules of engagement in a realistic manner without drawing these artificial boundaries. A consultant should strive to define the rules of engagement and the scope in this “fused” manner when selling the assessment. Similarly, organizations should combine these types of testing efforts into a single engagement, even if it means that two different teams need to collaborate and pool their budgets.

In a follow up to his post, Jake pointed out that talking to “the customer is ultimately the correct way to obtain ultimate resolution on the issue.” He explained that they report all issues they “find (even if they happen to be out of scope). … this provides a real value added to the client.”


Lenny Zeltser

3 Reasons Why People Choose to Ignore Security Recommendations

There are several reasons why information security recommendations are ignored. When I outlined the rationale for this in an earlier article, I did not account for one important reason that’s grounded in psychology: people often choose to ignore information, electing to stay ignorant. In the paper Information Avoidance: Who, What, When, and Why, researchers offer several explanations for such practices.

The researchers define information avoidance as “any behavior intended to prevent or delay the acquisition of available but potentially unwanted information.” According to the paper, people may choose to avoid information because:

(a) the information may demand a change in beliefs,
(b) the information may demand undesired action, and
(c) the information itself or the decision to learn information may cause unpleasant emotions or diminish pleasant emotions.

These reasons for information avoidance are frequently present in situations where the organization conducted or commissioned an information security assessment. The the assessment is likely to trigger the concerns that will motivate its recipients to avoid reading or understanding the assessment’s findings.

Beliefs that might be challenged by the assessment:

  • My IT infrastructure is secure
  • I can write code that’s free of bugs and vulnerabilities
  • My anti-malware defenses are working well
  • I am an unlikely target of computer attacks

Undesired actions that might be prompted by the assessment:

  • Security patches need to be applied throughout the environment
  • The software development process needs to be overhauled to incorporate security
  • Staff needs to be trained to improve information security-related skills
  • The budget for information security needs to be increased
  • The strategy defined for the information security program needs to be revamped

Unpleasant emotional situations that might arise due to the assessment:

  • I have two “fight” with the management team to increase the security budget
  • I don’t know how to secure information
  • I spend money on the wrong information security products
  • I look bad in front of my colleagues

The relevant importance of these concerns and the extent to which they come into play varies across situations. Yet, these psychological factors of information avoidance explain not only why the findings of a security assessment may be ignored, but also why organizations may be hesitant to conduct such an assessment in the first place. What can the organization do to avoid this? Can the people conducting the assessment do anything to combat this tendency?


Lenny Zeltser

4 Reasons Why Security Assessment Recommendations Get Ignored

If you’ve had the opportunity to perform a security assessment, you probably know the frustration of seeing your earnest recommendations being ignored. You may wonder whether the recipient even read your report. If they did, surely they would share your concerns regarding the risks you discovered. You are not alone in your grief.

You might be able to do something about this situation if you understand why security assessment findings are often dismissed:

  • Maybe your report was never read. There could be many reasons for this, including the reader commissioning the assessment to merely mark-off a check box to say it was done. Another reason might be that the report was too long, seemed too technical or too high-level, or was otherwise poorly written.
  • The reader may not have believed you. Perhaps you didn’t provide enough evidence to support your conclusions. Even if you were thorough, people often ignore the arguments that go against their point of view. As the result, the reader might have interpreted your findings in a way favorable to himself, or disagreed with the recommendations that went against his point of view.
  • Possibly, the organization hasn’t gotten around to acting upon your recommendations. IT personnel tends to get caught up fighting fires: responding to emergency issues, fixing problems and otherwise doing unplanned work. This doesn’t leave time for remediating security issues. Sometimes remediation doesn’t happen until a security incident.
  • Perhaps the advice in your report wasn’t practical. You meant well when you advised removing administrative rights from all user accounts, but the security leader might not have had the political power to pull that off. Or maybe you emphasized only important strategic issues that were too hard to handle without starting with easier, more tactical wins.

As you can see, some of the factors that affect whether the organization will follow your recommendations aren’t under your control. Yet, you can increase the likelihood that your findings will be acted upon if you write a strong report, offer practical advice, substantiate your findings, balance tactical and strategic recommendations, and go over your conclusions and remediation approaches in person or on the phone.

This note is part of a 4-post series on creating security assessment reports. For more, see:

For more on the topic, see my Tips for Creating an Information Security Assessment Report Cheat Sheet.

Lenny Zeltser

4 Tips for a Strong Executive Summary of a Security Assessment Report

A small subset of the audience for which a security assessment report is intended will actually read the whole document. The majority will only have the patience for the first page. That’s why it’s important that the report start with a strong executive summary. Here are 4 tips for writing an executive summary that gets read, understood and (hopefully) acted upon:

  • The summary has to make sense to a non-technical audience. Remember that it’s meant to be read by “executive” managers. Resist the urge to describe the details of exploits and avoid security jargon. At the same time, make sure that the accuracy of the statements you make there can hold water with the technical audience who will also read the report.
  • The summary should to have relevance to the company’s business. Outline the significance of your findings in the context important to an executive manager. That means referring to items such as risks, compliance requirements, metrics, contractual obligations, business processes. Otherwise, the reader might label the assessment’s findings irrelevant.
  • The summary must be brief, in most cases fitting into a single page. It’s much harder to write a short text than a long one, but they call it a “summary” for a reason. Write it in a way that allows the summary to stand on its own, as it might be distributed separately from the rest of the report. Use bullet points.
  • The summary needs to be specific. People put more trust into text that uses concrete statements. Avoid passive tense. Provide numbers rather using abstract words like “some” or “many.” Put effort into sounding confident, so that the reader accepts your findings. Be clear about your findings and your recommendations for addressing the issues.

The executive summary will be the part of the security assessment report that will be read most often. Take time to craft it so that it is readable by executives who care about business, have little time, and think in terms of actions. The effort invested into creating a strong executive summary will pay off at the end.

This note is part of a 4-post series on creating security assessment reports. For more, see:

For more on the topic, see my Tips for Creating an Information Security Assessment Report Cheat Sheet.

Lenny Zeltser

6 Qualities of a Good Information Security Assessment Report

Not all information security assessment reports equal. Many present irrelevant details and are tedious to read. They often miss the opportunity describe the risks and remediation approaches in a way that the assessment’s beneficiary—be it an external client or an internal group—can understand and act upon.

Even if the execution of the assessment tasks was flawless, the perceived value will be based to a large extent on the quality of the report that represents the project’s deliverable. Here is my list of 6 qualities of a good information security assessment report:

  • Starts with a strong executive summary that a non-technical reader can understand. Given people’s short attention span and time limitations, there’s a good chance that most readers won’t get past the executive summary. Moreover, the executive summary is often the part of the report that is distributed internally beyond the group that commissioned the security assessment.
  • Provides meaningful analysis, rather than merely presenting the output of assessment tools. The value that an experience assessor brings is in making sense and deriving meaning from the collected data. As the result, the report should narrate the assessor’s observations and conclusions.
  • Includes supporting figures to support the analysis. Such details should be included to substantiate the findings, so that the reader can confirm that the observations are based on factual data and, in some cases, to allow the reader to replicate the discovered vulnerabilities.
  • Describes assessment methodology and scope. Don’t assume that the reader will be aware of the initial discussions regarding what should be tested and how. Moreover, the report should describe the tools, approaches and techniques that the assessor employed, so that the reader can be confident in the professional and systemic approach to the project.
  • Looks professional and is without typos. Though the substance of the report isn’t directly affected by the document’s look-and-feel, it’s hard for the reader to take seriously a document that looks sloppy and unprofessional. Moreover, typos distract from absorbing the text’s meaning and can offer an excuse to cast doubt on the assessor’s capabilities.
  • Is structured in logical sections to accommodate the different groups who will need to read and act upon the report. Though some readers will be motivated to pay attention to the whole document, many might only care about some aspect of the assessment (e.g., application or infrastructure security). Also, the recipient might wish distribute the report’s contents on the need-to-know basis.

Of course, there’s more to a good security assessment report than the tips I offered above. My goal was to point out the qualities that I often see lacking missing from the reports that come across my desk. Keep these points in mind when creating a document to describe your findings and recommendations and when evaluating a group you might engage to perform a security assessment.

This note is part of a 4-post series on creating security assessment reports. For more, see:

For more on the topic, see my Tips for Creating an Information Security Assessment Report Cheat Sheet.

Lenny Zeltser

Security Assessment Report as Critique, Not Criticism

The goal of most information security assessments is to identify vulnerabilities and recommend ways to address them. The resulting report tends to be filled with criticism. Even when the document is filled with insightful observations and advice, it’s often viewed defensively by the readers, who feel like they are under a personal attack.

To create an assessment report that is more likely to be accepted by the readers and that provides more constructive advice, write it as a critique rather than criticism.

In an essay What is Critique?, Judith Butler pointed out philosopher Raymond Williams’ concern that the practice of criticism has been unduly restricted to “fault-finding,” which lead him to propose that we find a vocabulary that does not “assume the habit (or right or duty) of judgment.” The notion of critique involves providing a well-rounded assessment of the subject’s structure, rather than personalizing the identified issues.

A security assessment report that offers critique, comments on the factual findings, on the processes that contribute to the security issues and on the structure of the organization that may need to be adjusted to improve security. This means staying away from chastising specific individuals, unless you are prepared to deal with their anger and defensive counter-accusations. An angry reader will ignore the report’s key messages.

Another element of a critique-focused report involves the discussion of positive findings of the assessment. As the saying goes, a spoonful of sugar makes the medicine go down. Furthermore, seeing what aspects of security you liked, will help the organization learn from what is working, so it better understands how to address the processes that aren’t. Positive reinforcement is often even more effective than negative reinforcement in changing behavior.

This note is part of a 4-post series on creating security assessment reports. For more, see:

For more on the topic, see my Tips for Creating an Information Security Assessment Report Cheat Sheet.

Lenny Zeltser

6 Tips for Hiring and Working With Security Consultants

Sometimes organizations need outside help for getting their arms around information security challenges. That’s where security consultants come in. Here are a few tips for making sure that engaging a consultant—often in the form of a consulting company—brings the necessary benefits to justify the expense.

This advice isn’t specific to security consulting, but I present it on the basis of providing security consulting services for a fair bit of time:

  • Understand your requirements for security assistance. I’ve encountered many organizations, for example, that request a security assessment, but aren’t sure what type of an assessment they need. Good consultants will help you figure this out, but the more prepared you are, the more in control you will be.
  • Reach out to multiple security consultancies, so that you benefit from multiple perspectives on potential approaches to the project and to validate that the price estimates you receive are reasonable. Consider issuing an RFP to attract several qualified companies and to explain your needs.
  • Make it personal.  Treating the company as a generic entity may be fine from a contract’s perspective. However, establishing rapport with the individuals working on the project will help achieve results not only to the letter of the contract, but also in the spirit of your requirements.
  • Assess who will work on the project and what tasks they’ll be assigned to confirm that their expertise matches your needs. Look at their certifications and past project experiences. The consulting company might have a strong brand as a whole, but make sure that it extents to the consultants who will support your endeavor.
  • Request a high-level project plan that contains milestones for the expected deliverables. Organizations sometimes focus too much on the desired start date, forgetting to state their end date expectations and to incorporate intermediate check points into the project’s time line.
  • Understand the full cost of the project, rather than focusing on the hourly rate. Fixed-fee projects are least likely to have unexpected expenses. Pricing on Time & Material basis is may be necessary when the scope of work is unclear, but could be risky from a budgetary perspective. Also, don’t forget to account for travel expenses.
  • Dedicate time to oversight of the project’s delivery. The consulting company should offer such oversight, but you should also keep an eye on the project to make sure it is progressing according to your expectations and original commitments. This will help you identify and correct potential problems early before they escalate into major issues.

If you’d like to share additional tips, either from a security consultant’s or a client’s perspective, please leave a comment.

Lenny Zeltser

Perception of Value in Security Consulting Projects

For consultants, it’s not enough to do great work for their clients. The clients also need to understand the value received from the service to truly appreciate the work. For instance, a security consultant might have been highly skilled and thorough at performing a penetration test. Yet, the client might be unhappy unless the pen tester’s report and related communications clearly describe not only the project’s results, but also the methodology and effort that went into it.

Behavioral psychologist Dan Ariely pointed out that “perception of value is often not about what we’re getting. It’s about how much effort the other person is putting in.” Dan described a locksmith who would receive great tips and praise when he was still inexperienced and took a long time to open a lock. Now that the locksmith mastered the skill and can open locks in seconds, his customers complain about high fees and don’t tip.

Dan also described a study that assessed how much people were willing to pay for a service to recover data from a crashed computer. You might theorize that the amount would be tied to the amount of data the person was at risk of permanently losing. Instead, people’s willingness to pay was mostly a function of the time the specialist put into the recovery process.

Since clients are rarely able to understand the intricacies of the work that requires specialized skills, they seem to estimate value by assessing the effort (usually time) that went into the project. I’m not suggesting that you should artificially stretch the time to conduct a pen test. Rather, I recommend making sure that your written and verbal communications allow the client to understand the effort you put into it.

This is another reminder that communication abilities are no less important than elite hacker skills.

Lenny Zeltser