Security Testing - ilya-khadykin/notes-outdated GitHub Wiki

Tools:

Identifying helpful sources of information for test planning. A security test engineer must rely on information supplied by the application itself and must learn to extract that information from the application and its I/O interfaces. Functional testing involves testing an application under its intended functionality and typical user workflow, while in security testing the test engineer frequently deals with scenarios that may not be typical for the common user. Often, exploitation scenarios cannot be derived from an application’s technical specification, but anticipating these scenarios is vital when planning for security testing.

Choosing an appropriate set of inputs. During security testing the test engineer has to consider from where the inputs are coming, how much trust can be placed in an input source, and what could go wrong if this trust is misplaced. During functional testing the test engineer will most commonly use expected inputs derived from use cases, whereas during security testing the test engineer is required to use unexpected and malicious inputs that may expose a security problem.

Automating security tests. Interfaces that are not visible to the human eye are most interesting for security testing. In order to be viewed, such interfaces often require the use of specialized tools. Security test engineers must be familiar with these tools and their limitations. During functional testing it is simpler to automate a testing process, as results for a specific test are easily verified. Establishing whether a security defect has been triggered during security testing is much harder.

Deciding when testing is complete. Choosing when to ship a product is a business decision, and a security test engineer’s job is to provide data to assist the people who must make that business decision. When carrying out functional testing, it is straightforward to determine when testing is complete, because with the help of use cases it is clear how much testing has been done. When carrying out security testing, however, deciding when to stop is more complicated because the test engineer needs to help to quantify the risk and decide whether enough testing has been done to declare the product "safe".

Types of Vulnerabilities

Data and Logic Vulnerabilities

Format String Vulnerability

Type of attacks

Stack Overflow

SQL Injection

Buffer Overflow

Testing

Fuzz Testing

There two types of fuzz testing:

  • black-box fuzz testing
  • white-bo fuzz testing

A fuzzer can rely either on mutation or generation, depending on how test data is created.

There are numerous fuzzing frameworks (such as SPIKE and Peach) which you should consider using if you need to create your own fuzzer.

Input Testing

Some of the common categories of input that should be tested include:

  • SQL queries being passed to a database (SQL injection).
  • Script code being passed to a scripting engine or Web browser (cross-site scripting).
  • Commands being passed to an operating system or to a command-line interface (command injection).

The four main information sources for security testing are:

  • The user interface.
  • Error messages and outputs.
  • Abuse cases.
  • Knowledge of how the system is NOT supposed to work.

Abuse Cases as a Source of Information

Abuse cases can often be gleaned from sources of historical information such as:

  • CERT and Bugtraq, which contain security advisories and discussions about existing and new vulnerabilities.
  • Black-hat literature.

Input

Categorizing Dangerous Inputs

Dangerous input can be categorized into the following groups:

  • Long strings and buffers
  • Format strings
  • Numeric boundaries
  • Scripts
  • Code
  • OS commands
  • "Control" characters
  • Error codes
  • Return values

Input Valnerabilities

Buffer Overflow

Format String Valnerability

Data and Logic Vulnerabilities

Failures that can be potentially exploited

  • out-of-bounds memory access errors,
  • off-by-one errors,
  • segfaults, and
  • remote code execution.
  • deadlock (an error occurring when two processes are unable to continue because each is waiting for the other to complete a task)

Automated Security Testing

The Value of Automated Security Testing:

  • Larger coverage. Coverage refers to the percentage of security-critical functions that are exercised by different inputs during a security test. Security testing tools help achieve a substantially larger coverage.

  • Regression testing. Automated security testing allows regression testing to be performed easily. During regression testing an application is tested for newly introduced bugs, usually by re-running previous tests and checking whether previously fixed faults have re-emerged.

  • Efficiency. Automated security tests are more efficient than manual testing. Automated tests are repeatable with no human intervention, as opposed to manual testing, which is far more time-consuming. Automation allows for more targeted manual testing as they disclose areas of potential weakness in various execution, trust, and I/O boundaries, which means the test engineer need not try to find all weaknesses manually, but can focus on key areas and provide feedback to threat modeling and other phases of SDL.

Quantifying the Risk

To help quantify the risk, you have to ask the following:

  • Was the process which was established to prevent and detect security flaws fully implemented?
  • How many unmitigated vulnerabilities are currently extant?
  • Is additional testing necessary?
  • What can we do to make this product safer?

Assigning Severity

Critical

  • Remote, anonymous user escalation of privilege
  • Arbitrary code execution

Important

  • DoS (low bandwidth attack, blue-screen, or long duration)
  • Local elevation of privilege
  • Information disclosure with privacy implications
  • Tampering with user data
  • Spoofing a user or computer

Moderate

  • Temporary DoS
  • Information disclosure, general

Low

  • Non-persistent or hard-to-replicate issues

Shared Assets

Identify shared assets for testing by locating:

  • Shared synchronized objects, such as events, semaphores, mutexes, and lock files created by one application and/or shared by several applications.
  • Communication channels, such as sockets, pipes, and shared memory.

Fault Injection

  • Run-time fault injection. Tools are used to get between the application and the OS to control responses from system calls and deny resources such as memory, disk space, files, and libraries.
  • Source-based fault injection. Source statements are modified so that internal data can be set to values that cause exception conditions to evaluate to true. Source-based fault injection requires access to the source code.