penetration-testing-methodology

---
name: penetration-testing-methodology
description: Apply systematic penetration testing methodologies including reconnaissance, scanning, exploitation, and reporting. Use for security assessments, red team exercises, or vulnerability discovery.
license: MIT
---

# Penetration Testing Methodology

Apply systematic penetration testing methodologies to identify vulnerabilities and assess security posture through simulated attacks.

## When to Use This Skill

- Conducting security assessments or audits
- Performing red team exercises
- Validating security controls effectiveness
- Meeting compliance requirements (PCI DSS, SOC 2)
- Testing application security before deployment
- Assessing infrastructure security
- Training security teams

## Core Concepts

### Penetration Testing Phases

1. **Planning & Reconnaissance**: Define scope, gather intelligence
2. **Scanning**: Identify targets, enumerate services
3. **Gaining Access**: Exploit vulnerabilities
4. **Maintaining Access**: Establish persistence
5. **Analysis & Reporting**: Document findings and recommendations

### Testing Types

- **Black Box**: No prior knowledge of target
- **White Box**: Full knowledge and access
- **Gray Box**: Partial knowledge (most realistic)

## Implementation Patterns

### Pattern 1: Reconnaissance Framework

```python
from dataclasses import dataclass
from typing import List, Dict, Optional
import socket
import dns.resolver
import whois
import requests
from urllib.parse import urlparse

@dataclass
class Target:
    """Target information."""
    domain: str
    ip_addresses: List[str]
    subdomains: List[str]
    technologies: List[str]
    open_ports: List[int]
    email_addresses: List[str]
    metadata: Dict

class ReconnaissanceEngine:
    """Passive and active reconnaissance."""
    
    def __init__(self, target_domain: str):
        self.target = Target(
            domain=target_domain,
            ip_addresses=[],
            subdomains=[],
            technologies=[],
            open_ports=[],
            email_addresses=[],
            metadata={}
        )
    
    def passive_recon(self):
        """Passive reconnaissance without touching target."""
        # DNS enumeration
        self._dns_enumeration()
        
        # WHOIS lookup
        self._whois_lookup()
        
        # Search engines (Google dorking)
        self._search_engine_recon()
        
        # Social media reconnaissance (OSINT)
        self._social_media_recon()
        
        # Certificate transparency logs
        self._certificate_transparency()
    
    def active_recon(self):
        """Active reconnaissance (target interaction)."""
        # Port scanning
        self._port_scan()
        
        # Service enumeration
        self._service_enumeration()
        
        # Technology detection
        self._technology_detection()
        
        # Web crawling
        self._web_crawling()
    
    def _dns_enumeration(self):
        """DNS reconnaissance."""
        try:
            # A records
            answers = dns.resolver.resolve(self.target.domain, 'A')
            for rdata in answers:
                self.target.ip_addresses.append(str(rdata))
            
            # MX records (mail servers)
            mx_records = dns.resolver.resolve(self.target.domain, 'MX')
            for mx in mx_records:
                self.target.metadata['mx_records'] = self.target.metadata.get('mx_records', [])
                self.target.metadata['mx_records'].append(str(mx))
            
            # Subdomain enumeration
            common_subdomains = ['www', 'mail', 'ftp', 'admin', 'api', 'dev', 'staging', 'test']
            for sub in common_subdomains:
                try:
                    subdomain = f"{sub}.{self.target.domain}"
                    dns.resolver.resolve(subdomain, 'A')
                    self.target.subdomains.append(subdomain)
                except:
                    pass
        
        except Exception as e:
            print(f"DNS enumeration error: {e}")
    
    def _whois_lookup(self):
        """WHOIS information gathering."""
        try:
            w = whois.whois(self.target.domain)
            self.target.metadata['whois'] = {
                'registrar': w.registrar,
                'creation_date': str(w.creation_date),
                'expiration_date': str(w.expiration_date),
                'name_servers': w.name_servers
            }
        except Exception as e:
            print(f"WHOIS lookup error: {e}")
    
    def _port_scan(self):
        """Basic port scanning (use nmap for production)."""
        common_ports = [21, 22, 23, 25, 53, 80, 110, 143, 443, 445, 3306, 3389, 5432, 8080, 8443]
        
        for ip in self.target.ip_addresses[:1]:  # Scan first IP
            for port in common_ports:
                try:
                    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
                    sock.settimeout(1)
                    result = sock.connect_ex((ip, port))
                    if result == 0:
                        self.target.open_ports.append(port)
                    sock.close()
                except:
                    pass
    
    def _technology_detection(self):
        """Detect web technologies."""
        try:
            url = f"https://{self.target.domain}"
            response = requests.get(url, timeout=5)
            
            # Check headers
            headers = response.headers
            if 'Server' in headers:
                self.target.technologies.append(f"Server: {headers['Server']}")
            if 'X-Powered-By' in headers:
                self.target.technologies.append(f"Powered-By: {headers['X-Powered-By']}")
            
            # Check for common frameworks
            content = response.text.lower()
            if 'wp-content' in content:
                self.target.technologies.append("WordPress")
            if 'react' in content:
                self.target.technologies.append("React")
            if 'angular' in content:
                self.target.technologies.append("Angular")
        
        except Exception as e:
            print(f"Technology detection error: {e}")
    
    def _service_enumeration(self):
        """Enumerate services on open ports."""
        pass
    
    def _search_engine_recon(self):
        """Google dorking and search engine reconnaissance."""
        pass
    
    def _social_media_recon(self):
        """OSINT from social media."""
        pass
    
    def _certificate_transparency(self):
        """Check certificate transparency logs for subdomains."""
        pass
    
    def _web_crawling(self):
        """Crawl website for endpoints and parameters."""
        pass
    
    def generate_report(self) -> str:
        """Generate reconnaissance report."""
        report = [f"# Reconnaissance Report: {self.target.domain}\n"]
        
        report.append(f"## IP Addresses")
        for ip in self.target.ip_addresses:
            report.append(f"- {ip}")
        
        report.append(f"\n## Subdomains ({len(self.target.subdomains)})")
        for sub in self.target.subdomains:
            report.append(f"- {sub}")
        
        report.append(f"\n## Open Ports ({len(self.target.open_ports)})")
        for port in self.target.open_ports:
            report.append(f"- {port}")
        
        report.append(f"\n## Technologies Detected")
        for tech in self.target.technologies:
            report.append(f"- {tech}")
        
        return "\n".join(report)
```

### Pattern 2: Vulnerability Assessment

```python
from enum import Enum
from dataclasses import dataclass
from typing import List, Optional

class VulnerabilitySeverity(Enum):
    """CVSS severity ratings."""
    CRITICAL = "critical"  # 9.0-10.0
    HIGH = "high"  # 7.0-8.9
    MEDIUM = "medium"  # 4.0-6.9
    LOW = "low"  # 0.1-3.9
    INFORMATIONAL = "info"

@dataclass
class Vulnerability:
    """Vulnerability finding."""
    vuln_id: str
    title: str
    severity: VulnerabilitySeverity
    cvss_score: float
    description: str
    affected_component: str
    proof_of_concept: str
    remediation: str
    references: List[str]

class VulnerabilityScanner:
    """Vulnerability scanning and assessment."""
    
    def __init__(self):
        self.findings: List[Vulnerability] = []
    
    def scan_web_application(self, url: str):
        """Scan web application for vulnerabilities."""
        # SQL Injection
        self._test_sql_injection(url)
        
        # XSS (Cross-Site Scripting)
        self._test_xss(url)
        
        # CSRF (Cross-Site Request Forgery)
        self._test_csrf(url)
        
        # Authentication flaws
        self._test_authentication(url)
        
        # Authorization bypass
        self._test_authorization(url)
        
        # Sensitive data exposure
        self._test_data_exposure(url)
        
        # XXE (XML External Entity)
        self._test_xxe(url)
        
        # Insecure deserialization
        self._test_deserialization(url)
        
        # Security misconfiguration
        self._test_misconfiguration(url)
        
        # Known vulnerabilities
        self._test_known_vulns(url)
    
    def _test_sql_injection(self, url: str):
        """Test for SQL injection vulnerabilities."""
        payloads = [
            "' OR '1'='1",
            "' OR '1'='1' --",
            "' OR '1'='1' /*",
            "admin' --",
            "1' UNION SELECT NULL--"
        ]
        
        # Test each payload in parameters
        for payload in payloads:
            # Send request with payload
            # Check for SQL error messages or authentication bypass
            pass
    
    def _test_xss(self, url: str):
        """Test for Cross-Site Scripting."""
        payloads = [
            "<script>alert('XSS')</script>",
            "<img src=x onerror=alert('XSS')>",
            "<svg/onload=alert('XSS')>",
            "javascript:alert('XSS')"
        ]
        
        # Test reflected, stored, and DOM-based XSS
        pass
    
    def _test_csrf(self, url: str):
        """Test for CSRF vulnerabilities."""
        # Check for CSRF tokens
        # Test state-changing requests without token
        pass
    
    def _test_authentication(self, url: str):
        """Test authentication mechanisms."""
        # Brute force protection
        # Weak passwords
        # Session management
        # Password reset flaws
        pass
    
    def _test_authorization(self, url: str):
        """Test authorization and access controls."""
        # Insecure Direct Object References (IDOR)
        # Privilege escalation
        # Missing function-level access control
        pass
    
    def _test_data_exposure(self, url: str):
        """Test for sensitive data exposure."""
        # Unencrypted transmission
        # Weak encryption
        # Exposed API keys
        # Information disclosure in errors
        pass
    
    def _test_xxe(self, url: str):
        """Test for XML External Entity injection."""
        pass
    
    def _test_deserialization(self, url: str):
        """Test for insecure deserialization."""
        pass
    
    def _test_misconfiguration(self, url: str):
        """Test for security misconfigurations."""
        # Default credentials
        # Directory listing
        # Unnecessary services
        # Verbose error messages
        pass
    
    def _test_known_vulns(self, url: str):
        """Test for known vulnerabilities in components."""
        # Outdated libraries
        # Known CVEs
        pass
    
    def add_finding(self, vulnerability: Vulnerability):
        """Add vulnerability to findings."""
        self.findings.append(vulnerability)
    
    def get_critical_findings(self) -> List[Vulnerability]:
        """Get critical and high severity findings."""
        return [
            v for v in self.findings
            if v.severity in [VulnerabilitySeverity.CRITICAL, VulnerabilitySeverity.HIGH]
        ]
```

### Pattern 3: Exploitation Framework

```python
from dataclasses import dataclass
from typing import Optional, Dict
from enum import Enum

class ExploitType(Enum):
    """Types of exploits."""
    REMOTE_CODE_EXECUTION = "rce"
    SQL_INJECTION = "sqli"
    AUTHENTICATION_BYPASS = "auth_bypass"
    PRIVILEGE_ESCALATION = "privesc"
    FILE_INCLUSION = "file_inclusion"

@dataclass
class ExploitResult:
    """Result of exploitation attempt."""
    success: bool
    exploit_type: ExploitType
    target: str
    access_gained: str
    evidence: str
    timestamp: datetime

class ExploitationEngine:
    """Controlled exploitation for pentesting."""
    
    def __init__(self, target: str):
        self.target = target
        self.results: List[ExploitResult] = []
    
    def attempt_exploit(
        self,
        vulnerability: Vulnerability,
        exploit_type: ExploitType
    ) -> ExploitResult:
        """
        Attempt to exploit vulnerability.
        IMPORTANT: Only with explicit authorization.
        """
        result = ExploitResult(
            success=False,
            exploit_type=exploit_type,
            target=self.target,
            access_gained="None",
            evidence="",
            timestamp=datetime.utcnow()
        )
        
        try:
            if exploit_type == ExploitType.SQL_INJECTION:
                result = self._exploit_sql_injection(vulnerability)
            elif exploit_type == ExploitType.REMOTE_CODE_EXECUTION:
                result = self._exploit_rce(vulnerability)
            elif exploit_type == ExploitType.AUTHENTICATION_BYPASS:
                result = self._exploit_auth_bypass(vulnerability)
            
            self.results.append(result)
            
        except Exception as e:
            print(f"Exploitation failed: {e}")
        
        return result
    
    def _exploit_sql_injection(self, vuln: Vulnerability) -> ExploitResult:
        """Exploit SQL injection for data extraction."""
        # UNION-based injection
        # Boolean-based blind injection
        # Time-based blind injection
        return ExploitResult(
            success=False,
            exploit_type=ExploitType.SQL_INJECTION,
            target=self.target,
            access_gained="",
            evidence="",
            timestamp=datetime.utcnow()
        )
    
    def _exploit_rce(self, vuln: Vulnerability) -> ExploitResult:
        """Exploit remote code execution."""
        return ExploitResult(
            success=False,
            exploit_type=ExploitType.REMOTE_CODE_EXECUTION,
            target=self.target,
            access_gained="",
            evidence="",
            timestamp=datetime.utcnow()
        )
    
    def _exploit_auth_bypass(self, vuln: Vulnerability) -> ExploitResult:
        """Exploit authentication bypass."""
        return ExploitResult(
            success=False,
            exploit_type=ExploitType.AUTHENTICATION_BYPASS,
            target=self.target,
            access_gained="",
            evidence="",
            timestamp=datetime.utcnow()
        )
```

### Pattern 4: Penetration Test Report

```python
@dataclass
class PenetrationTestReport:
    """Comprehensive pentest report."""
    client_name: str
    test_date: datetime
    testers: List[str]
    scope: List[str]
    executive_summary: str
    findings: List[Vulnerability]
    recommendations: List[str]
    methodology: str

def generate_pentest_report(
    target: Target,
    findings: List[Vulnerability],
    exploit_results: List[ExploitResult]
) -> str:
    """Generate professional penetration testing report."""
    
    report = []
    
    # Executive Summary
    report.append("# Penetration Testing Report\n")
    report.append("## Executive Summary\n")
    
    critical_count = len([f for f in findings if f.severity == VulnerabilitySeverity.CRITICAL])
    high_count = len([f for f in findings if f.severity == VulnerabilitySeverity.HIGH])
    medium_count = len([f for f in findings if f.severity == VulnerabilitySeverity.MEDIUM])
    
    report.append(f"This assessment identified {len(findings)} security findings:")
    report.append(f"- Critical: {critical_count}")
    report.append(f"- High: {high_count}")
    report.append(f"- Medium: {medium_count}")
    
    # Scope
    report.append("\n## Scope\n")
    report.append(f"- Target: {target.domain}")
    report.append(f"- IP Addresses: {', '.join(target.ip_addresses)}")
    report.append(f"- Subdomains: {len(target.subdomains)}")
    
    # Methodology
    report.append("\n## Methodology\n")
    report.append("1. Reconnaissance and information gathering")
    report.append("2. Vulnerability scanning and assessment")
    report.append("3. Exploitation and privilege escalation")
    report.append("4. Post-exploitation and documentation")
    
    # Findings
    report.append("\n## Detailed Findings\n")
    
    for i, finding in enumerate(sorted(findings, key=lambda x: x.cvss_score, reverse=True), 1):
        report.append(f"\n### Finding #{i}: {finding.title}")
        report.append(f"**Severity**: {finding.severity.value.upper()} ({finding.cvss_score})")
        report.append(f"**Affected Component**: {finding.affected_component}")
        report.append(f"\n**Description**: {finding.description}")
        report.append(f"\n**Proof of Concept**:\n```\n{finding.proof_of_concept}\n```")
        report.append(f"\n**Remediation**: {finding.remediation}")
    
    # Recommendations
    report.append("\n## Recommendations\n")
    report.append("1. Address all Critical and High severity findings immediately")
    report.append("2. Implement security development lifecycle (SDL)")
    report.append("3. Regular security training for developers")
    report.append("4. Continuous security monitoring")
    report.append("5. Annual penetration testing")
    
    return "\n".join(report)
```

## Best Practices

### Testing Rules

1. **Authorization**: Get written authorization before testing
2. **Scope**: Stay within defined scope
3. **Timing**: Test during approved time windows
4. **Data Handling**: Protect any data exfiltrated during testing
5. **Communication**: Report critical findings immediately
6. **Documentation**: Document everything thoroughly

### Methodology

1. **Start Passive**: Begin with passive reconnaissance
2. **Progress Gradually**: Move from passive to active testing
3. **Be Methodical**: Follow structured methodology
4. **Document Everything**: Screenshots, commands, results
5. **Verify Findings**: Confirm all vulnerabilities
6. **Think Like Attacker**: Consider real-world attack scenarios

### Tools

- **Reconnaissance**: nmap, Shodan, Censys, TheHarvester
- **Scanning**: Nessus, OpenVAS, Burp Suite, OWASP ZAP
- **Exploitation**: Metasploit, SQLmap, BeEF
- **Post-Exploitation**: Mimikatz, PowerSploit
- **Reporting**: Dradis, Faraday

## Common Pitfalls

- **No Authorization**: Testing without written permission is illegal
- **Scope Creep**: Accidentally testing out-of-scope systems
- **Destructive Testing**: Breaking production systems
- **Data Leakage**: Exposing client data
- **Incomplete Documentation**: Missing evidence for findings
- **False Positives**: Reporting unverified vulnerabilities
- **Poor Communication**: Not reporting critical issues promptly

## Resources

- **OWASP Testing Guide**: Comprehensive web app testing methodology
- **PTES**: Penetration Testing Execution Standard
- **NIST SP 800-115**: Technical Guide to Information Security Testing
- **Kali Linux**: Penetration testing distribution
- **Bug Bounty Platforms**: HackerOne, Bugcrowd for practice