RequestShield - Intelligent Log Analysis for Threat Detection

RequestShield - Intelligent Log Analysis for Threat Detection

RequestShield is a free and open-source tool designed to analyze HTTP access logs and identify suspicious requests and potential security threats.

By leveraging factors like geolocation, abuse history, request volume, and suspicious request paths, it assigns a risk score to each IP address, providing actionable insights for security teams.

This guide will walk you through the purpose, features, setup, and usage of RequestShield.

Purpose

RequestShield is an ideal tool for security teams aiming to enhance their monitoring capabilities. Its primary use cases include:

  • Intrusion Detection: Identifying unauthorized access attempts.

  • Rate Limiting: Detecting high-volume traffic such as DoS/DDoS attacks.

  • Abuse Monitoring: Flagging IPs with a history of malicious activity.

Features

RequestShield offers the following key features:

  • Log Parsing: Analyzes logs in the Common Log Format.

  • Risk Scoring: Assesses IP risk based on:

    • Geolocation: Flags requests from high-risk countries.

    • Abuse History: Checks IPs against AbuseIPDB.

    • Request Volume: Detects unusually high request rates.

    • Suspicious Paths: Identifies risky endpoints (e.g., /admin).

  • HTML Reports: Generates detailed reports summarizing risks and anomalies.

Setup

Follow these steps to set up RequestShield:

1. Install Dependencies

Dependencies are automatically installed when you first run the tool. Simply execute:

python3 requestshield.py

This will install required libraries like requests, geoip2, and abuseipdb.

2. Obtain GeoLite2-Country Database

RequestShield uses the GeoLite2 database by MaxMind for geolocation. Here's how to set it up:

  1. Create an account on MaxMind's website.

  2. Download the GeoLite2-Country.mmdb database.

  3. Place the downloaded file in the main directory where requestshield.py is located.

3. Configure AbuseIPDB API Key

RequestShield integrates with AbuseIPDB to assess IP abuse history. To enable this feature:

  1. Create a free or premium account at AbuseIPDB.

  2. Generate an API key:

    • Log in to your account.

    • Navigate to the API section in the dashboard.

    • Click Create Key and configure permissions as needed.

  3. Copy the API key and save it securely.

4. Configure Settings

Edit the config.py file to customize settings:

  • Specify paths for your log file and GeoIP database.

  • Add your AbuseIPDB API key.

  • Set thresholds for request volume, risky countries, and unusual status codes.

Example configuration:

# DEFAULT Configuration

config_file["DEFAULT"] = {

    "geoip_db": str(path / "GeoLite2-Country.mmdb"),

    "log_file": str(path / "./logs/access.log"),

    "max_requests_per_minute": 100,

    "risky_country": "CN",

    "unusual_status_codes": "500,403"

}

# API Configuration

config_file["API"] = {

    "abuseipdb_api_key": "your_api_key_here"

}

# WHITELIST Configuration

config_file["WHITELIST"] = {

    "ips": "127.0.0.1"

}

Once edited, initialize the configuration with:

python3 config.py

Usage

After completing the setup, you can start using RequestShield with a single command:

python3 requestshield.py

Output

The tool processes your access logs and generates an HTML report (RequestShield_Report.html) that includes:

  • Risk Scoring: Each IP is assigned a risk score based on factors like geolocation, abuse history, and request patterns.

  • Summary of Anomalies: Highlights flagged IPs, unusual activity, and suspicious request paths.

Conclusion

RequestShield is a powerful tool for detecting and mitigating security threats by analyzing HTTP access logs in real-time. Its ability to assign risk scores and generate actionable reports makes it an essential addition to any security team’s toolkit.

By following this guide, you can easily set up and start using RequestShield to enhance your organization’s security posture. You can learn more and Download RequestShield in GitHub.

Upgrade Your Cybersecurity Skills EHA: Learn 150+ Practical Cyber Security Courses Online With Life Time Access - Enroll Here

 

Back to blog