Mirrors/hacktricks
2
0
Fork 0
mirror of https://github.com/carlospolop/hacktricks synced 2024-11-22 12:43:23 +00:00
Code Issues Projects Releases Packages Wiki Activity
hacktricks/network-services-pentesting/pentesting-web/uncovering-cloudflare.md

12 KiB
Raw Blame History

Uncovering CloudFlare

☁️ HackTricks Cloud ☁️ -🐦 Twitter 🐦 - 🎙️ Twitch 🎙️ - 🎥 Youtube 🎥

Techniques to try to uncover web servers behind cloudflare:

Techniques

  • You can also use some service that gives you the historical DNS records of the domain. Maybe the web page is running on an IP address used before.
    • Same could be achieve checking historical SSL certificates that could be pointing to the origin IP address.
    • Check also DNS records of other subdomains pointing directly to IPs, as it's possible that other subdomains are pointing to the same server (maybe to offer FTP, mail or any other service).
  • If you find a SSRF inside the web application you can abuse it to obtain the IP address of the server.
Search a unique string of the web page in browsers such as shodan (and maybe google and similar?). Maybe you can find an IP address with that content.

* In a similar way instead of looking for a uniq string you could search for the favicon icon with the tool: [https://github.com/karma9874/CloudFlare-IP](https://github.com/karma9874/CloudFlare-IP) or with [https://github.com/pielco11/fav-up](https://github.com/pielco11/fav-up)
* This won't work be very frequently because the server must send the same response when it's accessed by the IP address, but you never know.

Tools

# You can check if the tool is working with
prips 1.0.0.0/30 | hakoriginfinder -h one.one.one.one

# If you know the company is using AWS you could use the previous tool to search the
## web page inside the EC2 IPs
DOMAIN=something.com
WIDE_REGION=us
for ir in `curl https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.service=="EC2") | select(.region|test("^us")) | .ip_prefix'`; do 
    echo "Checking $ir"
    prips $ir | hakoriginfinder -h "$DOMAIN"
done

Uncovering Cloudflare from AWS machines

For a better description of this process check:

{% embed url="https://trickest.com/blog/cloudflare-bypass-discover-ip-addresses-aws/?utm_campaign=hacktrics&utm_medium=banner&utm_source=hacktricks" %}

# Find open ports
sudo masscan --max-rate 10000 -p80,443 $(curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.service=="EC2") | .ip_prefix' | tr '\n' ' ') | grep "open"  > all_open.txt
# Format results
cat all_open.txt | sed 's,.*port \(.*\)/tcp on \(.*\),\2:\1,' | tr -d " " > all_open_formated.txt
# Search actual web pages
httpx -silent -threads 200 -l all_open_formated.txt -random-agent -follow-redirects -json -no-color -o webs.json
# Format web results and remove eternal redirects
cat webs.json | jq -r "select((.failed==false) and (.chain_status_codes | length) < 9) | .url" | sort -u > aws_webs.json

# Search via Host header
httpx -json -no-color -list aws_webs.json -header Host: cloudflare.malwareworld.com -threads 250 -random-agent -follow-redirects -o web_checks.json

Bypassing Cloudflare through Cloudflare

Authenticated Origin Pulls

This mechanism relies on client SSL certificates to authenticate connections between Cloudflare’s reverse-proxy servers and the origin server, which is called mTLS.

Instead of configuring it's own certifica, Customer can simple use Cloudflare’s certificate to allow any connection from Cloudflare, regardless of the tenant.

{% hint style="danger" %} Therefore, an attacker could just set a domain in Cloudflare using CLoudflares certificate and point it to the victim domain IP address. This way, setting his domain completely unprotected, Cloudflare won't protect the requests sent. {% endhint %}

More info here.

Allowlist Cloudflare IP Addresses

This will reject connections that do not originate from Cloudflare’s IP address ranges. This is also vulnerable to the previous setup where an attacker just point his own domain in Cloudflare to the victims IP address and attack it.

More info here.

Bypass Cloudflare for scraping

Cache

Sometimes you just want to bypass Cloudflare to only scrape the web page. There are some options for this:

  • Use Google cache: https://webcache.googleusercontent.com/search?q=cache:https://www.petsathome.com/shop/en/pets/dog
  • Use other cache services such as https://archive.org/web/

Cloudflare Solvers

There have been a number of Cloudflare solvers developed:

Fortified Headless Browsers

The other option is to do the entire scraping job with a headless browser that has been fortified to look like a real users browser:

Smart Proxy With Cloudflare Built-In Bypass

The alternative to using open source Cloudflare bypasses, is to use smart proxies that develop and maintain their own private Cloudflare bypasses.

These are typically more reliable as it is harder for Cloudflare to develop patches for them, and they are developed by proxy companies who are financially motivated to stay 1 step ahead of Cloudflare and fix their bypasses the very minute they stop working.

Most smart proxy providers (ScraperAPI, Scrapingbee, Oxylabs, Smartproxy) have some form of Cloudflare bypass that work to varying degrees and vary in cost.

However, one of the best options is to use the ScrapeOps Proxy Aggregator as it integrates over 20 proxy providers into the same proxy API, and finds the best/cheapest proxy provider for your target domains.

Reverse Engineer Cloudflare Anti-Bot Protection

This approach works (and is what many smart proxy solutions do), however, it is not for the faint hearted.

Advantages: The advantage of this approach, is that if you are scraping at large scales and you don't want to run hundreds (if not thousands) of costly full headless browser instances. You can instead develop the most resource efficient Cloudflare bypass possible. One that is solely designed to pass the Cloudflare JS, TLS and IP fingerprint tests.

Disadvantages: The disadvantages to this approach is that you will have to dive deep into a anti-bot system that has been made purposedly hard to understand from the outside, and split test different techniques to trick their verification system. Then maintain this system as Cloudflare continue to develop their anti-bot protection.

References

☁️ HackTricks Cloud ☁️ -🐦 Twitter 🐦 - 🎙️ Twitch 🎙️ - 🎥 Youtube 🎥