In a simple/naive example, an HTTP web request from a customer is routed efficiently to a company or provider's web service infrastructure for processing. Traditionally internet routers would send the request relatively directly to a web server that is sat on a public IP address, or more commonly screened by the organisation's firewall, for processing. This is known as an origin server, since it is where served data originates.

More recently, other "screening" services may interdict and restrict the processing of the request under certain conditions - such devices include Web Application Firewalls (WAFs), Intrustion Detection Systems (IDSs) and Intrusion Prevention Systems (IPSs) - all located within an organisation's managed estate.

However the growth of cloud computing and the necessity of scaling web services to be able to handle very large volumes of web traffic has let to the use of various forms of "reverse proxy" - an intermediate connection point positioned at a network’s edge that receives HTTP connection requests, acting as if it were the actual serving webserver. Essentially acting as a "policeman" that analyses the web traffic, a reverse proxy serves as a gateway between customers/users and the "origin" webserver for the application.

1. What are CDNs?
2. What are DDos Mitigation Services?
3. What challenges do the above present to scanning?
4. If CDNs and DDoS mitigation services protect my origin web servers, why should I bother scanning them, aren't they safe?
5. What should I do to ensure that scanning by AppCheck is successful when screened by one of these services?
6. Further Information

What are CDNs?

A CDN is a type of reverse proxying service and stands for Content Delivery Network, or Content Distribution Network. It is a geographically distributed network of proxy servers and their data centers. The goal is to provide high availability and performance by distributing the service spatially relative to end users at the "network edge" (that is close to each requesting customer) rather than forwarding on all requests to a centralised and heavily contended target webserver in a single location or data centre.

CDN proxies use typically large number of "nodes" and are usually deployed in multiple locations, often over multiple Internet backbones. Benefits include reducing bandwidth costs, improving page load times, or increasing global availability of content. The number of nodes and servers making up a CDN varies, depending on the architecture, some reaching thousands of nodes with tens of thousands of servers on many remote points of presence (PoPs) as part of a global network.

Requests for content are typically directed to nodes that are optimal in some way, such as by choosing locations that are the fewest network "hops" from the requesting user/customer or the lowest number of network seconds away from the requesting client, so as to optimize delivery across local networks.

Examples of commonly used CDNs you may encounter include Cloudflare, Akamai, Amazon Cloudfront, Level 3, and Fastly.

What are DDos Mitigation Services?

DDoS mitigation is a set of network management techniques and/or tools for resisting or mitigating the impact of distributed denial-of-service (DDoS) attacks on networks attached to the Internet by protecting the target and relay networks. They typically operate at network "edge" on the same basis as a CDN (and are often combined into and built on the same platforms and using the same "nodes") so that traffic is never routed to the origin webserver if it is thought to be malicious.

The first thing to do in DDoS mitigation is to identify normal conditions for network traffic by defining "traffic patterns", which is necessary for threat detection and alerting. DDoS mitigation also requires identifying incoming traffic to separate human traffic from human-like bots and hijacked web browsers. The process is done by comparing signatures and examining different attributes of the traffic, including IP addresses, cookie variations, HTTP headers, and JavaScript footprints.

After the detection is made, the next process is filtering. Filtering can be done through anti-DDoS technology like connection tracking, IP reputation lists, deep packet inspection, allow-lists/deny-lists, or rate limiting.

Examples of prominent DDoS mitigation services include Cloudflare, Prolexic, Imperva, Incapsula. 

What challenges do the above present to scanning?

DDoS mitigation services as well as the Web Application Firewalls (WAFs) commonly built into commercial and open source CDNs will attempts to block requests that appear to have attack payloads within them, in order to protect the origin server. This means that traffic from AppCheck scanners, which emulates "hacking" activity in order to probe for vulnerabilities, may be detected as malicious and blocked, leaving AppCheck unable to effectively scan the requested target.

If CDNs and DDoS mitigation services protect my origin web servers, why should I bother scanning them, aren't they safe?

As with all devices and services offering web application firewall activity and application "screening", CDNs and DDoS mitigation services do not offer complete protection against malicious web based attacks and can often be bypassed - googling "bypass cloudflare waf xss" or "bypass cloudflare waf sqli" etc will show many examples.

Additionally, it cannot be guaranteed that a CDN or DDoS mitigation service will always be able to screen the origin server - the CDN may be down, or traffic management misconfigured so that the origin server can be accessed directly. Internet security has a principle of "defense in depth" whereby relying on a single panacea or measure for protection is foolhardy and provides incomplete protection - it is better to ensure that the application is screened and that it is scanned for vulnerabilities.

What should I do to ensure that scanning by AppCheck is successful when screened by one of these services?

The option that AppCheck recommends in most instances is to configure your traffic management such that the origin server is exposed directly on the public internet. Cloudflare is known to continue blocking traffic if it believes it's attempting something it believes is dangerous such as SQLInjection. If you have restricted the app so only the CDN/DDoS service can connect to it (which should be the case), then AppCheck would also need to have its IP range added in the same way. This means that AppCheck can then scan the server directly and not have traffic routed through a CDN or DDoS mitigation service at all. In order to do this, the origin service needs a publicly addressable (non-RFC1918) IP address as well as a public DNS entry creating if one is not already present.

If for any reason the above is not possible, then another option is to route traffic from AppCheck through the CDN or DDoS mitigation service, but to add our IP addresses to your CDN provider's allow-list such that the CDN service does not process the traffic for malicious pattern recognition. To find out how to do this for your specific CND or DDoS mitigation service provider, please see your provider's support pages, or contact your CDN/DDoS service account manager for more information.

Note: Some providers such as Cloudflare continue to block payloads it believes are malicious even if they come from addresses in your allow-list, which can prevent AppCheck from detecting vulnerabilities that are present in your application.

Further Information

• Allowing AppCheck Access to Your Network or Applications
• AppCheck's IP Range