After the installation of a brand new TMG Server (TMG stands for Threat Management Gateway which is the new name for Microsoft’s ISA Server Software of old) some of the internal clients complained of unusual connection issues – including web page dropouts, ftp access issues, IM applications connecting and disconnecting over and over etc. All these issues were sporadic, over time they would improve and then bam would return again..
After numerous hours of tracing packets and testing DNS, internal routes and all the way down to cables I finally made my way back to the TMG – If only I had started there I could have saved a lot of time.
After doing some log tracking ( i’ll talk about log file monitoring & configuration in another article) from a single IP address that was having this issue I noticed that every now and then the following errors would occur:
Status: A connection was rejected because the connection limit specifying the maximum number of connections that can be created for a rule during one second was exceeded.
(0x80074e23 FWX_E_CONCURRENT_CONNECTIONS_QUOTA_EXCEEDED_DROPPED)
Ok I was getting somewhere.. After asking my friend Mr Google I stumbled upon the answer..
TMG implements a new set of functionality to help to prevent denial of service (DOS) attacks and other types of malicious network flooding which could bring down your network. This approach is two tiered – blocking suspicious traffic from the outside world and additionally trying to prevent internal machines from flooding external\internal networks should they be originator of these style of attacks. These are succinclty called the ‘Intrusion Detection’ features of TMG.
While these functions are great and all the default limit was just too low for a site of the size we were working on, coupled with they had a basic rule set allowing all users certain protocols out, as soon as the internal users had MSN open, a few Web connections, some custom monitoring tools reaching external sources etc TMG would immediately deem these users as causing too many connections per rule and start dropping their packets.
Now every site will be different so you will have to work out the right limits for your organisation but all you have to do is increase these limits from the Forefront TMG Management Console and then check your logs and monitoring to see if your connections are no longer being dropped.
To access the Flood Mitigation Configuration:
- Log in to the TMG Console
- Navigate to Intrusion Prevention System in the Main Menu
- Open the Behavioural Intrusion Detection tab
- Click Configure Flood Mitigation Settings
Here you will find all the available flood protection settings, the ones to look at specifically are the Max TCP requests per minute per IP address, Max HTTP Requests per minute per IP address and Max new non-TCP session per second per rule as a good start. If you have one (trusted) specific machine which is overloading your TMG with connections (for a reason) you may want to add it as an exception via the IP Exceptions tab.
Microsoft TMG - Flood Mitigation Configuration
Additional information from Microsoft on configuring Flood Mitigation can be found here: http://technet.microsoft.com/en-us/library/dd441028.aspx
Hopefully this will help anyone having the same 80074e23 problem I was. Happy browsing!