How Good Are Ukraine’s Air Defense Systems?

We were captivated by reports out of Ukraine the morning of the 16th of May 2023 that Ukrainian air defenses intercepted a total of eighteen incoming Russian missiles, including six top-of-the line “hypersonics”[i]. This event appears to be yet another unexpected – and highly embarrassing – shortcoming for the Russian military. While details from the attack itself in the open press are developing as we write, the key detail that captivated our attention is that the Ukrainians achieved ‘raid annihilation.’  In more colloquial terms, their defense of Kyiv was a clean sweep.

While we don’t have a lot of details, we do have enough knowledge to ask the natural question:  How good did the Ukrainian air defenses need to be to achieve this result? For the sake of completeness, there are (at least) three ways that a defender can defeat an incoming missile:

  1. The defender can shoot it down (achieve a ‘kill’)
  2. The incoming missile can spontaneously fail (i.e., be a ‘dud’)
  3. “Something else” could happen. This something else could involve sabotage, cyber, aliens, etc.

Let’s focus on the first case, where the defender shoots down the incoming missile. We could think of each incoming missile as a coin flip, where the coin might be biased – i.e., not 50/50. We are left with a tradeoff between the effectiveness of the Ukrainian defenses and plain old luck. It turns out that with a little knowledge of statistics; we can discover quite a bit.

For example, if the effectiveness of the air defenses against each incoming missile was truly a coin flip, how lucky would the Ukrainians have to be? Amazingly lucky. The odds of shooting down all eighteen missiles if the per-missile effectiveness was 50% is approximately 1:262,000. If you were sitting down to play a five-card draw, these odds between being dealt a Royal Flush (1:650,000) and a Strait Flush (1:72,000).

How lucky do we think that the Ukrainians were with their air defenses? This is the type of question that is best turned around and left to the reader. In Figure 1 below, we show the tradeoff between how ‘lucky’ the Ukrainians were to achieve this result vs. how effective their air defenses were:[ii]

Harrison Schramm
Figure 1: Ukrainian Air Defenses (x-axis) vs. Probability of defeating all eighteen inbound Russian missiles. To have 95% confidence of defeating such a raid, the per-inbound effectiveness of the Ukrainian air defenses would have had to have been greater than 99.7%.

A tidal shift in Air Defense?

My friend and mentor, the late Professor Wayne Hughes admonished his students – including me – that “when there is a war, study the war.”  And the war in Ukraine – pitting Western defense technologies against Russian weapons – is teaching us valuable lessons about the future of conflict. Using the sort of ‘poker hand and coin flip’ arguments, it has always been my assumption that air defense problems favored the attacker over the defender because it is really hard to consistently be this good. However, seeing the trend in the effectiveness of Western-designed air defenses against incoming missiles should make all of us – including our adversaries – reevaluate what we consider the ‘art of the possible’ when it comes to air defense.

And, while I hate to do the Russians[iii] homework for them, we might ask ‘if the Ukrainian Air Defenses are that good, how many missiles would they need to fire for a successful raid (50% chance of 1 or more hits)’. I’m not going to say the number – they can figure it out for themselves – but it’s a lot more than eighteen.

Harrison Schramm is a professional statistician working at the intersection of data, AI, and public policy. A retired US Navy Commander, he is the 2018 recipient of the Clayton Thomas Prize for technical contributions to the Profession of Operations Research. He is Founder and CEO of Schramm and Associates, LLC, an applied mathematics consultancy.


[iii] To be fair, pre-Soviet Russians were titans of probability theory; much of my education came from studying the works of Andrey Markov and Pafnuty Chebyshev. I wonder what they would think of this current situation?