The "Curse of Dimensionality"

Name: Glyn Holton
Affiliation: Contingency Analysis
E-mail: glyn@contingencyanalysis.com
Date: 02 Mar 1997
Time: 15:32:42

Comments

André Levy recognizes that the calculation of VAR is nothing more than an integration over a (compounded) probability distribution. André asks: why not simply perform the integration with "good old numerical methods?" The answer is something called the "curse of dimensionality."

Most methods of numerical integration will not work on high-dimensional problems because the required number of calculations becomes huge as the number of dimensions grow. A perfect example of this is the Trapezoid Rule which is often taught in elementary calculus courses. If the Trapezoid Rule were used to perform an integration on the unit interval [0,1], the interval might be divided into 100 equal parts, requiring 100 separate sets of calculations to estimate the integral. If we expanded the problem to two dimensions, integrating over the unit square, [0.1]x[0,1], we might want to divide each side of the square into 100 parts. This would require 100x100 = 10,000 individual sets of calculations to estimate the integral. Finally, consider the 20-dimensional problem - integrating over the 20-dimensional hypercube ([0,1]x[0,1]x[0,1] ... [0,1]). Here, if we divided each interval into 100 parts, estimating the integral would require 100^20 (a one with 40 zeros after it) calculations.

In the above example, reducing the number of parts into which we divide each [0,1] interval will not help. For example, in the 20-dimensional case, even if we divided each of the 20 intervals into just 5 parts, we would still require 5^20 > 95,000,000,000,000 individual sets of calculations ... and, dividing each interval into just 5 parts, our precision would be awful.

Monte Carlo simulation was developed by the Los Alamos team (the people who developed the nuclear bomb for the US during the 1940's). They had high-dimensional integrals to solve, and traditional methods of numerical integration failed them because of the curse of dimensionality. What is so fantastic about Monte Carlo simulation is the fact that its precision is proportional to the square root of the number of scenarios used, and THIS RESULT IS ENTIRELY INDEPENDENT OF THE NUMBER OF DIMENSIONS OF THE PROBLEM. Effectively, Monte Carlo simulation was developed to break the curse of dimensionality. The history of World War II might have been different if it were never invented.

In many VAR calculations (perhaps involving multiple currencies and multiple yield curves) the dimensionality of the problem can easily reach 50 or 100. Monte Carlo techniques become the only viable method of integration (assuming you are not using closed form VAR or historical VAR).

Quasi Monte Carlo techniques are merely enhancements on the basic technique ... they can significantly improve the rate of convergence, and should be used whenever appropriate.

Regards,

Glyn Holton