Central Limit Theorem

<body onLoad="MM_checkPlugin('Shockwave Flash','','http://www.riskglossary.com/default_nofl.htm',false);return document.MM_returnValue" stylesrc="../_private/template.htm" bgcolor="#FFFFFF" link="#669933" vlink="#000000" alink="#000000"> <div align="left"> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table1"> <tr> <td width="740"> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" height="37" id="table2"> <tr> <td height="35" width="100%"> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" id="table3"> <tr> <td width="100%" align="left" valign="top"> <p class="header_gold">Central Limit Theorem</td> <td width="45" align="right" valign="bottom"> <map name="FPMap6"> <area href="http://www.contingencyanalysis.com" shape="rect" coords="1, 0, 44, 11" target="_top"> </map> <img border="0" src="../images/home.gif" usemap="#FPMap6" width="45" height="12"></td> </tr> </table> </td> </tr> <tr> <td width="100%" bgcolor="#CC9966" height="2"> <img border="0" src="../images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td width="0%" valign="top" bgcolor="#FFFFCC"> <table border="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" cellspacing="1" id="table4"> <tr> <td width="0%" align="left" valign="top"> <p class="header2_black">Explained:<br> <img border="0" src="../images/blank_1x100.gif" width="100" height="1"></td> <td width="98%" align="left" valign="top"><p class="word_list"> <a href="#central limit theorem">central limit theorem</a><br> <img border="0" src="../images/blank_1x160.gif" width="160" height="1"></td> </tr> </table> </td> </tr> </table> <table border="0" cellpadding="0" style="border-collapse: collapse" bordercolor="#111111" width="310" align="left" id="table5"> <tr> <td width="300" height="14"><font size="2"> </font><font size="4"> </font></td> <td width="9" height="14"></td> </tr> <tr> <td width="300"> <script type="text/javascript"></script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script>  </td> <td width="9"> </td> </tr> </table> <p class="body">The <a href="../articles/normal_distribution.htm">normal distribution</a> is useful for modeling various random quantities, such as people’s heights, asset <a href="../articles/return.htm">returns</a>, and test scores. This is no coincidence. If a process is additive—reflecting the combined influence of multiple random occurrences—the result is likely to be approximately normal. This famous result is known as the the <b> <a name="central limit theorem">central limit theorem</a></b>.</p> <p class="body">In a nutshell, the central limit theorem states that a sum of random variables will have a distribution that is approximately normal. The <a href="../articles/mean.htm">means</a> and <a href="../articles/standard_deviation.htm">standard deviations</a> of the random variables must exist, and other modest conditions must also be met. In practical applications, those modest conditions are met more often than not. </p> <p class="body">For an example, consider several independent <i>U</i>(–1, 1) random variables <img border="0" src="../symbols/x_i_cap.gif" align="absbottom" width="14" height="15"> (see the article <a href="../articles/uniform_distribution.htm">uniform distribution</a> for an explanation of this notation). Let <img border="0" src="../formulas/central_limit_theorem_X_average.gif" align="absbottom" width="18" height="19"> be the random variable equal to the average of the first <i>n</i> of the <img border="0" src="../symbols/x_i_cap.gif" align="absbottom" width="14" height="15">:</p> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" height="36" id="table6"> <tr> <td width="100%" align="center"> <img border="0" src="../formulas/central_limit_theorem_[1].gif" width="129" height="43"></td> <td width="5%" align="right">[1]</td> </tr> </table> <p class="body">Exhibit 1 indicates the probability density function (PDF) of <img border="0" src="../formulas/central_limit_theorem_X_average.gif" align="absbottom" width="18" height="19"> when <i>n</i> has values 1, 2, 3, 4 and then 5.</p> <div align="center"> <center> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="450" cellpadding="0" id="table7"> <tr> <td> <p class="exhibit_header"><span class="exhibit_header">Example</span><br> <span class="exhibit_subheader">Exhibit 1</span></td> </tr> <tr> <td bgcolor="#CC9966" height="2"> <img border="0" src="../images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td> <p style="margin-top: 6"> <img border="0" src="../formulas/central_limit_theorem_ex_1.gif" width="447" height="72"></td> </tr> <tr> <td> <p class="exhibit_legend">A random variable that equals an average of independent <i>U</i>(–1,1) random variables becomes more like a normal random variable as the number of <i><br> U</i>(–1,1) random variables in the average increases. This is illustrated with a progression of PDF's. Note that images have been scaled so all have similar widths.</td> </tr> </table> </center> </div> <p class="body">The first image in Exhibit 1 is simply the PDF of a <i>U</i>(–1, 1) random variable. The second is the PDF of a random variable that is an average of two independent <i>U</i>(–1, 1) random variables. That PDF has a triangular shape. Next, with an average of three independent <i>U</i>(–1, 1) random variables, the PDF takes on a bell shape. As <i>n</i> continues to grow, the shape of the PDF becomes increasingly like that of the normal distribution. This graphically illustrates the central limit theorem. Let's formalize this.</p> <p class="body">Let <b><i>X</i></b> be an <i>n</i>-dimensional random vector with independent and identically distributed (IID) components <img border="0" src="../symbols/x_i_cap.gif" align="absbottom" width="14" height="15">. It doesn’t matter what their common distribution is as long as its mean <img border="0" src="../symbols/mu.gif" align="absbottom" width="8" height="12"> and standard deviation <img border="0" src="../symbols/sigma.gif" align="absbottom" width="9" height="12"> exist. Let <img border="0" src="../formulas/central_limit_theorem_X_average.gif" align="absbottom" width="18" height="19"> be the random variable equal to the average of the <img border="0" src="../symbols/x_i_cap.gif" align="absbottom" width="14" height="15">. As a <a href="../articles/polynomial.htm">linear polynomial of a random vector</a>, <img border="0" src="../formulas/central_limit_theorem_X_average.gif" align="absbottom" width="18" height="19"> has mean <img border="0" src="../symbols/mu.gif" align="absbottom" width="8" height="12"> and standard deviation <img border="0" src="../formulas/central_limit_theorem_avg_std.gif" align="absbottom" width="38" height="19">. Accordingly, the normalized average</p> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" height="36" id="table8"> <tr> <td width="100%" align="center"> <img border="0" src="../formulas/central_limit_theorem_[2].gif" width="90" height="44"></td> <td width="5%" align="right">[2]</td> </tr> </table> <p class="body">has mean 0 and standard deviation 1. The central limit theorem tells us <img border="0" src="../formulas/central_limit_theorem_average_X_star.gif" align="absbottom" width="21" height="18"> is approximately <a href="../articles/normal_distribution.htm">standard normal</a>. Specifically, it states that, for any constant <i>x</i>,</p> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="0" height="36" id="table9"> <tr> <td width="100%" align="center"> <img border="0" src="../formulas/central_limit_theorem_[3].gif" width="150" height="30"></td> <td width="5%" align="right">[3]</td> </tr> </table> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="345" align="right" id="table10"> <tr> <td align="right" width="9" height="10"></td> <td width="336" height="10"> <font size="2"> </font></td> </tr> <tr> <td align="right" width="9"> </td> <td width="336"> <script type="text/javascript"></script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script>  </td> </tr> </table> <p class="body">where <img border="0" src="../formulas/central_limit_theorem_fi_x.gif" align="absbottom" width="28" height="15"> is the standard normal cumulative distribution function (CDF)</p> <p class="body">There are many versions of the central limit theorem. Several of these place additional restrictions on the <img border="0" src="../symbols/x_i_cap.gif" align="absbottom" width="14" height="15"> but do not require that they be identically distributed. The additional restrictions vary, but are generally designed to prevent one or a handful of random variables from dominating the average, which might happen if one random variable has a standard deviation far greater than the rest. </p> <p class="body">In Exhibit 2, probability distributions are illustrated for five independent random variables <img border="0" src="../symbols/x_i_cap.gif" align="absbottom" width="14" height="15">. All five distributions have mean 0 and standard deviation 1 and are dramatically non-normal. They were selected arbitrarily, but their normalized average is approximately normal. </p> <div align="center"> <center> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="450" cellpadding="0" id="table11"> <tr> <td> <p class="exhibit_header"><span class="exhibit_header">Example</span><br> <span class="exhibit_subheader">Exhibit 2</span></td> </tr> <tr> <td bgcolor="#CC9966" height="2"> <img border="0" src="../images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td> <img border="0" src="../formulas/central_limit_theorem_ex_2.gif" width="450" height="185"></td> </tr> <tr> <td> <p class="exhibit_legend">In this example, the normalized average of five independent random variables is still approximately normal despite the fact that the five random variables have very different distributions.</td> </tr> </table> </center> </div> <p class="body">Other versions of the central limit theorem modestly weaken the independence assumption for the <img border="0" src="../symbols/x_i_cap.gif" align="absbottom" width="14" height="15">. The central limit theorem generalizes to multiple dimensions.</p> <table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table12"> <tr> <td height="30" valign="bottom" colspan="2"> <p class="header2_black">Related Internal Links</td> </tr> <tr> <td colspan="2" bgcolor="#CC9966" height="2"> <img border="0" src="../images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td bgcolor="#FFFFCC" colspan="2"> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="6" id="table13"> <tr> <td width="100%"> <p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><strong style="font-weight: 400"><a href="../articles/cornish_fisher.htm">Cornish-Fisher expansion</a> A formula for approximating quintiles of a random variable based only on its first few cumulants.</strong><p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><strong style="font-weight: 400"><a href="../articles/joint_normal_distribution.htm">joint normal distribution</a> A multivariate distribution with normal marginal distributions.</strong><p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><a href="../articles/kurtosis.htm">kurtosis</a> A parameter describing the peakedness and tails of a distribution.<p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><a href="../articles/linear_polynomial_of_a_random_vecrtor.htm">linear polynomial of a random vector</a> A random variable or random vector that is defined as a linear polynomial of a random vector.<p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><a href="../articles/normal_distribution.htm"><strong style="font-weight: 400">normal distribution</strong></a><strong style="font-weight: 400"> Perhaps the most important probability distribution for probability and statistics.</strong><p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><strong style="font-weight: 400"><a href="../articles/stable_paretian_distributions.htm">stable Paretian distribution</a> A non-normal stable distribution.</strong><p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><a href="../articles/standard_deviation.htm">standard deviation</a> A parameter describing the dispersion of a distribution.<p class="explanatory_text"> <img border="0" src="../images/bullet_link_glossary.gif" width="26" height="15"><a href="../articles/uniform_distribution.htm">uniform distribution</a> <strong style="FONT-WEIGHT: 400">A continuous probability distribution that has constant probability on a finite interval.</strong></td> </tr> </table> </td> </tr> <tr> <td height="30" valign="bottom" colspan="2"> <p class="header2_black">Sponsored Links</td> </tr> <tr> <td colspan="2" bgcolor="#CC9966" height="2"> <img border="0" src="../images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td bgcolor="#FFFFCC" width="75"> <img border="0" src="../images/ad_side_banner_3.jpg" width="103" height="285"></td> <td bgcolor="#FFFFCC" width="99%">  <p class="ad_legend" style="margin-left: 0; margin-bottom:2"> <a target="_top" href="http://www.riskexpertise.com/link/advertising.htm"> <font color="#000000">Ads by Contingency Analysis</font></a></p> <p style="margin-top: 0; margin-bottom: 0"> <iframe src="http://servedbyadbutler.com/adserve/;ID=147917;size=336x265;setID=82436;type=iframe" style="margin: 0px; padding: 0px; border-width: 0px" width="336" height="265" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" name="I1"> <![if lt IE 4]> <script src="http://servedbyadbutler.com/adserve/;ID=147917;size=336x265;setID=82436;type=js" type="text/javascript"> </script> <noscript> <a href="http://servedbyadbutler.com/go2/;ID=147917;size=336x265;setID=82436"> <img src="http://servedbyadbutler.com/adserve.ibs/;ID=147917;size=336x265;setID=82436;type=img" border="0" height="265" width="336" alt=""></a> </noscript><![endif]></iframe>   </td> </tr> <tr> <td height="30" valign="bottom" colspan="2"> <p class="header2_black">Related Books</td> </tr> <tr> <td colspan="2" bgcolor="#CC9966" height="2"> <img border="0" src="../images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td bgcolor="#FFFFCC" colspan="2"> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="6" id="table14"> <tr> <td width="100%"> <p class="explanatory_text">Salsburg (<a target="_blank" href="http://www.riskbook.com/link/salsburg_d_(2001).htm">2001</a>) is a wonderful history of probability and statistics that describes the origins of the central limit theorem. Degroot and Schervish (<a target="_blank" href="http://www.riskbook.com/link/degroot_m_(2002).htm">2002</a>) is a standard university probability text. Holton (<a target="_blank" href="http://www.value-at-risk.net">2003</a>) describes the use of the central limit theorem in measuring portfolio market risk.</p> <table border="0" cellpadding="4" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table15"> <tr> <td width="100" align="left" valign="top"> <map name="FPMap431"> <area target="_blank" href="http://www.riskbook.com/link/salsburg_d_(2001).htm" shape="rect" coords="0, 0, 80, 124"> </map> <img border="0" src="../images/book_salsburg_d_(2001)_paper_125.jpg" usemap="#FPMap431" width="81" height="125"></td> <td align="left" valign="top"> <table border="0" cellpadding="1" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table16"> <tr> <td width="90%" colspan="3"> <p class="title"> <a target="_blank" href="http://www.riskbook.com/link/salsburg_d_(2001).htm"> Lady Tasting Tea</a></td> </tr> <tr> <td colspan="3"> <p class="author"><b>David Salsburg</b></td> </tr> <tr> <td width="10%"><map name="FPMap373"> <area target="_blank" href="http://www.riskbook.com/main/ratings.htm" shape="rect" coords="0, 0, 89, 15"> </map> <img border="0" src="../images/star_6.gif" usemap="#FPMap373" width="90" height="16"></td> <td width="10%" class="author">quality</td> <td width="70%"> <p class="author"> </td> </tr> <tr> <td width="10%"><map name="FPMap374"> <area target="_blank" href="http://www.riskbook.com/main/ratings.htm" shape="rect" coords="0, 0, 89, 15"> </map> <img border="0" src="../images/tech_1.gif" usemap="#FPMap374" width="90" height="16"></td> <td width="10%" class="author">technical</td> <td width="70%">  </td> </tr> <tr> <td width="10%"> <p class="author">2001</td> <td width="10%"> </td> <td width="70%"> <p align="center"><map name="FPMap430"> <area target="_blank" coords="0, 0, 145, 19" shape="rect" href="http://www.riskbook.com/link/salsburg_d_(2001).htm"> </map> <img border="0" src="../images/pointer_review.gif" usemap="#FPMap430" width="146" height="20"></td> </tr> </table> </td> </tr> </table> <table border="0" cellpadding="4" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table17"> <tr> <td width="100" align="left" valign="top"> <map name="FPMap432"> <area target="_blank" href="http://www.riskbook.com/link/degroot_m_(2002).htm" shape="rect" coords="0, 0, 101, 124"> </map> <img border="0" src="../images/book_degroot_and_schervish_(2002)_125.jpg" usemap="#FPMap432" width="102" height="125"></td> <td align="left" valign="top"> <table border="0" cellpadding="1" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table18"> <tr> <td width="90%" colspan="3"> <p class="title"> <a target="_blank" href="http://www.riskbook.com/link/degroot_m_(2002).htm"> Probability and Statistics</a></td> </tr> <tr> <td colspan="3"> <p class="author"><b>Morris H. Degroot and Mark J. Schervish </b></td> </tr> <tr> <span style="font-family: Times New Roman; letter-spacing: -0.2pt"> <td width="10%"> <span style="font-family: Times New Roman; letter-spacing: -0.2pt"> <map name="FPMap399"> <area target="_blank" href="http://www.riskbook.com/main/ratings.htm" shape="rect" coords="0, 0, 89, 15"> </map> <img border="0" src="../images/star_5.gif" usemap="#FPMap399" width="90" height="16"></span></td> <td width="10%" class="author">quality</td> <td width="70%"> <p class="author"> </td> </tr> <tr> <td width="10%"> <span style="font-family: Times New Roman; letter-spacing: -0.2pt"> <map name="FPMap402"> <area target="_blank" href="http://www.riskbook.com/main/ratings.htm" shape="rect" coords="1, 2, 89, 15"> </map> <img border="0" src="../images/tech_3.gif" usemap="#FPMap402" width="90" height="16"></span></td> <td width="10%" class="author">technical</td> <td width="70%">  </td> </tr> <tr> <td width="10%"> <p class="author">2002</td> <td width="10%"> </td> <td width="70%"> <p align="center"><map name="FPMap433"> <area target="_blank" coords="0, 0, 145, 19" shape="rect" href="http://www.riskbook.com/link/degroot_m_(2002).htm"> </map> <img border="0" src="../images/pointer_review.gif" usemap="#FPMap433" width="146" height="20"></td> </tr> </table> </td> </tr> </table> <table border="0" cellpadding="4" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table19"> <tr> <td width="100" align="left" valign="top"> <map name="FPMap611"> <area target="_blank" href="http://www.value-at-risk.net" shape="rect" coords="0, 0, 81, 124"> </map> <img border="0" src="../images/book_holton_(2003)_125.jpg" usemap="#FPMap611" width="82" height="125"></td> <td align="left" valign="top"> <table border="0" cellpadding="1" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" id="table20"> <tr> <td width="90%" colspan="3"> <p class="title"> <a target="_blank" href="http://www.value-at-risk.net">Value-at-Risk: Theory and Practice</a></td> </tr> <tr> <td colspan="3"> <p class="author"><b>Glyn Holton</b></td> </tr> <tr> <td width="10%"> <map name="FPMap612"> <area target="_blank" href="http://www.riskbook.com/main/ratings.htm" shape="rect" coords="0, 0, 89, 15"></map> <img border="0" src="../images/star_6.gif" usemap="#FPMap612" width="90" height="16"></td> <td width="10%" class="author">quality</td> <td width="70%"> <p class="author"> </td> </tr> <tr> <td width="10%"> <map name="FPMap613"> <area target="_blank" href="http://www.riskbook.com/main/ratings.htm" shape="rect" coords="0, 1, 89, 15"> </map><img border="0" src="../images/tech_4.gif" usemap="#FPMap613" width="90" height="16"></td> <td width="10%" class="author">technical</td> <td width="70%">  </td> </tr> <tr> <td width="10%"> <p class="author">2003</td> <td width="10%"> </td> <td width="70%"> <p align="center"><map name="FPMap614"> <area target="_blank" coords="0, 0, 145, 19" shape="rect" href="http://www.value-at-risk.net"> </map> <img border="0" src="../images/pointer_review.gif" usemap="#FPMap614" width="146" height="20"></td> </tr> </table> </td> </tr> </table> </td> </tr> </table> </td> </tr> <tr> <td height="30" valign="bottom" colspan="2"> <p class="header2_black">Sponsored Links</td> </tr> <tr> <td colspan="2" bgcolor="#CC9966" height="2"> <img border="0" src="../images/blank_2x2.gif" width="2" height="2"></td> </tr> <tr> <td bgcolor="#FFFFCC" width="75"> <img border="0" src="../images/ad_side_banner_2.jpg" width="103" height="285"></td> <td bgcolor="#FFFFCC" width="99%"> <script type="text/javascript"></script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script>  </td> </tr> <tr> <td bgcolor="#FFFFCC" colspan="2"> <table border="0" cellspacing="0" style="border-collapse: collapse" bordercolor="#111111" width="100%" cellpadding="6" id="table21"> <tr> <td width="100%" bgcolor="#FFFFFF"> <p align="center" style="margin-top: 18"><map name="FPMap365"> <area target="_top" href="http://www.riskglossary.com" shape="rect" coords="0, 0, 164, 102"> </map><img border="0" src="../images/logo.jpg" usemap="#FPMap365" width="165" height="103"></p> <p align="center"><span class="verdana10"> <a href="http://www.riskglossary.com/main/disclaimer.htm" target="_self"> Disclaimer</a></span></p> <p align="center">website: <span class="verdana10"> <a target="_top" href="http://www.contingencyanalysis.com"> http://www.contingencyanalysis.com</a></span><br> glossary direct link: <span class="verdana10"> <a target="_top" href="http://www.riskglossary.com"> http://www.riskglossary.com</a></span><br> copyright © Contingency Analysis, 2005<p align="center"><map name="FPMap1"> <area target="_top" href="http://www.contingencyanalysis.com" shape="rect" coords="0, 0, 240, 88"> </map><img border="0" src="../images/logo_glossary_white.gif" usemap="#FPMap1" width="241" height="89"></td> </tr> </table> </td> </tr> </table> </td> <td>  </td> </tr> </table> </div> </body>