Hfrhyu

Is it unprofessional to ask if a job posting on GlassDoor is real?

SSH "lag" in LAN on some machines, mixed distros

What to put in ESTA if staying in US for a few days before going on to Canada

Is it inappropriate for a student to attend their mentor's dissertation defense?

How to model explosives?

Infinite Abelian subgroup of infinite non Abelian group example

What exploit are these user agents trying to use?

Has there ever been an airliner design involving reducing generator load by installing solar panels?

Anagram holiday

Arrow those variables!

Why does Arabsat 6A need a Falcon Heavy to launch

Does a druid starting with a bow start with no arrows?

Assassin's bullet with mercury

What's the point of deactivating Num Lock on login screens?

In Romance of the Three Kingdoms why do people still use bamboo sticks when papers are already invented?

Why is consensus so controversial in Britain?

How to show the equivalence between the regularized regression and their constraint formulas using KKT

Is it legal for company to use my work email to pretend I still work there?

Why do I get two different answers for this counting problem?

A reference to a well-known characterization of scattered compact spaces

Why can't we play rap on piano?

Why are electrically insulating heatsinks so rare? Is it just cost?

What is the most common color to indicate the input-field is disabled?

Why is it a bad idea to hire a hitman to eliminate most corrupt politicians?

Variance of Monte Carlo integration with importance sampling

Monte Carlo Integration for non-square integrable functionsMonte Carlo integration aim for maximum varianceVariance reduction technique in Monte Carlo integrationMonte Carlo integration and varianceMonte Carlo Integration on the Real LineUse Importance Sampling and Monte carlo for estimating a summationSampling / Importance Resampling Poisson WeightsImportance SamplingFind the value of an integral using Monte-Carlo methodOptimal proposal for self-normalized importance sampling

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;

3

3

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked Apr 1 at 18:26

user1799323user1799323

1234

$endgroup$

add a comment | 

3

3

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked Apr 1 at 18:26

user1799323user1799323

1234

$endgroup$

add a comment | 

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked Apr 1 at 18:26

user1799323user1799323

1234

$endgroup$

share|cite|improve this question

asked Apr 1 at 18:26

user1799323user1799323

1234

share|cite|improve this question

asked Apr 1 at 18:26

user1799323user1799323

1234

asked Apr 1 at 18:26

user1799323user1799323

1234

asked Apr 1 at 18:26

user1799323user1799323

1234

1 Answer
1

active

oldest

votes

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited Apr 1 at 19:02

answered Apr 1 at 18:44

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f400628%2fvariance-of-monte-carlo-integration-with-importance-sampling%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited Apr 1 at 19:02

answered Apr 1 at 18:44

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited Apr 1 at 19:02

answered Apr 1 at 18:44

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited Apr 1 at 19:02

answered Apr 1 at 18:44

Xi'anXi'an

59.1k897365

$endgroup$

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

share|cite|improve this answer

edited Apr 1 at 19:02

answered Apr 1 at 18:44

Xi'anXi'an

59.1k897365

answered Apr 1 at 18:44

Xi'anXi'an

59.1k897365

answered Apr 1 at 18:44

Xi'anXi'an

59.1k897365