Is a distribution that is normal, but highly skewed, considered Gaussian? The Next CEO of Stack OverflowIs normality testing 'essentially useless'?What is the difference between zero-inflated and hurdle models?If my histogram shows a bell-shaped curve, can I say my data is normally distributed?How do I identify the “Long Tail” portion of my distribution?Skewed but bell-shaped still considered as normal distribution for ANOVA?Which Distribution Does the Data Point Belong to?Skewness - why is this distribution right skewed?log transform vs. resamplingIs it valid to remove the overhead of finding the current time for a computer program this way?Histograms for severely skewed dataWhat would the distribution of time spent per day on a given site look like?Distinguish between underlying Distribution and data shape in data transforming?Using bootstrap to estimate the 95th percentile and confidence interval for skewed data
Is it okay to majorly distort historical facts while writing a fiction story?
Avoiding the "not like other girls" trope?
Why can't we say "I have been having a dog"?
What did the word "leisure" mean in late 18th Century usage?
What does this strange code stamp on my passport mean?
Does the Idaho Potato Commission associate potato skins with healthy eating?
Is the 21st century's idea of "freedom of speech" based on precedent?
Finitely generated matrix groups whose eigenvalues are all algebraic
Strange use of "whether ... than ..." in official text
Could you use a laser beam as a modulated carrier wave for radio signal?
Is it "common practice in Fourier transform spectroscopy to multiply the measured interferogram by an apodizing function"? If so, why?
Can I cast Thunderwave and be at the center of its bottom face, but not be affected by it?
Why does the freezing point matter when picking cooler ice packs?
Is it a bad idea to plug the other end of ESD strap to wall ground?
Why did early computer designers eschew integers?
Incomplete cube
How should I connect my cat5 cable to connectors having an orange-green line?
Is the offspring between a demon and a celestial possible? If so what is it called and is it in a book somewhere?
Is this a new Fibonacci Identity?
Do I need to write [sic] when including a quotation with a number less than 10 that isn't written out?
Shortening a title without changing its meaning
Find the majority element, which appears more than half the time
How can I prove that a state of equilibrium is unstable?
That's an odd coin - I wonder why
Is a distribution that is normal, but highly skewed, considered Gaussian?
The Next CEO of Stack OverflowIs normality testing 'essentially useless'?What is the difference between zero-inflated and hurdle models?If my histogram shows a bell-shaped curve, can I say my data is normally distributed?How do I identify the “Long Tail” portion of my distribution?Skewed but bell-shaped still considered as normal distribution for ANOVA?Which Distribution Does the Data Point Belong to?Skewness - why is this distribution right skewed?log transform vs. resamplingIs it valid to remove the overhead of finding the current time for a computer program this way?Histograms for severely skewed dataWhat would the distribution of time spent per day on a given site look like?Distinguish between underlying Distribution and data shape in data transforming?Using bootstrap to estimate the 95th percentile and confidence interval for skewed data
$begingroup$
I have this question: What do you think the distribution of time spent per day on YouTube looks like?
My answer is that it is probably normally distributed and highly left skewed. I expect there is one mode where most users spend around some average time and then a long right tail since some users are overwhelming power users.
Is that a fair answer? Is there a better word for that distribution?
distributions normal-distribution skewness skew-normal
$endgroup$
add a comment |
$begingroup$
I have this question: What do you think the distribution of time spent per day on YouTube looks like?
My answer is that it is probably normally distributed and highly left skewed. I expect there is one mode where most users spend around some average time and then a long right tail since some users are overwhelming power users.
Is that a fair answer? Is there a better word for that distribution?
distributions normal-distribution skewness skew-normal
$endgroup$
3
$begingroup$
As some answers mention but do not emphasise, skewness is named informally for the longer tail if there is one, so right-skewed if a longer right tail. Left and right as used in this context both presuppose a display following a convention that magnitude is shown on the hoirizontal axis. If that sounds too obvious, consider displays in the Earth and environmental sciences in which the magnitude is height or depth and shown vertically. Small print: some measures of skewness can be zero even if a distribution is skewed geometrically.
$endgroup$
– Nick Cox
yesterday
$begingroup$
Total time per day for all users? or time per day per person? If the latter, then surely there's a moderately big spike at 0, in which case you probably need a 'spike and slab' style distribution with a Dirac delta at 0.
$endgroup$
– innisfree
23 hours ago
$begingroup$
"Normal" is synonymous with "Gaussian", and Gaussian distributions, also called normal distributions, are not skewed.
$endgroup$
– Michael Hardy
4 hours ago
add a comment |
$begingroup$
I have this question: What do you think the distribution of time spent per day on YouTube looks like?
My answer is that it is probably normally distributed and highly left skewed. I expect there is one mode where most users spend around some average time and then a long right tail since some users are overwhelming power users.
Is that a fair answer? Is there a better word for that distribution?
distributions normal-distribution skewness skew-normal
$endgroup$
I have this question: What do you think the distribution of time spent per day on YouTube looks like?
My answer is that it is probably normally distributed and highly left skewed. I expect there is one mode where most users spend around some average time and then a long right tail since some users are overwhelming power users.
Is that a fair answer? Is there a better word for that distribution?
distributions normal-distribution skewness skew-normal
distributions normal-distribution skewness skew-normal
edited yesterday
Nick Cox
39.1k587131
39.1k587131
asked 2 days ago
CauderCauder
8317
8317
3
$begingroup$
As some answers mention but do not emphasise, skewness is named informally for the longer tail if there is one, so right-skewed if a longer right tail. Left and right as used in this context both presuppose a display following a convention that magnitude is shown on the hoirizontal axis. If that sounds too obvious, consider displays in the Earth and environmental sciences in which the magnitude is height or depth and shown vertically. Small print: some measures of skewness can be zero even if a distribution is skewed geometrically.
$endgroup$
– Nick Cox
yesterday
$begingroup$
Total time per day for all users? or time per day per person? If the latter, then surely there's a moderately big spike at 0, in which case you probably need a 'spike and slab' style distribution with a Dirac delta at 0.
$endgroup$
– innisfree
23 hours ago
$begingroup$
"Normal" is synonymous with "Gaussian", and Gaussian distributions, also called normal distributions, are not skewed.
$endgroup$
– Michael Hardy
4 hours ago
add a comment |
3
$begingroup$
As some answers mention but do not emphasise, skewness is named informally for the longer tail if there is one, so right-skewed if a longer right tail. Left and right as used in this context both presuppose a display following a convention that magnitude is shown on the hoirizontal axis. If that sounds too obvious, consider displays in the Earth and environmental sciences in which the magnitude is height or depth and shown vertically. Small print: some measures of skewness can be zero even if a distribution is skewed geometrically.
$endgroup$
– Nick Cox
yesterday
$begingroup$
Total time per day for all users? or time per day per person? If the latter, then surely there's a moderately big spike at 0, in which case you probably need a 'spike and slab' style distribution with a Dirac delta at 0.
$endgroup$
– innisfree
23 hours ago
$begingroup$
"Normal" is synonymous with "Gaussian", and Gaussian distributions, also called normal distributions, are not skewed.
$endgroup$
– Michael Hardy
4 hours ago
3
3
$begingroup$
As some answers mention but do not emphasise, skewness is named informally for the longer tail if there is one, so right-skewed if a longer right tail. Left and right as used in this context both presuppose a display following a convention that magnitude is shown on the hoirizontal axis. If that sounds too obvious, consider displays in the Earth and environmental sciences in which the magnitude is height or depth and shown vertically. Small print: some measures of skewness can be zero even if a distribution is skewed geometrically.
$endgroup$
– Nick Cox
yesterday
$begingroup$
As some answers mention but do not emphasise, skewness is named informally for the longer tail if there is one, so right-skewed if a longer right tail. Left and right as used in this context both presuppose a display following a convention that magnitude is shown on the hoirizontal axis. If that sounds too obvious, consider displays in the Earth and environmental sciences in which the magnitude is height or depth and shown vertically. Small print: some measures of skewness can be zero even if a distribution is skewed geometrically.
$endgroup$
– Nick Cox
yesterday
$begingroup$
Total time per day for all users? or time per day per person? If the latter, then surely there's a moderately big spike at 0, in which case you probably need a 'spike and slab' style distribution with a Dirac delta at 0.
$endgroup$
– innisfree
23 hours ago
$begingroup$
Total time per day for all users? or time per day per person? If the latter, then surely there's a moderately big spike at 0, in which case you probably need a 'spike and slab' style distribution with a Dirac delta at 0.
$endgroup$
– innisfree
23 hours ago
$begingroup$
"Normal" is synonymous with "Gaussian", and Gaussian distributions, also called normal distributions, are not skewed.
$endgroup$
– Michael Hardy
4 hours ago
$begingroup$
"Normal" is synonymous with "Gaussian", and Gaussian distributions, also called normal distributions, are not skewed.
$endgroup$
– Michael Hardy
4 hours ago
add a comment |
9 Answers
9
active
oldest
votes
$begingroup$
A fraction per day is certainly not negative. This rules out the normal distribution, which has probability mass over the entire real axis - in particular over the negative half.
Power law distributions are often used to model things like income distributions, sizes of cities etc. They are nonnegative and typically highly skewed. These would be the first I would try in modeling time spent watching YouTube. (Or monitoring CrossValidated questions.)
More information on power laws can be found here or here, or in our power-law tag.
$endgroup$
12
$begingroup$
You're completely correct that normal distributions have support on the real line. And yet...they're no an awful model for some strictly positive qualities, like adults' height or weight, where the mean and variance are such that the negative values are very unlikely under the model.
$endgroup$
– Matt Krause
2 days ago
1
$begingroup$
@MattKrause That's actually a great question - is there a same probability I will be '10 cm above or below the mean height' or '10 percent above or below the mean height'? Only the first case could warrant normal distribution.
$endgroup$
– Tomáš Kafka
17 hours ago
$begingroup$
@MattKrause: I completely agree, in a general sense. Yet, the present question is about the proportion of daily time spent watching YouTube. We don't have any data, but I would be extremely surprised if the distribution was even remotely symmetric.
$endgroup$
– Stephan Kolassa
14 hours ago
add a comment |
$begingroup$
A distribution that is normal is not highly skewed. That is a contradiction. Normally distributed variables have skew = 0.
$endgroup$
1
$begingroup$
What is a better way to describe the distribution? Is there a word for that type of distribution where it centers around a mode and then has a long tail?
$endgroup$
– Cauder
2 days ago
11
$begingroup$
Unimodal and skewed is as close as I can come...
$endgroup$
– jbowman
2 days ago
8
$begingroup$
As an aside, it's just really incredible that people give their time to help other people get better at this stuff. I know it goes without saying, but it's so cool what you both do!
$endgroup$
– Cauder
2 days ago
5
$begingroup$
Yes, but it's worth clarifying that that statement pertains to the normally distributed population. A sample drawn from that population can be very skewed.
$endgroup$
– gung♦
2 days ago
$begingroup$
When the skew value is small ("small" being decided by the people dealing with the stats in question), you can still treat the population as normal, albeit with minor error as a result.
$endgroup$
– Carl Witthoft
12 hours ago
add a comment |
$begingroup$
If it has long right tail, then it's right skewed.
It can't be a normal distribution since skew !=0, it's perhaps a unimodal skew normal distribution:
https://en.wikipedia.org/wiki/Skew_normal_distribution
New contributor
$endgroup$
add a comment |
$begingroup$
It could be a log-normal distribution. As mentioned here:
Users' dwell time on online articles (jokes, news etc.) follows a log-normal distribution.
The reference given is: Yin, Peifeng; Luo, Ping; Lee, Wang-Chien; Wang, Min (2013). Silence is also evidence: interpreting dwell time for recommendation from psychological perspective. ACM International Conference on KDD.
$endgroup$
add a comment |
$begingroup$
The gamma distribution could be a good candidate to describe this kind of distribution over nonnegative, right-skewed data. See the green line in the image here:
https://en.m.wikipedia.org/wiki/Gamma_distribution
$endgroup$
add a comment |
$begingroup$
"Normal" and "Gaussian" mean exactly the same thing. As other answers explain, the distribution you're talking about is not normal/Gaussian, because that distribution assigns probabilities to every value on the real line, whereas your distribution only exists between $0$ and $24$.
$endgroup$
add a comment |
$begingroup$
In the case at hand, since the time spent per day is bound from $0$ to $1$ (if quantified as a fraction of the day), distributions that are unbounded above (e.g. Pareto, skew-normal, Gamma, log-normal) won't work, but Beta would.
$endgroup$
add a comment |
$begingroup$
How about a hurdle model?
A hurdle model has two parts. The first is Bernoulli experiment that determines whether you use YouTube at all. If you don't, then your usage time is obviously zero and you're done. If you do, you "pass that hurdle", then the usage time comes from some other strictly positive distribution.
A closely related concept are zero-inflated models. These are meant to deal with a situation where we observe a bunch of zeros, but can't distinguish between always-zeros and sometimes-zeros. For example, consider the number of cigarettes that a person smokes each day. For non-smokers, that number is always zero, but some smokers may not smoke on a given day (out of cigarettes? on a long flight?). Unlike the hurdle model, the "smoker" distribution here should include zero, but these counts are 'inflated' by the non-smokers' contribution too.
$endgroup$
add a comment |
$begingroup$
"Is there a better word for that distribution?"
There's a worthwhile distinction here between using words to describe the properties of the distribution, versus trying to find a "name" for the distribution so that you can identify it as (approximately) an instance of a particular standard distribution: one for which a formula or statistical tables might exist for its distribution function, and for which you could estimate its parameters. In this latter case, you are likely using the named distribution, e.g. "normal/Gaussian" (the two terms are generally synonymous), as a model that captures some of the key features of your data, rather than claiming the population your data is drawn from exactly follows that theoretical distribution. To slightly misquote George Box, all models are "wrong", but some are useful. If you are thinking about the modelling approach, it is worth considering what features you want to incorporate and how complicated or parsimonious you want your model to be.
Being positively skewed is an example of describing a property that the distribution has, but doesn't come close to specifying which off-the-shelf distribution is "the" appropriate model. It does rule out some candidates, for example the Gaussian (i.e. normal) distribution has zero skew so will not be appropriate to model your data if the skew is an important feature. There may be other properties of the data that are important to you too, e.g. that it's unimodal (has just one peak) or that it is bounded between 0 and 24 hours (or between 0 and 1, if you are writing it as a fraction of the day), or that there is a probability mass concentrated at zero (since there are people who do not watch youtube at all on a given day). You may also be interested in other properties like the kurtosis. And it is worth bearing in mind that even if your distribution had a "hump" or "bell-curve" shape and had zero or near-zero skew, it doesn't automatically follow that the normal distribution is "correct" for it! On the other hand, even if the population your data is drawn from actually did follow a particular distribution precisely, due to sampling error your dataset may not quite resemble it. Small data sets are likely to be "noisy", and it may be unclear whether certain features you can see, e.g. additional small humps or asymmetric tails, are properties of the underlying population the data was drawn from (and perhaps therefore ought to be incorporated in your model) or whether they are just artefacts from your particular sample (and for modelling purposes should be ignored). If you have a small data set and the skew is close to zero, then it is even plausible the underlying distribution is actually symmetric. The larger your data set and the larger the skewness, the less plausible this becomes — but while you could perform a significance test to see how convincing is the evidence your data provides for skewness in the population it was drawn from, this may be missing the point as to whether a normal (or other zero skew) distribution is appropriate as a model ...
Which properties of the data really matter for the purposes you are intending to model it? Note that if the skew is reasonably small and you do not care very much about it, even if the underlying population is genuinely skewed, then you might still find the normal distribution a useful model to approximate this true distribution of watching times. But you should check that this doesn't end up making silly predictions. Because a normal distribution has no highest or lowest possible value, then although extremely high or low values become increasingly unlikely, you will always find that your model predicts there is some probability of watching for a negative number of hours per day, or more than 24 hours. This gets more problematic for you if the predicted probability of such impossible events becomes high. If the skewness is so noteworthy you want to capture it as part of your model, then the skew normal distribution may be more appropriate. If you want to capture both skewness and kurtosis, then consider the skewed t. If you want to incorporate the physically possible upper and lower bounds, then consider using the truncated versions of these distributions. Many other probability distributions exist that can be skewed and unimodal (for appropriate parameter choices) such as the F or gamma distributions, and again you can truncate these so they do not predict impossibly high watching times. A beta distribution may be a good choice if you are modelling the fraction of the day spent watching, as this is always bounded between 0 and 1 without further truncation being necessary. If you want to incorporate the concentration of probability at exactly zero due to non-watchers, then consider building in a hurdle model.
But at the point you are trying to throw in every feature you can identify from your data, and build an ever more sophisticated model, perhaps you should ask yourself why you are doing this? Would there be an advantage to a simpler model, for example it being easier to work with mathematically or having fewer parameters to estimate? If you are concerned that such simplification will leave you unable to capture all of the properties of interest to you, it may well be that no "off-the-shelf" distribution does quite what you want. However, we are not restricted to working with named distributions whose mathematical properties have been elucidated previously. Instead, consider using your data to construct an empirical distribution function. This will capture all the behaviour that was present in your data, but you can no longer give it a name like "normal" or "gamma", nor can you apply mathematical properties that pertain only to a particular distribution. For instance, the "95% of the data lies within 1.96 standard deviations of the mean" rule is for normally distributed data and may not apply to your distribution; though note that some rules apply to all distributions, e.g. Chebyshev's inequality guarantees that at least 75% of your data must lie within two standard deviations of the mean, regardless of the skew. Unfortunately the empirical distribution will also inherit all of the properties of your data set that arose just by sampling error, not just those possessed by the underlying population, so you may find a histogram of your empirical distribution has some humps and dips that the population itself does not — if you want to improve matters, you can try taking a larger sample. You may want to investigate smoothed empirical distribution functions.
In summary: although the normal distribution has zero skew, the fact your data are skewed doesn't rule out the normal distribution as a useful model, though it does suggest some other distribution may be more appropriate. You should also consider other properties of the data when choosing your model, besides the skew, and consider too the purposes you are going to use the model for. It's safe to say that your true population of watching times does not exactly follow some famous, named distribution, but this does not necessarily mean that such a distribution is doomed to be useless as a model. However, for some purposes you may prefer to just use the empirical distribution itself, rather than to fit a standard distribution to it.
$endgroup$
add a comment |
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f400316%2fis-a-distribution-that-is-normal-but-highly-skewed-considered-gaussian%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
9 Answers
9
active
oldest
votes
9 Answers
9
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
A fraction per day is certainly not negative. This rules out the normal distribution, which has probability mass over the entire real axis - in particular over the negative half.
Power law distributions are often used to model things like income distributions, sizes of cities etc. They are nonnegative and typically highly skewed. These would be the first I would try in modeling time spent watching YouTube. (Or monitoring CrossValidated questions.)
More information on power laws can be found here or here, or in our power-law tag.
$endgroup$
12
$begingroup$
You're completely correct that normal distributions have support on the real line. And yet...they're no an awful model for some strictly positive qualities, like adults' height or weight, where the mean and variance are such that the negative values are very unlikely under the model.
$endgroup$
– Matt Krause
2 days ago
1
$begingroup$
@MattKrause That's actually a great question - is there a same probability I will be '10 cm above or below the mean height' or '10 percent above or below the mean height'? Only the first case could warrant normal distribution.
$endgroup$
– Tomáš Kafka
17 hours ago
$begingroup$
@MattKrause: I completely agree, in a general sense. Yet, the present question is about the proportion of daily time spent watching YouTube. We don't have any data, but I would be extremely surprised if the distribution was even remotely symmetric.
$endgroup$
– Stephan Kolassa
14 hours ago
add a comment |
$begingroup$
A fraction per day is certainly not negative. This rules out the normal distribution, which has probability mass over the entire real axis - in particular over the negative half.
Power law distributions are often used to model things like income distributions, sizes of cities etc. They are nonnegative and typically highly skewed. These would be the first I would try in modeling time spent watching YouTube. (Or monitoring CrossValidated questions.)
More information on power laws can be found here or here, or in our power-law tag.
$endgroup$
12
$begingroup$
You're completely correct that normal distributions have support on the real line. And yet...they're no an awful model for some strictly positive qualities, like adults' height or weight, where the mean and variance are such that the negative values are very unlikely under the model.
$endgroup$
– Matt Krause
2 days ago
1
$begingroup$
@MattKrause That's actually a great question - is there a same probability I will be '10 cm above or below the mean height' or '10 percent above or below the mean height'? Only the first case could warrant normal distribution.
$endgroup$
– Tomáš Kafka
17 hours ago
$begingroup$
@MattKrause: I completely agree, in a general sense. Yet, the present question is about the proportion of daily time spent watching YouTube. We don't have any data, but I would be extremely surprised if the distribution was even remotely symmetric.
$endgroup$
– Stephan Kolassa
14 hours ago
add a comment |
$begingroup$
A fraction per day is certainly not negative. This rules out the normal distribution, which has probability mass over the entire real axis - in particular over the negative half.
Power law distributions are often used to model things like income distributions, sizes of cities etc. They are nonnegative and typically highly skewed. These would be the first I would try in modeling time spent watching YouTube. (Or monitoring CrossValidated questions.)
More information on power laws can be found here or here, or in our power-law tag.
$endgroup$
A fraction per day is certainly not negative. This rules out the normal distribution, which has probability mass over the entire real axis - in particular over the negative half.
Power law distributions are often used to model things like income distributions, sizes of cities etc. They are nonnegative and typically highly skewed. These would be the first I would try in modeling time spent watching YouTube. (Or monitoring CrossValidated questions.)
More information on power laws can be found here or here, or in our power-law tag.
answered 2 days ago
Stephan KolassaStephan Kolassa
47.3k7100176
47.3k7100176
12
$begingroup$
You're completely correct that normal distributions have support on the real line. And yet...they're no an awful model for some strictly positive qualities, like adults' height or weight, where the mean and variance are such that the negative values are very unlikely under the model.
$endgroup$
– Matt Krause
2 days ago
1
$begingroup$
@MattKrause That's actually a great question - is there a same probability I will be '10 cm above or below the mean height' or '10 percent above or below the mean height'? Only the first case could warrant normal distribution.
$endgroup$
– Tomáš Kafka
17 hours ago
$begingroup$
@MattKrause: I completely agree, in a general sense. Yet, the present question is about the proportion of daily time spent watching YouTube. We don't have any data, but I would be extremely surprised if the distribution was even remotely symmetric.
$endgroup$
– Stephan Kolassa
14 hours ago
add a comment |
12
$begingroup$
You're completely correct that normal distributions have support on the real line. And yet...they're no an awful model for some strictly positive qualities, like adults' height or weight, where the mean and variance are such that the negative values are very unlikely under the model.
$endgroup$
– Matt Krause
2 days ago
1
$begingroup$
@MattKrause That's actually a great question - is there a same probability I will be '10 cm above or below the mean height' or '10 percent above or below the mean height'? Only the first case could warrant normal distribution.
$endgroup$
– Tomáš Kafka
17 hours ago
$begingroup$
@MattKrause: I completely agree, in a general sense. Yet, the present question is about the proportion of daily time spent watching YouTube. We don't have any data, but I would be extremely surprised if the distribution was even remotely symmetric.
$endgroup$
– Stephan Kolassa
14 hours ago
12
12
$begingroup$
You're completely correct that normal distributions have support on the real line. And yet...they're no an awful model for some strictly positive qualities, like adults' height or weight, where the mean and variance are such that the negative values are very unlikely under the model.
$endgroup$
– Matt Krause
2 days ago
$begingroup$
You're completely correct that normal distributions have support on the real line. And yet...they're no an awful model for some strictly positive qualities, like adults' height or weight, where the mean and variance are such that the negative values are very unlikely under the model.
$endgroup$
– Matt Krause
2 days ago
1
1
$begingroup$
@MattKrause That's actually a great question - is there a same probability I will be '10 cm above or below the mean height' or '10 percent above or below the mean height'? Only the first case could warrant normal distribution.
$endgroup$
– Tomáš Kafka
17 hours ago
$begingroup$
@MattKrause That's actually a great question - is there a same probability I will be '10 cm above or below the mean height' or '10 percent above or below the mean height'? Only the first case could warrant normal distribution.
$endgroup$
– Tomáš Kafka
17 hours ago
$begingroup$
@MattKrause: I completely agree, in a general sense. Yet, the present question is about the proportion of daily time spent watching YouTube. We don't have any data, but I would be extremely surprised if the distribution was even remotely symmetric.
$endgroup$
– Stephan Kolassa
14 hours ago
$begingroup$
@MattKrause: I completely agree, in a general sense. Yet, the present question is about the proportion of daily time spent watching YouTube. We don't have any data, but I would be extremely surprised if the distribution was even remotely symmetric.
$endgroup$
– Stephan Kolassa
14 hours ago
add a comment |
$begingroup$
A distribution that is normal is not highly skewed. That is a contradiction. Normally distributed variables have skew = 0.
$endgroup$
1
$begingroup$
What is a better way to describe the distribution? Is there a word for that type of distribution where it centers around a mode and then has a long tail?
$endgroup$
– Cauder
2 days ago
11
$begingroup$
Unimodal and skewed is as close as I can come...
$endgroup$
– jbowman
2 days ago
8
$begingroup$
As an aside, it's just really incredible that people give their time to help other people get better at this stuff. I know it goes without saying, but it's so cool what you both do!
$endgroup$
– Cauder
2 days ago
5
$begingroup$
Yes, but it's worth clarifying that that statement pertains to the normally distributed population. A sample drawn from that population can be very skewed.
$endgroup$
– gung♦
2 days ago
$begingroup$
When the skew value is small ("small" being decided by the people dealing with the stats in question), you can still treat the population as normal, albeit with minor error as a result.
$endgroup$
– Carl Witthoft
12 hours ago
add a comment |
$begingroup$
A distribution that is normal is not highly skewed. That is a contradiction. Normally distributed variables have skew = 0.
$endgroup$
1
$begingroup$
What is a better way to describe the distribution? Is there a word for that type of distribution where it centers around a mode and then has a long tail?
$endgroup$
– Cauder
2 days ago
11
$begingroup$
Unimodal and skewed is as close as I can come...
$endgroup$
– jbowman
2 days ago
8
$begingroup$
As an aside, it's just really incredible that people give their time to help other people get better at this stuff. I know it goes without saying, but it's so cool what you both do!
$endgroup$
– Cauder
2 days ago
5
$begingroup$
Yes, but it's worth clarifying that that statement pertains to the normally distributed population. A sample drawn from that population can be very skewed.
$endgroup$
– gung♦
2 days ago
$begingroup$
When the skew value is small ("small" being decided by the people dealing with the stats in question), you can still treat the population as normal, albeit with minor error as a result.
$endgroup$
– Carl Witthoft
12 hours ago
add a comment |
$begingroup$
A distribution that is normal is not highly skewed. That is a contradiction. Normally distributed variables have skew = 0.
$endgroup$
A distribution that is normal is not highly skewed. That is a contradiction. Normally distributed variables have skew = 0.
answered 2 days ago
Peter Flom♦Peter Flom
76.8k12109214
76.8k12109214
1
$begingroup$
What is a better way to describe the distribution? Is there a word for that type of distribution where it centers around a mode and then has a long tail?
$endgroup$
– Cauder
2 days ago
11
$begingroup$
Unimodal and skewed is as close as I can come...
$endgroup$
– jbowman
2 days ago
8
$begingroup$
As an aside, it's just really incredible that people give their time to help other people get better at this stuff. I know it goes without saying, but it's so cool what you both do!
$endgroup$
– Cauder
2 days ago
5
$begingroup$
Yes, but it's worth clarifying that that statement pertains to the normally distributed population. A sample drawn from that population can be very skewed.
$endgroup$
– gung♦
2 days ago
$begingroup$
When the skew value is small ("small" being decided by the people dealing with the stats in question), you can still treat the population as normal, albeit with minor error as a result.
$endgroup$
– Carl Witthoft
12 hours ago
add a comment |
1
$begingroup$
What is a better way to describe the distribution? Is there a word for that type of distribution where it centers around a mode and then has a long tail?
$endgroup$
– Cauder
2 days ago
11
$begingroup$
Unimodal and skewed is as close as I can come...
$endgroup$
– jbowman
2 days ago
8
$begingroup$
As an aside, it's just really incredible that people give their time to help other people get better at this stuff. I know it goes without saying, but it's so cool what you both do!
$endgroup$
– Cauder
2 days ago
5
$begingroup$
Yes, but it's worth clarifying that that statement pertains to the normally distributed population. A sample drawn from that population can be very skewed.
$endgroup$
– gung♦
2 days ago
$begingroup$
When the skew value is small ("small" being decided by the people dealing with the stats in question), you can still treat the population as normal, albeit with minor error as a result.
$endgroup$
– Carl Witthoft
12 hours ago
1
1
$begingroup$
What is a better way to describe the distribution? Is there a word for that type of distribution where it centers around a mode and then has a long tail?
$endgroup$
– Cauder
2 days ago
$begingroup$
What is a better way to describe the distribution? Is there a word for that type of distribution where it centers around a mode and then has a long tail?
$endgroup$
– Cauder
2 days ago
11
11
$begingroup$
Unimodal and skewed is as close as I can come...
$endgroup$
– jbowman
2 days ago
$begingroup$
Unimodal and skewed is as close as I can come...
$endgroup$
– jbowman
2 days ago
8
8
$begingroup$
As an aside, it's just really incredible that people give their time to help other people get better at this stuff. I know it goes without saying, but it's so cool what you both do!
$endgroup$
– Cauder
2 days ago
$begingroup$
As an aside, it's just really incredible that people give their time to help other people get better at this stuff. I know it goes without saying, but it's so cool what you both do!
$endgroup$
– Cauder
2 days ago
5
5
$begingroup$
Yes, but it's worth clarifying that that statement pertains to the normally distributed population. A sample drawn from that population can be very skewed.
$endgroup$
– gung♦
2 days ago
$begingroup$
Yes, but it's worth clarifying that that statement pertains to the normally distributed population. A sample drawn from that population can be very skewed.
$endgroup$
– gung♦
2 days ago
$begingroup$
When the skew value is small ("small" being decided by the people dealing with the stats in question), you can still treat the population as normal, albeit with minor error as a result.
$endgroup$
– Carl Witthoft
12 hours ago
$begingroup$
When the skew value is small ("small" being decided by the people dealing with the stats in question), you can still treat the population as normal, albeit with minor error as a result.
$endgroup$
– Carl Witthoft
12 hours ago
add a comment |
$begingroup$
If it has long right tail, then it's right skewed.
It can't be a normal distribution since skew !=0, it's perhaps a unimodal skew normal distribution:
https://en.wikipedia.org/wiki/Skew_normal_distribution
New contributor
$endgroup$
add a comment |
$begingroup$
If it has long right tail, then it's right skewed.
It can't be a normal distribution since skew !=0, it's perhaps a unimodal skew normal distribution:
https://en.wikipedia.org/wiki/Skew_normal_distribution
New contributor
$endgroup$
add a comment |
$begingroup$
If it has long right tail, then it's right skewed.
It can't be a normal distribution since skew !=0, it's perhaps a unimodal skew normal distribution:
https://en.wikipedia.org/wiki/Skew_normal_distribution
New contributor
$endgroup$
If it has long right tail, then it's right skewed.
It can't be a normal distribution since skew !=0, it's perhaps a unimodal skew normal distribution:
https://en.wikipedia.org/wiki/Skew_normal_distribution
New contributor
New contributor
answered 2 days ago
beholdbehold
1757
1757
New contributor
New contributor
add a comment |
add a comment |
$begingroup$
It could be a log-normal distribution. As mentioned here:
Users' dwell time on online articles (jokes, news etc.) follows a log-normal distribution.
The reference given is: Yin, Peifeng; Luo, Ping; Lee, Wang-Chien; Wang, Min (2013). Silence is also evidence: interpreting dwell time for recommendation from psychological perspective. ACM International Conference on KDD.
$endgroup$
add a comment |
$begingroup$
It could be a log-normal distribution. As mentioned here:
Users' dwell time on online articles (jokes, news etc.) follows a log-normal distribution.
The reference given is: Yin, Peifeng; Luo, Ping; Lee, Wang-Chien; Wang, Min (2013). Silence is also evidence: interpreting dwell time for recommendation from psychological perspective. ACM International Conference on KDD.
$endgroup$
add a comment |
$begingroup$
It could be a log-normal distribution. As mentioned here:
Users' dwell time on online articles (jokes, news etc.) follows a log-normal distribution.
The reference given is: Yin, Peifeng; Luo, Ping; Lee, Wang-Chien; Wang, Min (2013). Silence is also evidence: interpreting dwell time for recommendation from psychological perspective. ACM International Conference on KDD.
$endgroup$
It could be a log-normal distribution. As mentioned here:
Users' dwell time on online articles (jokes, news etc.) follows a log-normal distribution.
The reference given is: Yin, Peifeng; Luo, Ping; Lee, Wang-Chien; Wang, Min (2013). Silence is also evidence: interpreting dwell time for recommendation from psychological perspective. ACM International Conference on KDD.
answered 2 days ago
Count IblisCount Iblis
24113
24113
add a comment |
add a comment |
$begingroup$
The gamma distribution could be a good candidate to describe this kind of distribution over nonnegative, right-skewed data. See the green line in the image here:
https://en.m.wikipedia.org/wiki/Gamma_distribution
$endgroup$
add a comment |
$begingroup$
The gamma distribution could be a good candidate to describe this kind of distribution over nonnegative, right-skewed data. See the green line in the image here:
https://en.m.wikipedia.org/wiki/Gamma_distribution
$endgroup$
add a comment |
$begingroup$
The gamma distribution could be a good candidate to describe this kind of distribution over nonnegative, right-skewed data. See the green line in the image here:
https://en.m.wikipedia.org/wiki/Gamma_distribution
$endgroup$
The gamma distribution could be a good candidate to describe this kind of distribution over nonnegative, right-skewed data. See the green line in the image here:
https://en.m.wikipedia.org/wiki/Gamma_distribution
answered 2 days ago
mauricemaurice
18816
18816
add a comment |
add a comment |
$begingroup$
"Normal" and "Gaussian" mean exactly the same thing. As other answers explain, the distribution you're talking about is not normal/Gaussian, because that distribution assigns probabilities to every value on the real line, whereas your distribution only exists between $0$ and $24$.
$endgroup$
add a comment |
$begingroup$
"Normal" and "Gaussian" mean exactly the same thing. As other answers explain, the distribution you're talking about is not normal/Gaussian, because that distribution assigns probabilities to every value on the real line, whereas your distribution only exists between $0$ and $24$.
$endgroup$
add a comment |
$begingroup$
"Normal" and "Gaussian" mean exactly the same thing. As other answers explain, the distribution you're talking about is not normal/Gaussian, because that distribution assigns probabilities to every value on the real line, whereas your distribution only exists between $0$ and $24$.
$endgroup$
"Normal" and "Gaussian" mean exactly the same thing. As other answers explain, the distribution you're talking about is not normal/Gaussian, because that distribution assigns probabilities to every value on the real line, whereas your distribution only exists between $0$ and $24$.
answered 16 hours ago
David RicherbyDavid Richerby
1455
1455
add a comment |
add a comment |
$begingroup$
In the case at hand, since the time spent per day is bound from $0$ to $1$ (if quantified as a fraction of the day), distributions that are unbounded above (e.g. Pareto, skew-normal, Gamma, log-normal) won't work, but Beta would.
$endgroup$
add a comment |
$begingroup$
In the case at hand, since the time spent per day is bound from $0$ to $1$ (if quantified as a fraction of the day), distributions that are unbounded above (e.g. Pareto, skew-normal, Gamma, log-normal) won't work, but Beta would.
$endgroup$
add a comment |
$begingroup$
In the case at hand, since the time spent per day is bound from $0$ to $1$ (if quantified as a fraction of the day), distributions that are unbounded above (e.g. Pareto, skew-normal, Gamma, log-normal) won't work, but Beta would.
$endgroup$
In the case at hand, since the time spent per day is bound from $0$ to $1$ (if quantified as a fraction of the day), distributions that are unbounded above (e.g. Pareto, skew-normal, Gamma, log-normal) won't work, but Beta would.
answered 19 hours ago
J.G.J.G.
26616
26616
add a comment |
add a comment |
$begingroup$
How about a hurdle model?
A hurdle model has two parts. The first is Bernoulli experiment that determines whether you use YouTube at all. If you don't, then your usage time is obviously zero and you're done. If you do, you "pass that hurdle", then the usage time comes from some other strictly positive distribution.
A closely related concept are zero-inflated models. These are meant to deal with a situation where we observe a bunch of zeros, but can't distinguish between always-zeros and sometimes-zeros. For example, consider the number of cigarettes that a person smokes each day. For non-smokers, that number is always zero, but some smokers may not smoke on a given day (out of cigarettes? on a long flight?). Unlike the hurdle model, the "smoker" distribution here should include zero, but these counts are 'inflated' by the non-smokers' contribution too.
$endgroup$
add a comment |
$begingroup$
How about a hurdle model?
A hurdle model has two parts. The first is Bernoulli experiment that determines whether you use YouTube at all. If you don't, then your usage time is obviously zero and you're done. If you do, you "pass that hurdle", then the usage time comes from some other strictly positive distribution.
A closely related concept are zero-inflated models. These are meant to deal with a situation where we observe a bunch of zeros, but can't distinguish between always-zeros and sometimes-zeros. For example, consider the number of cigarettes that a person smokes each day. For non-smokers, that number is always zero, but some smokers may not smoke on a given day (out of cigarettes? on a long flight?). Unlike the hurdle model, the "smoker" distribution here should include zero, but these counts are 'inflated' by the non-smokers' contribution too.
$endgroup$
add a comment |
$begingroup$
How about a hurdle model?
A hurdle model has two parts. The first is Bernoulli experiment that determines whether you use YouTube at all. If you don't, then your usage time is obviously zero and you're done. If you do, you "pass that hurdle", then the usage time comes from some other strictly positive distribution.
A closely related concept are zero-inflated models. These are meant to deal with a situation where we observe a bunch of zeros, but can't distinguish between always-zeros and sometimes-zeros. For example, consider the number of cigarettes that a person smokes each day. For non-smokers, that number is always zero, but some smokers may not smoke on a given day (out of cigarettes? on a long flight?). Unlike the hurdle model, the "smoker" distribution here should include zero, but these counts are 'inflated' by the non-smokers' contribution too.
$endgroup$
How about a hurdle model?
A hurdle model has two parts. The first is Bernoulli experiment that determines whether you use YouTube at all. If you don't, then your usage time is obviously zero and you're done. If you do, you "pass that hurdle", then the usage time comes from some other strictly positive distribution.
A closely related concept are zero-inflated models. These are meant to deal with a situation where we observe a bunch of zeros, but can't distinguish between always-zeros and sometimes-zeros. For example, consider the number of cigarettes that a person smokes each day. For non-smokers, that number is always zero, but some smokers may not smoke on a given day (out of cigarettes? on a long flight?). Unlike the hurdle model, the "smoker" distribution here should include zero, but these counts are 'inflated' by the non-smokers' contribution too.
answered 16 hours ago
Matt KrauseMatt Krause
15k24380
15k24380
add a comment |
add a comment |
$begingroup$
"Is there a better word for that distribution?"
There's a worthwhile distinction here between using words to describe the properties of the distribution, versus trying to find a "name" for the distribution so that you can identify it as (approximately) an instance of a particular standard distribution: one for which a formula or statistical tables might exist for its distribution function, and for which you could estimate its parameters. In this latter case, you are likely using the named distribution, e.g. "normal/Gaussian" (the two terms are generally synonymous), as a model that captures some of the key features of your data, rather than claiming the population your data is drawn from exactly follows that theoretical distribution. To slightly misquote George Box, all models are "wrong", but some are useful. If you are thinking about the modelling approach, it is worth considering what features you want to incorporate and how complicated or parsimonious you want your model to be.
Being positively skewed is an example of describing a property that the distribution has, but doesn't come close to specifying which off-the-shelf distribution is "the" appropriate model. It does rule out some candidates, for example the Gaussian (i.e. normal) distribution has zero skew so will not be appropriate to model your data if the skew is an important feature. There may be other properties of the data that are important to you too, e.g. that it's unimodal (has just one peak) or that it is bounded between 0 and 24 hours (or between 0 and 1, if you are writing it as a fraction of the day), or that there is a probability mass concentrated at zero (since there are people who do not watch youtube at all on a given day). You may also be interested in other properties like the kurtosis. And it is worth bearing in mind that even if your distribution had a "hump" or "bell-curve" shape and had zero or near-zero skew, it doesn't automatically follow that the normal distribution is "correct" for it! On the other hand, even if the population your data is drawn from actually did follow a particular distribution precisely, due to sampling error your dataset may not quite resemble it. Small data sets are likely to be "noisy", and it may be unclear whether certain features you can see, e.g. additional small humps or asymmetric tails, are properties of the underlying population the data was drawn from (and perhaps therefore ought to be incorporated in your model) or whether they are just artefacts from your particular sample (and for modelling purposes should be ignored). If you have a small data set and the skew is close to zero, then it is even plausible the underlying distribution is actually symmetric. The larger your data set and the larger the skewness, the less plausible this becomes — but while you could perform a significance test to see how convincing is the evidence your data provides for skewness in the population it was drawn from, this may be missing the point as to whether a normal (or other zero skew) distribution is appropriate as a model ...
Which properties of the data really matter for the purposes you are intending to model it? Note that if the skew is reasonably small and you do not care very much about it, even if the underlying population is genuinely skewed, then you might still find the normal distribution a useful model to approximate this true distribution of watching times. But you should check that this doesn't end up making silly predictions. Because a normal distribution has no highest or lowest possible value, then although extremely high or low values become increasingly unlikely, you will always find that your model predicts there is some probability of watching for a negative number of hours per day, or more than 24 hours. This gets more problematic for you if the predicted probability of such impossible events becomes high. If the skewness is so noteworthy you want to capture it as part of your model, then the skew normal distribution may be more appropriate. If you want to capture both skewness and kurtosis, then consider the skewed t. If you want to incorporate the physically possible upper and lower bounds, then consider using the truncated versions of these distributions. Many other probability distributions exist that can be skewed and unimodal (for appropriate parameter choices) such as the F or gamma distributions, and again you can truncate these so they do not predict impossibly high watching times. A beta distribution may be a good choice if you are modelling the fraction of the day spent watching, as this is always bounded between 0 and 1 without further truncation being necessary. If you want to incorporate the concentration of probability at exactly zero due to non-watchers, then consider building in a hurdle model.
But at the point you are trying to throw in every feature you can identify from your data, and build an ever more sophisticated model, perhaps you should ask yourself why you are doing this? Would there be an advantage to a simpler model, for example it being easier to work with mathematically or having fewer parameters to estimate? If you are concerned that such simplification will leave you unable to capture all of the properties of interest to you, it may well be that no "off-the-shelf" distribution does quite what you want. However, we are not restricted to working with named distributions whose mathematical properties have been elucidated previously. Instead, consider using your data to construct an empirical distribution function. This will capture all the behaviour that was present in your data, but you can no longer give it a name like "normal" or "gamma", nor can you apply mathematical properties that pertain only to a particular distribution. For instance, the "95% of the data lies within 1.96 standard deviations of the mean" rule is for normally distributed data and may not apply to your distribution; though note that some rules apply to all distributions, e.g. Chebyshev's inequality guarantees that at least 75% of your data must lie within two standard deviations of the mean, regardless of the skew. Unfortunately the empirical distribution will also inherit all of the properties of your data set that arose just by sampling error, not just those possessed by the underlying population, so you may find a histogram of your empirical distribution has some humps and dips that the population itself does not — if you want to improve matters, you can try taking a larger sample. You may want to investigate smoothed empirical distribution functions.
In summary: although the normal distribution has zero skew, the fact your data are skewed doesn't rule out the normal distribution as a useful model, though it does suggest some other distribution may be more appropriate. You should also consider other properties of the data when choosing your model, besides the skew, and consider too the purposes you are going to use the model for. It's safe to say that your true population of watching times does not exactly follow some famous, named distribution, but this does not necessarily mean that such a distribution is doomed to be useless as a model. However, for some purposes you may prefer to just use the empirical distribution itself, rather than to fit a standard distribution to it.
$endgroup$
add a comment |
$begingroup$
"Is there a better word for that distribution?"
There's a worthwhile distinction here between using words to describe the properties of the distribution, versus trying to find a "name" for the distribution so that you can identify it as (approximately) an instance of a particular standard distribution: one for which a formula or statistical tables might exist for its distribution function, and for which you could estimate its parameters. In this latter case, you are likely using the named distribution, e.g. "normal/Gaussian" (the two terms are generally synonymous), as a model that captures some of the key features of your data, rather than claiming the population your data is drawn from exactly follows that theoretical distribution. To slightly misquote George Box, all models are "wrong", but some are useful. If you are thinking about the modelling approach, it is worth considering what features you want to incorporate and how complicated or parsimonious you want your model to be.
Being positively skewed is an example of describing a property that the distribution has, but doesn't come close to specifying which off-the-shelf distribution is "the" appropriate model. It does rule out some candidates, for example the Gaussian (i.e. normal) distribution has zero skew so will not be appropriate to model your data if the skew is an important feature. There may be other properties of the data that are important to you too, e.g. that it's unimodal (has just one peak) or that it is bounded between 0 and 24 hours (or between 0 and 1, if you are writing it as a fraction of the day), or that there is a probability mass concentrated at zero (since there are people who do not watch youtube at all on a given day). You may also be interested in other properties like the kurtosis. And it is worth bearing in mind that even if your distribution had a "hump" or "bell-curve" shape and had zero or near-zero skew, it doesn't automatically follow that the normal distribution is "correct" for it! On the other hand, even if the population your data is drawn from actually did follow a particular distribution precisely, due to sampling error your dataset may not quite resemble it. Small data sets are likely to be "noisy", and it may be unclear whether certain features you can see, e.g. additional small humps or asymmetric tails, are properties of the underlying population the data was drawn from (and perhaps therefore ought to be incorporated in your model) or whether they are just artefacts from your particular sample (and for modelling purposes should be ignored). If you have a small data set and the skew is close to zero, then it is even plausible the underlying distribution is actually symmetric. The larger your data set and the larger the skewness, the less plausible this becomes — but while you could perform a significance test to see how convincing is the evidence your data provides for skewness in the population it was drawn from, this may be missing the point as to whether a normal (or other zero skew) distribution is appropriate as a model ...
Which properties of the data really matter for the purposes you are intending to model it? Note that if the skew is reasonably small and you do not care very much about it, even if the underlying population is genuinely skewed, then you might still find the normal distribution a useful model to approximate this true distribution of watching times. But you should check that this doesn't end up making silly predictions. Because a normal distribution has no highest or lowest possible value, then although extremely high or low values become increasingly unlikely, you will always find that your model predicts there is some probability of watching for a negative number of hours per day, or more than 24 hours. This gets more problematic for you if the predicted probability of such impossible events becomes high. If the skewness is so noteworthy you want to capture it as part of your model, then the skew normal distribution may be more appropriate. If you want to capture both skewness and kurtosis, then consider the skewed t. If you want to incorporate the physically possible upper and lower bounds, then consider using the truncated versions of these distributions. Many other probability distributions exist that can be skewed and unimodal (for appropriate parameter choices) such as the F or gamma distributions, and again you can truncate these so they do not predict impossibly high watching times. A beta distribution may be a good choice if you are modelling the fraction of the day spent watching, as this is always bounded between 0 and 1 without further truncation being necessary. If you want to incorporate the concentration of probability at exactly zero due to non-watchers, then consider building in a hurdle model.
But at the point you are trying to throw in every feature you can identify from your data, and build an ever more sophisticated model, perhaps you should ask yourself why you are doing this? Would there be an advantage to a simpler model, for example it being easier to work with mathematically or having fewer parameters to estimate? If you are concerned that such simplification will leave you unable to capture all of the properties of interest to you, it may well be that no "off-the-shelf" distribution does quite what you want. However, we are not restricted to working with named distributions whose mathematical properties have been elucidated previously. Instead, consider using your data to construct an empirical distribution function. This will capture all the behaviour that was present in your data, but you can no longer give it a name like "normal" or "gamma", nor can you apply mathematical properties that pertain only to a particular distribution. For instance, the "95% of the data lies within 1.96 standard deviations of the mean" rule is for normally distributed data and may not apply to your distribution; though note that some rules apply to all distributions, e.g. Chebyshev's inequality guarantees that at least 75% of your data must lie within two standard deviations of the mean, regardless of the skew. Unfortunately the empirical distribution will also inherit all of the properties of your data set that arose just by sampling error, not just those possessed by the underlying population, so you may find a histogram of your empirical distribution has some humps and dips that the population itself does not — if you want to improve matters, you can try taking a larger sample. You may want to investigate smoothed empirical distribution functions.
In summary: although the normal distribution has zero skew, the fact your data are skewed doesn't rule out the normal distribution as a useful model, though it does suggest some other distribution may be more appropriate. You should also consider other properties of the data when choosing your model, besides the skew, and consider too the purposes you are going to use the model for. It's safe to say that your true population of watching times does not exactly follow some famous, named distribution, but this does not necessarily mean that such a distribution is doomed to be useless as a model. However, for some purposes you may prefer to just use the empirical distribution itself, rather than to fit a standard distribution to it.
$endgroup$
add a comment |
$begingroup$
"Is there a better word for that distribution?"
There's a worthwhile distinction here between using words to describe the properties of the distribution, versus trying to find a "name" for the distribution so that you can identify it as (approximately) an instance of a particular standard distribution: one for which a formula or statistical tables might exist for its distribution function, and for which you could estimate its parameters. In this latter case, you are likely using the named distribution, e.g. "normal/Gaussian" (the two terms are generally synonymous), as a model that captures some of the key features of your data, rather than claiming the population your data is drawn from exactly follows that theoretical distribution. To slightly misquote George Box, all models are "wrong", but some are useful. If you are thinking about the modelling approach, it is worth considering what features you want to incorporate and how complicated or parsimonious you want your model to be.
Being positively skewed is an example of describing a property that the distribution has, but doesn't come close to specifying which off-the-shelf distribution is "the" appropriate model. It does rule out some candidates, for example the Gaussian (i.e. normal) distribution has zero skew so will not be appropriate to model your data if the skew is an important feature. There may be other properties of the data that are important to you too, e.g. that it's unimodal (has just one peak) or that it is bounded between 0 and 24 hours (or between 0 and 1, if you are writing it as a fraction of the day), or that there is a probability mass concentrated at zero (since there are people who do not watch youtube at all on a given day). You may also be interested in other properties like the kurtosis. And it is worth bearing in mind that even if your distribution had a "hump" or "bell-curve" shape and had zero or near-zero skew, it doesn't automatically follow that the normal distribution is "correct" for it! On the other hand, even if the population your data is drawn from actually did follow a particular distribution precisely, due to sampling error your dataset may not quite resemble it. Small data sets are likely to be "noisy", and it may be unclear whether certain features you can see, e.g. additional small humps or asymmetric tails, are properties of the underlying population the data was drawn from (and perhaps therefore ought to be incorporated in your model) or whether they are just artefacts from your particular sample (and for modelling purposes should be ignored). If you have a small data set and the skew is close to zero, then it is even plausible the underlying distribution is actually symmetric. The larger your data set and the larger the skewness, the less plausible this becomes — but while you could perform a significance test to see how convincing is the evidence your data provides for skewness in the population it was drawn from, this may be missing the point as to whether a normal (or other zero skew) distribution is appropriate as a model ...
Which properties of the data really matter for the purposes you are intending to model it? Note that if the skew is reasonably small and you do not care very much about it, even if the underlying population is genuinely skewed, then you might still find the normal distribution a useful model to approximate this true distribution of watching times. But you should check that this doesn't end up making silly predictions. Because a normal distribution has no highest or lowest possible value, then although extremely high or low values become increasingly unlikely, you will always find that your model predicts there is some probability of watching for a negative number of hours per day, or more than 24 hours. This gets more problematic for you if the predicted probability of such impossible events becomes high. If the skewness is so noteworthy you want to capture it as part of your model, then the skew normal distribution may be more appropriate. If you want to capture both skewness and kurtosis, then consider the skewed t. If you want to incorporate the physically possible upper and lower bounds, then consider using the truncated versions of these distributions. Many other probability distributions exist that can be skewed and unimodal (for appropriate parameter choices) such as the F or gamma distributions, and again you can truncate these so they do not predict impossibly high watching times. A beta distribution may be a good choice if you are modelling the fraction of the day spent watching, as this is always bounded between 0 and 1 without further truncation being necessary. If you want to incorporate the concentration of probability at exactly zero due to non-watchers, then consider building in a hurdle model.
But at the point you are trying to throw in every feature you can identify from your data, and build an ever more sophisticated model, perhaps you should ask yourself why you are doing this? Would there be an advantage to a simpler model, for example it being easier to work with mathematically or having fewer parameters to estimate? If you are concerned that such simplification will leave you unable to capture all of the properties of interest to you, it may well be that no "off-the-shelf" distribution does quite what you want. However, we are not restricted to working with named distributions whose mathematical properties have been elucidated previously. Instead, consider using your data to construct an empirical distribution function. This will capture all the behaviour that was present in your data, but you can no longer give it a name like "normal" or "gamma", nor can you apply mathematical properties that pertain only to a particular distribution. For instance, the "95% of the data lies within 1.96 standard deviations of the mean" rule is for normally distributed data and may not apply to your distribution; though note that some rules apply to all distributions, e.g. Chebyshev's inequality guarantees that at least 75% of your data must lie within two standard deviations of the mean, regardless of the skew. Unfortunately the empirical distribution will also inherit all of the properties of your data set that arose just by sampling error, not just those possessed by the underlying population, so you may find a histogram of your empirical distribution has some humps and dips that the population itself does not — if you want to improve matters, you can try taking a larger sample. You may want to investigate smoothed empirical distribution functions.
In summary: although the normal distribution has zero skew, the fact your data are skewed doesn't rule out the normal distribution as a useful model, though it does suggest some other distribution may be more appropriate. You should also consider other properties of the data when choosing your model, besides the skew, and consider too the purposes you are going to use the model for. It's safe to say that your true population of watching times does not exactly follow some famous, named distribution, but this does not necessarily mean that such a distribution is doomed to be useless as a model. However, for some purposes you may prefer to just use the empirical distribution itself, rather than to fit a standard distribution to it.
$endgroup$
"Is there a better word for that distribution?"
There's a worthwhile distinction here between using words to describe the properties of the distribution, versus trying to find a "name" for the distribution so that you can identify it as (approximately) an instance of a particular standard distribution: one for which a formula or statistical tables might exist for its distribution function, and for which you could estimate its parameters. In this latter case, you are likely using the named distribution, e.g. "normal/Gaussian" (the two terms are generally synonymous), as a model that captures some of the key features of your data, rather than claiming the population your data is drawn from exactly follows that theoretical distribution. To slightly misquote George Box, all models are "wrong", but some are useful. If you are thinking about the modelling approach, it is worth considering what features you want to incorporate and how complicated or parsimonious you want your model to be.
Being positively skewed is an example of describing a property that the distribution has, but doesn't come close to specifying which off-the-shelf distribution is "the" appropriate model. It does rule out some candidates, for example the Gaussian (i.e. normal) distribution has zero skew so will not be appropriate to model your data if the skew is an important feature. There may be other properties of the data that are important to you too, e.g. that it's unimodal (has just one peak) or that it is bounded between 0 and 24 hours (or between 0 and 1, if you are writing it as a fraction of the day), or that there is a probability mass concentrated at zero (since there are people who do not watch youtube at all on a given day). You may also be interested in other properties like the kurtosis. And it is worth bearing in mind that even if your distribution had a "hump" or "bell-curve" shape and had zero or near-zero skew, it doesn't automatically follow that the normal distribution is "correct" for it! On the other hand, even if the population your data is drawn from actually did follow a particular distribution precisely, due to sampling error your dataset may not quite resemble it. Small data sets are likely to be "noisy", and it may be unclear whether certain features you can see, e.g. additional small humps or asymmetric tails, are properties of the underlying population the data was drawn from (and perhaps therefore ought to be incorporated in your model) or whether they are just artefacts from your particular sample (and for modelling purposes should be ignored). If you have a small data set and the skew is close to zero, then it is even plausible the underlying distribution is actually symmetric. The larger your data set and the larger the skewness, the less plausible this becomes — but while you could perform a significance test to see how convincing is the evidence your data provides for skewness in the population it was drawn from, this may be missing the point as to whether a normal (or other zero skew) distribution is appropriate as a model ...
Which properties of the data really matter for the purposes you are intending to model it? Note that if the skew is reasonably small and you do not care very much about it, even if the underlying population is genuinely skewed, then you might still find the normal distribution a useful model to approximate this true distribution of watching times. But you should check that this doesn't end up making silly predictions. Because a normal distribution has no highest or lowest possible value, then although extremely high or low values become increasingly unlikely, you will always find that your model predicts there is some probability of watching for a negative number of hours per day, or more than 24 hours. This gets more problematic for you if the predicted probability of such impossible events becomes high. If the skewness is so noteworthy you want to capture it as part of your model, then the skew normal distribution may be more appropriate. If you want to capture both skewness and kurtosis, then consider the skewed t. If you want to incorporate the physically possible upper and lower bounds, then consider using the truncated versions of these distributions. Many other probability distributions exist that can be skewed and unimodal (for appropriate parameter choices) such as the F or gamma distributions, and again you can truncate these so they do not predict impossibly high watching times. A beta distribution may be a good choice if you are modelling the fraction of the day spent watching, as this is always bounded between 0 and 1 without further truncation being necessary. If you want to incorporate the concentration of probability at exactly zero due to non-watchers, then consider building in a hurdle model.
But at the point you are trying to throw in every feature you can identify from your data, and build an ever more sophisticated model, perhaps you should ask yourself why you are doing this? Would there be an advantage to a simpler model, for example it being easier to work with mathematically or having fewer parameters to estimate? If you are concerned that such simplification will leave you unable to capture all of the properties of interest to you, it may well be that no "off-the-shelf" distribution does quite what you want. However, we are not restricted to working with named distributions whose mathematical properties have been elucidated previously. Instead, consider using your data to construct an empirical distribution function. This will capture all the behaviour that was present in your data, but you can no longer give it a name like "normal" or "gamma", nor can you apply mathematical properties that pertain only to a particular distribution. For instance, the "95% of the data lies within 1.96 standard deviations of the mean" rule is for normally distributed data and may not apply to your distribution; though note that some rules apply to all distributions, e.g. Chebyshev's inequality guarantees that at least 75% of your data must lie within two standard deviations of the mean, regardless of the skew. Unfortunately the empirical distribution will also inherit all of the properties of your data set that arose just by sampling error, not just those possessed by the underlying population, so you may find a histogram of your empirical distribution has some humps and dips that the population itself does not — if you want to improve matters, you can try taking a larger sample. You may want to investigate smoothed empirical distribution functions.
In summary: although the normal distribution has zero skew, the fact your data are skewed doesn't rule out the normal distribution as a useful model, though it does suggest some other distribution may be more appropriate. You should also consider other properties of the data when choosing your model, besides the skew, and consider too the purposes you are going to use the model for. It's safe to say that your true population of watching times does not exactly follow some famous, named distribution, but this does not necessarily mean that such a distribution is doomed to be useless as a model. However, for some purposes you may prefer to just use the empirical distribution itself, rather than to fit a standard distribution to it.
edited 2 hours ago
answered 5 hours ago
SilverfishSilverfish
15.1k1567147
15.1k1567147
add a comment |
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f400316%2fis-a-distribution-that-is-normal-but-highly-skewed-considered-gaussian%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
$begingroup$
As some answers mention but do not emphasise, skewness is named informally for the longer tail if there is one, so right-skewed if a longer right tail. Left and right as used in this context both presuppose a display following a convention that magnitude is shown on the hoirizontal axis. If that sounds too obvious, consider displays in the Earth and environmental sciences in which the magnitude is height or depth and shown vertically. Small print: some measures of skewness can be zero even if a distribution is skewed geometrically.
$endgroup$
– Nick Cox
yesterday
$begingroup$
Total time per day for all users? or time per day per person? If the latter, then surely there's a moderately big spike at 0, in which case you probably need a 'spike and slab' style distribution with a Dirac delta at 0.
$endgroup$
– innisfree
23 hours ago
$begingroup$
"Normal" is synonymous with "Gaussian", and Gaussian distributions, also called normal distributions, are not skewed.
$endgroup$
– Michael Hardy
4 hours ago