Probabilities

[drkitten]

At any rate drkitten explained it in terms that make sense to me -- that the distinction isn't really static, it is dynamic and just happens to be related to what you know and don't know at a given point in time.

Glad to be of help.

So technically I can't go back and "remove" knowledge already acquired, I.E. the fact that a normal die is six sided will forever be "baked" into any prior I can come up with from this point onward. Is that about right, drkitten?

More or less. I mean, you can go back and "remove" the knowledge mathematically, essentially creating and uninformative prior for the distribution that the die generates, and then throw it lots of times until you observe that it never seems to throw above a six and that within the range 1..6 it approximates a uniform distribution to whatever degree of certainty you like.

But what a horrid waste of effort to re-learn what you already know. The only reason I can imagine wanting to do that is if I suspected that the die was NOT really random. But in this case, I can approximate the same thing by using as a prior the fact that the die is uniformly random and then generating a new distribution for the (possibly loaded) die as a posterior, which gives me the same results for less work....

 

[Towlie]

We don't know if the world is "truly" random - and if it's not, all probabilities are simply quantifications of our ignorance.

So I don't think one can really draw this distinction.

It appears to me that you just invalidated just about every branch of science that exists. The existence of randomness is certainly, at the very least, an elegant scientific theory that does an excellent job of describing observations and making predictions.

But even if you're right and randomness doesn't really exist, there's still a difference between probability and confidence, just as there's a difference between a horse and a unicorn. The central point is that the two words have different meanings.

 

[sol invictus]

It appears to me that you just invalidated just about every branch of science that exists.

I did!!!

I certainly didn't intend to...

The existence of randomness is certainly, at the very least, an elegant scientific theory that does an excellent job of describing observations and making predictions.

Sure - but my point was that it works for parametrizing ignorance too. For example, surely you would agree that flipping a coin is not "truly" random - if you knew the parameters of the flip and catch, standard Newtonian dynamics would tell you how it ends up. But because you don't know them, you parametrize your ignorance with a probability - and it works very well, even though there's little - or perhaps no - true randomness involved.

Same goes for pseudo-random number generators on a computer.

But even if you're right and randomness doesn't really exist, there's still a difference between probability and confidence, just as there's a difference between a horse and a unicorn. The central point is that the two words have different meanings.

Obviously. I just don't think it's what you said it was.

 

[Towlie]

In a nutshell, probability is an objective concept that applies to the outcome of an event that may take place in the future. Confidence is a subjective property of an observer concerning a state of nature.

I have a similar complaint about the phrase "he looks suspicious", which I think has a confusing double meaning. To me, a person looks suspicious when he has a doubting expression on his face, suggesting that he suspects something. A person does not "look suspicious" when he's sneaking around and looking at other people's valuables as if he intends to steal them. We need another word for that. When you spot someone sneaking around like that, he's not suspicious, you are. It's another case of applying an attribute to an external object that is really an attribute of yourself.

 

[rsaavedra]

I have a similar complaint about the phrase "he looks suspicious", which I think has a confusing double meaning.

In Spanish we have "suspicaz" as opposed to "sospechoso".

Someone that is "suspicaz" thinks that someone else might be guilty of something, or distrusts that other person. Someone that is "sospechoso" is the person being thought of as possibly guilty of something, or the person not being trusted. You look up either suspicaz or sospechoso in a Spanish-English dictionary, and for both you get "suspicious".

Doesn't make much sense indeed to use the same word to describe both of them, but so are languages I guess.

 

[fls]

We Bayesians are quite aware that what the frequentist means by probability and what we mean by probability are not the same thing, but that they coincide in a broad range of circumstances and obey essentially the same mathematics, which is why we choose to call our level of belief in something a probability.

I don't think it's fair to criticise Bayesian methods for their choice of wording rather than for their actual value in decision making, assessing data and so on.

If it helps, the prior and posterior probabilities when Bayesian analysis is used for diagnostic tests are measured frequencies.

As for 'prior probabilities' always being 0.5 - no. That reads like a somewhat incomplete formulation of a statement that in the lack of better founded priors you should choose a uniform prior.

I think the recommendation is that you should choose an uninformative prior when faced with a lack of knowledge. Sometimes a uniform prior is uninformative, but sometimes it is not. However, the recommendation of a uniform prior (no matter how misguided) is something I have previously encountered from someone claiming to be a statistician, so I suspect that you are correct about the possible source of the statement.

There's certainly no reason to use 0.5, it's quite inconsistent with the other questions you might ask (will I roll a 4? will I roll a 17?) - I'd argue much more that if you have no idea what to expect then you should wait till your first results are in before using any prior at all (arguably a uniform prior with limits tending to infinity), and then use those to generate some very broad initial prior. And work onwards from there carefully, taking past posteriors as future priors.

The purpose of an uninformative prior is to allow for the least influence of the choice of prior on the posterior probability. This means that the posterior probability is allowed to mostly reflect the observation/condition. This is essentially the same as saying that one shouldn't bother with priors in the face of uncertainty, but rather rely on gathering information through observation instead (i.e. what you said).

Linda

 

[fls]

Let me illustrate what I mean about the problem that arises when discussing "probability" in connection with the identity of a playing card laying face down on a table, or any other existing state of nature for that matter.

The probability that a standard die will land with six spots up when rolled randomly is 1/6. That is a property of dice and is not subjective.

Does this statement reflect the problem that Bayesians have with Frequentists, and the problem referred to in the OP - is it too easy to confuse the number of outcomes with the frequency of those outcomes?

If I ask three guys what that probability is, they should agree that it's 1/6, and if so, they'll all be correct. If one of them says otherwise he'll be wrong.

You know this is incorrect, right? We ask that standard dice conform to an ideal within a certain margin of error, but we recognize that there's no objective reason for the dice to land with each number up an equal number of times. This simply reflects a measured property.

Probability is an absolute objective state of nature that can often, although not always, be approximated by experiment or determined by analysis. Probability itself is not subject to opinion or differing points of view, although degree of confidence in an assessment of probability is.

Probability is not frequency, it is only the expected value of a frequency. If a die is rolled several times, it's unlikely that the frequency that a six appears will be exactly 1 out of 6. It's more likely be either higher or lower.

Confidence is different. Confidence is a property of a person and is affected by the knowledge that person has and his ability to properly analyze that knowledge, so different people can have different levels of confidence about an existing state of nature, based on what they know and how they analyze it.

Imagine that you're sitting around a card table with Andy, Bill, and Charlie, who, incidentally, aren't aware of any difference between confidence and probability. You remove the four aces from a deck of cards and set the rest of the deck aside. You stipulate that the aces of Hearts and Diamonds are "red cards" and the aces of Spades and Clubs are "black cards". You ask what the probability is that a randomly drawn card from that set will be red. Everyone agrees that the probability is 1/2.

You then shuffle the cards thoroughly so that it's impossible for anyone present to know which card is which, and you place one card face down in the center of the table. You then deal one card each to Andy, Bill, and Charlie, but warn them not to look at their cards yet. You ask them what the probability is that the card in the center of the table is red. They all impatiently repeat that the probability is 1/2. They feel that they've already answered that question.

You then instruct Andy and Charlie, but not Bill, to each secretly peek at the card he's been dealt. You give each guy a pencil and a piece of paper, and instructions to privately write down his name and the probability that the card in the center of the table is red, then fold up his paper and give it to you.

You then compare the papers. As it turns out, all three are different. Andy says 1/3, Bill says 1/2, and Charlie says 2/3. You then announce that not all answers are the same, and you ask how this could be. An argument breaks out because each person feels absolutely justified in his assessment of a property of a certain playing card, yet they can't all be correct. They begin to grasp at awkward phrases like "probability for you" and probability for me", but since they're talking about a property of a certain playing card, this sounds as weak and irrational as if they were speaking of "your truth" and "my truth".

The bottom line is that there is no probability that the card in the center of the table is red. It's either red or it isn't. At that point in the exercise, it would only have made sense to ask each person to write down his own level of confidence that the card is red, and in that case, what each person wrote would have been correct and there would have been no conflict.

Confidence is a measure of personal certainty about a state of nature.

Probability is a state of nature.

I still don't understand the distinction you are making. If probability represents the possible outcomes and the frequency of those outcomes, then your description of confidence is no different from a description of probability. All you have done is to use knowledge to trim away some of the possible outcomes, which changes the frequency of the remaining outcomes. Confidence (as you are using it) is simply identical to knowledge about which outcomes are no longer possible and their frequency - the same information that is used to represent probability.

Linda

 

[Towlie]

Does this statement reflect the problem that Bayesians have with Frequentists, and the problem referred to in the OP - is it too easy to confuse the number of outcomes with the frequency of those outcomes?

I'll pass on that one. I have no idea what you're asking me.

You know this is incorrect, right?

No, if I knew it to be incorrect I wouldn't have written it. That's something that trolls do and I'm not a troll.

We ask that standard dice conform to an ideal within a certain margin of error, but we recognize that there's no objective reason for the dice to land with each number up an equal number of times. This simply reflects a measured property.

Only superstitious gamblers ask dice to do anything. But yes, we do recognize that there's no objective reason for a die to land with each number up an equal number of times. That's pretty much what I said in my fourth paragraph. Probability determines the expected outcome, not the actual outcome, and the expected outcome of multiple rolls of a die is for it to land with each number up an equal number of times.

But that doesn't mean we actually expect that to happen. Indeed, unless the number of rolls is divisible by six, it's impossible. We only recognize that as the number of trials increases, the ratio of the actual outcome to the expected outcome tends to approach unity, the difference between the actual outcome and the expected outcome tends to increase without limit, and the probability of the actual outcome equaling the expected outcome (i.e. a die landing with each number up an equal number of times) asymptotically approaches zero.

I still don't understand the distinction you are making. If probability represents the possible outcomes and the frequency of those outcomes, then your description of confidence is no different from a description of probability.

But as I said before, it doesn't represent the frequency of outcomes, it only represents the expected frequency of outcomes. On the other hand, confidence (as I am using it) doesn't even apply to outcomes. It applies to already existing but uncertain states of nature.

All you have done is to use knowledge to trim away some of the possible outcomes, which changes the frequency of the remaining outcomes. Confidence (as you are using it) is simply identical to knowledge about which outcomes are no longer possible and their frequency - the same information that is used to represent probability.

Sorry, I guess we're just not communicating very well. There's not much more I can say to make myself more clear.

 

[fls]

I'll pass on that one. I have no idea what you're asking me.

It wasn't addressed to you specifically, but rather to anyone that may have some insight. I don't understand why there is any supposed conflict between Frequentism and Bayesianism.

No, if I knew it to be incorrect I wouldn't have written it. That's something that trolls do and I'm not a troll.

I meant something like saying things that are incorrect or incomplete as a sort of shorthand for getting to the main point. For example, I make reference to "statistically significant", but that phrase is at best incomplete and sometimes downright incorrect. But the meaning it conveys under the circumstances that I use it is correct, so it serves as a shorthand representation of a longer and more convoluted statement.

Only superstitious gamblers ask dice to do anything. But yes, we do recognize that there's no objective reason for a die to land with each number up an equal number of times. That's pretty much what I said in my fourth paragraph. Probability determines the expected outcome, not the actual outcome, and the expected outcome of multiple rolls of a die is for it to land with each number up an equal number of times.

I understand what you are saying, but that is not the distinction that I was making. We don't expect each number to land up an equal (or roughly equal) number of times either. As an exaggerated example, if the dice were loaded, then we would expect them to land with one number up most of the time. But even if there isn't deliberate manipulation, there isn't any particular reason to think that a physical object that isn't compositionally symmetrical will exhibit symmetry in outcomes.

But that doesn't mean we actually expect that to happen. Indeed, unless the number of rolls is divisible by six, it's impossible. We only recognize that as the number of trials increases, the ratio of the actual outcome to the expected outcome tends to approach unity, the difference between the actual outcome and the expected outcome tends to increase without limit, and the probability of the actual outcome equaling the expected outcome (i.e. a die landing with each number up an equal number of times) asymptotically approaches zero.But as I said before, it doesn't represent the frequency of outcomes, it only represents the expected frequency of outcomes.

My point was only that your expectation as to the expected frequency of outcomes should be based on observation, not on counting up the number of different outcomes and assuming that all have equal frequency. And I wondered if this was a tendency among Frequentists (which then justifies Bayesians' misgivings).

On the other hand, confidence (as I am using it) doesn't even apply to outcomes. It applies to already existing but uncertain states of nature.

But your distinction between expectation and already existing but uncertain seems artificial. What difference is there between a state we don't have information on because it hasn't happened yet and a state that has happened but we don't have information on because it is hidden? In both cases we are basing our expectation upon an understanding of the frequency of various possible outcomes. Regardless of why the outcome is unknown (i.e. it hasn't yet taken place vs. it has been hidden), the point is that guesses about what that outcome will be or is are based on the same information.

Sorry, I guess we're just not communicating very well. There's not much more I can say to make myself more clear.

I hope you keep trying. I have seen this argument before and I would like to understand where it comes from.

Linda

 

[Ivor the Engineer]

Frequentists are just jealous because Bayesians have bigger computers.

http://mh1823.com/frequentists_and_bayesians.htm

The frequentists definition sees probability as the long-run expected frequency of occurrence. P(A) = n/N, where n is the number of times event A occurs in N opportunities. The Bayesian view of probability is related to degree of belief. It is a measure of the plausibility of an event given incomplete knowledge.

Thus a frequentist believes that a population mean is real, but unknown, and unknowable, and can only be estimated from the data. Knowing the distribution for the sample mean, he constructs a confidence interval, centered at the sample mean.

...

Bayesians have an altogether different world-view. They say that only the data are real. The population mean is an abstraction, and as such some values are more believable than others based on the data and their prior beliefs. (Sometimes the prior belief is very non-informative, however.) The Bayesian constructs a credible interval, centered near the sample mean, but tempered by "prior" beliefs concerning the mean.

Now the Bayesian can say what the frequentist cannot: "There is a 95% probability(2) that this interval contains the mean."

 

[fls]

Frequentists are just jealous because Bayesians have bigger computers.

http://mh1823.com/frequentists_and_bayesians.htm

I don't understand how the two statements you have highlighted are different. They seem to be saying the same thing - that sample means are used to estimate population means, and that an interval can be formed which is likely to contain the mean that would be obtained if the entire population were measured.

Linda

 

[Careyp74]

Confidence is a little different in statistics than what Towlie is describing. It is not subjective. When asked of a specific incident, three statisticians are going to give you the same confidence interval for any given confidence level. It is not how confident you are that the card is red, whether you looked at it or not, it is based on probability.

For example, if there was a class room and 100 students were going to walk into the room, at any time, based on the student population, you can give a probability of whether the next person that walks in is male or female, and the probability would be different based on the confidence level. You could say, I am 25% confident that the next person is male. Or you could say I am 75% confident that out of the next 10 people 5 of them will be male. Or you could say I am 95% confident that of the 100 people that are going to walk into the room, 50 or them will be male.

 

[Jorghnassen]

When asked of a specific incident, three statisticians are going to give you the same confidence interval for any given confidence level.

Not necessarily... There's more than one way to construct a confidence interval, and you could have 3 different intervals with the same confidence for the same quantity, using the same sample. But you are right that confidence is not subjective.

 

[Ivor the Engineer]

I don't understand how the two statements you have highlighted are different. They seem to be saying the same thing - that sample means are used to estimate population means, and that an interval can be formed which is likely to contain the mean that would be obtained if the entire population were measured.

Linda

Not quite. The Frequentist point of view excludes the possibility of saying there is a 95% chance the population value lies within the 95% confidence interval. But that's generally what we want to be able to say. The Bayesian credible interval does allow such statements to be made.

 

[fls]

Not quite. The Frequentist point of view excludes the possibility of saying there is a 95% chance the population value lies within the 95% confidence interval. But that's generally what we want to be able to say. The Bayesian credible interval does allow such statements to be made.

There was an example in a thread here which adequately illustrated the difference. I have to think on where that was.

Linda

 

[69dodge]

I don't understand how the two statements you have highlighted are different. They seem to be saying the same thing - that sample means are used to estimate population means, and that an interval can be formed which is likely to contain the mean that would be obtained if the entire population were measured.

According to frequentists:

The population mean is unknown, but fixed, so one cannot make any probability statements about it. A sample drawn from the population is the result of a random process (i.e., sampling randomly), so one can make probability statements about such samples.

The endpoints of a confidence interval are computed from a random sample. The confidence interval is therefore random too, so one can make probability statements about it too.

Before sampling, one can say, "the probability is 95% that a future random sample will be such as to yield a confidence interval that surrounds the fixed population mean." After sampling, the sample and the confidence interval computed from it are known, and the population mean is still fixed (though still unknown), so, strictly speaking, one cannot make any probability statement at all about their relation. Either the mean does lie within the computed interval or it doesn't; we just don't know which is the case.

 

[fls]

According to frequentists:

The population mean is unknown, but fixed, so one cannot make any probability statements about it. A sample drawn from the population is the result of a random process (i.e., sampling randomly), so one can make probability statements about such samples.

The endpoints of a confidence interval are computed from a random sample. The confidence interval is therefore random too, so one can make probability statements about it too.

Before sampling, one can say, "the probability is 95% that a future random sample will be such as to yield a confidence interval that surrounds the fixed population mean." After sampling, the sample and the confidence interval computed from it are known, and the population mean is still fixed (though still unknown), so, strictly speaking, one cannot make any probability statement at all about their relation. Either the mean does lie within the computed interval or it doesn't; we just don't know which is the case.

The prior thread I was thinking of spoke to a peripherally related point, so it won't be of use.

The Bayesian is taking essentially the same information that the Frequentist uses, but modifying it based on prior information. What does this really mean when there is no prior information? I realize that this is of some value when the prior information is of a type that is unusable to a Frequentist (e.g. a guesstimate as to low, intermediate or high probability of pulmonary embolism based on clinical factors), but what if we are talking about the situation described in the OP, where there isn't really any prior information?

Linda

 

[Ivor the Engineer]

The prior thread I was thinking of spoke to a peripherally related point, so it won't be of use.

The Bayesian is taking essentially the same information that the Frequentist uses, but modifying it based on prior information. What does this really mean when there is no prior information? I realize that this is of some value when the prior information is of a type that is unusable to a Frequentist (e.g. a guesstimate as to low, intermediate or high probability of pulmonary embolism based on clinical factors), but what if we are talking about the situation described in the OP, where there isn't really any prior information?

Linda

With flat priors, Bayesian estimates are equivalent to Maximum likelihood estimates.

 

[fls]

With flat priors, Bayesian estimates are equivalent to Maximum likelihood estimates.

Yeah, that's an example of what I mean.

Linda

 

[edd]

With flat priors, Bayesian estimates are equivalent to Maximum likelihood estimates.

Numerically they might be equal, but they have a quite different meaning. And there are not infrequently difficulties in deciding what measurement your prior should be flat in.