• Quick note - the problem with Youtube videos not embedding on the forum appears to have been fixed, thanks to ZiprHead. If you do still see problems let me know.

Probabilities

rocketdodger

Philosopher
rocketdodger
Joined
Jun 22, 2005
Messages
6,946
I have a question regarding prior vs posterior probabilities -- can it be said that all prior probabilities are in fact 0.5, since any knowledge at all could be argued to contribute to only posteriors?

For instance, if I roll a typical die, is it valid to say the prior probability of rolling a 6 is actually 0.5, whereas only the posterior is 1/6, since knowing the die has 6 possible values is "further" knowledge?
 
rocketdodger said:
I have a question regarding prior vs posterior probabilities -- can it be said that all prior probabilities are in fact 0.5, since any knowledge at all could be argued to contribute to only posteriors?
Click to expand...

No. Both "prior" and "posterior" refer to your knowledge at a particular point in time. As time advances, your "posterior" probability from one observation will become the "prior" probability for the next one.

For instance, if I roll a typical die, is it valid to say the prior probability of rolling a 6 is actually 0.5, whereas only the posterior is 1/6, since knowing the die has 6 possible values is "further" knowledge?
Click to expand...

Not unless you're the sort of person who starts jokes with "So, a person was born, and attended seminary, and became a priest, while a second person was born, attended yeshiva, and became a rabbi, and a plot of land was cleared, and building was built on it that became a bar...."
 
I've read statistics textbooks that are critical of Bayesian Statistics and I have difficulty with it myself. It seems to confuse probability, which concerns random events that will take place in the future, such as the roll of a pair of dice, with confidence, which concerns personal belief about an existing state of nature, such as the identity of a playing card that is laying face down on a table.
 
An uninformative prior is one that is uniform in a space where the likelihood is data translated. Or something like that. I haven't really touched Bayesian stats outside of computational stats course in a long time...

/many people love Bayesian stats until they deal with nonparametric Bayesian
 
How do you get 0.5? I'd be okay if you said it was "unknown" because you didn't know anything about the event until after you saw the die you rolled. But you could just as easily look at the die right before rolling and figure out the odds. In that case, I don't any other answer besides 1:6 is valid. That is, until you find the die has been tampered with, but I think that's beyond the scope of your question.
 
Towlie said:
I've read statistics textbooks that are critical of Bayesian Statistics and I have difficulty with it myself. It seems to confuse probability, which concerns random events that will take place in the future, such as the roll of a pair of dice, with confidence, which concerns personal belief about an existing state of nature, such as the identity of a playing card that is laying face down on a table.
Click to expand...

We Bayesians are quite aware that what the frequentist means by probability and what we mean by probability are not the same thing, but that they coincide in a broad range of circumstances and obey essentially the same mathematics, which is why we choose to call our level of belief in something a probability.

I don't think it's fair to criticise Bayesian methods for their choice of wording rather than for their actual value in decision making, assessing data and so on.

As for 'prior probabilities' always being 0.5 - no. That reads like a somewhat incomplete formulation of a statement that in the lack of better founded priors you should choose a uniform prior.

There's certainly no reason to use 0.5, it's quite inconsistent with the other questions you might ask (will I roll a 4? will I roll a 17?) - I'd argue much more that if you have no idea what to expect then you should wait till your first results are in before using any prior at all (arguably a uniform prior with limits tending to infinity), and then use those to generate some very broad initial prior. And work onwards from there carefully, taking past posteriors as future priors.
 
Towlie said:
I've read statistics textbooks that are critical of Bayesian Statistics and I have difficulty with it myself. It seems to confuse probability, which concerns random events that will take place in the future, such as the roll of a pair of dice, with confidence, which concerns personal belief about an existing state of nature, such as the identity of a playing card that is laying face down on a table.
Click to expand...

That's because that's not what "probability" means.

Random events that will take place in the future are properly described using "frequency." If you believe that probability and frequency are identical, then you're a frequentist, which is the catch-all term for most non-Bayesians.

If you believe that terms like "likelihood" apply to one-off events (I figure there's at least a 25% chance that Lance Armstrong will win this year's Tour de France) then you're not, properly speaking, a frequentist but a Bayesian.
 
I think he is getting .5 from the idea of binary statistics, either it rolls a 6, or it doesn't. The roll of a die, however, does not meet the criteria for using binary statistics, so the usage is false.
 
Careyp74 said:
I think he is getting .5 from the idea of binary statistics, either it rolls a 6, or it doesn't. The roll of a die, however, does not meet the criteria for using binary statistics, so the usage is false.
Click to expand...

Yeah, that is what I was thinking.

As edd pointed out a uniform prior would be more rational, because it would occur to any rational human that additional numbers might be just as likely, but that just begs my question -- wouldn't the thought "well, it might also end up 17, or 25, etc" count as further knowledge?

At any rate drkitten explained it in terms that make sense to me -- that the distinction isn't really static, it is dynamic and just happens to be related to what you know and don't know at a given point in time. So technically I can't go back and "remove" knowledge already acquired, I.E. the fact that a normal die is six sided will forever be "baked" into any prior I can come up with from this point onward. Is that about right, drkitten?
 
Let me illustrate what I mean about the problem that arises when discussing "probability" in connection with the identity of a playing card laying face down on a table, or any other existing state of nature for that matter.

The probability that a standard die will land with six spots up when rolled randomly is 1/6. That is a property of dice and is not subjective. If I ask three guys what that probability is, they should agree that it's 1/6, and if so, they'll all be correct. If one of them says otherwise he'll be wrong.

Probability is an absolute objective state of nature that can often, although not always, be approximated by experiment or determined by analysis. Probability itself is not subject to opinion or differing points of view, although degree of confidence in an assessment of probability is.

Probability is not frequency, it is only the expected value of a frequency. If a die is rolled several times, it's unlikely that the frequency that a six appears will be exactly 1 out of 6. It's more likely be either higher or lower.

Confidence is different. Confidence is a property of a person and is affected by the knowledge that person has and his ability to properly analyze that knowledge, so different people can have different levels of confidence about an existing state of nature, based on what they know and how they analyze it.

Imagine that you're sitting around a card table with Andy, Bill, and Charlie, who, incidentally, aren't aware of any difference between confidence and probability. You remove the four aces from a deck of cards and set the rest of the deck aside. You stipulate that the aces of Hearts and Diamonds are "red cards" and the aces of Spades and Clubs are "black cards". You ask what the probability is that a randomly drawn card from that set will be red. Everyone agrees that the probability is 1/2.

You then shuffle the cards thoroughly so that it's impossible for anyone present to know which card is which, and you place one card face down in the center of the table. You then deal one card each to Andy, Bill, and Charlie, but warn them not to look at their cards yet. You ask them what the probability is that the card in the center of the table is red. They all impatiently repeat that the probability is 1/2. They feel that they've already answered that question.

You then instruct Andy and Charlie, but not Bill, to each secretly peek at the card he's been dealt. You give each guy a pencil and a piece of paper, and instructions to privately write down his name and the probability that the card in the center of the table is red, then fold up his paper and give it to you.

You then compare the papers. As it turns out, all three are different. Andy says 1/3, Bill says 1/2, and Charlie says 2/3. You then announce that not all answers are the same, and you ask how this could be. An argument breaks out because each person feels absolutely justified in his assessment of a property of a certain playing card, yet they can't all be correct. They begin to grasp at awkward phrases like "probability for you" and probability for me", but since they're talking about a property of a certain playing card, this sounds as weak and irrational as if they were speaking of "your truth" and "my truth".

The bottom line is that there is no probability that the card in the center of the table is red. It's either red or it isn't. At that point in the exercise, it would only have made sense to ask each person to write down his own level of confidence that the card is red, and in that case, what each person wrote would have been correct and there would have been no conflict.

Confidence is a measure of personal certainty about a state of nature.

Probability is a state of nature.
 
Towlie said:
The bottom line is that there is no probability that the card in the center of the table is red. It's either red or it isn't.
Click to expand...

Why can't one take the same point of view before the card is chosen at random from the four and placed on the table? Either it will be red or it won't be; we just don't know which. Why is there such a big difference between not knowing because it hasn't yet been chosen and not knowing because we haven't yet looked at it? Either way, it's the same card, and either way, we don't know which card it is.
 
Towlie said:
Confidence is a measure of personal certainty about a state of nature.

Probability is a state of nature.
Click to expand...

Hmm, I'm not sure I like the word 'confidence' to express this quantity. I'd prefer 'belief' or something like it.

If I get to have a peek at a card that is red, my belief that the card is black drops to zero. If I said my confidence was zero, it sounds too much like I've dropped back to a completely uninformed state rather than being in a completely informed state.

We definitely need a nice new word for Bayesian probability/belief. And then we can go round telling everyone about this word and what it means and thereby teach the entire world the joys of Bayesian statistics ;)

Bayesability perhaps? :)
 
edd said:
Hmm, I'm not sure I like the word 'confidence' to express this quantity. I'd prefer 'belief' or something like it.
Click to expand...
I understand your point, but I'm not making this stuff up. "Confidence" is the term used in the science of Statistics. You could also protest mathematical terms like "irrational" and "imaginary" numbers, and "improper" fractions.
 
Last edited:
Yeah confidence is already used in statistics but doesn't it usually mean something somewhat different from probability? Certainly in the way I use it the two terms are not interchangeable.
What I meant more was that I would prefer confidence was restricted to the way it is currently used in frequentist statistics already, and wasn't taken to be a synonym for Bayesian probability.
 
Towlie said:
Confidence is a measure of personal certainty about a state of nature.

Probability is a state of nature.
Click to expand...

We don't know if the world is "truly" random - and if it's not, all probabilities are simply quantifications of our ignorance.

So I don't think one can really draw this distinction.
 
You must log in or register to reply here.

Back
Top Bottom