Introduction
to Probability
We live in the world of
uncertainties. Man is surrounded by situations which are not fully under his
control. The nature commands these situations. A person on a road does not know
whether or not he will reach his destination safely. A patient in the hospital
is never sure about his survival after a delicate operation. What will be the
weather conditions tomorrow, nothing is known with certainty but we always like
to have an idea about the weather conditions in future. A flight will be in
time, the road will be clear or there will be some traffic jam. We face this of
problem in our daily life. Man is always curious to know as to what will happen
in future. The things which happen are important for the man today. These
things are based on what is called chance or probability. If we have some
numerical measure of uncertainty, this measure is called probability. We may
find a numerical measure for a bulb to be defective, some numerical measure for
the rain to fall. The belief or confidence associated with a certain situation
can also be measured. It is also called probability. In statistics there are
various situations where uncertainty in involved.
Such situations need the
application of probability. Probability is widely and rightly used in
statistical decisions. The areas of statistics where probability is used are
called the areas of statistical inference. Statistical inference is not
possible without the use of probability. Probability is also used in different
fields of life where uncertainty is involved. Knowledge of probability is used
in space research, astronomy, business, weather studies, economics, genetics
and various other fields of life. It is simple to explain various concepts of
probability with the help of set theory. Thus we shall use here the set theory
notation.
Definition
of Probability
Probability is something
strange and it has been defined in different manners. We can define probability
in objective or subjective manner. Let us first use objective approach to
define probability.
The
Classical Definition of Probability:
This definition is for
equally likely outcomes. If an experiment can produced N mutually exclusive and equally likely
outcomes out of which n outcomes are
favourable to the occurrence of event A, then the probability of A is denoted by P(A) and is defined as the ratio n/N.
Thus the probability of A is
given by
This definition can be applied in
a situation in which all possible outcomes and the outcomes in the events A can be counted. This definition is due to P.S.
Laplace (1749 – 1827). The classical definition is also called the priori
definition of probability. The word priori is from prior and is used because
the definition of base on the previous knowledge that the outcomes are equally
likely. When a coin is tossed, the probability of head is assumed to be ½. This
probability of ½ is based on this classical
definition of probability. It is assumed that the two faces of the coin are
equally likely. In practical life the people do believe that a coin will do
justice when it is tossed. In the playgrounds, the participating teams toss the
coin to start the match. A coin in which probability of head is assumed to be
equal to the probability of tail is called a true or uniform or an unbiased
coin. But it is an all assumption. The probability of a certain event is a
number which lies between 0 and 1. If the event does not contain any outcome, it is
called impossible event and its probability is zero. If the event is as big as
the sample space, the probability of the event is one because
When probability of an event is one, it is called a “Sure” or “Certain”, event.
Criticism:
The classical definition of probability has always been criticized for the following reasons:
The classical definition of probability has always been criticized for the following reasons:
1. This
definition assumes that the outcomes are equally likely. The term equally
likely is almost as difficult as the word probability itself. Thus the
definition uses the circular reasoning.
2. The
definition is not applicable when the numbers of outcomes are not equally
likely.
3. The
definition is also not applicable when the total number of outcomes is infinite
or it is difficult to count the total outcomes or the outcomes favourable to
the event. It is difficult to count the fish in the ocean. Thus it is difficult
to find the probability of catching a fish of some weight say more than one
kilogram.
Example:
One day 20 files were presented to an income tax officer for disposal. Five files contained bogus entries. All the files were thoroughly mixed and there was no indication about bogus files. What is the probability that one file with bogus entries is selected.
One day 20 files were presented to an income tax officer for disposal. Five files contained bogus entries. All the files were thoroughly mixed and there was no indication about bogus files. What is the probability that one file with bogus entries is selected.
Solution:
Here all possible outcomes
= 20
Let A be the event that the
file has bogus entries.
Thus, number of
favourable outcomes = 5
Here we shall apply
the classical definition of probability. All the 20 files are assumed to be equally likely
for the purpose of selecting a file.
Probability of
selecting a file with bogus entries is written as P(A)
Four
Perspectives on Probability
Four perspectives on probability are commonly used: Classical, Empirical, Subjective, and Axiomatic.
1)
Classical (sometimes called "A priori" or
"Theoretical")
This is the perspective on
probability that most people first encounter in formal education (although they
may encounter the subjective perspective in informal education).
For example, suppose we consider
tossing a fair die. There are six possible numbers that could come up
("outcomes"), and, since the die is fair, each one is equally likely
to occur. So we say each of these outcomes has probability 1/6. Since the event
"an odd number comes up" consists of exactly three of these basic
outcomes, we say the probability of "odd" is 3/6, i.e. 1/2.
More generally, if we have a
situation (a "random process") in which there are n equally likely
outcomes, and the event A consists of exactly m of these outcomes, we say that
the probability of A is m/n. We may write this as "P(A) = m/n" for
short.
This perspective has the
advantage that it is conceptually simple for many situations. However, it is
limited, since many situations do not have finitely many equally likely
outcomes. Tossing a weighted die is an example where we have finitely many
outcomes, but they are not equally likely. Studying people's incomes over time
would be a situation where we need to consider infinitely many possible
outcomes, since there is no way to say what a maximum possible income would be,
especially if we are interested in the future.
This
perspective defines probability via a thought experiment.
To get the idea, suppose that we
have a die which we are told is weighted, but we don't know how it is weighted.
We could get a rough idea of the probability of each outcome by tossing the die
a large number of times and using the proportion of times that the die gives
that outcome to estimate the probability of that outcome.
This idea
is formalized to define the probability of the event A as
P (A) =
the limit as n approaches infinity of m/n,
Where n is the number of times
the process (e.g., tossing the die) is performed, and m is the number of times
the outcome A happens.
(Notice that m and n stand for different
things in this definition from what they meant in Perspective 1.)
In other words, imagine tossing
the die 100 times, 1000 times, 10,000 times ... Each time we expect to get a
better and better approximation to the true probability of the event A. The
mathematical way of describing this is that the true probability is the limit of
the approximations, as the number of tosses "approaches infinity"
(that just means that the number of tosses gets bigger and bigger
indefinitely).
This view of probability
generalizes the first view: If we indeed have a fair die, we expect that the
number we will get from this definition is the same as we will get from the
first definition (e.g., P(getting 1) = 1/6; P(getting an odd number) = 1/2). In
addition, this second definition also works for cases when outcomes are not
equally likely, such as the weighted die. It also works in cases where it
doesn't make sense to talk about the probability of an individual outcome. For
example, we may consider randomly picking a positive integer (1, 2, 3 ...) and
ask, "What is the probability that the number we pick is odd?"
Intuitively, the answer should be 1/2, since every other integer (when counted
in order) is odd. To apply this definition, we consider randomly picking 100
integers, then 1000 integers, then 10,000 integers, ... . Each time we
calculate what fraction of these chosen integers are odd. The resulting
sequence of fractions should give better and better approximations to 1/2.
However, the empirical
perspective does have some disadvantages. First, it involves a thought
experiment. In some cases, the experiment could never in practice be carried
out more than once. Consider, for example the probability that the Dow Jones
average will go up tomorrow. There is only one today and one tomorrow. Going
from today to tomorrow is not at all like rolling a die. We can only imagine
all possibilities of going from today to a tomorrow (whatever that means). We can't
actually get an approximation.
A second disadvantage of the
empirical perspective is that it leaves open the question of how large n has to
be before we get a good approximation. The example linked above shows that, as
n increases, we may have some wobbling away from the true value, followed by
some wobbling back toward it, so it's not even a steady process.
The empirical view of probability
is the one that is used in most statistical inference procedures. These are
called frequentist statistics.
Subjective probability is an
individual person's measure of belief that an event will occur. With this view
of probability, it makes perfectly good sense intuitively to talk about the
probability that the Dow Jones average will go up tomorrow. You can quite
rationally take your subjective view to agree with the classical or empirical
views when they apply, so the subjective perspective can be taken as an
expansion of these other views.
However, subjective probability
also has its downsides. First, since it is subjective, one person's probability
(e.g., that the Dow Jones will go up tomorrow) may differ from another's. This
is disturbing to many people. Still, it models the reality that often people do
differ in their judgments of probability.
The second downside is that
subjective probabilities must obey certain "coherence" (consistency)
conditions in order to be workable. For example, if you believe that the
probability that the Dow Jones will go up tomorrow is 60%, then to be
consistent you cannot believe that the probability that the Dow Jones will
do down tomorrow is also 60%. It is easy to fall into
subjective probabilities that are not coherent.
The
subjective perspective of probability fits well with Bayesian statistics, which
are an alternative to the more common frequentist statistical methods. (This
website will mainly focus on frequentist statistics.)
This is a unifying perspective.
The coherence conditions needed for subjective probability can be proved to
hold for the classical and empirical definitions. The axiomatic perspective
codifies these coherence conditions, so can be used with any of the above three
perspectives.
The axiomatic perspective says
that probability is any function (we'll call it P) from events to numbers
satisfying the three conditions (axioms) below. (Just what constitutes events
will depend on the situation where probability is being used.)
The three
axioms of probability:
i)
0 ≤ P(E) ≤ 1 for every allowable event E. (In other
words, 0 is the smallest allowable probability and 1 is the largest allowable
probability).
ii)
The certain event has probability 1. (The certain
event is the event "some outcome occurs." For example, in
rolling a die, the certain event is "One of 1, 2, 3, 4, 5, 6 comes
up." In considering the stock market, the certain event is "The Dow
Jones either goes up or goes down or stays the same.")
iii)
The probability of the union of mutually exclusive
events is the sum of the probabilities of the individual events. (Two events
are called mutually exclusive if they cannot both occur
simultaneously. For example, the events "the die comes up 1" and
"the die comes up 4" are mutually exclusive, assuming we are talking
about the same toss of the same die. The union of events is
the event that at least one of the events occurs. For example, if E is the
event "a 1 comes up on the die" and F is the event "an even
number comes up on the die," then the union of E and F is the event
"the number that comes up on the die is either 1 or even."
If we have a fair die, the axioms
of probability require that each number comes up with
probability 1/6: Since the die is fair, each number comes up with the same
probability. Since the outcomes "1 comes up," "2 comes up,"
..."6 come up" are mutually exclusive and their union is the certain
event, Axiom III says that
P (1 comes up) + P (2 comes up) +
... + P (6 comes up) = P (the certain event),
Which is 1 (by Axiom 2), since
all six probabilities on the left are equal, that common probability must be
1/6.
No comments:
Post a Comment