Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to characterize this probability. Next we give a more mathematical description of the problem. Now define your fortune at time n ≥ 1 by - Xn = Xn-1+ I{Y=1} = I{Yn=0} with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk = L}. Finally, let Q(x0) = P{XT = . L}. Show that X is a Markov chain and obtain its transition probability matrix. Is it irreducible? Use conditional expectations to prove that Q(x)=pQ(x+1) + (1 − p)Q(x − 1), - x = 1, L-1, (1) with Q(0) 0 and Q(L) = 1. = d) Solve the difference equation in (1) analytically. Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1. Hint: Use the same form of the difference equation and analytical solution as in (d) but use different initial conditions. f) Show that all intermediate states are transient and that both absorbing states are recurrent. Use (f) to show that the gambler's fortune converges almost surely to a binary random variable and characterize the limit.
Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to characterize this probability. Next we give a more mathematical description of the problem. Now define your fortune at time n ≥ 1 by - Xn = Xn-1+ I{Y=1} = I{Yn=0} with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk = L}. Finally, let Q(x0) = P{XT = . L}. Show that X is a Markov chain and obtain its transition probability matrix. Is it irreducible? Use conditional expectations to prove that Q(x)=pQ(x+1) + (1 − p)Q(x − 1), - x = 1, L-1, (1) with Q(0) 0 and Q(L) = 1. = d) Solve the difference equation in (1) analytically. Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1. Hint: Use the same form of the difference equation and analytical solution as in (d) but use different initial conditions. f) Show that all intermediate states are transient and that both absorbing states are recurrent. Use (f) to show that the gambler's fortune converges almost surely to a binary random variable and characterize the limit.
College Algebra (MindTap Course List)
12th Edition
ISBN:9781305652231
Author:R. David Gustafson, Jeff Hughes
Publisher:R. David Gustafson, Jeff Hughes
Chapter5: Exponential And Logarithmic Functions
Section5.2: Applications Of Exponential Functions
Problem 35E
Related questions
Question
Transcribed Image Text:Suppose that you start with an initial fortune of 20 dollars, and you bet one dollar
each time. The probability of winning each hand is p. You quit if you reach your goal of L dollars
or when you go broke. Let Q(xo) denote that probability that you eventually win. We need to
characterize this probability. Next we give a more mathematical description of the problem.
Now define your fortune at time n ≥ 1 by
-
Xn = Xn-1+ I{Y=1} = I{Yn=0}
with the initial condition X = x0 ≥0. Let L = N be given, and define T inf{k : Xk = 0 or Xk =
L}. Finally, let Q(x0) = P{XT = .
L}.
Show that X is a Markov chain and obtain its transition probability matrix.
Is it irreducible?
Use conditional expectations to prove that
Q(x)=pQ(x+1) + (1 − p)Q(x − 1),
-
x = 1, L-1,
(1)
with Q(0) 0 and Q(L) = 1.
=
d) Solve the difference equation in (1) analytically.
Let P(x0) P{XT = 0}. Determine P(x) directly and show that P(x) + Q(x) = 1.
Hint: Use the same form of the difference equation and analytical solution as in (d) but use
different initial conditions.
f) Show that all intermediate states are transient and that both absorbing states are recurrent.
Use (f) to show that the gambler's fortune converges almost surely to a binary random
variable and characterize the limit.
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
This is a popular solution!
Trending now
This is a popular solution!
Step by step
Solved in 2 steps
Recommended textbooks for you
College Algebra (MindTap Course List)
Algebra
ISBN:
9781305652231
Author:
R. David Gustafson, Jeff Hughes
Publisher:
Cengage Learning
Algebra & Trigonometry with Analytic Geometry
Algebra
ISBN:
9781133382119
Author:
Swokowski
Publisher:
Cengage
Algebra: Structure And Method, Book 1
Algebra
ISBN:
9780395977224
Author:
Richard G. Brown, Mary P. Dolciani, Robert H. Sorgenfrey, William L. Cole
Publisher:
McDougal Littell
College Algebra (MindTap Course List)
Algebra
ISBN:
9781305652231
Author:
R. David Gustafson, Jeff Hughes
Publisher:
Cengage Learning
Algebra & Trigonometry with Analytic Geometry
Algebra
ISBN:
9781133382119
Author:
Swokowski
Publisher:
Cengage
Algebra: Structure And Method, Book 1
Algebra
ISBN:
9780395977224
Author:
Richard G. Brown, Mary P. Dolciani, Robert H. Sorgenfrey, William L. Cole
Publisher:
McDougal Littell
Calculus For The Life Sciences
Calculus
ISBN:
9780321964038
Author:
GREENWELL, Raymond N., RITCHEY, Nathan P., Lial, Margaret L.
Publisher:
Pearson Addison Wesley,
Glencoe Algebra 1, Student Edition, 9780079039897…
Algebra
ISBN:
9780079039897
Author:
Carter
Publisher:
McGraw Hill