Stochastic Process

Topics covered

• Discrete-time Markov chains: Transition probabilities, hitting probabilities and expected hitting times, recurrence, transience, and communicating classes.
• Long-run behavior: Stationary distributions, limiting behavior, and time reversibility.
• Continuous-time processes: Markov chains, generator matrices, Poisson processes, and birth-death processes.
• Queueing: Queueing systems, Little’s law, and queueing networks.