Since it looks like that there is a very high chance that I’ll have to use Markov Chains in my research, I’ve been searching and reading up on the subject.
The following are the links & articles on Markov Chains that I’ve found useful.
- Markov Chains, a visual explanation is a great introduction to Markov Chains. Like its name implies, the blog post contained a lot of great interactive graphs that offer immense help in understanding what Markov Chains are and how they work. Also, the Hacker News and Reddit threads on the post also contain a lot of interesting discussions, my favorite of which is the following short and concise explanation of what is a Markov Chain: “Markov chain = probabilistic (finite) state machine”.
- Analysis of Chutes and Ladders and Candyland are a great pair of articles that show how to use Markov Chains to analyze complex games of chance.
- The Model Thinking course on Coursera has a nice set of video lectures on the subject.
- This Stack Overflow thread describes how Markov Chains and (finite) state machines are related (in short, Markov Chains can be represented by (finite) state machines).