A stochastic journey to stochastic games

In stochastic games, the games people play today depend on their previous decisions and on chance. The story of our article is a perfect example.

Go to the profile of Christian Hilbe
Jul 04, 2018
Upvote 3 Comment

The story of our article“The evolution of cooperation in stochastic games”, now published in Nature, could already have started eight years ago. It would have started without me, and most likely I wouldn’t have played a part in it. 

Back then in 2010, Krish was invited to deliver a lecture in Martin’s lab, the Program of Evolutionary Dynamics (PED). While the two of them are close collaborators now, they hardly knew each other back then. Krish asked Martin which kind of talk would be most relevant to the diverse audience at PED, offering either a talk on auctions or on stochastic games. Martin found the first topic to be more appealing, so Krish held a lecture on auctions. It was the beginning of a fruitful collaboration at the interface of computer science and biology. In the meanwhile, the other topic of stochastic games pretty much ended up in a time-out state. It remained there for the next five years.

Krish, Christian, and Martin, eight years after the project on stochastic games could have started.

In 2015, I moved from Boston to Vienna. After a two-year postdoc at PED, I was looking forward to working in Krish’s group at IST Austria. Due to a coincidence, Martin happened to be in Austria, too. On my first workday at IST we all sat together and discussed possible research topics. After contemplating various ideas on extortion in repeated games, a topic I had been working on for years, Krish took another chance on proposing stochastic games. This time we were excited immediately. 

 The topic seemed to be a perfect fit. Just like repeated games, stochastic games deal with strategic situations where individuals interact over multiple rounds. However, the exact strategic situation that individuals face could change from one round to the next, depending on the outcome of the previous round and on chance. We were intrigued by this basic premise, the idea that past decisions could affect future possibilities. The framework seemed to be amenable to several open questions in direct reciprocity. Can cooperation evolve if individuals interact in several prisoner’s dilemma games over time? What if the length of a repeated game is not predetermined, but affected by the players’ decisions? 


One of our first results: players are more cooperative in the prisoner’s dilemma if their decisions affect the number of rounds to be played. 

Within a few weeks, we obtained first promising results. We noted that certain stochastic games could facilitate the evolution of cooperation in a way we did not anticipate. At the same time, we noted that the framework offered immense possibilities. A systematic analysis would take time, and we would need all the help we could get. Fortunately, Štěpán, an extremely talented math student from Charles University, had just started an internship at IST to work on his bachelor’s thesis. He happily joined the project, and quickly derived beautiful results on the effect of chance events. Over time, we collected more and more examples highlighting crucial differences between stochastic games and repeated games. By that time, we were sure we had taken the right path, and that a goal was in sight. 

Štěpán discovered the positive impact of chance events on cooperation.

In stochastic games, a simple decision made today can alter the future. The consequences of such a decision may not be immediately apparent, and they may depend on further coincidences, and quite often, on luck. In many ways, our journey to explore stochastic games resembled the very object we were setting out to explore.   

Go to the profile of Christian Hilbe

Christian Hilbe

Post-Doc, IST Austria

No comments yet.