Markov decision processes for ad network optimization

Abstract

In this paper we examine a central problem in a particular advertising scheme: we are concerned with matching marketing campaigns that produce advertisements (“ads”), to impressions — where “impression” is a general term for any space in the internet that can display an ad. In this paper we propose a new take on the problem by resorting to planning techniques based on Markov Decision Processes, and by resorting to plan generation techniques that have been developed in the AI literature. We present a detailed formulation of the Markov Decision Process approach and results of simulated experiments