# setting up a dynamic programming problem with multiple states and controls

For an optimization trouble with numerous states ($x$), controls ($y$), and also arbitrary disruptions ($z$), the Euler formula for a stochastic vibrant shows trouble is:

$D_yU(x,y,z)+\beta E D_xU(x,y,z)=0$

where $D_x$ and also $D_y$ are the slopes relative to the states and also controls, specifically, $\beta$ is the price cut variable, $U$ is the unbiased function, and also $E$ is the assumption driver. This is very easy sufficient to do in the solitary - state instance, yet I'm having troubles obtaining a clear image of the multivariate instance, and also none of the books I have include really lucid instances.

My trouble is:

$\max \quad E_0 \sum_{t=0}^\infty\beta^t u(c_t,N_t)$

based on:

$c_t+K_{t+1}+G_t=f(K_t,N_t,\zeta_t)$

and also

$N_t\leq1$

About the symbols - $u$ is the unbiased function (an energy function), $K$ is the resources supply, $N$ is the percentage of readily available time provided as labor, $\zeta$ is an arbitrary performance disruption, and also $G$ is the federal government is spending plan (exogenous). $f$ is a manufacturing function, and also complete manufacturing relies on both inputs ($N,K$), along with the disruption $\zeta$. Choices are made after the disruption is understood, so the spending plan constraint is constantly binding. $G$ and also $\zeta$ both adhere to Markov procedures. Last but not least, resources entirely decreases every duration, which is why $K_t$ does not turn up on the left hand side of the restraint.

OK, with that said off the beaten track, what I actually need to know is just how to set the trouble up to make sure that I can get first order problems for a remedy. As I see it, if I replace the spending plan restraint right into the unbiased function, the states for the trouble are $\{K,G,\zeta\}$, the controls are $\{K^+,N\}$, and also the arbitrary disruptions are $\{G,\zeta\}$. If I replace these right into the Euler formula I get:

$\left[\begin{matrix} U_{K^+} \\ U_N \end{matrix}\right]+\beta E(U_{K^+})|G,\zeta=0,$

where $U_{K^+}$ and also $U_N$ are the partial by-products of the unbiased function relative to $K^+$ (i.e., $K_{t+1}$) and also $N$ (i.e., $N_t$) specifically. Does this appearance proper, or am I missing out on something? My largest fear is that the measurements of the slopes do not match - - is that a trouble? I would certainly value any kind of aid in all.

On closer examination, it appears that the Euler formula at the end of the inquiry is the proper expression for the essential problems for a remedy. It is made up of 2 problems-- an intertemporal problem on the development of the state variable, $K$:

$U_{K^+}+\beta E\left[(U_{K^+})|G,\zeta \right]=0$

In various other words, the sacrifice in energy obtained by raising $K^+$ needs to be made up for by the increase in predicted future energy that is obtained. To put it simply, raising $K^+$ this duration lowers existing energy, yet raises the anticipated value of future energy moves, and also an optimum is attained (given an indoor remedy exists) when both of these specifically countered each other.

The 2nd problem is the intratemporal problem on $N$:

$U_N=\frac{\partial u}{\partial c}\times\frac{\partial c}{\partial N} + \frac{\partial u}{\partial N}=0$

$u$ is raising in $N$ in its first argument (greater $N$, even more manufacturing), yet lowering in its 2nd argument (lower $N$, even more energy from recreation). Consequently, it is an essential problem for an optimum that the low increase in energy obtained by raising $N$ be countered by the low decrease in energy shed.

Related questions