\documentclass[10pt]{article}
\usepackage{fullpage}
\usepackage{setspace}
\usepackage{parskip}
\usepackage{titlesec}
\usepackage[section]{placeins}
\usepackage{xcolor}
\usepackage{breakcites}
\usepackage{lineno}
\usepackage{hyphenat}
\PassOptionsToPackage{hyphens}{url}
\usepackage[colorlinks = true,
linkcolor = blue,
urlcolor = blue,
citecolor = blue,
anchorcolor = blue]{hyperref}
\usepackage{etoolbox}
\makeatletter
\patchcmd\@combinedblfloats{\box\@outputbox}{\unvbox\@outputbox}{}{%
\errmessage{\noexpand\@combinedblfloats could not be patched}%
}%
\makeatother
\usepackage[round]{natbib}
\let\cite\citep
\renewenvironment{abstract}
{{\bfseries\noindent{\abstractname}\par\nobreak}\footnotesize}
{\bigskip}
\titlespacing{\section}{0pt}{*3}{*1}
\titlespacing{\subsection}{0pt}{*2}{*0.5}
\titlespacing{\subsubsection}{0pt}{*1.5}{0pt}
\usepackage{authblk}
\usepackage{graphicx}
\usepackage[space]{grffile}
\usepackage{latexsym}
\usepackage{textcomp}
\usepackage{longtable}
\usepackage{tabulary}
\usepackage{booktabs,array,multirow}
\usepackage{amsfonts,amsmath,amssymb}
\providecommand\citet{\cite}
\providecommand\citep{\cite}
\providecommand\citealt{\cite}
% You can conditionalize code for latexml or normal latex using this.
\newif\iflatexml\latexmlfalse
\AtBeginDocument{\DeclareGraphicsExtensions{.pdf,.PDF,.eps,.EPS,.png,.PNG,.tif,.TIF,.jpg,.JPG,.jpeg,.JPEG}}
\usepackage[utf8]{inputenc}
\usepackage[ngerman,greek,english]{babel}
\begin{document}
\title{Constrained optimization}
\author[1]{Lennart Stern}%
\affil[1]{École normale supérieure}%
\vspace{-1em}
\date{\today}
\begingroup
\let\center\flushleft
\let\endcenter\endflushleft
\maketitle
\endgroup
\sloppy
Description: In the coming capsules we will often be faced with
constrained optimization problems. In this capsule we will motivate,
define and practice the Lagrangian method for solving these problems.
Approximate duration: 25 minutes
Resource:
In this resource we give an overview. Feel free to skip reading it, the questions do not
require the resource.
In the coming capsules we will often be faced with problems such as that
of finding which consumption bundles \(x\) maximize \(u(x)\) subject to
\(p\ x \leq y\). We will follow the following procedure: We will write
down the Lagrangian \(L = u - \lambda\ p\ x\), differentiate with
respect to each component of \(x\) and then consider the set of
conditions
\(\{\frac{\partial L}{\partial x_{1}} = 0,\ \ldots,\ \frac{\partial L}{\partial x_{1}} = 0,px = y \}\).
These are \(n + 1\) conditions in the \(n + 1\) unkwowns
\(\{ x_{1},\ldots,x_{n},\lambda\}\). We then find the solutions to this
system of equations. Typically (i.e. in the applications in economics),
the expressions \((x_{1},\ldots,x_{n})\) that we find in this way is a
global maximum of \(L = u - \lambda\ p\ x\). In this capsule we will see
why any such solution is also a solution to the original problem of
maximizing \(u(x)\) subject to \(p\ x \leq y\).
For illustration we will at the end of this capsule study the special case
of above the problem where there are two goods and
\(u\left( x_{1},x_{2} \right) = \log{(x_{1})} + log(x_{2})\).
\emph{Question 1}
Consider a consumer who is trying to choose the quantities \((x_{1},x_{2})\) he consumes of two goods so as to maximize his utility \(u(x_{1},x_{2})\).
If he did not take into account the constraint that
\(p_{1}x_{1} + p_{2}x_{2} \leq y\), the consumer would want to choose
both \(x_{1}\) and \(x_{2}\) as high as possible. How could the consumer
mathematically try to take into account the consideration that his
expenditure \(p_{1}x_{1} + p_{2}x_{2}\) should not be too high? A
natural thing to try is to add a penalty term to the consumer's
objective. Thus instead of maximizing \(u(x_{1},x_{2})\) without any concern
for his expenditure the consumer could consider maximizing
\[u(x_{1},x_{2}) - \lambda\left( p_{1}x_{1} + p_{2}x_{2} \right)\]
Here \(\lambda\) could be called `the penalty weight'. Let us denote by
\(\left( x_{1}\left( \lambda \right),x_{2}\left( \lambda \right) \right)\)
the solution to the problem of maximizing
\(u(x_{1},x_{2}) - \lambda\left( p_{1}x_{1} + p_{2}x_{2} \right)\).
Intuitively (before doing any computations), which of the following
statements would you expect to be true?
True: The higher the \(\lambda\), the lower the
\(\left( p_{1}x_{1}\left( \lambda \right) + p_{2}x_{2}\left( \lambda \right) \right)\)
False: The higher the \(\lambda\), the higher the
\(\left( p_{1}x_{1}(\lambda) + p_{2}x_{2}(\lambda) \right)\)
True: The higher the \(\lambda\), the lower the
\(u(x_{1}\left( \lambda \right),x_{1}\left( \lambda \right))\)
False: The higher the \(\lambda\), the higher the
\(u(x_{1}\left( \lambda \right),x_{1}\left( \lambda \right))\)
Explanation:
Choosing a high \(\lambda\) means penalizing oneself a lot for spending
money. Intuitively, if one increases the penalty for the expenditures
\(p_{1}x_{1} + p_{2}x_{2}\), one will end up making a choice with lower
expenditures.
Intuitively, by choosing a higher penalty rate \(\lambda\), one
prioritizes more the consideration of reducing expenditure. This must
necessarily come at the expense of the utility.
Both of these intuitively plausible statements can be proved formally.
\emph{Question 2}
The considerations from the previous question make it plausible that there will be
some value \(\lambda^{*}\) for \(\lambda\) such that
\(\left( p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) \right) = y\).
Intuitively, this value for \(\lambda^{*}\) is just the right level of
penalty that the consumer should impose himself for spending money.
Consider any other pair \((x_{1},x_{2})\). What do we know about
\((x_{1},x_{2})\)?
True: We must have \(u\left( x_{1},x_{2} \right) - \lambda^{*}\left( x_{1}p_{1} + x_{2}p_{2} \right) \leq u(x_{1}\left( \lambda^{*} \right),x_{2}(\lambda^{*})) - \lambda^{*}\left( p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) \right)\)
False: We must have
\(u\left( x_{1},x_{2} \right) - \lambda^{*}\left( x_{1}p_{1} + x_{2}p_{2} \right) \geq u(x_{1}\left( \lambda^{*} \right),x_{2}(\lambda^{*})) - \lambda^{*}\left( p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) \right)\)
True: If \(u(x_{1},x_{2}) > u(x_{1}(\lambda^{*}),x_{2}(\lambda^{*}))\) then \(x_{1}p_{1} + x_{2}p_{2} > p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right)\)
False: If \(u(x_{1},x_{2}) > u(x_{1}(\lambda^{*}),x_{2}(\lambda^{*}))\) then \(x_{1}p_{1} + x_{2}p_{2} < p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right)\)
True: If \(u(x_{1},x_{2}) > u(x_{1}(\lambda^{*}),x_{2}(\lambda^{*}))\) then \(x_{1}p_{1} + x_{2}p_{2} > y\)
False: If \(u(x_{1},x_{2}) > u(x_{1}(\lambda^{*}),x_{2}(\lambda^{*}))\) then \(x_{1}p_{1} + x_{2}p_{2} < y\)
True: If the pair \((x_{1},x_{2})\) gives strictly higher utility than the
pair
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\),
then \((x_{1},x_{2})\) is not affordable.
True: If the pair \((x_{1},x_{2})\) gives strictly higher utility than the
pair
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\),
then \((x_{1},x_{2})\) is more expensive than
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\).
Explanation:
First we note that we have\(u\left( x_{1},x_{2} \right) - \lambda^{*}\left( x_{1}p_{1} + x_{2}p_{2} \right) \leq u(x_{1}\left( \lambda^{*} \right),x_{2}(\lambda^{*})) - \lambda^{*}\left( p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) \right)\) by the definition of
\((x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right))\)
From this we deduce: If \(u(x_{1},x_{2}) > u(x_{1}(\lambda^{*}),x_{2}(\lambda^{*}))\) then \(x_{1}p_{1} + x_{2}p_{2} > p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right)\)
Since \(\left( p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) \right) = y\) we deduce:
If \(u(x_{1},x_{2}) > u(x_{1}(\lambda^{*}),x_{2}(\lambda^{*}))\) then \(x_{1}p_{1} + x_{2}p_{2} > y\)
\emph{Question 3}
Let us ponder what we have found: For any pair \((x_{1},x_{2})\) we know
that if it gives strictly higher utility than the pair
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\),
then \((x_{1},x_{2})\) is not affordable.
What can we conclude from this?
True
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\)
is a solution to our original problem of maximizing
\(u\left( x_{1},x_{2} \right) \) under the budget constraint
\(p_{1}x_{1} + p_{2}x_{2} \leq y\).
True No pair \((x_{1},x_{2})\) that is affordable can give a strictly
higher utility than
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\).
\emph{Question 4}
So far we have shown that if we find a value \(\lambda^{*}\) such that
the pair
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\)
that maximizes
\(L = u(x_{1},x_{2}) - \lambda^{*}\left( p_{1}x_{1} + p_{2}x_{2} \right)\)
satisfies the budget constraint with equality (i.e. such that
\(p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) = y\)
then
\(\left( x_{1}\left( \lambda^{*} \right),x_{2}\left( \lambda^{*} \right) \right)\)
is a solution to the original problem of maximizing
\(u\left( x_{1},x_{2} \right)\) under the budget constraint
\(p_{1}x_{1} + p_{2}x_{2} \leq y\).
Now let us try to actually find this value \(\lambda^{*}\) for our
particular problem with
\(u\left( x_{1},x_{2} \right) = \log\left( x_{1} \right) + log(x_{2})\).
First let us first find
\(\left( x_{1}\left( \lambda \right),x_{2}\left( \lambda \right) \right)\)
for any \(\lambda\). (As we have said above,
\(\left( x_{1}\left( \lambda \right),x_{2}\left( \lambda \right) \right)\)
denotes the solution to the problem of maximizing
\(L = \log\left( x_{1} \right) + log(x_{2}) - \lambda\left( p_{1}x_{1} + p_{2}x_{2} \right))\).
Drag and drop the equations into the categories:
Category 1: First order condition for \(x_{1}\)
\(\frac{\partial L}{\partial x_{1}} = 0\)
\(\frac{1}{x_{1}} - \lambda p_{1} = 0\)
Category 2: First order condition for \(x_{2}\)
\(\frac{\partial L}{\partial x_{2}} = 0\)
\(\frac{1}{x_{2}} - \lambda p_{2} = 0\)
\emph{Question 5}
What can we conclude from the first order conditions
\(x_{2} - \lambda p_{1} = 0\) and \(x_{1} - \lambda p_{2} = 0\)?
True: The function
\(L\left( x_{1},x_{2} \right) = \log\left( x_{1} \right) + log(x_{2}) - \lambda\left( p_{1}x_{1} + p_{2}x_{2} \right))\)
has a unique local extremum given by
\(\left( x_{1},x_{2} \right) = (\lambda p_{2},\lambda p_{1})\)
False: The function
\(L\left( x_{1},x_{2} \right) = \log\left( x_{1} \right) + log(x_{2}) - \lambda\left( p_{1}x_{1} + p_{2}x_{2} \right))\)
has a unique local extremum given by
\(\left( x_{1},x_{2} \right) = (p_{1},p_{2})\)
\emph{Question 6}
It turns out that the function
\(L\left( x_{1},x_{2} \right) = \log\left( x_{1} \right) + log(x_{2}) - \lambda\left( p_{1}x_{1} + p_{2}x_{2} \right))\)
has the shape indicated below:\selectlanguage{english}
\begin{figure}[h!]
\begin{center}
\includegraphics[width=0.70\columnwidth]{figures/image1/image1}
\caption{{Couldn't find a caption, edit here to supply one.%
}}
\end{center}
\end{figure}
It follows that \((\lambda p_{2},\lambda p_{1})\) is actually the unique
global maximum of \(L\left( x_{1},x_{2} \right)\) so we have
\(\left( x_{1}(\lambda),x_{2}(\lambda) \right) = (\lambda p_{2},\lambda p_{1})\).
Now let us find the value \(\lambda^{*}\) for \(\lambda\) for which the
budget constraint holds with equality:
\(p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) = y\)
.
What is \(\lambda^{*}\)?
True: \(\lambda^{*} = \frac{y}{2\ p_{1}p_{2}}\)
False: \(\lambda^{*} = \frac{y}{p_{1}p_{2}}\)
Explanation:
Plugging
\(\left( x_{1}(\lambda^{*}),x_{2}(\lambda^{*}) \right) = (\lambda^{*}p_{2},\lambda^{*}p_{1})\)
into
\(p_{1}x_{1}\left( \lambda^{*} \right) + p_{2}x_{2}\left( \lambda^{*} \right) = y\)
yields:
\[p_{1}\lambda^{*}p_{2} + p_{2}\lambda^{*}\ p_{1} = y\]
\[ \lambda^{*} = \frac{y}{2\ p_{1}p_{2}} \]
\emph{Question 7}
What is the optimal consumption bundle, i.e. the solution to the
original problem of maximizing \(log(x_{1}) + log(x_{2})\) subject to
the budget constraint \(p_{1}x_{1} + p_{2}x_{2}\)?
True
\(\left( x_{1},x_{2} \right) = \left( \frac{y}{2\ p_{1}},\frac{y}{2\ p_{2}} \right)\)
False
\(\left( x_{1},x_{2} \right) = (\frac{y}{\ p_{1}},\frac{y}{\ p_{2}})\)
Explanation:
Plugging \(\lambda^{*} = \frac{y}{2\ p_{1}p_{2}}\) into
\(\left( x_{1}(\lambda),x_{2}(\lambda) \right) = (\lambda p_{2},\lambda p_{1})\)
gives
\[\left( x_{1}(\lambda^{*}),x_{2}(\lambda^{*}) \right) = (\frac{y}{2\ p_{1}p_{2}}p_{2},\frac{y}{2\ p_{1}p_{2}}p_{1}) = (\frac{y}{2\ p_{1}},\frac{y}{2\ p_{2}})\]
From our reasoning in the previous exercises, we know that
\(\left( x_{1}(\lambda^{*}),x_{2}(\lambda^{*}) \right)\) is a solution
to the original problem of maximizing \(log(x_{1}) + log(x_{2})\)
subject to the budget constraint \(p_{1}x_{1} + p_{2}x_{2}\). One can
also show that there is only one solution to this problem, so
\(\left( x_{1}(\lambda^{*}),x_{2}(\lambda^{*}) \right)\) is the unique
solution.
\emph{Question 10}
Now let us carry out the following procedure:
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
Write down
\(L = u - \lambda\left( p_{1}x_{1} + \ldots + p_{n}x_{n} \right)\).
(\(L\) is often called `the Lagrangian' and \(\lambda\) is called `a
Lagrange multiplier')
\item
For each \(i \in \{ 1,\ldots,n\}\) compute
\(\frac{\partial L}{\partial x_{\text{i\ }}} = 0\), i.e.
\(\frac{\partial u}{\partial x_{i}} - \lambda p_{i} = 0\) (These
equations are called the first order conditions for
\(x_{1},\ldots x_{n}\))
\item
Solve the system of \(n + 1\) equations
\(\{\frac{\partial u}{\partial x_{1}} - \lambda p_{1} = 0,\ldots,\frac{\partial u}{\partial x_{n}} - \lambda p_{n} = 0,p_{1}x_{1} + \ldots + p_{n}x_{n} = y\}\)
in the \(n + 1\) unknowns.
\end{enumerate}
To solve this system, use the following procedure: \\
3.1 Solve \(\frac{\partial u}{\partial x_{1}} - \lambda p_{1} = 0\) for
\(x_{1}\), solve
\(\frac{\partial u}{\partial x_{2}} - \lambda p_{2} = 0\) for
\(x_{2}\ldots,\ \text{solve\ }\frac{\partial u}{\partial x_{n}} - \lambda p_{n} = 0\text{\ for\ }x_{n}\) \\
3.2 substitute all the expression obtained in step 3.1 for
\(x_{1},\ x_{2},\ldots,x_{n}\) into
\(p_{1}x_{1} + \ldots + p_{n}x_{n} = y\). \\
3.3 Then solve for \(\lambda\) the equation obtained in step 3.2 \\
3.4 Substitute the expression obtained for \(\lambda\) in step 3.3 into
the expression obtained for \(x_{1}\) in step 3.1. This gives an
expression for \(x_{1}\) not involving \(\selectlanguage{greek}\text{λ.}\selectlanguage{english}\) Similarly,
proceed for \(x_{2},\ldots x_{n}\).\\
Applying this procedure to our problem with
\(u\left( x_{1},x_{2} \right) = \log{\left( x_{1} \right) + log(x_{2})}\),
what do we obtain?
True \(x_{1} = \frac{y}{2p_{1}}\)
False \(x_{1} = \frac{y}{p_{1}}\)
False \(x_{1} = \frac{p_{2}}{p_{1} + p_{2}}\frac{y}{p_{1}}\)
Explanation
We follow the steps of the procedure:
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
\(L = \log\left( x_{1} \right) + \log\left( x_{2} \right) - \lambda\left( p_{1}x_{1} + p_{2}x_{2} \right)\)
\item
\(\frac{\partial L}{\partial x_{1}} = 0,\frac{\partial L}{\partial x_{2}} = 0\)
becomes
\(\frac{1}{x_{1}} - \lambda p_{1} = 0,\ \frac{1}{x_{2}} - \lambda p_{2} = 0\)
\item
We have the following system of \(3\) equations in the 3 unkwowns
\((x_{1},\ x_{2},\lambda)\):
\end{enumerate}
\[\frac{1}{x_{1}} - \lambda p_{1} = 0,\ \frac{1}{x_{2}} - \lambda p_{2} = 0,p_{1}x_{1} + p_{2}x_{2} = y\]
\\
3.1 Solving \(\frac{1}{x_{1}} - \lambda p_{1}\) for \(x_{1}\) gives
\(x_{1} = \frac{1}{\lambda p_{1}}\). Similarly, we obtain
\(x_{2} = \frac{1}{\lambda p_{2}}\).\\
3.2 Now substituting this into \(p_{1}x_{1} + p_{2}x_{2} = y\) gives:
\(p_{1}\frac{1}{\lambda p_{1}} + p_{2}\frac{1}{\lambda p_{2}} = y\)\\
3.3 Solving
\(p_{1}\frac{1}{\lambda p_{1}} + p_{2}\frac{1}{\lambda p_{2}} = y\)
for \(\lambda\) gives: \(\lambda = \frac{2}{y}\)\\
3.4 Substituting \(\lambda = \frac{2}{y}\) into
\(x_{1} = \frac{1}{\lambda p_{1}}\) gives \(x_{1} = \frac{y}{2p_{1}}\)
\\
\emph{Question 11:}
We have seen that the result for \((x_{1},x_{2})\) that we obtained
above using the notation with \(\lambda^{*}\) and so on is the same as
the result obtained by the procedure in the preceding question. Why is
that?
True: We actually carried out the same procedure in the two cases, just
with slightly different notation.
False: It is because of the utility function
\(u = \log{(x_{1})} + log(x_{2})\) that we used. For other utility
function, the two results obtained using the two procedures will not
always be the same.
Explanation:
Going through what we did before, one sees that we actually carried out
the same procedure in the two cases, just with slightly different
notation.
A side note: Actually, for any problem of maximizing \(u(x)\) under some
constraint \(g\left( x \right) \leq 0\) we can apply the procedure
\begin{enumerate}
\def\labelenumi{\arabic{enumi}.}
\item
Write down \(L = u - \lambda g(x)\)
\item
For each \(i \in \{ 1,\ldots,n\}\) compute
\(\frac{\partial L}{\partial x_{\text{i\ }}} = 0\)
\item
Solve the system of \(n + 1\) equations
\(\{\frac{\partial u}{\partial x_{1}} - \lambda\frac{\partial g}{\partial x_{1}} = 0,\ldots,\frac{\partial u}{\partial x_{n}} - \lambda\frac{\partial g}{\partial x_{n}} = 0,g(x) = y\}\)
in the \(n + 1\) unknowns.
\end{enumerate}
The only place where we used the specific properties of the consumer
choice problem was in the way we solved step 3 using the steps
\(3.1,\ 3.2\), 3.3 and \(3.4\).
Our analysis from above can be slightly extended to show that under very
general conditions (always satisfied in the applications we will
encounter in these capsules) the following: if this procedure yields a
solution \((x_{1},\ldots,x_{n})\) then this solution will be also be a
solution to the problem of maximizing \(u(x)\) und some constraint
\(g\left( x \right) \leq 0\).
\emph{Recap:}
We considered the constrained optimization of of finding which
consumption bundles \(x\) maximize \(u(x)\) subject to \(p\ x \leq y\).
We intuitively motivated a procedure for computing expressions for
\(x_{1},\ldots,x_{n}\) purely in terms of disposable money y and prices.
For our illustrative example with
\(u\left( x_{1},x_{2} \right) = \log{\left( x_{1} \right) + log(x_{2})}\)
we showed that the expressions for \((x_{1},x_{2})\) found following
this procedure are a solution to the problem of maximizing \(u(x)\)
subject to \(p\ x \leq y\). Actually, this turns out to be the unique
solution to the problem. This typically turns out to be true in economic
applications. In practice we can therefore simply carry out the
procedure we defined here. In the other capsules we will often apply
this procedure.
\selectlanguage{english}
\FloatBarrier
\end{document}