Consider a problem (ICP) with equality and inequality constraints:
minimizef(x)subject tohi(x)=0,i=1,…,mgj(x)≤0,j=1,…,r where f,hi,gj are continuously differentiable functions from Rn to R.
With Lagrange function
L(x,λ,μ)=f(x)+m∑i=1λihi(x)+r∑j=1μjgj(x)
Define the set of active inequality constraints:
A(x)={j|gj(x)=0} A feasible vector x is regular if the equality constraint gradients ∇hi(x),i=1,…,m, and the active inequality constraint gradients ∇gj(x),j∈A(x) are linearly independent.
KKT optimality conditions
KKT necessary conditions for regular x∗
Let x∗ be a local minimum of the problem
minimizef(x)subject tohi(x)=0,i=1,…,mgj(x)≤0,j=1,…,r where f,hi,gj are continuously differentiable functions from Rn to R and x∗ is regular. There exist unique Lagrange multiplier vectors λ∗=(λ∗1,…,λ∗m), μ∗=(μ∗1,…,μ∗r) such that
∇xL(x∗,λ∗,μ∗)=0,μ∗j≥0,j=1,…,r,μ∗j=0,∀j∉A(x∗) Note: The condition can be written as
μ∗jgj(x∗)=0,j=1,…,r and aptly referred to as complementary slackness condition.
If in addition f, h, and g are twice continuously differentiable, there holds
yT∇2xxL(x∗,λ∗,μ∗)y≥0 for all y∈Rn such that
∇hi(x∗)Ty=0,∀i=1,…,m∇gj(x∗)Ty=0,∀j∈A(x∗)