Structural uncertainty arises when key features of the behavior of a system are not well understood. Observational uncertainty arises when key variables in a system are not directly observed. Both types of uncertainty lead to problems for standard dynamic optimization approaches. The replacement of uncertainties by belief distributions over those uncertainties is one approach to addressing the problem. The use of this and other approaches are reviewed, with an emphasis on applications to environmental and resource management problems.