Decision-making With Vector-valued Objectives Under Model Uncertainty

We had the honor and privilege to be among the first participants to benefit from the SIAM Postdoctoral Support Program — an initiative backed by the SIAM Postdoctoral Support Fund, which was established through a generous gift from Drs. Martin Golubitsky and Barbara Keyfitz. The program provides targeted funding for mentor-mentee pairs from distinct institutions with the goal of fostering research collaboration and supporting professional development. Thanks to this support, we—Gabriela Kováčová as the mentee and Igor Cialenco as the mentor—were able to expand our research pursuits in decision-making with a more fundamental approach, exploring deeper theoretical questions while building a strong collaborative foundation. Although we had worked with each other prior, the research visits made possible by this program allowed us to tackle a broad and substantive research problem together, rather than a more narrowly-scoped project. During these visits, we engaged in productive brainstorming sessions to generate new ideas and gain a deeper understanding of the structure of the underlying problems. These interactions also provided an opportunity to discuss various aspects of careers in the mathematical sciences at length.

Our motivation for this line of research is rooted in two central themes in decision-making: time inconsistency and model uncertainty. Both of these topics have been studied extensively, particularly over the past two decades. In the context of stochastic control, time inconsistency refers to a class of finite-horizon problems for which the dynamic programming principle (DPP), or Bellman principle of optimality, does not hold. As a result, such problems cannot be solved by a backward recursive procedure. A prominent example is the multi-period portfolio optimization problem under mean-risk criteria. This approach to asset management was originally proposed in the 1950s by the Nobel laureate Harry Markowitz, who argued that investors should evaluate portfolios not only based on their expected returns, but also on their risk, which could be measured, for example, by the variance or standard deviation of returns. Since maximizing the mean return and minimizing the risk are competing objectives, the mean-risk problem is naturally a bi-objective problem. Within the last decade, multi-objective dynamic problems were successfully tackled by deriving a counterpart of DPP that was appropriate for multi-objective problems [8], thus making the problem time consistent. This development is our main reason for considering problems with vector-valued objectives. Other approaches to time-inconsistent stochastic control problems include the so-called sub-game perfect approach [3] or dynamic adjustment of the optimization criteria [6].

Model uncertainty refers to the inability to accurately model the dynamics of the underlying stochastic system due to incomplete information, ambiguity about the model, or presence of unobservable factors. It is also referred to as Knightian uncertainty after Knight [7]. For stochastic controlled systems, this means that the controller does not know the true law of the underlying stochastic process a priori, but only knows that it belongs to a given family of probability laws. Here, the controller faces not only the randomness of the controlled system, but also the Knightian uncertainty. A significant body of literature is devoted to this problem, particularly for problems in finance and economics wherein flawed models can lead to erroneous investment decisions, ineffective risk management strategies, and inaccurate pricing of financial instruments. One approach to tackle this challenge is through robust optimization or a minimax approach, which seeks controls that perform well across various possible models [5]. Other approaches include adaptive control, Bayesian control, adaptive-robust control, and strong robust control [1]. All such methods fundamentally rely on time consistency of the original problem without model uncertainty, however the literature on time-inconsistent control problems that are subject to model uncertainty is sparse. In [2] the authors use the sub-game perfect approach to time-inconsistency and adaptive robust control for model uncertainty. To the best of our knowledge, studying time-inconsistent problems under model uncertainty through the lens of multi-objective stochastic control—while a natural and conceptually appealing approach—remains largely unexplored, with our work being the first to systematically develop this perspective. This enabled us to utilize our combined experiences in model uncertainty and multi-objective control, fostering the development of a novel research direction.

Formulation of the Problem

We assume that the model uncertainty is modeled as a family of distributions \(\mathbb{Q} (\Theta) = \{ \mathbb{Q}^{\theta} \, : \, \theta \in \Theta \}\) of the stochastic factor \((Z_t, t = 0,1,\ldots,T)\), parametrized by \(\theta\) belonging to the model uncertainty set \(\Theta\subset \mathbb{R}^k\). Consider the controlled dynamics of the state process \((X_t)_{t \in \mathcal{T}}\) and control \((\varphi_t)_{t \in \mathcal{T}}\) given by

\[X_{t+1} = F(t, X_t, \varphi_t, Z_{t+1}), \ \varphi_t \in A_t(X_t),  \quad t = 0,1,\ldots, T,\]

where \(A_t(x)\) is the feasible set. For simplicity, consider a risk-neutral decision maker interested in the expected terminal multi-loss \(\mathbb{E} [\ell (X_T)]\), where \(\ell: \mathbb{X} \to \mathbb{R}^d\). If the true model \(\theta^* \in \Theta\) for the stochastic factor was known, we would be studying a multi-objective control, without model uncertainty, of the form

\[\underset{(\varphi_t, \dots, \varphi_{T-1}) \in \mathcal{A}^t (X_t)}{\text{ minimize }} \mathbb{E}^{\theta^*}_t [\ell (X_T)] \quad \text{ with respect to } \preceq, \]

where \(\preceq\) is a pre-order on the space of \(\mathscr{F}_t\)-measurable random vectors with corresponding ordering cone \(\mathscr{L}_t (C)\). The most commonly used pre-order is the component-wise partial order \(\leq\) generated by \(\mathscr{L}_t (\mathbb{R}^d_+)\) (see Figures 1 and 2). The value function of a multi-objective control problem should be a set-valued functional that attains the so-called upper image of the multi-objective problem [8, 9],

\[\mathcal{V}^{\theta^*}_t (X_t) = \text{cl} \left( \left\lbrace \mathbb{E}_t^{\theta^*} [\ell (X_T)] \; \vert \; (\varphi_t, \dots, \varphi_{T-1}) \in \mathcal{A}^t (X_t) \right\rbrace + \mathscr{L}_t (\mathbb{R}^d_+) \right).\]

Moreover, it is also known that the appropriate counterpart of DPP for a multi-objective problem is a recursion of the set-valued value function known as the set-valued Bellman’s principle:

\[\begin{equation}\label{eq:Bellman}
\mathcal{V}^{\theta^*}_t (X_t) = \text{cl} \left\lbrace \mathbb{E}^{\theta^*}_t [Y] \; \vert \; \varphi_t \in A_t (X_t), \, Y \in \mathcal{V}^{\theta^*}_{t+1} (F(t, X_t, \varphi_t, Z_{t+1})) \right\rbrace.
\end{equation}\]

These results are significant because, when viewed through the lens of set optimization, they uncover a structure that directly parallels the single-objective case, thereby extending familiar principles to the multi-objective setting. \((1)\) represents the (set-optimization) infimum of the multi-objective problem, and the set-valued Bellman principle \((2)\) can be interpreted analogously as a recursive (set-optimization) infimum that reduces the problem to a sequence of one-step problems. Additionally, the time consistency properties of the problem can be also established within the framework.

A Robust Perspective on Vector-valued Control

As a first attempt at handling multi-objective control under model uncertainty, in [4] we opted to explore the robust approach and establish the corresponding set-valued Bellman’s principle of optimality.

The first question that needed to be tackled was the interpretation of the supremum in the arising min-max multi-objective problem. The existing literature on static robust vector optimization consists of two main approaches in interpreting the supremum of vectors: as a vector (ideal point) or as a set. To gain initial insight into the problem, we began by studying the structurally simpler, even if more conservative, option of interpreting the supremum of collection of vectors as a single vector (called ideal point supremum) with the property of being the smallest among the upper bounds (see Figure 1, on page 2). Some of the nontrivial challenges that we had to address were the existence, uniqueness, and the attainability of the supremum when taking across the models \(\theta\in\Theta\); all being obvious in the scalar case. We note that when saying an ideal point supremum, represented through the operators v-sup, we refer to a point corresponding to an idealized scenario or model.

With these issues sorted out, we formulate the robust multi-objective control problem as follows:

\[\underset{(\varphi_t, \dots, \varphi_{T-1}) \in \mathcal{A}^t (X_t)}{\text{ minimize }} \qquad \underset{\theta \in \Theta}{\text{vsup}} \enspace \mathbb{E}^{\theta} [\ell (X_T)] \quad \text{ with respect to } \leq.\]

The corresponding set-valued value function is defined as

\[\mathcal{V}^{\Theta}_t (X_t) = \text{cl} \left( \left\lbrace \underset{\theta \in \Theta}{\text{vsup}} \enspace \mathbb{E}^{\theta} [\ell (X_T)] \; \vert \; (\varphi_t, \dots, \varphi_{T-1}) \in \mathcal{A}^t (X_t) \right\rbrace + \mathscr{L}_t (\mathbb{R}^d_+) \right).\]

The next key question was whether the set-valued Bellman’s principle \((2)\), or a version of it, is still valid for the robust framework. Without any further assumptions, the short answer is yes, but only in its weaker form. Namely, the DPP is a set relation, corresponding to an inclusion, between the true value function and its recursive version. Moreover, by means of counter examples, we showed that this result is sharp (see Figure 2).

Even in the scalar case, the DPP for robust stochastic control problems holds only if the set of probability measures \(\mathbb{Q}(\Theta)\) satisfies an additional structural condition, known as rectangularity property (or \(m\)-stability). This property essentially requires the family of models to be sufficiently rich to ensure dynamic consistency, thus we had to establish an analogous property for vector-valued problems that ensured a stronger and fully appropriate version of the Bellman principle. The main technical difficulties arose from the non-uniqueness and, in general, the non-attainability of the v-sup. As a result, we prove that under this new notion of rectangularity for \(\mathbb{Q}(\theta)\) the following strong set-valued Bellman’s principle holds

\[\mathcal{V}^{\Theta}_t (X_t) = \text{cl} \left\lbrace \underset{\theta \in \Theta}{\text{vsup}} \enspace \mathbb{E}^{\theta}_t [Y] \; \vert \; \varphi_t \in A_t (X_t), \, Y \in \mathcal{V}^{\Theta}_{t+1} (F(t, X_t, \varphi_t, Z_{t+1})) \right\rbrace, \]

as well as a time consistency property of the dynamic robust problem. 

Current Developments and Future Directions

The results obtained for robust multi-objective control are encouraging and highlight the potential of this new research direction on multi-objective control under model uncertainty. Our work represents a first glimpse into the potential research endeavors on the subject, with numerous interesting and challenging open problems. For instance, it is well known that the robust approach described tends to be overly conservative, and exploring alternative methods for handling model uncertainty in the multi-objective setting is not merely a theoretical exercise, but also of significant practical importance.

In an ongoing follow-up project, we are currently investigating the Bayesian approach within the context of multi-objective control. This involves formulating the control problem as a Markov decision process, performing Bayesian updating of posterior beliefs about the underlying distribution \(\mathbb{Q}(\theta)\) and understanding how to extend the state space to include these beliefs as part of the value function’s arguments.

Another important research direction where substantial progress is still needed is the development of efficient algorithms for solving multi-objective control problems. Addressing these computational challenges is also part of our planned work.

Concluding Remarks

The SIAM Postdoctoral Support Program was an enriching experience that elevated the process of postdoctoral training and encouraged academic career growth. We are especially grateful for the program’s travel support for research meetings and conferences. This facilitation is particularly important in the current landscape, where many postdoctoral positions come without a dedicated budget for travel or research. We wish to express our sincere gratitude to Drs. Martin Golubitsky and Barbara Keyfitz for their generous gift that made this experience possible, as well as to the SIAM staff for their guidance throughout this program.

For more information and to apply, visit the SIAM Postdoctoral Support Program webpage. The next priority deadline will be November 1, 2026.

The SIAM Postdoctoral Support Program is made possible by gifts to the SIAM Postdoctoral Support Fund, which was established by Drs. Martin Golubitsky and Barbara Keyfitz. If you would like to make a contribution to the SIAM Postdoctoral Support Fund, please visit the SIAM website.

Acknowledgements: Igor Cialenco also received support for this work from U.S. National Science Foundation Grant DMS-2407549.

References
[1] Bielecki, T.R., Chen, T., Cialenco, I., Cousin, A., & Jeanblanc, M. (2019). Adaptive robust control under model uncertainty. SIAM J. Control Optim., 57(2), 925-946. 
[2] Bielecki, T.R., Chen, T., & Cialenco, I. (2021). Time-inconsistent Markovian control problems under model uncertainty with application to the mean-variance portfolio selection. Int. J. Theor. Appl. Finance, 24(01), 2150003.
[3] Björk, T., & Murgoci, A. (2014). A theory of Markovian time-inconsistent stochastic control in discrete time. Finance Stoch., 18(3), 545-592.
[4] Cialenco, I. & Kováčová, G. (2025). Vector-valued robust stochastic control. Preprint, arXiv:2407.00266.  
[5] Hansen, L.P., & Sargent, T.J. (2008). Robustness. Princeton, NJ: Princeton University Press.
[6] Karnam, C., Ma, J., & Zhang, J. (2017). Dynamic approaches for some time-inconsistent optimization problems. Ann. Appl. Probab., 27(6), 3435-3477. 
[7] Knight, F.H. (1921). Risk, uncertainty, and profit. Boston, MA: Houghton Mifflin Company.
[8] Kováčová, G., & Rudloff, B. (2021). Time consistency of the mean-risk problem. Oper. Res., 69(4), 1100-1117.
[9] Löhne, A. (2011). Vector optimization with infimum and supremum. Heidelberg, Germany: Springer Berlin Heidelberg.