Abstract.     To make safe decisions in real-world environments, algorithms must account for the inherent uncertainty in perception systems and agent dynamics, resulting in high-dimensional problems. The use of surrogate models to replace hand-crafted planning heuristics and avoid running the computationally expensive true system has shown promise in enabling large-scale, safe planning. This work introduces four main contributions to address the challenges of safe planning under uncertainty. To improve planning efficiency over beliefs in partially observable Markov decision processes (POMDPs), we introduce batched belief-state MDPs, which abstract belief-state planning using parallelizable batches of the underlying models. This abstraction requires models that can be easily parallelized; therefore, we learn surrogate transition and observation models from data and propose the inversion variational autoencoder ($\mathcal{I}$-VAE) to sample from the posterior of the updated belief given partial observations during inference. To replace planning heuristics and enable long-horizon planning, we introduce BetaZero, a policy iteration algorithm that combines offline learning with online belief-state planning. Extending BetaZero to safety-critical problems, we propose ConstrainedZero, which solves chance-constrained POMDPs by optimizing the balance between utility and a target level of safety. Finally, given a learned safe policy, we develop a Bayesian safety validation method to estimate the failure probability of a black-box system using probabilistic surrogate models. We apply these algorithms to real-world problems, including aircraft collision avoidance, safe carbon capture and storage, critical mineral exploration, robot navigation, and wildfire resource allocation.