Artificial Intelligence (AI) and Machine Learning hold immense potential to accelerate scientific discovery and engineering design. A fundamental challenge in these domains involves efficiently exploring a large space of designs or hypotheses using expensive experiments in a resource-efficient manner. This paper surveys novel adaptive experimental design methods to address this broad challenge. Specifically, we discuss new probabilistic modeling and decision-making techniques that are applicable in small data settings. These approaches have shown substantial improvements in sample-efficiency, particularly for black-box optimization over high-dimensional combinatorial spaces (e.g., sequences and graphs) and a variety of goals ranging from multiobjective to multi-fidelity optimization. This paper outlines key methods and their real-world sustainability applications in areas such as nanoporous materials discovery, hardware design, surfactant design, and additive manufacturing.