To create large-scale habitat and ancillary infrastructure, it will be necessary to create utility structures that can protect sensitive hardware and provide shelter. Due to the lack of on-site energy and the extreme expense of shipping equipment and materials, the use of prefabricated parts or onsite materials for 3D printing is prohibitively costly, both financially and energetically. However, there is already an abundance of potential construction material present on the lunar surface, as loose soil or irregularly shaped objects. This project aims to use these objects directly as raw construction material without further processing them. We propose using autonomous mobile manipulator systems that can segment, reconstruct, and rearrange single objects in their environment to build complex structures. A key capability is to plan and evaluate the composition of the found material to follow the desired target design. Different structures might need varying characteristics, e.g., support structures can be made slender to minimize material and energy consumption, whereas protection walls would be dense to block radiation. The structural stability of the planned composition of found material is evaluated using sophisticated machine learning and optimization tools and physics simulators. The major advantage of using raw material is that there is no energy-intensive transformation of the construction material. A minimal setup of a single robot (or several) can build arbitrarily large constructions without human operation. Such a solution can be used to build a variety of utility structures on an architectural- and landscape-scale, such as walls and domes for habitat infrastructure, ramps for mobility infrastructure, or safe landing sites.