Open Source Summit + AI_dev Japan 2024

As AI and ML become popular in container (K8s) environment, enormous computational resources are required more and more. On the other hand, efforts toward energy efficiency are also required for the realization of a sustainable society. It is expected to achieve the conflicting requirements that providing higher performance and reducing power consumption simultaneously. Recently, a new server architecture called Composable Disaggregated Infrastructure (CDI) is emerged. CDI can provide custom made servers by composing devices such as compute, memory, storage and GPU connected to PCIe or CXL switch fabric as a resource pool on demand. CDI can provide appropriate type and number of devices on demand depending on container workloads, therefore it would be a solution for this requirement. However, there is currently no standard way to control CDI from K8s. In this talk, we show how K8s and CDI could be a solution. We provide CDI operator and its Custom Resource Definition. Also, we introduce advanced vertical and horizontal cluster auto scaler for CDI. We demonstrate how K8s dynamically attach or detach devices to nodes combined with Dynamic Resource Allocation (DRA) function.