Open Source Summit + AI_dev Japan 2024

The demand for running LLMs in the cloud is growing exponentially. In this keynote, we will explore developer’s and enterprises’ urging need for open source LLMs and the best practices for deploying them in cloud-native environments. Three key approaches for LLM deployment: Python-based solutions, native runtimes like llama.cpp or vLLM, and WebAssembly as an abstraction layer. Miley will discuss the benefits and challenges of each approach, focusing on real-world applications, integration ease, portability, and resource efficiency and talk about the CNCF CNAI ecosystem landscape. This keynote aims to demystify cloud-native AI. Attendees will have practical advice and a clear roadmap for deploying LLMs in the cloud, learning about the strengths and trade-offs of different approaches, to have a better idea to select and implement the most suitable strategy