Open Source Summit + AI_dev Japan 2024

Recent IT paradigms like DevOps, CD, and IaC have shortened release cycles but burdened developers with mastering many tools. Platform Engineering, a field of software engineering frequently discussed in the CNCF, addresses this by providing Internal Developer Platforms (IDPs) that automate non-essential tasks. Kubernetes is widely used to run the various components that make up an IDP, thanks to its consistent API experience and high extensibility. However, Kubernetes was initially created as a general foundation for running workloads, not as part of platforms. Therefore, IDP builders must add Cloud Native technologies and customizations to Kubernetes. Despite three years since the rise of platform engineering, making the right architectural and technology choices remains challenging and requires a deep understanding of the organization's context and technologies. In this session, he will explain various architectural patterns for building IDPs using Kubernetes, including access control and multitenancy. Additionally, based on years of experience providing Kubernetes-based IDPs, he will discuss the pros/cons, operational considerations, and suitable organizational structures.

The demand for running LLMs in the cloud is growing exponentially. In this keynote, we will explore developer’s and enterprises’ urging need for open source LLMs and the best practices for deploying them in cloud-native environments. Three key approaches for LLM deployment: Python-based solutions, native runtimes like llama.cpp or vLLM, and WebAssembly as an abstraction layer. Miley will discuss the benefits and challenges of each approach, focusing on real-world applications, integration ease, portability, and resource efficiency and talk about the CNCF CNAI ecosystem landscape. This keynote aims to demystify cloud-native AI. Attendees will have practical advice and a clear roadmap for deploying LLMs in the cloud, learning about the strengths and trade-offs of different approaches, to have a better idea to select and implement the most suitable strategy

Developers often face the complex challenges of designing, debugging, and testing distributed systems like microservices. Understanding failures, identifying performance issues, and ensuring system reliability from the early stages of design and development can be daunting. Observability technologies provide valuable insights not just in production but also during design and development. In this session, we will explore OpenTelemetry, a cutting-edge observability framework, and its practical applications in the design, debugging, and testing of distributed systems. Key topics include: - Assessing the impact of incorporating a cache server on system behavior during the design phase. - Evaluating how database failures affect both backend and frontend applications during fault testing. - Detecting performance bottlenecks for specific requests during load testing. Participants will gain a clear understanding of how OpenTelemetry can revolutionize their debugging and testing processes, leading to more effective experiments and increased reliability in their distributed systems.

This presentation will explain the concept of Docker and demonstrate how it shortens the development iteration cycle. Additionally, a demonstration will showcase how Kubernetes can enhance CI/CD performance in the development process.

With the advancements in the fields of Applied AI/ML, computational power, FinOps practices, and cloud services, organizations are increasingly building and deploying AI/ML workloads on Kubernetes. Cloud Computing gives the impression that the underlying resources (like Virtual Machines, CPU, RAM, Storage, etc.) are infinite, and hence many individuals don’t focus on the count of machines and resources when training their AI/ML models and proceed with training fairly large models with millions and billions of parameters. The energy-intensive nature of model training and iterations, coupled with the growing scale of AI/ML deployments, is contributing to a considerable environmental impact in the form of Carbon Emissions.

This talk introduces some key open-source cloud agnostic tools and techniques to build eBPF-powered observability solutions for practicing Sustainable Computing. By analyzing and tuning performance and energy metrics from Kubernetes resources like Pods and Nodes, AI/ML Practitioners and Platform Engineers can optimize workloads, reduce energy consumption, lower costs, and minimize environmental impact while maintaining performance.

As the importance of data utilization for AI grows, enterprises aim to securely exchange data with their customers and leverage external data.Many services and open-source software related to data sharing and governance have attracted attention and extensive research and development, such as Snowflake Marketplace and Databricks Delta Sharing, among others. However, sharing data between different companies presents numerous challenges in terms of data security and efficiency, including efficient access to geographically dispersed data and access control for data managed by multiple organizations. To address these challenges, we have developed virtual data lake system that achieves efficient and secure data sharing using federated cloud storage. In this approach, virtual data integration is performed by collecting and managing only metadata without collecting the original data. In this session, we’ll take a look at how to build a safe and efficent data lake system by using existing OSS for data governance and data federation tools, such as LinkedIn DataHub and Alluxio ..etc.

Traditional testing methods like unit and integration tests are great for functional validation in isolation, but are they enough for distributed systems? Distributed systems need to deal with real-world failures such as network issues, hardware errors, and race conditions. One of the proven ways to test these systems is to inject failures during testing and see if the system still works as expected. This is called robustness testing, where you run the system like it would be used in real life. Jespen is one of the first frameworks to test distributed systems by simulating such real-world scenarios and validating the operational history. Inspired by Jespen, etcd, the backbone of Kubernetes, built its own testing framework. This framework is written in Golang and for Golang projects, allowing even more failure types on the fly and verifying if the data stays consistent using Porcupine. As etcd contributors, we will share our challenges in writing tests to force failure via gofail and our journey of developing, leveraging, and debugging issues caught by this ever-evolving framework, so that you can apply the findings to your projects with minimal tweaks.

Authn/authz are some of the most important considerations for cloud-native applications, which is clear from OWASP Top 10, and at the same time, these are big challenges for many implementers today. Fortunately, regarding "authn," there are standards such as OpenID Connect and there is a CNCF project, Keycloak, so the concerns are somewhat alleviated. On the other hand, regarding "authz," there was no clear standard, and there are several superior OSSs such as OPA, OpenFGA, and Topaz, so implementers are faced with a difficult choice.
Currently, OpenID Foundation AuthZEN WG works on authz standards, and it is difficult to predict the situation after the standards are established, which also makes the choice difficult.
In this session, Yoshiyuki Tabata introduces how to authorize requests using the OSSs including the AuthZEN perspective, and integrate them with Keycloak. It helps implementors explore a best practice for implementing authn/authz in a cloud-native environment.

The migration of data-intensive analytics applications to cloud-native environments promises enhanced scalability and flexibility but introduces complex cost models that pose new challenges to traditional optimization strategies. While on-premises setups focused on speed, cloud deployments require a more nuanced approach, factoring in cloud storage operations costs, which can escalate rapidly in real-world scenarios. In this presentation, Bin will analyze these challenges through a case study on Uber's large deployment analytics SQL platform on HDFS and GCS. They will show their findings of unexpected cost implications with standard I/O optimizations like table scans, filters, and broadcast joins when implemented in cloud environments. He will also highlight the need for a paradigm shift in optimizing data-intensive applications for the cloud and advocate for developing new I/O strategies, balancing performance and costs while tailored to cloud ecosystems' unique demands.

Modern applications increasingly require the output of large files such as videos and images, and the generation of more comprehensive logs. As a result, the use of RAM disks - ephemeral disks backed by memory, which are faster than traditional HDDs and SSDs - has become more common. In Kubernetes, RAM disks can be utilized through the abstraction of memory-backed emptyDir, which is based on tmpfs. However, if used carelessly, memory-backed emptyDir can pose significant risks to the stability of the Kubernetes cluster. When used as storage, the use case differs from typical memory usage by applications. Sharing files between different applications can obscure the responsibility for file garbage collection, increasing the risk of files not being deleted. If tmpfs usage excessively consumes node memory, the application containers using this tmpfs may be killed. In some cases, the deletion of tmpfs might not be timely, leading to memory exhaustion and potentially causing node crashes. This talk will share risk mitigation strategies for node crashes from the perspectives of cluster administrators, namespace administrators, and application developers.

Edge computing is an important technology for a data-driven society that makes decisions in real time from data. LF Edge, a project of The Linux Foundation, is working to establish an open and interoperable edge computing framework that is independent of hardware, processor, cloud, and OS. LF Edge is also focused on enabling edge AI because of the growing need for real-time AI processing. However, there are challenges to achieving an infrastructure that is both energy efficient and flexible, such as increasing data volumes, increasing the computational complexity of AI models, and meeting application-specific requirements. So we need innovation in infrastructure technology. Therefore, The Linux Foundation and THE IOWN GF signed a basic agreement in June 2023 to integrate the Linux Foundation’s software on the infrastructure proposed by IOWN GF to develop a common infrastructure that improves performance, reduces latency, and improves energy efficiency. Based on this agreement, we planned the IOWN GF/LF Edge Joint PoC to demonstrate the convergence of the two technologies and the performance improvements. This presentation describes the content of the PoC.

Confidential Computing has been around for a while, and is now widely available from most hyperscalers and CSPs, but over the past year, two use cases have raised its visibility enormously: Confidential AI and Secure Multi-party Collaboration. Both of these require the confidentiality and integrity that Confidential Computing provides, but the addition of another property - cryptographic attestation - allows trust in these workloads and their inputs and outputs - to be established in ways that increase their value significantly. Learn about the technology underpinning these developments, how organisations are creating new business models, and how the Confidential Computing Consortium is promoting open source in this area.

"Decoding Serverless and FaaS: Knative vs. OpenFaaS vs. Kubeless vs. Pulumi" delves into the complexity of serverless computing and highlight the strengths of today’s most compelling open-source tools. We’ll dive deep into Knative’s seamless Kubernetes-native integration that automates scaling and manages workloads effortlessly, OpenFaaS’s intuitive interface that empowers developers to deploy multi-language functions, Kubeless’s straightforward approach to deploying functions as Kubernetes objects, and Pulumi’s innovative infrastructure as code that merges serverless capabilities with familiar programming languages. We will also compare these tools in real-world scenarios, providing clear guidance on when and why to choose each one based on specific project needs and operational goals. Whether you’re looking to streamline microservices, handle event-driven workloads, or simply enhance your development efficiency, this talk will provide you with the insights and practical comparisons you need to choose the right tool for your serverless journey. Let’s navigate the future of serverless together and empower our next big project!