Open Source Summit + AI_dev Japan 2024

The talk “Democratizing Diffusion Models with Diffusers” will explore the diverse applications of the open-source Python library Diffusers in the image and video generation space. The talk will showcase how Diffusers, based on diffusion models, enables fast and high-quality image and video generation, making it accessible to a wide range of users. The presentation will cover various use cases, including image inpainting, image editing, and scene composition, demonstrating the capabilities of Diffusers in enabling users to create and edit photo-realistic images with minimum effort. The audience will gain insights into the potential of Diffusers in revolutionizing the way images and videos are generated and edited, making it a must-attend session for anyone interested in the latest advancements in this field.

Every conversation on AI starts with models and ends with data. Data preparation is emerging as a very important phase of the GenAI journey, as high quantity and quality text and code corpora for GenAI model training have shown to play a crucial role in producing high performing Large Language Models (LLMs). The data preparation phase in the Generative AI lifecycle aims to clean, filter, and transform the datasets of text and code that are acquired from various sources into a tokenized form that is suitable for the training of LLMs, be it pre-training, or constructing LLM apps via fine-tuning or instruct tuning. The latter poses unique challenges, as each use case may necessitate tailored data preparation approaches. Given the enduring and evolving demand for data preparation techniques in LLM applications, we are introducing Data Prep Kit as an open-source software asset. This endeavour is geared towards fostering collaborative efforts within the community, enabling collective development and utilization, and ultimately reducing time to value. DPK has been instrumental in powering the IBM open-source Granite models.

GPU Cloud has become a ubiquitous component of contemporary AI infrastructure, especially for distributed machine learning scenarios. While conversations around AI infrastructure optimization typically revolve around the application layer, such as machine learning tasks and distributed job schedulers, delving into the underhood of the GPU cloud is essential. Numerous factors, including POD Scheduler, Device Plugin, GPU/NUMA topology, ROCE/NCCL Stack, and more, can significantly impact performance.

This session will thoroughly explore the tuning of various machine models(CNN/RNN/Transformer) from MLPerf using an H100 Cluster as a reference. We will analyze the correlation between model performance and device operator configuration in nodes by presenting first-hand experimental results to unveil the hidden potential within a K8S GPU Cloud.

In this session, we will introduce the construction of a comprehensive platform that uses Edge AI and sensors to cover everything from devices to the cloud. The platform enables advanced cooperation between sensors and AI control, and emphasizes seamless and dynamic replacement of AI models by using WebAssembly (Wasm). Furthermore, through open sourcing, we aim to expand the ecosystem and form a technical community. Through technical details and real-world scenarios, we will provide insights that participants can apply to their own projects.

This talk will share the story of quantization, its rise in popularity, and its current status in the open-source community. We'll begin by reviewing key quantization papers, such as QLoRA by Tim Dettmers and GPTQ by Elias Frantar. Next, we'll demonstrate how quantization can be applied at various stages of model development, including pre-training, fine-tuning, and inference. Specifically, we'll share our experience in pre-training a 1.58-bit model, show how fine-tuning is achievable using PEFT + QLoRA, and discuss optimizing inference performance with torch.compile or custom kernels. Finally, we'll highlight efforts within the community to make quantized models more accessible, including how transformers incorporate state-of-the-art quantization schemes and how to run GGUF models from llama.cpp.

What does it take to design, implement, and roll out a comprehensive AI governance program? What challenges and opportunities do companies encounter when scaling AI governance across diverse products and AI applications, and how can AI governance programs be designed to stay agile in a dynamic technological and regulatory environment? Insights on scaling AI governance programs progressively across multiple jurisdictions while keeping them agile in a dynamic technological and regulatory landscape. Practical learnings, effective solutions and challenges to watch out for. Topics discussed: - What are the building blocks of your company’s AI Governance program? - What challenges did we face when building an AI Governance program? - The AI Governance landscape is evolving rapidly – both through technical innovation and regulatory advances. How to keep an agile approach to AI governance? - What kinds of risks are front and center when you’re building an AI Governance program? What kinds of opportunities?

End point devices are resource constrained, either in terms of power, memory or communication capabilities - sometimes all three. However, being able to apply machine learning on these end point devices is possible and when applied strategically enables system wide efficiencies to be realized. This talk will explore the requirements and tradeoffs for such system to be considered when using the Zephyr RTOS and Tensorflow Lite for Embedded Microcontrollers projects.

As GPUs become increasingly powerful, the separation between compute and storage often results in underutilized GPUs waiting for data. Meanwhile, high-performance components on GPU machines, such as NVMe storage and fast networks leveraging InfiniBand or special NICs, remain idle. Effectively leveraging these hardware resources to address GPU underutilization is a critical challenge. In this talk, we introduce a Kubernetes-native distributed caching layer that leverages NVMe disks and fast networks to optimize PyTorch training data access. Utilizing stateless workers for scalability and ETCD for membership services, this caching layer efficiently manages and serves data. Cached data is rapidly and efficiently fed into GPU memory using NVIDIA's DALI data loader, GPUDirect Storage (GDS), and Remote Direct Memory Access (RDMA), significantly reducing data transfer bottlenecks and improving overall training performance.