Open Source Summit + AI_dev Japan 2024

The talk “Democratizing Diffusion Models with Diffusers” will explore the diverse applications of the open-source Python library Diffusers in the image and video generation space. The talk will showcase how Diffusers, based on diffusion models, enables fast and high-quality image and video generation, making it accessible to a wide range of users. The presentation will cover various use cases, including image inpainting, image editing, and scene composition, demonstrating the capabilities of Diffusers in enabling users to create and edit photo-realistic images with minimum effort. The audience will gain insights into the potential of Diffusers in revolutionizing the way images and videos are generated and edited, making it a must-attend session for anyone interested in the latest advancements in this field.

Every conversation on AI starts with models and ends with data. Data preparation is emerging as a very important phase of the GenAI journey, as high quantity and quality text and code corpora for GenAI model training have shown to play a crucial role in producing high performing Large Language Models (LLMs). The data preparation phase in the Generative AI lifecycle aims to clean, filter, and transform the datasets of text and code that are acquired from various sources into a tokenized form that is suitable for the training of LLMs, be it pre-training, or constructing LLM apps via fine-tuning or instruct tuning. The latter poses unique challenges, as each use case may necessitate tailored data preparation approaches. Given the enduring and evolving demand for data preparation techniques in LLM applications, we are introducing Data Prep Kit as an open-source software asset. This endeavour is geared towards fostering collaborative efforts within the community, enabling collective development and utilization, and ultimately reducing time to value. DPK has been instrumental in powering the IBM open-source Granite models.

GPU Cloud has become a ubiquitous component of contemporary AI infrastructure, especially for distributed machine learning scenarios. While conversations around AI infrastructure optimization typically revolve around the application layer, such as machine learning tasks and distributed job schedulers, delving into the underhood of the GPU cloud is essential. Numerous factors, including POD Scheduler, Device Plugin, GPU/NUMA topology, ROCE/NCCL Stack, and more, can significantly impact performance.

This session will thoroughly explore the tuning of various machine models(CNN/RNN/Transformer) from MLPerf using an H100 Cluster as a reference. We will analyze the correlation between model performance and device operator configuration in nodes by presenting first-hand experimental results to unveil the hidden potential within a K8S GPU Cloud.

In this session, we will introduce the construction of a comprehensive platform that uses Edge AI and sensors to cover everything from devices to the cloud. The platform enables advanced cooperation between sensors and AI control, and emphasizes seamless and dynamic replacement of AI models by using WebAssembly (Wasm). Furthermore, through open sourcing, we aim to expand the ecosystem and form a technical community. Through technical details and real-world scenarios, we will provide insights that participants can apply to their own projects.

This talk will share the story of quantization, its rise in popularity, and its current status in the open-source community. We'll begin by reviewing key quantization papers, such as QLoRA by Tim Dettmers and GPTQ by Elias Frantar. Next, we'll demonstrate how quantization can be applied at various stages of model development, including pre-training, fine-tuning, and inference. Specifically, we'll share our experience in pre-training a 1.58-bit model, show how fine-tuning is achievable using PEFT + QLoRA, and discuss optimizing inference performance with torch.compile or custom kernels. Finally, we'll highlight efforts within the community to make quantized models more accessible, including how transformers incorporate state-of-the-art quantization schemes and how to run GGUF models from llama.cpp.

What does it take to design, implement, and roll out a comprehensive AI governance program? What challenges and opportunities do companies encounter when scaling AI governance across diverse products and AI applications, and how can AI governance programs be designed to stay agile in a dynamic technological and regulatory environment? Insights on scaling AI governance programs progressively across multiple jurisdictions while keeping them agile in a dynamic technological and regulatory landscape. Practical learnings, effective solutions and challenges to watch out for. Topics discussed: - What are the building blocks of your company’s AI Governance program? - What challenges did we face when building an AI Governance program? - The AI Governance landscape is evolving rapidly – both through technical innovation and regulatory advances. How to keep an agile approach to AI governance? - What kinds of risks are front and center when you’re building an AI Governance program? What kinds of opportunities?

End point devices are resource constrained, either in terms of power, memory or communication capabilities - sometimes all three. However, being able to apply machine learning on these end point devices is possible and when applied strategically enables system wide efficiencies to be realized. This talk will explore the requirements and tradeoffs for such system to be considered when using the Zephyr RTOS and Tensorflow Lite for Embedded Microcontrollers projects.

As GPUs become increasingly powerful, the separation between compute and storage often results in underutilized GPUs waiting for data. Meanwhile, high-performance components on GPU machines, such as NVMe storage and fast networks leveraging InfiniBand or special NICs, remain idle. Effectively leveraging these hardware resources to address GPU underutilization is a critical challenge. In this talk, we introduce a Kubernetes-native distributed caching layer that leverages NVMe disks and fast networks to optimize PyTorch training data access. Utilizing stateless workers for scalability and ETCD for membership services, this caching layer efficiently manages and serves data. Cached data is rapidly and efficiently fed into GPU memory using NVIDIA's DALI data loader, GPUDirect Storage (GDS), and Remote Direct Memory Access (RDMA), significantly reducing data transfer bottlenecks and improving overall training performance.

Love developing AI/ML systems, but don’t want to become the next front-page cyberattack? We got you! In this fast-paced, meme-a-liscious, hands-on workshop, we’ll take a deep dive into the murky waters of the AI/ML supply chain, explore its many threats and terrors, and then – with our trusty box of supply chain security tools – build an island of safety for our AI/ML systems! Participants will come away with the skills and knowledge to significantly improve AI/ML supply chain security at their organization, as well as the unpleasant awareness about what might happen if the industry doesn’t do the same. Sorry! Hands-on activities will include vulnerability scanning, creating/consuming SBOMs/AIBOMs, digital signing using Sigstore tools, and provenance tracking. We will also provide a conceptual background on AI/ML supply chain components, attack categories, and global regulation and standards related to AI/ML security. Led by software supply chain and AI deployment experts at Chainguard, this workshop will enable participants to harden their AI/ML systems and evangelize others to do the same.

Due to the ever-increasing adoption of AI into the lives of daily users, trustworthy AI is of utmost priority. Even though advocates of AI globally have started talking about ethical considerations during ML model building, in reality, very few people know how to create robust, privacy-preserving, and fair AI models. In this talk, I'll explore 2 concrete technical concepts of trustworthy AI, namely ensuring robustness and fairness in ML models. Robustness: 1. Attendees will go through an in-depth understanding of critical vulnerabilities of common AI models and how to exploit them to adversarially attack the model (e.g., inference attacks, data poisoning). 2. This will be followed by simple defence strategies to increase robustness (e.g., gradient obfuscation, transformations). 3. This will be further followed by adaptive attacks on previous defence strategies thereby motivating the concept of certified robustness of AI models. Fairness: 1. Attendees will get to know how they can unconsciously encode bias (representational bias, model bias, etc) during training AI models. 2. This is followed by strategies to correct this bias using domain knowledge to create fair AI models.

Ever feel like contributing to groundbreaking AI projects but unsure where to start? Have a cool project that you are looking to find a neutral entity for? Through the session learn how the LF AI & Data foundation empowers open-source projects. Discover the framework to support for open development, governance models, and resources like legal assistance, marketing, and events. Gain exclusive updates from the LF AI and Data Technical Advisory Council (TAC). Learn about their vision, technical roadmap, success stories and how you can contribute. The session will cover practical tips and tricks to dive into project contributions, navigate the initiation process, and ultimately guide you towards project graduation. Also discover the diverse work streams within LF AI and Data foundation and how you can leverage your skills to make a real difference in the open-source AI community.

Step into the future of AI as we break free from the constraints of cloud-based processing and unlock the immense potential of edge computing. This cutting-edge talk explores the revolutionary shift of AI from centralized data centers to the devices in your pocket, on your wrist, and all around you. Discover how this paradigm shift is not just changing the game – it's rewriting the rules. We'll journey through the landscape of on-device AI, revealing how it's transforming user experiences, supercharging privacy, and pushing the boundaries of what's possible in real-time applications. From smart homes to autonomous vehicles, from augmented reality to personalized healthcare – learn how on-device AI is the key to unlocking a world of intelligent, responsive, and secure applications.

Bitol is one of the newest LF AI & Data projects focusing on open standards and open source tools for modern data engineering. Bitol's flagship standard is ODCS (Open Data Contract Standard). In this talk, as the chair of the Bitol TSC, I will first explain what a data contract is, the work of the Bitol project, and focus on the benefits of data contracts for AI & analytics. I will conclude with an end-to-end demo of creating and enforcing data contracts using open-source and free tools.

As global regulations on AI software tighten, developers face a complex set of new, ambiguous rules. The AI Software Bill of Materials (AI BOM), especially the new SPDX 3.0 with AI and dataset profiles, offers a promising solution for compliance, providing detailed, machine-readable documentation of AI systems. Despite its benefits, adoption has been slow, hindered by gaps in developer knowledge and the complex nature of AI systems. Many AI BOMs are incomplete or inaccurate, limiting their utility for compliance. Our talk will tackle these issues, drawing on our experience with SPDX 3.0 and AI BOM implementation. We'll share best practices and strategies to improve AI BOM accuracy and utility, equipping professionals with the insights to ensure their AI applications are compliant and prepared for future regulations.