GenAI Processors Project Details

Project Overview

GenAI Processors is a lightweight Python library for building modular, asynchronous, and composable AI processing pipelines, specifically designed for generative AI applications. Launched by Google, this project aims to simplify the development of complex AI applications, especially those requiring multimodal inputs and real-time responsiveness.

Core Concepts

Processor

At the core of GenAI Processors is the concept of a Processor: a fundamental building block that encapsulates a specific unit of work. It receives an input stream, performs operations, and outputs a result stream. Each Processor has a simple, unified API:

# Any class inheriting from processor.Processor and implementing this function is a Processor
async def call(
    content: AsyncIterable[ProcessorPart]
) -> AsyncIterable[ProcessorPartTypes]

ProcessorPart

ProcessorPart is a wrapper around genai.types.Part with rich metadata such as MIME type, role, and custom attributes. It supports various content types (text, image, audio, custom JSON).

Key Features

1. Modular Design

• Decomposes complex tasks into reusable Processor and PartProcessor units
• Can be easily chained (+) or parallelized (//) to create complex data flows and agent behaviors

2. Integration with GenAI API

• Includes ready-to-use processors, such as GenaiModel for turn-based API calls
• Supports LiveProcessor for real-time streaming interactions

3. Extensibility

• Create custom processors by inheriting from base classes or using simple function decorators
• Supports community-contributed processor extensions

4. Asynchronous and Concurrency

• Based on Python's familiar asyncio framework
• Coordinates concurrent tasks (including network I/O and communication with computationally intensive sub-threads)

5. Stream Management

• Provides utilities for splitting, joining, and merging ProcessorPart asynchronous streams
• Supports streaming to reduce latency and Time to First Token (TTFT)

Installation Requirements

The library requires Python 3.10+ version.

Installation command:

pip install genai-processors

Usage Example

Basic Usage

from genai_processors import content_api
from genai_processors import streams

# Create an input stream (strings are automatically converted to Parts)
input_parts = ["Hello", content_api.ProcessorPart("World")]
input_stream = streams.stream_content(input_parts)

# Apply the processor to the part stream and iterate over the results
async for part in simple_text_processor(input_stream):
    print(part.text)

Learning Resources

The project provides a series of Colab notebooks to help users familiarize themselves with GenAI Processors (recommended to study in order):

1. Content API Colab - Explains the basics of ProcessorPart, ProcessorContent, and how to create them
2. Processor Intro Colab - An introduction to the core concepts of GenAI Processors
3. Create Your Own Processor - A walkthrough of the typical steps to create a Processor or PartProcessor
4. Work with the Live API - An example of building real-time processors using the LiveProcessor class from the Gemini Live API

Practical Application Examples

1. Real-time Live Example

examples/realtime_simple_cli.py - An audio-in audio-out Live agent that integrates Google Search as a tool. This is a client-side implementation of a Live processor, demonstrating GenAI Processors' streaming and orchestration capabilities.

2. Research Agent Example

examples/research/README.md - A research agent built using Processors, featuring 3 sub-processors, chained processing, ProcessorPart creation, and more.

3. Real-time Commentary Example

examples/live/README.md - A real-time commentary agent built using the Gemini Live API, composed of two agents: one for event detection and another for managing conversations.

Project Structure

Core Directory

The core/ directory contains a set of fundamental processors that can be used in your own applications. It includes common building blocks required for most real-time applications and will evolve over time to include more core components.

Contrib Directory

The contrib/ directory contains community-contributed processor extensions to expand the built-in processor collection.

Technical Advantages

1. Low-Latency Processing

Even for non-streaming use cases, processing data immediately as it becomes available can significantly reduce latency and Time to First Token (TTFT), which is crucial for building a good user experience.

2. Responsive Application Development

While many LLM APIs prioritize synchronous, simplified interfaces, GenAI Processors offers a way to write responsive applications by leveraging native Python features without making the code overly complex.

3. Concurrent Processing Capabilities

The travel planner and research agent examples demonstrate how turn-based agents can use GenAI Processors' concurrency features to improve responsiveness.

Conclusion

GenAI Processors provides developers with a powerful and flexible framework for building complex generative AI applications. Its modular design, asynchronous processing capabilities, and deep integration with Google AI services make it an ideal choice for developing modern AI applications. Whether building real-time conversational systems, multimodal processing pipelines, or complex AI agents, GenAI Processors offers the necessary tools and abstraction layers to simplify the development process.