Free and open-source image inpainting tool based on state-of-the-art AI models, capable of removing unwanted objects or replacing anything in pictures.

IOPaint Project Details

Project Overview

IOPaint is a free and open-source image inpainting and outpainting tool based on state-of-the-art AI model technology. Developed and maintained by Sanster, the project aims to provide users with powerful and convenient image editing capabilities.

Project Address: https://github.com/Sanster/IOPaint Official Website: https://www.iopaint.com/

Core Features

1. Image Inpainting

• Object Removal: Remove unwanted objects, blemishes, watermarks, people, etc., from images.
• Content Replacement: Replace any content in images using Stable Diffusion technology.
• Intelligent Filling: Automatically identify and fill removed areas, maintaining the naturalness of the image.

2. Image Outpainting

• Canvas Extension: Extend image boundaries to generate larger images.
• Background Generation: Intelligently generate background content consistent with the original image style.

3. Text Drawing

• Text Addition: Add text of various styles to images.
• Font Customization: Supports multiple font and style selections.

Supported AI Models

Erase Models

These models are specifically designed for removing unwanted objects, blemishes, watermarks, and people from images, including:

• LAMA: Lightweight and efficient image inpainting model.
• LDM: Inpainting technology based on latent diffusion models.
• ZITS: Professional image inpainting algorithm.
• MAT: Mask-Aware Transformer model.

Diffusion Models

These models are used for object replacement and image outpainting tasks, including:

• PowerPaint: Stable Diffusion model optimized for inpainting/outpainting.
• Stable Diffusion: Classic image generation model.
• SDXL: Higher quality extended version.
• Kandinsky 2.2: High-quality image generation model developed in Russia.

Plugin System

IOPaint provides a rich plugin ecosystem:

Image Segmentation Plugins

• Segment Anything (SAM): Precise and fast interactive object segmentation.
• MobileSAM: Lightweight version, faster and less resource-intensive.
• Anime Segmentation: Segmentation model optimized for anime images.

Background Processing Plugins

• RemoveBG: Remove image backgrounds or generate masks for foreground objects.

Image Enhancement Plugins

• RealESRGAN: Super-resolution technology to improve image clarity.
• GFPGAN: Face restoration and enhancement.
• RestoreFormer: Another face restoration solution.

File Management Plugins

• FileManager: Convenient image browsing and management tool.

Technical Features

Cross-Platform Support

• Completely Free and Open Source: Follows the open-source license, allowing users to freely use and modify.
• Self-Hosted: Fully local deployment, protecting user privacy.
• Multi-Device Support:
  • CPU: Suitable for ordinary computers.
  • GPU: Supports CUDA acceleration.
  • Apple Silicon: Native support for M-series chips.

Web Interface

• Modern UI: Provides an intuitive and easy-to-use web interface.
• Real-time Preview: Supports real-time viewing of editing effects.
• Batch Processing: Supports batch processing of multiple images.

Installation and Usage

Quick Installation

# Install IOPaint
pip3 install iopaint

# Start the service (using LAMA model, CPU mode)
iopaint start --model=lama --device=cpu --port=8080

GPU Accelerated Installation

# First install the CUDA version of PyTorch
pip3 install torch==2.1.2 torchvision==0.16.2 --index-url https://download.pytorch.org/whl/cu118

# Then install IOPaint
pip3 install iopaint

AMD GPU Support

# AMD GPU users (Linux only)
pip3 install torch==2.1.2 torchvision==0.16.2 --index-url https://download.pytorch.org/whl/rocm5.6

Start the Service

After installation, visit http://localhost:8080 to start using.

Command-Line Batch Processing

iopaint run --model=lama --device=cpu \
--image=/path/to/image_folder \
--mask=/path/to/mask_folder \
--output=output_dir

Plugin Activation

# Enable interactive segmentation plugin
iopaint start --enable-interactive-seg --interactive-seg-device=cuda

Development Environment Setup

Frontend Development

# Clone the project
git clone https://github.com/Sanster/IOPaint.git
cd IOPaint/web_app

# Install dependencies
npm install

# Build the frontend
npm run build
cp -r dist/ ../iopaint/web_app

# Development mode
npm run dev

Backend Development

# Install backend dependencies
pip install -r requirements.txt

# Start the backend service
python3 main.py start --model lama --port 8080

Visit http://localhost:5173/ in the development environment for frontend development.

Mobile Application

IOPaint also launched a mobile application:

• OptiClean: Object eraser application for macOS and iOS, available for download on the App Store.

Application Scenarios

1. Photo Post-Processing: Remove unwanted passers-by, objects, or watermarks.
2. Product Image Optimization: Clean up blemishes in product photos.
3. Artistic Creation: Perform creative image editing and content replacement.
4. Image Restoration: Restore old photos or damaged images.
5. Background Processing: Replace or remove image backgrounds.
6. Image Extension: Expand the image canvas.

Project Advantages

1. Open Source and Free: Completely open source, with no usage restrictions.
2. Privacy Protection: Local deployment, data is not uploaded to the cloud.
3. Rich Models: Supports a variety of state-of-the-art AI models.
4. Easy to Use: Provides an intuitive web interface.
5. Highly Customizable: Supports plugin extensions and model selection.
6. Cross-Platform: Supports multiple operating systems and hardware architectures.
7. Continuous Updates: Active development community and regular updates.

IOPaint is one of the best open-source image inpainting tools currently available, providing powerful image editing capabilities for both individual users and professional developers.