Image Super-Resolution with SMFANet

Summary

• Introduction
• Parameters
• Examples
• Usage for developers
• Reference
• License
• Citation

Introduction

Image Super-Resolution (SR) aims to reconstruct high-resolution images from their low-resolution counterparts, restoring fine details and enhancing visual quality. SMFANet (Self-Modulation Feature Aggregation Network) is a lightweight yet effective neural network designed specifically for this task. The architecture of SMFANet is composed of three main components: a shallow feature extraction module, multiple feature modulation blocks, and a compact image reconstruction module.

At the core of SMFANet lies the Feature Modulation Block, which integrates two key innovations: the Self-Modulation Feature Aggregation (SMFA) module and the Partial Convolution-based Feed-Forward Network (PCFN). The SMFA module adaptively aggregates features across different scales, enhancing context understanding, while PCFN improves efficiency and generalization by selectively updating feature regions. Together, these components enable SMFANet to achieve high-quality super-resolution with minimal computational overhead, making it well-suited for real-time and resource-constrained applications.

Parameters

Inputs

• input_image - (image -.png|.jpg|.jpeg): The input to the model is a low-resolution image that needs to be upscaled and enhanced in detail and clarity.

Output

• output_image - (image -.png): The output is a high-resolution version of the input image, generated using the SMFANet super-resolution model.

Examples

input	output

Usage for developers

Please find below the details to track the information and access the code for processing the model on our platform.

Requirements

pip install -r requirements.txt

Code based on AIOZ structure

import torch
from .smfanet_arch import SMFANet
from .utils import load_model, load_yaml_config, load_image, save_output_image

...

def do_ai_task(
        input_image: Union[str, Path],
        model_storage_directory: Union[str, Path],
        device: Literal["cpu", "cuda", "gpu"] = "cpu",
        *args, **kwargs) -> Any:
    """Define AI task: load model, pre-process, post-process, etc ..."""
    # Define AI task workflow. Below is an example
    model_dir = Path(model_storage_directory)
    config_path = model_dir / "..."
    weights_path = model_dir / "..."

    cfg = load_yaml_config(config_path)
    torch_device = torch.device("cuda" if torch.cuda.is_available() and device != "cpu" else "cpu")

    # Initialize model
    model = SMFANet(
        dim=cfg.model.dim,
        n_blocks=cfg.model.n_blocks,
        ffn_scale=cfg.model.ffn_scale,
        upscaling_factor=cfg.model.upscaling_factor
    ).to(torch_device)

    load_model(model, weights_path)

    # Prepare input image
    img_np = load_image(input_image)
    input_tensor = torch.from_numpy(img_np).permute(2, 0, 1).unsqueeze(0).to(torch_device)

    with torch.no_grad():
        output_tensor = model(input_tensor)

    output_np = (
        output_tensor.squeeze()
        .permute(1, 2, 0)
        .clamp(0, 1)
        .cpu()
        .numpy()
    )

    output_path = save_output_image(output_np, "output.png")
    return open(output_path, "rb")

Reference

This repository is based on and inspired by Zheng-MJ's work. We sincerely appreciate their generosity in sharing the code.

License

We respect and comply with the terms of the author's license cited in the Reference section.

Citation

@inproceedings{smfanet,
    title={SMFANet: A Lightweight Self-Modulation Feature Aggregation Network for Efficient Image Super-Resolution},
    author={Zheng, Mingjun and Sun, Long and Dong, Jiangxin and Pan, Jinshan},
    booktitle={ECCV},
    year={2024}
 }