DWPose: Whole-Body Pose Estimation

Summary

• Introduction
• Parameters
• Examples
• Usage for developers
• Reference
• License

Introduction

DWPose (Effective Whole-body Pose Estimation with Two-stages Distillation) is a state-of-the-art model for whole-body pose estimation. It accurately extracts 133 keypoints—including body, face, hands, and feet—from a single image. By utilizing a two-stage pose distillation strategy and building upon the RTMPose framework and YOLOX object detector, DWPose achieves an excellent balance between accuracy and inference speed.

Packaged for the AIOZ AI Node runtime, this workflow accepts a local image path and generates an annotated output image containing the rendered whole-body pose skeletons. The extracted poses can be used directly for downstream tasks such as ControlNet OpenPose-guided generation, motion capture, and human pose analysis.

Parameters

Inputs

• input - (image -.png|.jpg|.jpeg): The input image containing human pose(s) to estimate.

Output

• output - (image -.png): The output image with whole-body pose skeletons (body, hands, face, and feet keypoints) rendered on a canvas.

Examples

input	output

Usage for developers

Please find below the details to track the information and access the code for processing the model on our platform.

Requirements

• Python 3.10 or newer
• Git LFS for the model checkpoints
• A CUDA-capable GPU is optional

Fetch LFS objects if the repository was cloned without them, then install the dependencies:

git lfs pull
pip install -r requirements.txt

Run the demo

python demo.py

Code based on AIOZ structure

import os
from pathlib import Path
from typing import Any, Union, Literal
import tempfile

import cv2
import numpy as np

# DWPose
from controlnet_dwpose import DWposeDetector, draw_pose

...
def do_ai_task(
    input: Union[str, Path],
    model_storage_directory: Union[str, Path],
    device: Literal["cpu", "cuda", "gpu"] = "cpu",
    *args,
    **kwargs,
) -> Any:
    """Define AI task: load model, pre-process, post-process"""

    device_name = "cuda" if device in ["cuda", "gpu"] else "cpu"

    yolox_model = os.path.join(model_storage_directory, "yolox_l.onnx")
    dwpose_model = os.path.join(model_storage_directory, "dw-ll_ucoco_384.onnx")

    # Initialize the DWPose detector
    detector = DWposeDetector(
        model_det=yolox_model,
        model_pose=dwpose_model,
        device=device_name
    )

    # Load image via opencv
    image = cv2.imread(str(input))
    if image is None:
        raise ValueError(f"Could not read image from {input}")

    # Process image with DWPose
    # The detector returns the raw pose points
    res = detector(image)

    # Draw pose skeleton
    result_image = draw_pose(res, image.shape[0], image.shape[1])
    # The drawn image is (C, H, W) format, so transpose to (H, W, C) for OpenCV
    result_image = np.transpose(result_image, (1, 2, 0))

    # Save to a temporary file
    fd, fp = tempfile.mkstemp(suffix=".png")
    os.close(fd)

    cv2.imwrite(fp, result_image)

    return fp

Reference

This repository is based on and inspired by IDEA-Research/DWPose and utilizes ONNX model weights from fashn-ai/DWPose. We sincerely appreciate their generosity in sharing the models and research.

• Paper: "Effective Whole-body Pose Estimation with Two-stages Distillation" (ICCV 2023)
• Authors: Zhendong Yang, Ailing Zeng, Chun Yuan, Yu Li

License

We respect and comply with the terms of the author's apache-2.0 license cited in the Reference section.

Citation

@inproceedings{yang2023effective,
  title={Effective Whole-body Pose Estimation with Two-stages Distillation},
  author={Yang, Zhendong and Zeng, Ailing and Yuan, Chun and Li, Yu},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
  pages={4210--4220},
  year={2023}
}