Image_Gen_Server/.github/copilot-instructions.md

AI Coding Agent Instructions

Project Overview

This is an OpenPose-based image generation server that integrates with ComfyUI for AI image generation. The system processes pose coordinates to generate skeleton images and sends them to ComfyUI for final image generation.

Architecture & Key Components

Core Service Flow

1. Flask API (app.py) receives pose coordinates via REST endpoints
2. OpenPose Generation (openpose_gen.py) converts coordinates to skeleton images
3. ComfyUI Integration uploads images and queues generation workflows
4. Skeleton Library (skeleton_lib.py) handles pose drawing with different formats (COCO, Body25)

Critical Dependencies

• ComfyUI Server: External service at localhost:8188 for AI image generation
• OpenCV: For image processing and skeleton rendering
• Flask: REST API server
• requests-toolbelt: For multipart file uploads to ComfyUI

Development Patterns

Coordinate System Convention

• Input coordinates are flat arrays: [x1, y1, confidence1, x2, y2, confidence2, ...]
• Use coordinates_to_keypoints() to convert to Keypoint objects
• Support both single pose (/gen_image) and multi-pose (/gen_group_pic) workflows

File Naming & Counters

• Output images use incremental counters: body_pose_output0000.png, body_pose_output0001.png
• Each function maintains its own static counter using hasattr() pattern
• Circular queue naming for ComfyUI uploads with hash-based names

MCP Integration

• Use context7 defined in .vscode/mcp.json for coding tasks
• Use python-language-server MCP for Python coding assistance
• Use analyzer MCP for code analysis tasks

ComfyUI Workflow Integration

• Workflow templates stored as JSON: fencerAPI.json, group_pic.json
• Modify seed values for randomization: prompt["3"]["inputs"]["seed"] = random.randint(0, 10000000000)
• Reference uploaded images by name in workflow nodes: prompt["17"]["inputs"]['image'] = openpose_image_name

Error Handling Pattern

if not coordinates or not canvas_size:
    return jsonify({"status": "error", "message": "Missing data"}), 422

Key Functions to Understand

save_bodypose() / save_bodypose_mulit()

• Converts coordinates to skeleton images using CV2
• Creates output directory if missing
• Returns image path for ComfyUI upload

upload_image_circular_queue()

• Manages unique image names per user/session using SHA256 hash
• Implements circular queue to prevent infinite file accumulation
• Essential for ComfyUI integration

queue_prompt()

• Sends workflow JSON to ComfyUI /prompt endpoint
• Triggers actual AI image generation

Development Workflow

Testing API Endpoints

# Single pose generation
curl -X POST -H "Content-Type: application/json" \
  -d '{"coordinates": [x1,y1,conf1,...], "canvas_size": [width,height], "pid": "user123"}' \
  http://localhost:5000/gen_image

Running the Server

python app.py  # Starts Flask in debug mode on localhost:5000

Skeleton Format Support

• COCO format: 18 keypoints (default for single poses)
• Body25 format: 25 keypoints (used in main() function)
• Use corresponding limbSeq and colors arrays from skeleton_lib.py

Integration Points

ComfyUI Server Requirements

• Must be running on localhost:8188
• Requires /upload/image and /prompt endpoints
• Workflow JSON files must match ComfyUI node structure

Output Directory Structure

output/                    # Generated skeleton images
embeddings/               # ComfyUI embeddings and models
script_examples/          # API usage examples

Common Modifications

Adding New API Endpoints

1. Define new Flask route in app.py
2. Create corresponding handler function
3. Follow existing patterns for input validation, image generation, and ComfyUI queuing

Adding New Pose Formats

1. Define new limbSeq and colors in skeleton_lib.py
2. Update coordinate conversion in coordinates_to_keypoints()
3. Modify canvas drawing in save_bodypose()

New ComfyUI Workflows

1. Export workflow from ComfyUI as JSON
2. Save in project root (e.g., new_workflow.json)
3. Create API function following gen_fencer_prompt() pattern
4. Add Flask endpoint in app.py

Debugging ComfyUI Integration

• Check ComfyUI server status at http://localhost:8188
• Verify uploaded images in ComfyUI interface
• Monitor workflow queue for errors
• Use script_examples/ for isolated testing