• CUDAbuilt on top of ROCmby AMD

IMAGE GENERATION

• parent: stable_diffusion

• upscale model database: https://openmodeldb.info/ old version site

WORKFLOWS

• https://comfyworkflows.com/

• others: cooking_repertoire

• reposer plus comfyui workflow for pose face(ip-adapter) clothing

• https://learn.thinkdiffusion.com/a-list-of-the-best-comfyui-workflows/

• Stable Cascade Canny ControlNet https://github.com/ZHO-ZHO-ZHO/ComfyUI-Workflows-ZHO

KEYWORDS

• sdxl artists-repertoire keywords

• nai artistswith examples

SD GUIDES

• entry guide (old): https://imgur.com/a/VjFi5uM

• all the links about stable diffusion categorized https://rentry.co/RentrySD

• https://gitgud.io/gayshit/makesomefuckingporn (lora index)

• https://rentry.org/sdg-link (index of everything)

• https://rentry.org/hdgrecipes (model merging index)

  • block merging (unet) https://rentry.org/BlockMergeExplained

• neuralnomicon node: cooking_repertoire

LORA

• https://rentry.org/lora_train (how to lora)

  • https://rentry.org/ezlora

• CharFrameworkexplanationsframework to create character loras

  • charturnerseveral faces of one character

SIDE TOOLS

• generate: spritesage of empires

• real time paintingdiffusing while painting): https://github.com/houseofsecrets/SdPaint

• SVGcode Convertcolor bitmap images to color SVG vector images, auto versionvector graphics

• read the stable diffusion stored(the prompts) metadata from images: prompt-reader

• inpainter (watermark remover) https://github.com/advimman/lama huggingface

• DiffusionToolkit Metadata-indexer and Viewer for generated images

• RatioScope bucketing images for dataset

FACE GENERATION - SWAP

• loraless

  • FaceChain now two images = any angle, sd-webui support

• FaceSwapLab stable diffusion, webui

• an IP Adapterface model

• FaceFusion face swapperand enhancer for video ==best==

• Roop High-performanceface swapper (Opal)

• COMFYUI INSTANTID

• InsightFace opensource alternative: LivePortrait(Bring portraits to life)

HELPERS

• DeepBump generate normal & height maps from single pictures

• ProPainter removalfor videos using metaclip and sam

• VCHITECT

• supervision people-object trakers ready to use (reusable computer vision tools) employees at the zone

2SKETCH, TO SKETCH

• Anime2Sketch generate sketch from anime image

• Stylized FaceSketch Extraction via Generative Prior with Limited Data

  • generate sketch from image, using sketch-image-example to set style

STORYTELLING CAPTIONING

• The MangaWhisperer: Automatically Generating Transcriptions for Comics (magi) storytelling, storyboard

  • generate a transcript, ocr, order panels, cluster characters

OPENPOSE EDITOR

• online3d openpose editor: https://zhuyu1997.github.io/open-pose-editor/

• PMX model - MMD(mikumikudance): https://civitai.com/models/28916/openpose-pmx-model-mmd

• blender https://toyxyz.gumroad.com/l/ciojz

• OpenPose Man lora: https://civitai.com/models/76718

• Stable SegMap using unity on web

• ComfyUI-openpose-editor

TAGGER

WD TAGGER

• https://github.com/picobyte/stable-diffusion-webui-wd14-tagger/tree/master

• model: https://huggingface.co/SmilingWolf/wd-v1-4-vit-tagger/tree/main

• model: https://github.com/KichangKim/DeepDanbooru/releases/tag/v3-20211112-sgd-e28

• cli: https://github.com/KichangKim/DeepDanbooru

CAPTIONING MODELS

• CogVLM and Moonshot2 both are insanely good at captioning

• Qwen-VL-Max #1, THUDM/cogagent-vqa-hf #2, liuhaotian/llava-v1.6-vicuna-13b #3.

• tagguifor cog - https://github.com/jhc13/taggui/releases

• For llava 1.6 - https://github.com/DEVAIEXP/image-interrogator

• Qwen-VL-Max - https://huggingface.co/spaces/Qwen/Qwen-VL-Max

• VISION MODEL, IMAGE TO PROMPT

• JoyCaption Free, Open, Uncensored VLM (Early pre-alpha release) (nsfw captioning)

UNLABELED

• https://github.com/autodistill/autodistill

DETECTORS (COMPUTER VISION)

https://github.com/hysts/anime-face-detector

ANTI GLOW

• Nightshade Antidote remove poison pill from image

UIs

FRONT-ENDS

• ComfyUI

  • comfyui but from python-emacs: https://codeberg.org/tekakutli/apapach

  • StableSwarmUI making comfyui easily accessible

• Auto1111 Webui

• Fooocus

  • Focus on prompting and generating, manual tweaking is not needed

  • Refocus Alternative ui for Fooocus

• ENFUGUE Stable Diffusion web app

• Chibi comfyui gui in Vue

MAKE YOUR GUI

• https://github.com/space-nuko/ComfyBox

CODE

• Diffusers (python pipelines): https://huggingface.co/docs/diffusers/index

  • https://github.com/ddPn08/Radiata

    • Stable diffusion webui based on diffusers

• nodejs: https://github.com/dakenf/stable-diffusion-nodejs

• fastembed lightweightPython for embedding generation

CPU

• ggml: inference in pure c/c++ (interoperability, no python dependency hell)

  • https://github.com/leejet/stable-diffusion.cpp

  • Running StableDiffusion XL 1.0 in 298MB of RAM (Raspberry Pi Zero 2)

    • OnnxStream consumes 55x less memory than OnnxRuntime while being only 0.5-2x slower

• FastSD CPU Faster version of stable diffusion running on CPU

  • FastSD CPU beta 16 release with 2 steps fast inference

FASTER

• Stable-Fast on NVIDIA GPU

• ComfyUI-AIT faster inference using cpp/cuda.

TRAINING

• LoRAtraining extentionfor Web-UI

• kohya training scripts: https://github.com/kohya-ss/sd-scripts

  • https://github.com/bmaltais/kohya_ss

• SCEPTER training, fine-tuning, and inference with generative models

• OneTrainer one-stop solutionfor all your stable diffusion training needs

FINETUNING

• SimpleTuner fine-tuning kit geared toward Stable Diffusion 2.1 and SDXL

  • StableTuner 1.5

• EveryDream2trainer

• SensorialSystem's Stable Diffusion

  • automate all the steps of finetuning Stable Diffusion models.

MODELS

• Taiyi-Stable-Diffusion finetuned in chinese

CONTROLNET

• ALL SDXL CONTROLNET MODELS : https://civitai.com/models/136070?modelVersionId=154923

  • controlllite: https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite

• PREPROCESSORS

• face landmarks get landmarks from face

• MasaCtrl change pose by changing prompt of input image, optionally with controlnet

• Würstchen: more controlnets

• Freecontrol wireframe, rag doll, lidar, face mesh

SDXL-1.0

• openpose t2i-adapter: https://huggingface.co/TencentARC/T2I-Adapter/tree/main/models_XL

• list of them all: https://six-loganberry-ba7.notion.site/23-08-23-SDXL-ControlNet-619fdd7fff954df2ae918c69e2814fe1

  • all on: https://huggingface.co/lllyasviel/sd_control_collection/tree/main

  • https://huggingface.co/collections/TencentARC/t2i-adapter-sdxl-64fac9cbf393f30370eeb02f

• TTPLanet_SDXL_Controlnet_Tile_Realistic_V1

  • adds feature details

SMOL

• controlnet-loras instead: https://huggingface.co/stabilityai/control-lora

  • seems to extract the difference between the model and ControlNet with svd

• controlnet-lltite (for now only sdxl) by kohya

  • controlnet as a hypernetwork.

  • comfyui node: https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI

  • ==what's the difference between them?==

    • by furusu: https://twitter.com/gcem156/status/1693597263912899046 <button class="pull-tweet" value=https://twitter.com/gcem156/status/1693597263912899046>pull</button>/photo/1

CONTROLNET CANNY

• https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0/tree/main

COMFY

• krita plugin

• AP Workflow complexworkflow with everything and organized, including interoperability with oobabooga

INSTALLATION SNIPPET

NEGATIVE LORAS

• feeding Stable Diffusion XL examples of bad images that it itself generated as a lora makes SDXL behave much better to the spirit of the prompt

  • https://twitter.com/minimaxir/status/1693657050990227640 <button class="pull-tweet" value=https://twitter.com/minimaxir/status/1693657050990227640>pull</button>

    • https://minimaxir.com/2023/08/stable-diffusion-xl-wrong/

PROGRAMMATIC

PYTHON

• https://github.com/pydn/ComfyUI-to-Python-Extension

• python coderemotely

• ComfyScript workflows, A Python front end for ComfyUI

• Comfy Runner Automatically install ComfyUI nodes and models and use it as a backend (like diffusers)

CUSHY

• programmatic pipelines using typescript

• https://github.com/rvion/CushyStudio

COMFY NODES

• https://registry.comfy.org/ << best tracker

• Mathnodes

• diffdiff: Differential Diffusion

• Core MLmodels: leverage Apple Silicon

• bundlednodes, lower node count (like highresfix)

• LCMSampler-ComfyUI In order to take advantage of the high-speed generation by LCM lora, loras

  • Alternative

  • comfyui-tcd-scheduler default value 0.3, use higher eta when more inference steps

• AnyText text generation on the image

• InstructIR image restoration watermark removal, fuzziness removal

• StableSR superresolution

• ComfyUI_VLM_nodes querying(llava, kosmos), captioning(joytag)

• ComfyUI_FaceAnalysis evaluate the similarity between two faces

• 3D Text Comfyroll Studio

• dynamicprompts combinatorial prompts, prompt enhancement

• ComfyUI-DragAnything

• Controller floating panel which reproduces widget inputs from all the red nodes in your workflow.

AUDIO

• ComfyUI-VoiceCraft Zero-Shot Speech Editing and Text-to-Speech in the Wild

• comfyui-sound-lab A node collection for sound design, supporting MusicGen and Stable Audio.

• AudioReactive create animations that follow sounds-bands

  • Particle systems! Optical flow! Temporal masks!

REGIONAL EDITING

• RMBG background removal

• Inpaint Nodes better inpaint

• Prompt-To-Prompt change words

• OOTDiffusion integrates the OOTDiffusion

• BrushNet better inpainting

NATIVE OFFSET NOISE

• VectorscopeCC: Offset Noise* natively (control over light, contrast, shadows)

UI MANAGER

• comfy-browser An image/video/workflow browser and manager for ComfyUI

• AIGODLIKE-ComfyUI-Studio loading models more intuitive, create model thumbnails

• ComfyUI-N-Sidebar for fav nodes

UPSCALE

• upscale

  • SuperResolution

  • ComfyUI-CCSR

  • SUPIR

  • clarity-upscaler

  • ComfyUI-APISR anime upscaler

OPTIMIZATION

• comfy-todo Token Downsampling for Efficient Generation of High-Resolution Images

3D

• ComfyUI-3D-Pack process 3D inputs (Mesh & UV Texture, etc) using cutting edge algorithms (3DGS, NeRF, etc)

• ComfyTextures Unreal Engine ⚔️ ComfyUI - Automatic texturing using generative diffusion models

  • ComfyUI-Texture-Simple

• ComfyUI-Flowty-CRM generate meshes one image to 3d

STYLE

• VisualStylePrompting style from example image ==best==

• ComfyUI-PixelArt-Detector Generate, downscale, change palletes and restore pixel art images

TARGET STYLE-SUBJECT

• StyleAligned consistent style to all images in a batch

• faceswap

• ComfyUI PortraitMaster: generates prompts for skin color, expresion, shape, light direction

PLUGIN LISTS

• https://github.com/WASasquatch/comfyui-plugins/

IMAGE PROCESSING

• ComfyUI-YOLO Ultralytics-Powered Object Recognition for ComfyUI

MASKING

• ComfyUI-BiRefNet best

EDITING MASK

• ComfyUI-KJNodes RBG to mask, grow mask with blur

IMPACT PACK

• inpainting, masking, sam, automasking face

  • select face minus hair or the inverse

• https://github.com/ltdrdata/ComfyUI-Impact-Pack

• hair restyling

  • tutorial: https://www.youtube.com/watch?v=5SCP9ijDuuA

  • workflow: https://raw.githubusercontent.com/ltdrdata/ComfyUI-extension-tutorials/Main/ComfyUI-Impact-Pack/workflow/hair-restyle.json

PER-INSTANCE MASK

• YOLO-World+ EfficientSAM for ComfyUI

GENERATE WITH TRANSPARENCY

• Layer Diffusioncustom nodes

PREPROCESSORS

• CONTROLNET

• turn image into canny, openpose, etc

  • pose editor

• it also has an inpaint node

• old preprocessorsnew preprocessors

  • HandRefiner Support

• DepthFM monocular depth estimation

LIGHTING

• comfyUI_TJ_NormalLighting Custom Node for comfyUI for virtual lighting based on normal map

  • relightening, based on animatediff

• Line2Normalmap

• ComfyUI-DiffusionLight method of creating light probes

TEXT

LLM

• ComfyUI_LLM_Node deployment of models like T5, GPT-2

• Tara- ComfyUI Node for LLM Integration

• ComfyUI-Gemini Gemini in ComfyUI

TEXT ENCODERS

• promptcontrol: example: a [large::0.1] [cat|dog:0.05] [<lora:somelora:0.5:0.6>::0.5]

• ComfyUI-ScenarioPrompt prompt creation helper

• ComfyUI_ELLA llm instead of clip

PROMPT ENHANCE

• Plush-for-ComfyUI prompt enhancing using llm

• Comfyui-Superprompt-Unofficial make dull prompts detailed

AUTO1111 TOKENS ON COMFY

A1111's token normalization and weighting in ComfyU This means you can reproduce the same images generated on stable-diffusion-webui on ComfyUI.

<h7 id="org-ca5685abf2">LINK</h7>

https://github.com/shiimizu/ComfyUI_smZNodes

ADVANCED TOKEN WEIGHTS

https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb

IMAGE ENCODING

• clip-vision model

• SEECODERSComfy

• ComfyUI-InstantID no more lora per subject, just one picture is enough

  • ComfyUI PhotoMakerPlus

• Vector_Sculptor_ComfyUI

  • Gather similar vectors within the CLIP weights and use them to redirect the original weights

IP ADAPTER

• IP-Adapter

  • you need clip-vision model

  • comfyui examples

  • ip-composition-adapter general composition of an image while ignoring the style and content

    • like controlnet but less acurate

• ComfyUI_IPAdapter_plus

COMFYUI INSTANTID

• old: ComfyUI InstantIDFaceswapper

• native: ComfyUI_InstantID

VISION MODEL, IMAGE TO PROMPT

• VLM nodesexamples

• comfyui-moondream tiny vision language model; image to prompt

• Comfyui_image2prompt image to prompt by vikhyatk/moondream1

• ComfyUI_DanTagGen LLM model designed for generating Danboou Tags with provided informations, trained on Danbooru datasets

VIDEO

• Comfy-SVDTools

• Jovimetrix Nodes for procedural masking, live composition and video manipulation

• DynamiCrafter diffusionpriors

  • native DynamiCrafter

• Font toAnimation

DANCING

• AnimateAnyone dancing

  • ComfyUI-AnimateAnyone

• Champ Controllableand Consistent Human Image Animation with 3D Parametric Guidance

  • incorporates depth, normal maps, semantic maps from SMPL sequences, skeleton-based motion guidance

MOTION

• MotionCtrl Flexible Motion Controller for Video Generation

• DragNUWA manipulate backgrounds or objects motions

  • LightGluerequired)

• LiveDirector use reference video to drive motion

LIP SYNC

• ComfyUI-MuseTalk

NOT JUST IMAGES

• AnimateDiff

  • SteerableMotion: for steering videos with batches of images

• Stable Zero123for ComfyUI

• fastblend smooth out video frames

• ComfyUI_cspnodes ZeroScope nodes

ONLINE SERVICES

• main registry of loras: https://www.civitai.com/

  • chinese alternative: https://www.liblib.art/

  • vietnam hosted: https://tensor.art/

• comfy pipelines https://comfyworkflows.com/

THE HORDE

• https://www.stablehorde.net

  • distributed cluster, built on top of ComfyUI, you can use any lora on CivitAI

  • https://dbzer0.com/blog/state-of-the-ai-horde-july-2023/

  • clients: Lucid Creations, ArtBot

    • https://github.com/daveschumaker/artbot-for-stable-diffusion/

    • krita plugin: https://github.com/dunkeroni/krita-stable-horde

OTHERS VISUAL

• PALLAIDIUM Generative AI for the Blender VSE(Blender Video Sequence Editor)

  • Text, video or image to video, image and audio

• blender-stable-diffusion-render addon for using Stable Diffusion to render texture bakes for objects

SEGMENTATION

• https://github.com/ltdrdata/ComfyUI-Impact-Pack

  • facedetailer

    • uses

    • https://github.com/hysts/anime-face-detector

    • https://github.com/open-mmlab/mmdetection

    • https://github.com/open-mmlab/mmpose

• https://github.com/facebookresearch/segment-anything/blob/main/notebooks/predictor_example.ipynb

  • hd finetune https://github.com/SysCV/sam-hq

  • with text prompt https://github.com/IDEA-Research/Grounded-Segment-Anything

    • merged segment-anything and grounding-dino

      • grounding = get bounding box(or mask) from text prompt

• https://github.com/biegert/ComfyUI-CLIPSeg

3D

MESH GENERATION

• threestudio A unifiedframework for 3D content generation

  • ProlificDreamer, DreamFusion, Magic3D, SJC, Latent-NeRF, Fantasia3D, TextMesh, Zero-1-to-3, Magic123 InstructNeRF2NeRF, and Control4D are all implemented in this framework.

• GSGEN: Text-to-3Dusing Gaussian Splatting

• 3DTopia Two-stage text-to-3D generation model (5 minutes)

NERF

• https://github.com/nerfstudio-project/nerfstudio

GAUSSIAN

• JavaScript Gaussian Splatting library

  • https://github.com/dylanebert/gsplat.js

ANIMATEDIFF

• https://github.com/JaredTherriault/sd-webui-animatediff

TEXT

• aphrodite chat bots, roleplay (by the horde)

• LLaMA-Factory Easy-to-useLLM fine-tuning framework (LLaMA, BLOOM, Mistral, Baichuan, Qwen, ChatGLM)

INFERENCE

• https://github.com/simonw/llm (cli)

  • online and local

• ollama golang, uses llama.cpp, local models

  • llama.cpp

    • intel-enhancedllama.cpp

    • ChatGLMcpp (chinese, llama.cpp derived)

  • https://github.com/s-kostyaev/ellama (emacs)

    • LLM-Bash Wrapper for llm & Ollama to be used by your code editor

• fast-llmgit

UI

• oobabooga main, supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models

• koboldcpp-rocm various GGML models with KoboldAI's UI with AMD ROCm offloading

• ChuanhuChatGPT webui gradio

CODE

• Open Interpreter an open-source Code Interpreter

  • create and edit photos, summarize pdfs, control your browser, plot and analyze large datasets

• DeepSeek Coder several models

DATASET

• lewd roleplay dataset: https://huggingface.co/datasets/lemonilia/LimaRP

VOICE GENERATION

• parent: voice

REALTIME

• Realtime Voice Changer

  • https://github.com/w-okada/voice-changer/tree/master

  • AI covers(music): INSTANTLY make AI covers with ANY voice https://www.youtube.com/watch?v=pdlhk4vVHQk

    • https://github.com/SociallyIneptWeeb/AICoverGen