:PROPERTIES:
:ID:       75929071-e62b-4c0a-8374-8ca322d0a020
:END:
#+title: software
#+filetags: :neuralnomicon:
#+SETUPFILE: https://fniessen.github.io/org-html-themes/org/theme-readtheorg.setup
- [[https://github.com/vosen/ZLUDA][CUDA]] built on top of [[https://lemmy.dbzer0.com/post/14289810][ROCm]] by AMD
* IMAGE GENERATION
- parent: [[id:c7fe7e79-73d3-4cc7-a673-2c2e259ab5b5][stable_diffusion]]
- upscale model database: https://openmodeldb.info/ [[https://upscale.wiki/w/index.php?title=Model_Database&oldid=1571][old version site]]
** WORKFLOWS
- https://comfyworkflows.com/
- others: [[id:848d0172-a0a9-47a9-8524-7cdd2f60c6bd][cooking_repertoire]]
- [[https://www.youtube.com/watch?v=ZcCfwTkYSz8][reposer plus]]: comfyui workflow for pose + face(ip-adapter) + clothing
- https://learn.thinkdiffusion.com/a-list-of-the-best-comfyui-workflows/
- Stable Cascade Canny ControlNet https://github.com/ZHO-ZHO-ZHO/ComfyUI-Workflows-ZHO
*** KEYWORDS
- [[https://weirdwonderfulai.art/resources/stable-diffusion-xl-sdxl-artist-study/][sdxl artists-repertoire]]: keywords
- [[https://rentry.org/NAIDv3artisttagtest][nai artists]] with examples
** SD GUIDES
- entry guide (old): https://imgur.com/a/VjFi5uM
- all the links about stable diffusion categorized https://rentry.co/RentrySD
- https://gitgud.io/gayshit/makesomefuckingporn (lora index)
- https://rentry.org/sdg-link (index of everything)
- https://rentry.org/hdgrecipes (model merging index)
  - block merging (unet) https://rentry.org/BlockMergeExplained
- neuralnomicon node: [[id:848d0172-a0a9-47a9-8524-7cdd2f60c6bd][cooking_repertoire]]
*** LORA
- https://rentry.org/lora_train (how to lora)
  - https://rentry.org/ezlora
- [[https://github.com/Zuntan03/CharFramework][CharFramework]] [[https://twitter.com/Zuntan03/status/1640240599323541504][explanations]] framework to create character loras
  - [[https://note.com/stelsray/n/n1b1f12e17c5d][charturner]] several faces of one character
** SIDE TOOLS
- generate: [[https://www.engine.study/blog/modding-age-of-empires-ii-with-a-sprite-diffuser/][sprites]] age of empires
- real time [[https://www.reddit.com/r/StableDiffusion/comments/12qlg3b/who_needs_photoshop_anyway_ms_paint_sd/][painting]](diffusing while painting): https://github.com/houseofsecrets/SdPaint
- [[https://github.com/tomayac/SVGcode][SVGcode]]: [[https://svgco.de/][Convert]] color bitmap images to color SVG vector images, [[https://github.com/GeorgLegato/stable-diffusion-webui-vectorstudio][auto version]] vector graphics
- read the stable diffusion stored(the prompts) metadata from images: [[https://github.com/receyuki/stable-diffusion-prompt-reader][prompt-reader]]
- inpainter (watermark remover) https://github.com/advimman/lama [[https://huggingface.co/spaces/Sanster/Lama-Cleaner-lama][huggingface]]
- [[https://github.com/RupertAvery/DiffusionToolkit][DiffusionToolkit]]: Metadata-indexer and Viewer for generated images
- [[https://github.com/sanmilano/RatioScope][RatioScope]]: bucketing images for dataset
*** FACE GENERATION - SWAP
- loraless
  - [[https://github.com/modelscope/facechain][FaceChain]]: now two images = any angle, sd-webui support
- [[https://glucauze.github.io/sd-webui-faceswaplab/][FaceSwapLab]]: stable diffusion, webui
- an [[https://huggingface.co/h94/IP-Adapter/blob/main/models/ip-adapter-full-face_sd15.safetensors][IP Adapter]] face model
- [[https://github.com/facefusion/facefusion][FaceFusion]]: [[https://twitter.com/marcelpociot/status/1694483184992297017][face swapper]] and enhancer for video ==best==
- [[https://github.com/Hillobar/Rope][Roop]]: [[https://github.com/Hillobar/Rope][High-performance]] face swapper (Opal)
- [[COMFYUI INSTANTID]]
- InsightFace opensource alternative: [[https://github.com/KwaiVGI/LivePortrait][LivePortrait]] (Bring portraits to life)
** HELPERS
- [[https://github.com/HugoTini/DeepBump][DeepBump]]: generate normal & height maps from single pictures
- [[https://twitter.com/fffiloni/status/1720179934134481398][ProPainter]]: [[https://twitter.com/skalskip92/status/1720631569193246767][removal]] for videos using metaclip and sam
- [[id:89b433ed-e943-4fe6-8c18-bdaa834298fa][VCHITECT]]
- [[https://github.com/roboflow/supervision][supervision]]: people-object trakers ready to use (reusable computer vision tools) employees at the zone
*** 2SKETCH, TO SKETCH
- [[https://github.com/Mukosame/Anime2Sketch][Anime2Sketch]]: generate sketch from anime image
- [[https://github.com/kwanyun/StyleSketch/][Stylized Face]] Sketch Extraction via Generative Prior with Limited Data
  - generate sketch from image, using sketch-image-example to set style
*** STORYTELLING CAPTIONING
:PROPERTIES:
:ID:       bdd9160a-2438-4af0-a6f9-618b87096727
:END:
- [[https://github.com/ragavsachdeva/magi][The Manga]] Whisperer: Automatically Generating Transcriptions for Comics (magi) storytelling, storyboard
  - generate a transcript, ocr, order panels, cluster characters
*** OPENPOSE EDITOR
- [[https://github.com/ZhUyU1997/open-pose-editor][online]] 3d openpose editor: https://zhuyu1997.github.io/open-pose-editor/
- PMX model - MMD(mikumikudance): https://civitai.com/models/28916/openpose-pmx-model-mmd
- blender https://toyxyz.gumroad.com/l/ciojz
- OpenPose Man lora: https://civitai.com/models/76718
- [[https://github.com/makeplayhappy/stable-segmap#stable-segmap][Stable SegMap]]: using unity on web
- [[https://github.com/huchenlei/ComfyUI-openpose-editor?tab=readme-ov-file][ComfyUI-openpose-editor]]
*** TAGGER
**** WD TAGGER
- https://github.com/picobyte/stable-diffusion-webui-wd14-tagger/tree/master
- model: https://huggingface.co/SmilingWolf/wd-v1-4-vit-tagger/tree/main
- model: https://github.com/KichangKim/DeepDanbooru/releases/tag/v3-20211112-sgd-e28
- cli: https://github.com/KichangKim/DeepDanbooru
**** CAPTIONING MODELS
- CogVLM and Moonshot2 both are insanely good at captioning
- Qwen-VL-Max #1, THUDM/cogagent-vqa-hf #2, liuhaotian/llava-v1.6-vicuna-13b #3.
- [[https://github.com/jhc13/taggui][taggui]] for cog - https://github.com/jhc13/taggui/releases
- For llava 1.6 - https://github.com/DEVAIEXP/image-interrogator
- Qwen-VL-Max - https://huggingface.co/spaces/Qwen/Qwen-VL-Max
- [[VISION MODEL, IMAGE TO PROMPT]]
- [[https://www.reddit.com/r/StableDiffusion/comments/1egwgfk/joycaption_free_open_uncensored_vlm_early/][JoyCaption]]: Free, Open, Uncensored VLM (Early pre-alpha release) (nsfw captioning)
***** UNLABELED
- https://github.com/autodistill/autodistill
*** DETECTORS (COMPUTER VISION)
https://github.com/hysts/anime-face-detector
*** ANTI GLOW
- [[https://github.com/RichardAragon/NightshadeAntidote][Nightshade Antidote]]: remove poison pill from image
** UIs
*** FRONT-ENDS
- [[https://github.com/comfyanonymous/ComfyUI][ComfyUI]]
  - comfyui but from python-emacs: https://codeberg.org/tekakutli/apapach
  - [[https://github.com/Stability-AI/StableSwarmUI][StableSwarmUI]]: making comfyui easily accessible
- [[https://github.com/AUTOMATIC1111/stable-diffusion-webui][Auto1111 Webui]]
- [[https://github.com/lllyasviel/Fooocus][Fooocus]]
  - Focus on prompting and generating, manual tweaking is not needed
  - [[https://github.com/diffus-me/Refocus][Refocus]]: Alternative ui for Fooocus
- [[https://github.com/painebenjamin/app.enfugue.ai][ENFUGUE]]: Stable Diffusion web app
- [[https://github.com/bedovyy/chibi-client][Chibi]]: comfyui gui in Vue
**** MAKE YOUR GUI
- https://github.com/space-nuko/ComfyBox
**** CODE
- Diffusers (python pipelines): https://huggingface.co/docs/diffusers/index
  - https://github.com/ddPn08/Radiata
    - Stable diffusion webui based on diffusers
- nodejs: https://github.com/dakenf/stable-diffusion-nodejs
- [[https://twitter.com/_akhaliq/status/1715004501155864831][fastembed]]: [[https://github.com/qdrant/fastembed][lightweight]] Python for embedding generation
**** CPU
- ggml: inference in pure c/c++ (interoperability, no python dependency hell)
  - https://github.com/leejet/stable-diffusion.cpp
  - [[https://github.com/vitoplantamura/OnnxStream/tree/846da873570a737b49154e8f835704264864b0fe][Running Stable]] Diffusion XL 1.0 in 298MB of RAM (Raspberry Pi Zero 2)
    - OnnxStream consumes 55x less memory than OnnxRuntime while being only 0.5-2x slower
- [[https://github.com/rupeshs/fastsdcpu#fastsd-cpu][FastSD CPU]]: Faster version of stable diffusion running on CPU
  - FastSD CPU beta 16 release with 2 steps fast inference
**** FASTER
- [[https://github.com/chengzeyi/stable-fast][Stable-Fast]]: on NVIDIA GPU
- [[https://github.com/FizzleDorf/ComfyUI-AIT][ComfyUI-AIT]]: faster inference using cpp/cuda.
** TRAINING
- [[https://lemmy.dbzer0.com/post/12711492][LoRA]] [[https://github.com/hako-mikan/sd-webui-traintrain][training extention]] for Web-UI
- kohya training scripts: https://github.com/kohya-ss/sd-scripts
  - https://github.com/bmaltais/kohya_ss
- [[https://github.com/modelscope/scepter][SCEPTER]]: training, fine-tuning, and inference with generative models
- [[https://github.com/Nerogar/OneTrainer][OneTrainer]]: [[https://github.com/Nerogar/OneTrainer][one-stop solution]] for all your stable diffusion training needs
*** FINETUNING
- [[https://github.com/bghira/SimpleTuner][SimpleTuner]]: fine-tuning kit geared toward Stable Diffusion 2.1 and SDXL
  - [[https://github.com/devilismyfriend/StableTuner][StableTuner]]: 1.5
- [[https://github.com/victorchall/EveryDream2trainer][EveryDream2trainer]]
- [[https://github.com/sensorial-systems/stable-diffusion][Sensorial]] System's Stable Diffusion
  - automate all the steps of finetuning Stable Diffusion models.
** MODELS
- Taiyi-[[https://huggingface.co/IDEA-CCNL/Taiyi-Stable-Diffusion-XL-3.5B][Stable-Diffusion]]: finetuned in chinese
*** CONTROLNET
- **ALL SDXL CONTROLNET MODELS** : https://civitai.com/models/136070?modelVersionId=154923
  - controlllite: https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite
- [[PREPROCESSORS]]
- [[https://huggingface.co/georgefen/Face-Landmark-ControlNet][face landmarks]]: get landmarks from face
- [[https://github.com/TencentARC/MasaCtrl/][MasaCtrl]]: change pose by changing prompt of input image, optionally with controlnet
- Würstchen: [[https://twitter.com/dome_271/status/1749494484897796355][more controlnets]]
- [[https://github.com/kijai/ComfyUI-Diffusers-freecontrol?tab=readme-ov-file][Freecontrol]]: wireframe, rag doll, lidar, face mesh
**** SDXL-1.0
- openpose t2i-adapter: https://huggingface.co/TencentARC/T2I-Adapter/tree/main/models_XL
- list of them all: https://six-loganberry-ba7.notion.site/23-08-23-SDXL-ControlNet-619fdd7fff954df2ae918c69e2814fe1
  - all on: https://huggingface.co/lllyasviel/sd_control_collection/tree/main
  - https://huggingface.co/collections/TencentARC/t2i-adapter-sdxl-64fac9cbf393f30370eeb02f
- [[https://civitai.com/models/330313/tplanetsdxlcontrolnettilerealisticv1][TTPLanet_SDXL_Controlnet_Tile_Realistic_V1]]
  - adds feature details
***** SMOL
- controlnet-loras instead: https://huggingface.co/stabilityai/control-lora
  - seems to extract the difference between the model and ControlNet with svd
- controlnet-lltite (for now only sdxl) by [[https://github.com/kohya-ss/sd-scripts/blob/sdxl/docs/train_lllite_README.md][kohya]]
  - controlnet as a hypernetwork.
  - comfyui node: https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI
  - ==what's the difference between them?==
    - by furusu: https://twitter.com/gcem156/status/1693597263912899046/photo/1
***** CONTROLNET CANNY
- https://huggingface.co/diffusers/controlnet-canny-sdxl-1.0/tree/main
** COMFY
:PROPERTIES:
:ID:       0b531c2f-a0c0-418e-97ce-d2afcc626834
:END:
- [[https://github.com/Acly/krita-ai-diffusion/releases/tag/v1.14.0][krita plugin]]
- [[https://perilli.com/ai/comfyui/#whatsnew][AP Workflow]]: [[https://perilli.com/ai/comfyui/?8][complex]] workflow with everything and organized, including interoperability with oobabooga
*** INSTALLATION SNIPPET
#+transclude: [[id:51b11067-a042-46d0-b233-faef36a633df][orgbook://commandomoire/stable_diffusion::COMFY INSTALL]] :level 4
*** NEGATIVE LORAS
- feeding Stable Diffusion XL examples of bad images that it itself generated as a lora makes SDXL behave much better to the spirit of the prompt
  - https://twitter.com/minimaxir/status/1693657050990227640
    - https://minimaxir.com/2023/08/stable-diffusion-xl-wrong/
*** PROGRAMMATIC
**** PYTHON
- https://github.com/pydn/ComfyUI-to-Python-Extension
- [[https://github.com/seanlynch/srl-nodes/][python code]] remotely
- [[https://github.com/Chaoses-Ib/ComfyScript][ComfyScript]]: workflows, A Python front end for ComfyUI
- [[https://github.com/piyushK52/comfy-runner][Comfy Runner]]: Automatically install ComfyUI nodes and models and use it as a backend (like diffusers)
**** CUSHY
- programmatic pipelines using typescript
- https://github.com/rvion/CushyStudio
*** COMFY NODES
- https://registry.comfy.org/ << best tracker
- [[https://github.com/evanspearman/ComfyMath][Math]] nodes
- diffdiff: Differential Diffusion
- [[https://github.com/aszc-dev/ComfyUI-CoreMLSuite][Core ML]] models: leverage Apple Silicon
- [[https://github.com/LucianoCirino/efficiency-nodes-comfyui/tree/main][bundled]] nodes, lower node count (like highresfix)
- [[https://github.com/laksjdjf/LCMSampler-ComfyUI#lcmsampler-comfyui][LCMSampler-ComfyUI]]: In order to take advantage of the high-speed generation by LCM lora, [[https://huggingface.co/latent-consistency/lcm-lora-sdxl][loras]]
  - [[https://github.com/jojkaart/ComfyUI-sampler-lcm-alternative/tree/main][Alternative]]
  - [[https://github.com/dfl/comfyui-tcd-scheduler][comfyui-tcd-scheduler]], default value 0.3, use higher eta when more inference steps
- [[https://github.com/ZHO-ZHO-ZHO/ComfyUI-AnyText][AnyText]]: text generation on the image
- [[https://github.com/zhongpei/ComfyUI-InstructIR?tab=readme-ov-file][InstructIR]]: [[https://github.com/zhongpei/ComfyUI-InstructIR][image restoration]], watermark removal, fuzziness removal
- [[https://github.com/gameltb/Comfyui-StableSR][StableSR]]: superresolution
- [[https://github.com/gokayfem/ComfyUI_VLM_nodes][ComfyUI_VLM_nodes]]: querying(llava, kosmos), captioning(joytag)
- [[https://github.com/cubiq/ComfyUI_FaceAnalysis][ComfyUI_FaceAnalysis]]: evaluate the similarity between two faces
- [[https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes][3D Text]], Comfyroll Studio
- [[https://github.com/adieyal/comfyui-dynamicprompts][dynamicprompts]]: combinatorial prompts, prompt enhancement
- [[https://github.com/chaojie/ComfyUI-DragAnything][ComfyUI-DragAnything]]
- [[https://github.com/chrisgoringe/cg-controller][Controller]]: floating panel which reproduces widget inputs from all the red nodes in your workflow.
**** AUDIO
- [[https://github.com/kijai/ComfyUI-VoiceCraft][ComfyUI-VoiceCraft]]: Zero-Shot Speech Editing and Text-to-Speech in the Wild
- [[https://github.com/shadowcz007/comfyui-sound-lab][comfyui-sound-lab]]: A node collection for sound design, supporting MusicGen and Stable Audio.
- [[https://www.youtube.com/watch?v=BYFkEdgG4ks][Audio]] [[https://github.com/ryanontheinside/ComfyUI_RyanOnTheInside][Reactive]]: create animations that follow sounds-bands
  - Particle systems! Optical flow! Temporal masks!
**** REGIONAL EDITING
- [[https://github.com/ZHO-ZHO-ZHO/ComfyUI-BRIA_AI-RMBG][RMBG]]: background removal
- [[https://github.com/Acly/comfyui-inpaint-nodes][Inpaint Nodes]]: better inpaint
- [[https://github.com/SadaleNet/ComfyUI-Prompt-To-Prompt][Prompt-To-Prompt]]: change words
- [[https://github.com/AuroBit/ComfyUI-OOTDiffusion][OOTDiffusion]]: integrates the OOTDiffusion
- [[https://github.com/nullquant/ComfyUI-BrushNet][BrushNet]]: better inpainting
**** NATIVE OFFSET NOISE
- [[https://github.com/Haoming02/sd-webui-vectorscope-cc][Vectorscope]] CC: Offset Noise* natively (control over light, contrast, shadows)
**** UI MANAGER
- [[https://github.com/talesofai/comfyui-browser][comfy-browser]]: An image/video/workflow browser and manager for ComfyUI
- [[https://github.com/AIGODLIKE/AIGODLIKE-ComfyUI-Studio][AIGODLIKE-ComfyUI-Studio]]: loading models more intuitive, create model thumbnails
- [[https://github.com/Nuked88/ComfyUI-N-Sidebar][ComfyUI-N-Sidebar]]: for fav nodes
**** UPSCALE
- [[https://github.com/ssitu/ComfyUI_UltimateSDUpscale][upscale]]
  - [[https://github.com/WSJUSA/Comfyui-StableSR][SuperResolution]]
  - [[https://github.com/kijai/ComfyUI-CCSR][ComfyUI-CCSR]]
  - [[https://github.com/kijai/ComfyUI-SUPIR][SUPIR]]
  - [[https://github.com/philz1337x/clarity-upscaler][clarity-upscaler]]
  - [[https://github.com/kijai/ComfyUI-APISR][ComfyUI-APISR]]: anime upscaler
**** OPTIMIZATION
- [[https://github.com/ethansmith2000/comfy-todo][comfy-todo]]: Token Downsampling for Efficient Generation of High-Resolution Images
**** 3D
- [[https://github.com/MrForExample/ComfyUI-3D-Pack][ComfyUI-3D-Pack]]: process 3D inputs (Mesh & UV Texture, etc) using cutting edge algorithms (3DGS, NeRF, etc)
- [[https://github.com/AlexanderDzhoganov/ComfyTextures][ComfyTextures]]: Unreal Engine ⚔️ ComfyUI - Automatic texturing using generative diffusion models
  - [[https://github.com/gokayfem/ComfyUI-Texture-Simple/][ComfyUI-Texture-Simple]]
- [[https://github.com/flowtyone/ComfyUI-Flowty-CRM][ComfyUI-Flowty-CRM]]: generate meshes one image to 3d
**** STYLE
- [[https://github.com/ExponentialML/ComfyUI_VisualStylePrompting][VisualStylePrompting]]: style from example image ==best==
- [[https://github.com/dimtoneff/ComfyUI-PixelArt-Detector][ComfyUI-PixelArt-Detector]]: Generate, downscale, change palletes and restore pixel art images
***** TARGET STYLE-SUBJECT
- [[https://github.com/brianfitzgerald/style_aligned_comfy][StyleAligned]]: consistent style to all images in a batch
- [[https://github.com/Gourieff/comfyui-reactor-node#troubleshooting][face]] swap
- [[https://github.com/florestefano1975/comfyui-portrait-master][ComfyUI Portrait]] Master: generates prompts for skin color, expresion, shape, light direction
**** PLUGIN LISTS
- https://github.com/WASasquatch/comfyui-plugins/
**** IMAGE PROCESSING
- [[https://github.com/kadirnar/ComfyUI-YOLO][ComfyUI-YOLO]]: Ultralytics-Powered Object Recognition for ComfyUI
***** MASKING
- [[https://github.com/viperyl/ComfyUI-BiRefNet][ComfyUI-BiRefNet]]: best
****** EDITING MASK
- [[https://github.com/kijai/ComfyUI-KJNodes][ComfyUI-KJNodes]]: RBG to mask, grow mask with blur
****** IMPACT PACK
- inpainting, masking, sam, automasking face
  - select face minus hair or the inverse
- https://github.com/ltdrdata/ComfyUI-Impact-Pack
- hair restyling
  - tutorial: https://www.youtube.com/watch?v=5SCP9ijDuuA
  - workflow: https://raw.githubusercontent.com/ltdrdata/ComfyUI-extension-tutorials/Main/ComfyUI-Impact-Pack/workflow/hair-restyle.json
****** PER-INSTANCE MASK
  - [[https://github.com/ZHO-ZHO-ZHO/ComfyUI-YoloWorld-EfficientSAM][YOLO-World]] + EfficientSAM for ComfyUI
****** GENERATE WITH TRANSPARENCY
 - [[https://github.com/huchenlei/ComfyUI-layerdiffusion][Layer Diffusion]] custom nodes
***** PREPROCESSORS
- [[CONTROLNET]]
- turn image into canny, openpose, etc
  - pose [[https://github.com/hinablue/ComfyUI_3dPoseEditor][editor]]
- it also has an inpaint node
- [[https://github.com/Fannovel16/comfy_controlnet_preprocessors][old preprocessors]] [[https://github.com/Fannovel16/comfyui_controlnet_aux][new preprocessors]]
  - HandRefiner Support
- [[https://github.com/kijai/ComfyUI-depth-fm][DepthFM]]: monocular depth estimation
****** LIGHTING
- [[https://github.com/TJ16th/comfyUI_TJ_NormalLighting][comfyUI_TJ_NormalLighting]]: Custom Node for comfyUI for virtual lighting based on normal map
  - relightening, based on animatediff
- [[https://github.com/tori29umai0123/Line2Normalmap/tree/main][Line2Normalmap]]
- [[https://github.com/kijai/ComfyUI-DiffusionLight][ComfyUI-DiffusionLight]]: method of creating light probes
**** TEXT
***** LLM
- [[https://github.com/Big-Idea-Technology/ComfyUI_LLM_Node][ComfyUI_LLM_Node]]: deployment of models like T5, GPT-2
- [[https://github.com/ronniebasak/ComfyUI-Tara-LLM-Integration/blob/main/README.md][Tara]] - ComfyUI Node for LLM Integration
- [[https://github.com/ZHO-ZHO-ZHO/ComfyUI-Gemini][ComfyUI-Gemini]]: Gemini in ComfyUI
***** TEXT ENCODERS
- [[https://github.com/asagi4/comfyui-prompt-control][prompt]] control: example: a [large::0.1] [cat|dog:0.05] [<lora:somelora:0.5:0.6>::0.5]
- [[https://github.com/Ludobico/ComfyUI-ScenarioPrompt][ComfyUI-ScenarioPrompt]]: prompt creation helper
- [[https://github.com/ExponentialML/ComfyUI_ELLA][ComfyUI_ELLA]]: llm instead of clip
****** PROMPT ENHANCE
- [[https://github.com/glibsonoran/Plush-for-ComfyUI][Plush-for-ComfyUI]]: prompt enhancing using llm
- [[https://github.com/NeuralSamurAI/Comfyui-Superprompt-Unofficial][Comfyui-Superprompt-Unofficial]]: make dull prompts detailed
****** AUTO1111 TOKENS ON COMFY
A1111's token normalization and weighting in ComfyU
This means you can reproduce the same images generated on stable-diffusion-webui on ComfyUI.
******* LINK
https://github.com/shiimizu/ComfyUI_smZNodes
****** ADVANCED TOKEN WEIGHTS
https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb
**** IMAGE ENCODING
:PROPERTIES:
:ID:       a6bdca7d-3455-4b9d-93f4-bab9de716dfb
:END:
- [[https://huggingface.co/openai/clip-vit-large-patch14/blob/main/pytorch_model.bin][clip-vision model]]
- [[id:1c014bca-d8db-4d28-9c49-5297626d4484][SEECODERS]] [[https://github.com/BlenderNeko/ComfyUI_SeeCoder][Comfy]]
- [[https://github.com/ZHO-ZHO-ZHO/ComfyUI-InstantID][ComfyUI-InstantID]]: no more lora per subject, just one picture is enough
  - [[https://github.com/shiimizu/ComfyUI-PhotoMaker-Plus][ComfyUI PhotoMaker]] Plus
- [[https://github.com/Extraltodeus/Vector_Sculptor_ComfyUI][Vector_Sculptor_ComfyUI]]
  - Gather similar vectors within the CLIP weights and use them to redirect the original weights
***** IP ADAPTER
- [[https://github.com/laksjdjf/IPAdapter-ComfyUI][IP-Adapter]]
  - you need clip-vision model
  - comfyui [[https://comfyanonymous.github.io/ComfyUI_examples/sdxl/][examples]]
  - [[https://huggingface.co/ostris/ip-composition-adapter][ip-composition-adapter]]: general composition of an image while ignoring the style and content
    - like controlnet but less acurate
- [[https://github.com/cubiq/ComfyUI_IPAdapter_plus][ComfyUI_IPAdapter_plus]]
***** COMFYUI INSTANTID
- old: [[https://github.com/nosiu/comfyui-instantId-faceswap][ComfyUI InstantID]] Faceswapper
- native: [[https://github.com/cubiq/ComfyUI_InstantID][ComfyUI_InstantID]]
***** VISION MODEL, IMAGE TO PROMPT
- [[https://github.com/gokayfem/ComfyUI_VLM_nodes][VLM nodes]] [[https://github.com/gokayfem/ComfyUI_VLM_nodes/tree/main/examples][examples]]
- [[https://github.com/shadowcz007/comfyui-moondream][comfyui-moondream]]: tiny vision language model; image to prompt
- [[https://github.com/zhongpei/Comfyui_image2prompt][Comfyui_image2prompt]]: image to prompt by vikhyatk/moondream1
- [[https://github.com/huchenlei/ComfyUI_DanTagGen][ComfyUI_DanTagGen]]: LLM model designed for generating Danboou Tags with provided informations,  trained on Danbooru datasets
**** VIDEO
- [[https://github.com/brianfitzgerald/Comfy-SVDTools][Comfy-SVDTools]]
- [[https://github.com/Amorano/Jovimetrix][Jovimetrix]]: Nodes for procedural masking, live composition and video manipulation
- [[https://github.com/chaojie/ComfyUI-DynamiCrafter][DynamiCrafter]]: [[https://github.com/kijai/ComfyUI-DynamiCrafterWrapper][diffusion]] priors
  - [[https://github.com/ExponentialML/ComfyUI_Native_DynamiCrafter][native DynamiCrafter]]
- [[https://github.com/ForeignGods/ComfyUI-Mana-Nodes][Font to]] Animation
***** DANCING
- [[https://github.com/MrForExample/ComfyUI-AnimateAnyone-Evolved][AnimateAnyone]]: dancing
  - [[https://github.com/AuroBit/ComfyUI-AnimateAnyone-reproduction][ComfyUI-AnimateAnyone]]
- [[https://github.com/kijai/ComfyUI-champWrapper][Champ]]: [[https://twitter.com/Gradio/status/1777352076106056002][Controllable]] and Consistent Human Image Animation with 3D Parametric Guidance
  - incorporates depth, normal maps, semantic maps from SMPL sequences, skeleton-based motion guidance
***** MOTION
- [[https://github.com/chaojie/ComfyUI-MotionCtrl][MotionCtrl]]: Flexible Motion Controller for Video Generation
- [[https://github.com/chaojie/ComfyUI-DragNUWA][DragNUWA]]: manipulate backgrounds or objects motions
  - [[https://github.com/chaojie/ComfyUI-LightGlue][LightGlue]](required)
- [[https://github.com/ExponentialML/ComfyUI_LiveDirector][LiveDirector]]: use reference video to drive motion
****** LIP SYNC
- [[https://github.com/chaojie/ComfyUI-MuseTalk][ComfyUI-MuseTalk]]
***** NOT JUST IMAGES
- [[https://github.com/Kosinkadink/ComfyUI-AnimateDiff][AnimateDiff]]
  - [[https://github.com/banodoco/steerable-motion][Steerable]] Motion: for steering videos with batches of images
- [[https://comfyanonymous.github.io/ComfyUI_examples/3d/][Stable Zero123]] for ComfyUI
- [[https://github.com/AInseven/ComfyUI-fastblend][fastblend]]: smooth out video frames
- [[https://github.com/cerspense/ComfyUI_cspnodes][ComfyUI_cspnodes]]: ZeroScope nodes
** ONLINE SERVICES
- main registry of loras: https://www.civitai.com/
  - chinese alternative: https://www.liblib.art/
  - vietnam hosted: https://tensor.art/
- comfy pipelines https://comfyworkflows.com/
*** THE HORDE
- https://www.stablehorde.net
  - distributed cluster, built on top of ComfyUI, you can use any lora on CivitAI
  - https://dbzer0.com/blog/state-of-the-ai-horde-july-2023/
  - clients: Lucid Creations, ArtBot
    - https://github.com/daveschumaker/artbot-for-stable-diffusion/
    - krita plugin: https://github.com/dunkeroni/krita-stable-horde
* OTHERS VISUAL
- [[https://github.com/tin2tin/Pallaidium][PALLAIDIUM]]: Generative AI for the Blender VSE(Blender Video Sequence Editor)
  - Text, video or image to video, image and audio
- [[https://github.com/neph1/blender-stable-diffusion-render][blender-stable-diffusion-render]]: addon for using Stable Diffusion to render texture bakes for objects
** SEGMENTATION
- https://github.com/ltdrdata/ComfyUI-Impact-Pack
  - facedetailer
    - uses
    - https://github.com/hysts/anime-face-detector
    - https://github.com/open-mmlab/mmdetection
    - https://github.com/open-mmlab/mmpose
- https://github.com/facebookresearch/segment-anything/blob/main/notebooks/predictor_example.ipynb
  - hd finetune https://github.com/SysCV/sam-hq
  - with text prompt https://github.com/IDEA-Research/Grounded-Segment-Anything
    - merged segment-anything and grounding-dino
      - grounding = get bounding box(or mask) from text prompt
- https://github.com/biegert/ComfyUI-CLIPSeg
** 3D
*** MESH GENERATION
- [[https://www.reddit.com/r/StableDiffusion/comments/1635cb0/threestudio_a_unified_framework_for_3d_content/][threestudio]]: [[https://github.com/threestudio-project/threestudio#magic123-][A unified]] framework for 3D content generation
  - ProlificDreamer, DreamFusion, Magic3D, SJC, Latent-NeRF, Fantasia3D, TextMesh, Zero-1-to-3, [[https://github.com/threestudio-project/threestudio#magic123-][Magic123]], InstructNeRF2NeRF, and Control4D are all implemented in this framework.
- GSGEN: [[https://github.com/gsgen3d/gsgen][Text-to-3D]] using Gaussian Splatting
- [[https://github.com/3DTopia/3DTopia][3DTopia]]: Two-stage text-to-3D generation model (5 minutes)
*** NERF
- https://github.com/nerfstudio-project/nerfstudio
*** GAUSSIAN
- JavaScript Gaussian Splatting library
  - https://github.com/dylanebert/gsplat.js
** ANIMATEDIFF
- https://github.com/JaredTherriault/sd-webui-animatediff
* TEXT
- [[https://github.com/PygmalionAI/aphrodite-engine][aphrodite]]: chat bots, roleplay (by the horde)
- [[https://twitter.com/_akhaliq/status/1724456693378040195][LLaMA-Factory]]: [[https://github.com/hiyouga/LLaMA-Factory][Easy-to-use]] LLM fine-tuning framework (LLaMA, BLOOM, Mistral, Baichuan, Qwen, ChatGLM)
** INFERENCE
- https://github.com/simonw/llm (cli)
  - online and local
- [[https://github.com/jmorganca/ollama][ollama]]: golang, uses llama.cpp, local models
  - [[https://github.com/ggerganov/llama.cpp][llama.cpp]]
    - [[https://twitter.com/HaihaoShen/status/1715569980568449348][intel-enhanced]] llama.cpp
    - [[https://github.com/li-plus/chatglm.cpp][ChatGLM]].cpp (chinese, llama.cpp derived)
  - https://github.com/s-kostyaev/ellama (emacs)
    - [[https://github.com/wearedevx/llm-bash][LLM-Bash]]: Wrapper for llm & Ollama to be used by your code editor
- [[https://twitter.com/reach_vb/status/1757519118306037995][fast-llm]] [[https://github. com/Vaibhavs10/fast-llm.rs/][git]]
*** UI
- [[https://github.com/oobabooga/text-generation-webui][oobabooga]]: main, supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models
- [[https://github.com/YellowRoseCx/koboldcpp-rocm][koboldcpp-rocm]]: various GGML models with KoboldAI's UI with AMD ROCm offloading
- [[https://twitter.com/_akhaliq/status/1723144861182808312][ChuanhuChatGPT]]: [[https://github.com/GaiZhenbiao/ChuanhuChatGPT][webui]], gradio
** CODE
- [[https://github.com/KillianLucas/open-interpreter][Open Interpreter]], an open-source Code Interpreter
  - create and edit photos, summarize pdfs, control your browser, plot and analyze large datasets
- [[https://twitter.com/_akhaliq/status/1721552195957444765][DeepSeek Coder]]: several models
** DATASET
- lewd roleplay dataset: https://huggingface.co/datasets/lemonilia/LimaRP
* VOICE GENERATION
- parent: [[id:73ac7415-61d5-4266-964a-647a4243ac6c][voice]]
** REALTIME
- Realtime Voice Changer
  - https://github.com/w-okada/voice-changer/tree/master
  - AI covers(music): INSTANTLY make AI covers with ANY voice https://www.youtube.com/watch?v=pdlhk4vVHQk
    - https://github.com/SociallyIneptWeeb/AICoverGen