:PROPERTIES: :ID: f5d2ef09-1412-4955-a3c5-c22f6fff8d11 :END: #+title: nerf #+filetags: :neuralnomicon: #+SETUPFILE: https://fniessen.github.io/org-html-themes/org/theme-readtheorg.setup - https://jonbarron.info/ - [[id:d66a336f-083e-4515-b68e-67141ae4776c][NERF SEGMENTATION]] * LIGHT - [[Relightable Gaussian Codec]] - [[https://twitter.com/_akhaliq/status/1696109616105771453][Relighting]] Neural Radiance Fields with Shadow and Highlight Hints - moving point light source - second multi layer perceptron which takes shadow and highlight hints ** REFLECTIONS - nerf reflections: https://youtu.be/qrdRH9irAlk - https://arxiv.org/pdf/2112.03907.pdf - [[https://twitter.com/_akhaliq/status/1688793965011636224][Mirror-NeRF]]: Learning Neural Radiance Fields for Mirrors with Whitted-Style Ray Tracing - introducing the reflection probability - [[https://twitter.com/_akhaliq/status/1737688417633181844][SpecNeRF]]: Gaussian Directional Encoding for Specular Reflections - learnable Gaussian directional encoding to model view-dependent effects under near-field light - [[https://twitter.com/_akhaliq/status/1737689964085723497][UniSDF]]: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections - faster reconstructions; explicitly blending these representations in 3D space *** SNAP-IT :PROPERTIES: :ID: cf480119-9ee7-41c5-8b55-4f51303baaad :END: - [[https://twitter.com/_akhaliq/status/1774640724090183741][Snap-it]], Tap-it, Splat-it: Tactile-Informed 3D Gaussian Splatting for Reconstructing Challenging Surfaces - incorporates touch data (local depth maps) with multi-view vision data to achieve surface reconstruction and novel view synthesis - fewer images required ** MESH INTEGRATION - [[https://twitter.com/_akhaliq/status/1701468962108604771][Dynamic Mesh-Aware]] [[https://mesh-aware-rf.github.io/][Radiance Fields]] - two-way rendering-simulation coupling between mesh and NeRF - realistic light from NeRF media onto surfaces, cast shadows on the NeRF = enhanced realism * QUALITY - [[https://twitter.com/_akhaliq/status/1665951945012412417][Neuralangelo]]: High-Fidelity Neural Surface Reconstruction (3d mesh augmented) - [[https://twitter.com/_akhaliq/status/1731508015419814271][PyNeRF]]: Pyramidal Neural Radiance Fields - modification to grid-based models by training model heads at different spatial grid resolutions - reduce error rates by 20% while training over 60x faster against Mip-NeRF - [[https://twitter.com/_akhaliq/status/1732232045969965433][ReconFusion]]: 3D Reconstruction with Diffusion Priors - leverages diffusion for novel view synthesis, trained on multiview; few photos - [[https://twitter.com/_akhaliq/status/1738061362452975625][DyBluRF]]: Dynamic Deblurring Neural Radiance Fields for Blurry Monocular Video - deals with challenge of view synthesis from motion blur ** CLEAN-UP - [[https://twitter.com/_akhaliq/status/1668088138785341441][GANeRF]]: Leveraging Discriminators to Optimize Neural Radiance Fields - using gan, no floating ghost artifacts - [[https://twitter.com/_akhaliq/status/1699639628754518329][Bayes']] Rays: Uncertainty Quantification for Neural Radiance Fields - evaluate uncertainty in any pre-trained NeRF, then clean - [[IMPROVED GAUSSIAN]] * CREATING NERFS - [[https://arxiv.org/abs/2402.14586][FrameNeRF]]: A Simple and Efficient Framework for Few-shot Novel View Synthesis - regularization model (sd) as a data generator to produce dense views from sparse inputs ** POSE PREDICTION - [[https://melon-nerf.github.io/][MELON]]: NeRF with Unposed Images Using Equivalence Class Estimation (no poses) - [[https://arxiv.org/abs/2211.16630][DINER]]: Depth-aware Image-based NEural Radiance fields - [[https://twitter.com/_akhaliq/status/1734803566802407901][COLMAP-Free]] 3D Gaussian Splatting - continuity of the input video = no need for camera poses ** FROM VIDEO - [[https://arxiv.org/pdf/2112.01517.pdf][Efficient Neural]] Radiance Fields for Interactive Free-viewpoint Video (people) - [[https://localrf.github.io/][Progressively Optimized]] Local Radiance Fields for Robust View Synthesis - turn video into nerf - [[https://twitter.com/liuziwei7/status/1640968549748584453][F2-NeRF]]: [[https://twitter.com/_akhaliq/status/1640891354149531648][Fast Neural]] Radiance Field Training with Free Camera Trajectories - [[https://showlab.github.io/HOSNeRF/][HOSNeRF]]: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Video - free 360% viewpoint - [[https://twitter.com/_akhaliq/status/1721789209202094374][Consistent4D]]: Consistent 360° Dynamic Object Generation from Monocular Video - no need for multi-view data collection and camera calibration - to train DyNeRF *** VIDEO NERF - [[https://twitter.com/_akhaliq/status/1692073793995600335][SceNeRFlow]]: Time-Consistent Reconstruction of General Dynamic Scenes (dynamic-NeRF method) - correspondences even for long-term, long-range motions - [[https://twitter.com/_akhaliq/status/1699632839178776975][ResFields]]: Residual Neural Fields for Spatiotemporal Signals - effectively represent complex temporal signals - matrix factorization technique to reduce the number of trainable parameters ** FROM TEXT :PROPERTIES: :ID: 2a24e0fc-aa45-416c-bcce-7ecf55409e88 :END: - [[https://twitter.com/_akhaliq/status/1717740144101380438][HyperFields]]: Towards Zero-Shot Generation of NeRFs from Text - distills scenes encoded in individual NeRFs into one dynamic hypernetwork * OPERATING UPON, EDITING - ARF: Artistic Radiance Fields https://www.cs.cornell.edu/projects/arf/ - nerf style transfer - [[https://twitter.com/bilawalsidhu/status/1638919452392583169][Instruct-NeRF2NeRF]]: [[https://arxiv.org/pdf/2303.12789.pdf][Editing]] [[https://instruct-nerf2nerf.github.io/][3D]] Scenes with Instructions <> - training to NeRF in an iterative fashion - integrated to [[https://github.com/nerfstudio-project/nerfstudio][nerfstudio]] - [[https://twitter.com/_akhaliq/status/1683344812446220288][FaceCLIPNeRF]]: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fields - train a latent code-conditional deformable NeRF, over a dynamic scene - learns to represent a manipulated scene with spatially varying latent codes using clip - [[https://twitter.com/_akhaliq/status/1685895591971405824][Seal-3D]]: Interactive Pixel-Level Editing for Neural Radiance Fields - preview, instantly; local pretraining and global finetuning - proxy function mapping the editing instructions to the original space - [[https://twitter.com/_akhaliq/status/1742752753003143262][SIGNeRF]]: Scene Integrated Generation for Neural Radiance Fields - Generatively edits NeRF scenes - [[https://arxiv.org/abs/2401.14828][TIP-Editor]]: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts - accepts text and image prompts and a 3D bounding box to specify the editing region - [[https://twitter.com/_akhaliq/status/1775769263669756076][Freditor]]: High-Fidelity and Transferable NeRF Editing by Frequency Decomposition - lift 2D stylization results to 3D scenes - enabling stable intensity control and novel scene transfer ** INPAINTING - [[https://twitter.com/ashmrz10/status/1693683425176322063][Reference-guided]] [[https://twitter.com/_akhaliq/status/1648984579708010500][Controllable]] Inpainting of Neural Radiance Fields - use a mask and a single view image to force it on - [[https://twitter.com/_akhaliq/status/1732979577062838429][NeRFiller]]: Completing Scenes via Generative 3D Inpainting - leveraging a 2D inpainting diffusion model - [[https://mohamad-shahbazi.github.io/inserf/][InseRF]]: Text-Driven Generative Object Insertion in Neural 3D Scenes - [[https://arxiv.org/abs/2401.05750][GO-NeRF]]: Generating Virtual Objects in Neural Radiance Fields - utilizing scene context for high-quality and harmonious 3D object generation within an existing NeRF - [[https://twitter.com/_akhaliq/status/1752911147173335246][ReplaceAnything3D]]: Text-Guided 3D Scene Editing with Compositional Neural Radiance Fields - replace while maintaining 3D consistency across multiple viewpoints * OUTPUT ** TEXTURE :PROPERTIES: :ID: 63fb85fb-85cf-469c-be65-726fc0f3ff5d :END: - [[https://www.anjiecheng.me/TUVF][TUVF]] : Learning Generalizable Texture UV Radiance Fields - nerf baked to texture - [[https://twitter.com/_akhaliq/status/1672061068276011008][Blended-NeRF]]: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields - change nerf objects, multiple nerf views - [[https://meshnca.github.io/][Mesh Neural]] Cellular Automata, instead of uv map, 3d texture feel ** NERF FACE :PROPERTIES: :ID: 5a81561e-9e1c-4fc8-bfba-de467b4de033 :END: - [[https://tobias-kirschstein.github.io/nersemble/][NeRSemble]]: Multi-view Radiance Field Reconstruction of Human Heads - nerf to texture to faces, very realistic (73 fps) - fix faces generated no texture-sticking issue https://www.youtube.com/watch?v=j1ZY7LInN9g&t=272s - https://nvlabs.github.io/stylegan3/ - https://github.com/rosinality/alias-free-gan-pytorch ** GRID - [[https://jonbarron.info/zipnerf/][Zip-NeRF]]: [[https://arxiv.org/abs/2304.06706][Anti-Aliased]] Grid-Based Neural Radiance Fields (house interiors) - [[https://twitter.com/_akhaliq/status/1669381548011814930][Progressively]] Optimized Local Radiance Fields for Robust View Synthesis - dynamically allocate new local radiance *** URBAN - CITY :PROPERTIES: :ID: cd1f222a-024a-43d4-83dd-6accc8913a1e :END: - city nerf: https://city-super.github.io/gridnerf/ - [[https://arxiv.org/abs/2304.03266][Neural Fields]] [[https://nv-tlabs.github.io/fegr/][meet]] Explicit Geometric Representation for Inverse Rendering of Urban Scenes - nerf inserting 3d things - (not nerf) [[https://twitter.com/_akhaliq/status/1669533542391357442][UrbanIR]]: Large-Scale Urban Scene Inverse Rendering from a Single Video (google maps) - [[https://twitter.com/_akhaliq/status/1740589735024975967][City-on-Web]]: Real-time Neural Rendering of Large-scale Scenes on the Web - real life stream - [[https://twitter.com/_akhaliq/status/1762697888369381456][VastGaussian]]: Vast 3D Gaussians for Large Scene Reconstruction - large scenes 3D Gaussian Splatting - [[https://github.com/kcheng1021/GaussianPro][GaussianPro]]: 3D Gaussian Splatting with Progressive Propagation - guide the densification of the 3D Gaussians across large scenes - [[https://twitter.com/_akhaliq/status/1772836157983797574][Octree-GS]]: Towards Consistent Real-time Rendering with LOD-Structured 3D Gaussians - LOD-structured 3D Gaussian approach * FASTNESS - parent: [[id:ce14c053-d6bd-467d-84b6-11172ad3a8bc][computer_illumination]] - [[GAUSSIAN]] - NERF for real-time view synthesis https://arxiv.org/abs/2103.14645 - AdaNeRF https://arxiv.org/pdf/2207.10312.pdf (40 ms, two nerfs) - [[https://twitter.com/_akhaliq/status/1721790661978894451][VR-NeRF]]: High-Fidelity Virtualized Walkable Spaces (36 Hz) ** GEOMETRY - [[https://arxiv.org/abs/2212.01959][INGeo]]: Accelerating Instant Neural Scene Reconstruction with Noisy Geometry Priors, faster than Instant-NGP - [[https://nvlabs.github.io/instant-ngp/][instant-ngp]]: fractions of screen-space repeatedly, all neural primitives in seconds - cuda-less: https://github.com/taichi-dev/taichi https://github.com/Linyou/taichi-ngp-renderer - [[https://github.com/sxyu/plenoctree][PlenOctree]]: https://github.com/sxyu/volrend 150 fps - [[https://jkulhanek.com/tetra-nerf/][Tetra-NeRF]]: [[https://arxiv.org/pdf/2304.09987.pdf][Representing]] Neural Radiance Fields Using Tetrahedra - [[https://twitter.com/_akhaliq/status/1718831873017643345][Reconstructive]] Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations - nerf with autoencoder latent field, 13 times faster rendering - [[https://twitter.com/_akhaliq/status/1731883546157965766][VideoRF]]: Rendering Dynamic Radiance Fields as 2D Feature Video Streams ==best== - streaming-rendering nerf online or mobile devices - feature image stream can be efficiently compressed by 2D video codecs - [[https://twitter.com/_akhaliq/status/1732599791031357476][HybridNeRF]]: Efficient Neural Rendering via Adaptive Volumetric Surfaces - ==surfaces instead of volumes==, real time better speed *** HARMONICS - Plenoxels (plenoptic voxels), 3D grid with spherical harmonics https://arxiv.org/pdf/2112.05131 - [[https://arxiv.org/abs/2304.12670][Patch-based]] [[https://github.com/wyysf-98/Sin3DGen][3D]] [[https://twitter.com/weiyuli99072112/status/1651069926634053633][Natural]] [[https://weiyuli.xyz/Sin3DGen/][Scene]] Generation from a Single Example (3d patches as codebook) - content + structure separation *** TRIANGLES - MobileNERF = small neural network (Small MLP) for view dependant per pixel, deferred neural shader - https://youtu.be/ofVgAEb1FiE - https://youtu.be/nIqmuylmpFY - 10 minutes - [[https://arxiv.org/pdf/2208.00277v2.pdf][mobileNERF]] [[https://github.com/google-research/jax3d/tree/main/jax3d/projects/mobilenerf][(polygons, triangles)]] 124.3 fps - [[https://arxiv.org/abs/2303.08717][Re-ReND]]: Real-time Rendering of NeRFs across Devices (facebook) 329.6 fps - using rendering pipeline gpu geometry (like that one which used triangles) * NOT NERF :PROPERTIES: :ID: 6b33609a-fcac-4712-bcd7-6bee873fbe81 :END: - [[id:88e29751-d7d6-41e4-8375-3c7ac24cb77b][OPTICAL FLOW]] [[id:7ab592e4-5e55-40eb-8c93-9d779ea2bcf7][INTERACTIVE]] ** 4D GENERATION - [[GAUSSIAN GENERATION]] - [[https://twitter.com/_akhaliq/status/1714486520759861296][4K4D]]: Real-Time 4D View Synthesis at 4K Resolution (30x faster than previous) - 4D feature grid with points naturally regularized and optimized - learn the proposed model from RGB videos - [[https://vveicao.github.io/projects/Motion2VecSets/][Motion2VecSets]]: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking - dynamic reconstruction from point cloud sequences ** NERF ALIKES :PROPERTIES: :ID: 8f2a9969-2221-4993-9113-6e5e3e5874f5 :END: *** VIDEO - [[https://arxiv.org/pdf/2110.13903.pdf][NeRV]]: Neural Representations for Videos (nerf video) - https://github.com/haochen-rye/NeRV - https://github.com/haochen-rye/HNeRV - [[https://arxiv.org/abs/2212.12294][FFNeRV]]: Flow-Guided Frame-Wise Neural Representations for Videos - incorporates flow information - [[https://arxiv.org/abs/2303.17228][Streaming]] [[https://github.com/yuzhms/Streaming-Video-Model][Video Model]] - [[https://twitter.com/_akhaliq/status/1731887180815941641][Fast]] View Synthesis of Casual Videos - synthesize high-quality novel views from a monocular video efficiently, real time *** IMAGES - [[https://twitter.com/_akhaliq/status/1707291551276187685][NeuRBF]]: A Neural Fields Representation with Adaptive Radial Basis Functions (hd images) - general radial bases with flexible kernel position and shape, to fit target signals - [[https://twitter.com/_akhaliq/status/1717383187205099905][LightSpeed]]: Light and Fast Neural Light Fields on Mobile Devices - a direct mapping from a ray representation to the pixel color, neural light field using a light slab representation ** NERF ALTERNATIVE - parent: [[id:f5d2ef09-1412-4955-a3c5-c22f6fff8d11][nerf]] - ==nerf alternative== [[id:3c4595ca-1c60-4c23-b305-9068e85dc22d][ROOMDREAMER]] [[id:6b33609a-fcac-4712-bcd7-6bee873fbe81][NOT NERF]] [[id:35add1fe-b835-49c7-99f4-8aa4321a3904][SUPERPRIMITIVE]] - [[https://arxiv.org/abs/2203.10157][ViewFormer]]: no NeRF, instead Transformers - Geometry-Free View Synthesis: Transformers and no 3D Priors; [[https://arxiv.org/pdf/2104.07652.pdf][no 3d prior]] - [[https://github.com/MetaSLAM/AutoMerge_Docker][AutoMerge]]: A Framework for Map Assembling and Smoothing in City-scale Environments (google maps) without gps - M-SDF: [[https://twitter.com/_akhaliq/status/1735519575083458925][Mosaic-SDF]] for 3D Generative Models ==best== - approximates the Signed Distance Function (SDF) of shape by using set of local grids spread near the boundary - [[id:b95dc13d-0509-4f97-9597-34393c075799][VOLUME DIFFUSION]] ==best== *** GIBR ==best== :PROPERTIES: :ID: fa7469f5-948a-42b2-8787-14109bc9ed5a :END: - GIBR: [[https://anciukevicius.github.io/generative-image-based-rendering/][Denoising]] Diffusion via Image-Based Rendering - IB-planes, ==new neural scene representation== accurately represent large 3D scenes dynamically allocating more capacity as needed for details - denoising-diffusion framework to learn prior over IB-planes - only 2D images no need for masks or depths - single image as input, synthesises plausible details in hidden regions *** GAUSSIAN :PROPERTIES: :ID: e05c8e78-1579-42a1-8202-30c58b96e504 :END: - [[GAUSSIAN GENERATION]] - a point cloud > gaussian cloud - ray tracing(nerf) > ray marching - vs mobilenerf? which seems faster with lower system requirements - [[https://twitter.com/_akhaliq/status/1689147758744104961][3D Gaussian]] [[https://twitter.com/dylan_ebert_/status/1690041338572550161][Splatting]] [[https://github.com/graphdeco-inria/gaussian-splatting][for Real-Time]] [[https://github.com/MrNeRF/gaussian-splatting-cuda][Radiance]] Field Rendering - represent the scene with 3D Gaussians - it has NO neural networks at all - ==best nerf== far better than instant-ngp - [[https://twitter.com/rsasaki0109/status/1729334980013408287][Mip-Splatting]]: Alias-free 3D Gaussian Splatting - smoothing filter eliminating multiple artifacts and achieving alias-free renderings - [[https://twitter.com/_akhaliq/status/1731508732368990591][Scaffold-GS]]: Structured 3D Gaussians for View-Adaptive Rendering ==best== - reduces redundant Gaussians while delivering higher-quality rendering - [[https://twitter.com/_akhaliq/status/1760884260817240267][GaussianPro]]: 3D Gaussian Splatting with Progressive Propagation (vs 3DGS) - guide the densification of the 3D Gaussians - [[https://twitter.com/_akhaliq/status/1770669543565734194][RadSplat]]: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS - [[https://twitter.com/Gradio/status/1771100962003804590][GRM]]: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation - recovering a 3D asset from sparse-view images in around 0.1s - reconstructs 3d gaussians-meshes from various sources: zero123++, instant3d, v3d, and sv3d **** IMPROVED GAUSSIAN - [[https://twitter.com/_akhaliq/status/1745647440592568753][TRIPS]]: Trilinear Point Splatting for Real-Time Radiance Field Rendering - approach that combines ideas from both Gaussian Splatting and ADOP(crisper images) - real-time frame rate of 60fps - [[https://twitter.com/_akhaliq/status/1777209144778346544][Robust Gaussian]] Splatting - fixing blur, imperfect camera poses, color inconsistencies(caused by ambient light, shadows) ***** GAUSSIAN QUANTIZATION - FisherRF: Active View Selection and Uncertainty Quantification for Radiance Fields using Fisher Informations - view selections at 70~fps, better quality - by leveraging fisher information, longer needing density distribution assumptions ***** MEMORY - [[https://abdullahamdi.com/ges/][GES]]: Generalized Exponential Splatting for Efficient Radiance Field Rendering - requiring far fewer particles to represent a scene, half the memory seed up to 40% ***** TRAIN - KEP-SVGP: [[https://browse.arxiv.org/abs/2402.01476][Self-Attention]] through Kernel-Eigen Pair Sparse Variational Gaussian Processes - attention kernels are in essence asymmetric, thus KEP-SVGP as attention kernel to fully characterizes the asymmetry **** 4D GAUSSIAN - [[https://twitter.com/Gradio/status/1713978753888698685][4D Gaussian]] Splatting for Real-Time Dynamic Scene Rendering - video and in real time, 20 min - [[https://twitter.com/_akhaliq/status/1727195991411925398][PhysGaussian]]: Physics-Integrated 3D Gaussians for Generative Dynamics - integrates Newtonian dynamics within 3D Gaussians for motion synthesis - negates the necessity for triangle/tetrahedron meshing - [[https://vita-group.github.io/4DGen/][4DGen]]: Grounded 4D Content Generation with Spatial-temporal Consistency - 4D representation using dynamic 3D Gaussians, generation from images or videos - specify geometry and motion offering superior control over content creation - [[https://arxiv.org/abs/2402.03307][4D Gaussian]] Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes - 4DGS, anisotropic 4D XYZT Gaussian - modeling complicated dynamics and fine details, especially for scenes with abrupt motions - [[https://github.com/oppo-us-research/SpacetimeGaussians#spacetime-gaussian-feature-splatting-for-real-time-dynamic-view-synthesis][Spacetime]] Gaussian Feature Splatting for Real-Time Dynamic View Synthesis - 3d gaussians enhanced with temporal opacity and parametric motion/rotation - replaces spherical harmonics with neural features, so small size and fast at 60 FPS - [[https://zerg-overmind.github.io/GaussianFlow.github.io/][GaussianFlow]]: Splatting Gaussian Dynamics for 4D Content Creation - smooth and natural, even in highly dynamic regions, no artifacts ***** MESH CONTROL - [[https://github.com/Anttwo/SuGaR][SuGaR]]: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering - hybrid gaussian-mesh for easy animation by manipulating the mesh - [[https://browse.arxiv.org/abs/2402.01459][GaMeS]]: Mesh-Based Adapting and Modification of Gaussian Splatting - hybrid of mesh and gaussian, that pin all gaussians splats on the object surface (mesh) - allowing for adjustments in position, scale, and rotation during animation - [[SPLATTING AVATAR]] - [[https://github.com/lizhe00/AnimatableGaussians][Animatable Gaussians]]: [[https://github.com/lizhe00/AnimatableGaussians][Learning]] Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling - 2D CNNs(StyleGAN-based) and 3D Gaussian splatting - [[https://twitter.com/_akhaliq/status/1771010842680520808][Gaussian Frosting]]: Editable Complex Radiance Fields with Real-Time Rendering - extracting a base mesh from gaussians; the fuzzier the material, the thicker the frosting - editing and animation by modifying the mesh **** HUMAN BODY - [[https://twitter.com/_akhaliq/status/1731881239928340868][GPS-Gaussian]]: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis - lift 2D parameter maps(depth estimation) to 3D space - [[https://twitter.com/_akhaliq/status/1732603200291659794][HiFi4G]]: High-Fidelity Human Performance Rendering via Compact Gaussian Splatting - gaussian from non-rigid tracking - compression rate of approximately 25 times, less than 2MB of storage per frame - [[https://github.com/ShunyuanZheng/GPS-Gaussian][GPS-Gaussian]]: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis - synthesizing novel views of any unseen characters instantly without any fine-tuning or optimization - [[https://twitter.com/_akhaliq/status/1739884783939748040][Human101]]: Training 100+FPS Human Gaussians in 100s from 1 View (Gaussian Animation method) ***** BODY AVATAR :PROPERTIES: :ID: a64f621e-a7e1-4a42-b05e-7c3757b8bce3 :END: - [[https://github.com/computational-imaging/GSM][GSM]]: Gaussian Shell Maps for Efficient 3D Human Generation - 3D Gaussian rendering primitives for controllable poses and diverse appearances - [[https://github.com/mikeqzy/3dgs-avatar-release][3DGS-Avatar]]: Animatable Avatars via Deformable 3D Gaussian Splatting - [[https://arxiv.org/abs/2404.01053][HAHA]]: [[https://david-svitov.github.io/HAHA_project_page/][Highly]] Articulated Gaussian Human Avatars with Textured Mesh Prior - model learns to apply Gaussian splatting only in areas of mesh where it is necessary - like hair and out-of-mesh clothing - so it can handle the animation of small body parts such as fingers ***** GAUSSIAN FACE :PROPERTIES: :ID: ea70f586-4414-4af8-8328-a3d7e891d7f2 :END: - [[https://twitter.com/_akhaliq/status/1732583744618328103][Relightable]] Gaussian Codec Avatars <> - high-fidelity relightable(real time) head avatars, eye reflections, animated to novel expressions - [[https://twitter.com/_akhaliq/status/1732604961081434261][Gaussian]] Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians - controllable 3D Gaussians, webcam controlled expressions - [[https://arxiv.org/abs/2401.12900][PSAvatar]]: A Point-based Morphable Shape Model for Real-Time Head Avatar Creation with 3D Gaussian Splatting - parametric morphable for poses and expressions ****** MAGICMIRROR :PROPERTIES: :ID: a8c7edef-f00a-42ad-9511-086c23948c86 :END: - [[https://arxiv.org/abs/2404.01296][MagicMirror]]: Fast and High-Quality Avatar Generation with a Constrained Search Space - conditional NeRF and stable diffusion geometric prior - creation of custom avatars with unparalleled quality and better adherence to input text prompts ****** SPLATTING AVATAR - [[https://initialneil.github.io/SplattingAvatar][SplattingAvatar]]: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting - disentangle the motion and appearance of a virtual human - control the rotation and translation of the Gaussians directly by mesh **** GAUSSIAN GENERATION :PROPERTIES: :ID: 1b2337de-9064-40e4-a82c-3e8e48084ad0 :END: - [[https://twitter.com/_akhaliq/status/1712840565920862550][GaussianDreamer]]: Fast Generation from Text to 3D Gaussian Splatting with Point Cloud Priors - 3D diffusion makes point cloud priors and then 2D model enriches the geometry and appearance, 25 min - D3GA: [[https://twitter.com/_akhaliq/status/1725032680683708862][Drivable]] 3D Gaussian Avatars ==best== - multi-view videos as input - [[https://me.kiui.moe/lgm/][LGM]]: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation ==best== - 3D models from text prompts or single-view images, 5 seconds - [[https://taoranyi.com/gaussiandreamer/][GaussianDreamer]]: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models - 3D diffusion model provides priors for initialization, 2D model enriches the geometry and appearance; 15 minutes - [[id:2c09738a-baef-4db0-a7d8-e9551c3dd2df][FDGAUSSIAN]] ***** FASTER - [[https://twitter.com/_akhaliq/status/1707605750070100304][DreamGaussian]]: [[https://twitter.com/liuziwei7/status/1707772233291342090][Generative]] [[https://twitter.com/dylan_ebert_/status/1729240128726438051][Gaussian]] [[https://twitter.com/liuziwei7/status/1754151216534487484][Splatting]] for Efficient 3D Content Creation (1 min) - gaussian splatting with mesh extraction and texture refinement in uv space - high-quality textured meshes, just 2 minutes from a single-view image - [[https://twitter.com/_akhaliq/status/1731637967762894946][FSGS]]: Real-Time Few-shot View Synthesis using Gaussian Splatting - real-time and photo-realistic synthesis with three training views ***** LUCIDDREAMER :PROPERTIES: :ID: d9dfa74e-0833-46af-8b79-1135cd38e21e :END: - [[https://twitter.com/_akhaliq/status/1727570137576833112][LucidDreamer]]: [[https://twitter.com/_ironjr_/status/1727677682463518890][Domain-free]] [[https://luciddreamer-cvlab.github.io/][Generation]] [[https://twitter.com/_akhaliq/status/1729455373617185101][of 3D]] [[https://twitter.com/camenduru/status/1732609892391813462][Gaussian]] [[https://twitter.com/_ironjr_/status/1732811804169150963][Splatting]] Scenes - [[https://twitter.com/_akhaliq/status/1726825565309608144][LucidDreamer]]: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching - interval-based score matching to counteract over-smoothing - generated from any text or image prompt (pseudo-depth alignment algorithm) - incorporated 3D Gaussian Splatting - project a portion of point cloud to the desired view and provide the projection - painted images are lifted to 3D space with estimated depth maps, composing a new points - [[https://twitter.com/_akhaliq/status/1772834816003580168#m][DreamPolisher]]: Towards High-Quality Text-to-3D Generation via Geometric Diffusion - coarse 3D generation refined via geometric optimization - then ControlNet driven refiner coupled with the geometric consistency to improve texture and consistency