layout: post title: Here come iTunes purchased video created: 1115708299 categories:

• Movie
• Mac

AppleInsider broke the news that iTunes 4.8 (not yet in Software Update, you can get it from Apple directly) supports Quicktime content. Specifically, you can play videos directly in iTunes, either in the main window, a separate window (Engadget as a good screenshot), or…full screen.

As you can see from the screenshot, movie content is stored directly within iTunes, much like a regular audio file. The fields don’t make as much sense, but that’s easy enough to fix in later versions.

The AppleInsider forum thread is filled with various bits of dis-information, but there were some good tidbits.

One of the lines of discussion was about the different apps and how they fit together. iPhoto 5 supports storing/organizing videos from your camera, and iMovie is for editing movies; why in iTunes as well? Well, one view is that iTunes is for paid content (movie/TV downloads!) and organizing (think video playlists/search/etc.) not importing/editing of your created content.

Oh right, and of course, iTunes is the only app that plays with Windows, so that’s where movie download/sales stuff has to live.

There might even be a smoking gun. The excellent ArsTechnica forum had this:

I just looked through the strings in the iTunes executable file.

The first one I found was “Has Video,” mixed in with attributes like “Has Variable Bit Rate” and “Genre.”

The second is a bit more interesting. After “Purchased Music”: “Purchased Videos”!

Update: Christopher Allen points to “AirVideo” — video delivered via an upgrade to the AirPort Express. Scoble isn’t sure it’s an Apple employee where the comments originated on Slashdot. I’ve always thought that there’s more than meets the eye in the AirPort Express (like VoIP).

And that should be enough for the rumour mill tonight — now lovingly collected under the iTunes Movies rumour section.

layout: post title: Video iPod or….Airport Express Video? created: 1128581421 categories:

• airport express
• video
• Apple
• speculation
• Mac
• Movie

Ever since the Airport Express came out, I’ve been speculating about what else could be done with it. Surely that USB port wasn’t just for something as mundance as printer sharing.

Well, there’s another Apple Special Event coming up. They might be announcing the bundling of the Google Toolbar with Safari….or it might be a video version of the Airport Express to coincide with the opening of the iTunes Movie Store.

Yep, put me down for not believing in the fabled video iPod. Well, actually, I do believe in it, but Travis figures the Airport Express Video edition is a safer bet. Travis said I *had* to blog about this, so I am.

So, the special event will announce the opening of for-pay video downloads via the iTunes store, at the same time as they announce how to connect it to your TV via Airport. Apparently, there has been no new music video content in iTunes for the past couple of weeks, perhaps indicating a build up of new content. With Google announcing streaming of UPN shows, Apple’s timing is probably about right.

On quality: well, this is meant to be the year of HD, but this probably just means a crisper, clearer version than regular cable TV. Could an Airport device have anything more than an SVideo interface? I can’t picture it, although I suppose a component output is possible. 

A dock that streams video content wirelessely directly from your iPod to this Airport device is a potential as well, but I can’t quite picture the price point on it. Bundling it with the Airport might be interesting, to catch all those people that only have iPods but no computer at home? Not sure…

According to Eris, the special event is probably the next version of Keynote :P 

layout: post title: Save us from Flash video created: 1160603507 categories:

• video
• Flash
• Quicktime
• MPEG4
• Google Video
• Democracy
• DivX
• Google
• YouTube
• Web Development

I've been spending a lot of time lately looking at video online. Part of it has been technical, and part of it has been thinking about what is needed and where it is headed. I was involved with IPTV / video over IP back at Nortel, and a lot of things actually haven't changed.

As if video on the Internet wasn't hot enough already, the whole Google buys YouTube has upped the fever pitch to…well, something hotter than fever :P

So, for starters, go and read (and link to!) Kevin Marks' five point plan to save use from Flash video. He included "crappy" in the title, and he's right, but…well, it's what works today.

And that's what will stop us from getting any further. Getting stuck in the "works today" mindset is a bad thing. When you sit down to build something new, you're really never building it for today…you're building it for today AND for the next little while. Somebody has to take a chance on figuring out the future. 

In any case, I very much agree with all of Kevin's points. The phrase "detection script" makes me shiver a bit, and Flash is still the only platform solution if you want one integrated application that has interactivity built into it as well, but for straight up video, we can do better, and should be trying to.

A closer alliance between Google and Apple wouldn't surprise me… 

layout: post title: "More Digital Ethnography: Information R/Evolution Video" created: 1192489746 categories:

• Digital Ethnography
• Michael Wesch
• Common Craft
• video
• Web 2.0

I don’t think I ever posted the “Web 2.0” video created by Michael Wesch, a professor at Kansas State University doing “Digital Ethnography”. I was really, well, I’d have to say *moved* by the original video. Much like Lee and Sachi’s Common Craft videos in plain English, Michael seems to have taken many things that I’ve long thought instinctively and explained them a little more fully. Still way more geeky than CC’s stuff, but I think understandable and interesting to watch.

In any case, here is the follow up video, Information R/Evolution. It walks us through how we are still exploring how to break free from the concepts of physical organization. The summary seems to be that we participants on the web are helping to build this R/Evolution, and it’s based around links and tags, and allow us to put “stuff” at multiple “locations”. Take the time to watch the video (and the previous one if you haven’t), and tell me what you think.

• tags :: zettelkasten

• source

• Literature notes should be brief, atomic, and not contain quotes

• The video also illustrates that literature notes should contain page numbers, like so:

  - p. 19
    - some note that occurs on page 19
  

  • This may be difficult if reading from an e-reader, where pages are relative!

• if taking notes on an e-reader, review your highlights or in-app notes and translate them into roam notes later

• permanent notes should come directly from literature notes

  • in the video he highlights "journaling is the easiest way of getting feedback" and turns it into a note unto itself

• tags and keywords should be framed as "how do I want to find / use this information?"

• permanent notes should be fully formed

• permanent notes should also contain keywords, references, etc.

• cogvideox1.5-5B Lora finetuning on 1.5-5B-I2V inference

  • https://github.com/a-r-r-o-w/cogvideox-factory

• parent: stable_diffusion

• SSM MeetsVideo DiffusionModels: Efficient Video Generation with Structured State Spaces

  • no longer exponential for more frames

TUNING

• Tuning-Free NoiseRectification for High Fidelity Image-to-Video Generation (dataset alleviation)

  • prevent loss of image details and the noise prediction biases during the denoising process

  • adds noise then denoises the noisy latent with proper rectification to alleviate the noise prediction biases

• Attention PromptTuning Parameter-Efficient Adaptation of Pre-Trained Models for Action Recognition

  • efficient prompt tuning for video applications such as action recognition

ANIMATION

• Keyframer Empowering Animation Design using Large Language Models

  • animating static images (SVGs) with natural language

• AnimatedStickers: Bringing Stickers to Life with Video Diffusion (animated emojis)

4D CONTROL

• BIGGER COHERENCEFACEARTIST EDITING

• DiffDreamer ConsistentSingle-view Perpetual View Generation with Conditional Diffusion Models

  • landscape(mountains) fly overs

• DisCo Disentangled Control for Referring Human Dance Generation in Real World

  • human dance(movement) images and videos (using skelleton rigs)

• paintsundo instead of just diffusing, ai paints like humans paint

  • make speedpaints and (extract) sketches, fake drawing process

GIF

• Hotshot-XL text-to-GIF model for Stable Diffusion XL

• GenerativeImage Dynamics, interactive gifs(looping dynamic videos)

  • frequency-coordinated diffusion sampling process

  • neural stochastic motion texture

• Pix2Gif Motion-Guided Diffusion for GIF Generation

  • transformed feature map (motion) remains within the same space as the target, thus consistency-coherence

• dynamicrafter generative frame interpolation and looping video generation (320x512)

• ExplorativeInbetweening of Time and Space

  • bounded generation of a pre-trained image-to-video model without any tuning and optimization

  • two images that capture a subject motion, translation between different viewpoints, or looping

INTERACTIVE

COLORIZATION

• LearningInclusion Matching for Animation Paint Bucket Colorization

  • for hand-drawn cel animation

  • comprehend the inclusion relationships between segments

  • paint based on previous frame

DRAG

• ==DragGAN==: Drag YourGAN Interactive Point-based Manipulation on the Generative Image Manifold

  • dragging as input primitive, using pairs of points, excellent results, stylegan derivative

• DragonDiffusion Enabling Drag-style Manipulation on Diffusion Models

  • moving, resizing, appearance replacement, dragging

• StableDrag Stable Dragging for Point-based Image Editing

  • models: StableDrag-GAN and StableDrag-Diff

  • confidence-based latent enhancement strategy for motion supervision

DRAG DIFFUSION

• DragDiffusion HarnessingDiffusion Models for Interactive Point-based Image Editing

  • RotationDrag Point-basedImage Editing with Rotated Diffusion Features

    • utilizing the feature map to rotate-move images

• DragonDiffusion Enabling Drag-style Manipulation on Diffusion Models

• DragNUWA Fine-grainedControl in Video Generation by Integrating Text, Image, and Trajectory

  • control trajectories in different granularities

• Drag YourNoise Interactive Point-based Editing via Diffusion Semantic Propagation

  • superior control and semantic retention, reducing the optimization time 50% compared to DragDiffusion

• Tora Trajectory-oriented Diffusion Transformer for Video Generation

HEAD POSE

• TALKING FACES

• Control4D DynamicPortrait Editing by Learning 4D GAN from 2D Diffusion-based Editor <<Control4D>>

  • 4d gan, 2D diffusion, consistent 4D, ==best one==

  • change face of video

• AniPortraitGAN Animatable 3D Portrait Generation from 2D Image Collections

  • facial expression, head pose, and shoulder movements

  • trained on unstructured 2D images

• MagiCapture High-Resolution Multi-Concept Portrait Customization

  • generate high-resolution portrait images given a handful of random selfies

• DiffPortrait3D: ControllableDiffusion for Zero-Shot Portrait View Synthesis

  • input: unposed portrait image, retains identity and facial expression

• MorphableDiffusion: 3D-Consistent Diffusion for Single-image Avatar Creation

  • novel view synthesis; input: single image and morphable mesh for desired facial expression (emotion)

SEMANTICALLY DEFORMED

• VideoLDM hd, but still semantically deformed (nvidia)

SEMANTICAL FIELD

• TokenFlow ConsistentDiffusion Features for Consistent Video Editing

  • consistency in edited video can be obtained by enforcing consistency in the diffusion feature space

• CoDeF Content Deformation Fields for Temporally Consistent Video Processing

  • video to video, frame consistency

  • aggregating the entire video and then using deformation field on one image ==best one==

• S2DM Sector-Shaped Diffusion Models for Video Generation ==best==

  • explore the use of optical flow as temporal conditions

  • prompt correctness while keeping semantical consistenc, can integrate with another temporal conditions

  • decouple the generation of temporal features from semantic-content features

SD BASED

• Latent-Shift LatentDiffusion with Temporal Shift for Efficient Text-to-Video Generation

  • temporal shift module that can leverage the spatial unet as is

• Rerender AVideo Zero-ShotText-Guided Video-to-Video Translation

  • compatible with existing diffusion ==best one==

  • hierarchical cross-frame constraints applied to enforce coherence

• Tune-A-Video One-ShotTuning of Image Diffusion Models for Text-to-Video Generation

  • inflated sd model into video

  • FROZEN SD

    • Fate/Zero Fusing AttentionsMIT) for Zero-shot Text-based Video Editing

      • most fluid one, without training

      • RAVE RandomizedNoise Shuffling for Fast and Consistent Video Editing with Diffusion Models ==best==

        • employs novel noise shuffling strategy to leverage temporal interactions (coherence)

        • guidance with ControlNet

• FlowVid Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis

  • doesnt strictly adhere to optical flow

  • first frame = supplementary reference in the diffusion model

  • works seamlessly with existing I2I models

I2VGEN-XL

• I2VGen-XLMS-Image2Videonon commercial good consistency and continuity, animate image

  • built on sd; designed UNet to perform spatiotemporal modeling in the latent space;

  • pre-trained on video and images

  • I2VGen-XL High-QualityImage-to-Video Synthesis via Cascaded Diffusion Models

    • utilizing static images as a form taining guidance

ANIMATEDIFF

• ==best one==

• AnimateDiff AnimateYourPersonalizedText-to-Image Diffusion Models without Specific Tuning

  • insert motion module into frozen(normal sd) text-to-image model

• examples: (nsfw) video1video2video3video4>>96101928notnsfw: video1>>96052859sword and sun>>96155685

  • best one: video6video7video8video9

• current state ways: https://banodoco.ai/Animatediff more insight

  • techniques:

    • Animatediff-cli-prompt-travelUpscale: https://twitter.com/toyxyz3/status/1695134607317012749 <button class="pull-tweet" value=https://twitter.com/toyxyz3/status/1695134607317012749>pull</button>

    • Controlling AnimatedDiffusing starting and ending frames (from Twitter user @TDS_95514874

• AnimateZero Video Diffusion Models are Zero-Shot Image Animators

  • T2I generation is more controllable and efficient compared to T2V

  • we can transform pre-trained T2V models into I2V models

• LongAnimateDiff now 64 frames

• FreeNoise Tuning-Free Longer Video Diffusion via Noise Rescheduling (FreeNoise-AnimateDiff)

  • removed the semantic flickering

• AnimateDiff-Lightning fast text-to-video model; can generate videos ten times than ANIMATEDIFF

DIFFDIRECTOR

• DiffDirector AnimateDiff-MotionDirector, MotionDirector Train a MotionLoRA and run it on any compatible AnimateDiff UI

PIA

• PIA YourPersonalized Image Animator via Plug-and-Play Modules in Text-to-Image Models

  • motion controllability by text: temporal alignment layers (TA) out of token

ANIMATELCM

• AnimateLCM decouplesthe distillationof image generation priors and motion generation priors

CMD

• EfficientVideo Diffusion Models via Content-Frame Motion-Latent Decomposition ==best==

  • content-motion latent diffusion model (CMD)

    • autoencoder that succinctly encodes a video as a combination of image and a low-dimensional motion latent representation

    • pretrained image diffusion model plus lightweight diffusion motion model

3D SD

• VideoCrafter Open Diffusion Models for High-Quality Video Generation and Editing (A Toolkitfor Text-to-Video

  • has loras and controlnet, 3d unet; deeper lesson

• VideoFusion: damo/text-to-video-synthesis summarytiny paper

  • https://rentry.org/f34hy license change commit

BY INPUT

VIDEO COHERENCE

• BIGGER COHERENCEfrom normal sd image generation

• InstructVideo Instructing Video Diffusion Models with Human Feedback

  • recast reward fine-tuning as editing: process corrupted video rated by image reward model

VCHITECT

• Vchitect LaVieText2Video), SEINEImage2Video)

IMAGES

• SEINE Short-to-LongVideo Diffusion Model for Generative Transition and Prediction ==best== <<SEINE>>

  • SEINE images of different scenes as inputs, plus text-based control, generates transition videos

• DynamiCrafter AnimatingOpen-domainImageswith Video Diffusion Priors (prompt and image) ==best==

• AtomoVideo High Fidelity Image-to-Video Generation ==best==

  • from input images, motion intensity and consistency; compatible with sd models without specific tuning

  • pre-trained sd, add 1D temporal convolution, temporal attention

• ToonCrafter Generative Cartoon Interpolation ==best==

DANCING

• CLOTH

• PixelDance Make Pixels Dance: High-Dynamic Video Generation

  • synthesizing videos with complex scenes and intricate motions

  • incorporates image instructions (not just text instructions)

• MagicAnimate Temporally Consistent Human Image Animation using Diffusion Model

  • video diffusion model to encode temporal information

• REFERENENET

• Text2AC-Zero Consistent Synthesis of Animated Characters using 2D Diffusion

  • zero shot on existing t2i, no training or fine-tuning

  • pixel-wise guidance to steer the diffusion to minimizes visual discrepancies

• DreaMoving A HumanDanceVideo Generation Framework based on Diffusion Models

  • Video ControlNet for motion-controlling and a Content Guider for identity preserving

• Motionshop An applicationof replacing the human motion in the video with a virtual 3D human

  • segment retarget and and inpaint (with light awareness)

• Diffutoon High-ResolutionEditableToon Shading via Diffusion Models

  • aiming to directly render(turn) photorealistic videos into anime styles; keeping consistency

• AnaMoDiff 2D Analogical Motion Diffusion via Disentangled Denoising

  • best trade-off between motion analogy and identity preservation

• MagicDance Realistic Human Dance Video Generation with Motions & Facial Expressions Transfer ==best==

  • real people references

TALKING FACES

• DreamTalk When Expressive Talking Head Generation Meets Diffusion Probabilistic Models

  • inputs: songs, speech in multiple languages, noisy audio, and out-of-domain portraits

• EmoSpeaker One-shot Fine-grained Emotion-Controlled Talking Face Generation

  • emotion input, different emotional intensities by adjusting the fine-grained emotion

• HeadStudio Text to Animatable Head Avatars with 3D Gaussian Splatting

  • generating animatable avatars from textual prompts, visually appealing

• PEGASUS Personalized Generative 3D Avatars with Composable Attributes

  • disentangled controls while preserving the identity, realistic

  • trained using synthetic data at first

VIDEO INPUT

• MagicProp Diffusion-based Video Editing via Motion-aware Appearance Propagation

  • edit one frame, then propagate to all

• HierarchicalMasked 3D Diffusion Model for Video Outpainting

• FRESCO Spatial-Temporal Correspondence for Zero-Shot Video Translation

  • Zero shot and EBsynth come together for a new vid2vid

BY PROMPT

• Text2Video-Zero Text-to-ImageDiffusion Modelsare Zero-Shot Video Generators

  • DDIM enhanced with motion dynamics, after cross-frame attention to protect identity

  • Zero-Shot VideoEditing UsingOff-The-Shelf Image Diffusion Models (vid2vid zero)

• Edit-A-Video Single Video Editing with Object-Aware Consistency

• video-p2pcrossattention control (more coherance than instruct-pix2pix) (Adobe)

  • VidEdit Zero-Shot and Spatially Aware Text-Driven Video Editing (temporal smoothness)

• StableVideo Text-drivenConsistency-aware Diffusion Video Editing (14 gb vram)

  • temporal dependency = consistent appearance for the edited objects ==best one==

• FreeNoise Tuning-FreeLonger Video Diffusion Via Noise Rescheduling Video Diffusion Via Noise Rescheduling ==best==

  • reschedule a sequence of noises peforming window-based function = longer videos conditioned on multiple texts

MODELS

• Stable VideoDiffusion

  • loras for camara control, multiview generation

• MagicVideo-V2 Multi-Stage High-Aesthetic Video Generation ==best==

  • more coherent movements

LATENT OF BOTH IMAGES AND VIDEO

• Phenaki

  • C-vit is the video encoder, Vivitrepo

  • single images are treated like videos

• PhotorealisticVideo Generation with Diffusion Models

  • compress images and videos within a unified latent space

• I2VGEN-XL

WITH ARCHITECTURE STRUCTURE

• Show-1 Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation

  • first pixel-based t2v generation then latent-based upscaling

CASCADED

• LAVIE High-QualityVideoGenerationwith Cascaded Latent Diffusion Models <<LAVIE>>

  • cascaded video latent diffusion models, temporal interpolation model

  • incorporation of simple temporal self-attentions with rotary positional encoding, captures correlations inherent in video ==best one==

• I2VGen-XL High-Quality Image-to-Video Synthesis via Cascaded Diffusion Models ==best==

  • utilizing static images as a form of crucial guidance

    • guarantee coherent semantics by using two hierarchical encoders

EXTRA PRIORS

• DRAG DIFFUSIONIDENTITY IN VIDEO

• Dual-StreamDiffusion Net for Text-to-Video Generation

  • two diffusion streams, video content and motion branches = video variations; continuous with no flickers

STYLECRAFTER

• StyleCrafter EnhancingStylized Text-to-Video Generation with Style Adapter ==best==

  • high-quality stylized videos that align with the content of the texts

  • train a style control adapter from image dataset then transfer to video

MOTION

• MCDiff Motion-Conditioned Diffusion Model for Controllable Video Synthesis

• VideoComposer Compositional Video Synthesis with Motion Controllability models (temporal consistency)

  • motion vector from as control signal

• MotionDirector Motion Customization of Text-to-Video Diffusion Models

  • dual-path LoRAs architecture to decouple the learning of appearance and motion

• ANIMATEDIFF

  • LAMP Learn A Motion Pattern for Few-Shot-Based Video Generation (8~16 videos = 1 Motion)

    • expand pretrained 2D T2I convolution layers to temporal-spatial motion learning layers

    • shared-noise sampling = improve the stability of videos

• MotionDirector Motion Customization of Text-to-Video Diffusion Models ==best==

• DreamVideo Composing Your Dream Videos with Customized Subject and Motion

  • desired subject and a few videos of target motion (subject, motion learning on top of video model)

SVD

• AnimateAnyghing Fine Grained Open Domain Image Animation with Motion Guidance (anything)

  • finetuning stable diffusion video

CONTROLER

• MagicStick Controllable Video Editing via Control Handle Transformations

  • keyframe transformations can easily propagate to other frames to provide generation guidance

  • inflate image model and ControlNet to temporal dimension, train lora to fit the specific scenes

• CustomizingMotion in Text-to-Video Diffusion Models

  • map depicted motion to a new unique token, and can invoke the motion in combination with other motions

• Peekaboo Interactive Video Generation via Masked-Diffusion

  • based on masking attention, control size and position

• DIFFDIRECTORANIMATELCM

• Motion Guidance Diffusion-Based Image Editing with Differentiable Motion Estimators

  • a guidance loss that encourages the sample to have the desired motion

• TrailBlazer Trajectory Control for Diffusion-Based Video Generation

  • pre-trained model without further model training (bounding boxes to guide)

• Boximator Generating Rich and Controllable Motions for Video Synthesis

  • hard box and soft box

  • plug-in for existing video diffusion models, training only a module

• Follow-Your-Click Open-domainRegional Image Animation via Short Prompts

  • locally aware and not moving the entire scene

• CameraCtrl Enabling Camera Control for Text-to-Video Generation

  • camera pose control, parameterizing the camera trajector

  • AnimateDiffmoremore

MOTION FROM VIDEO

• Spectral MotionAlignment for Video Motion Transfer using Diffusion Models

  • aligns motion vectors using Fourier and wavelet transforms

  • maintaining computational efficiency and compatibility with other customizations

• Motion Inversionfor Video Customization

  • Motion Embeddings: temporally coherent derived from a given video

  • less than 10 minutes of training time

DRAGANYTHING

• DragAnything Motion Control for Anything using Entity Representation

  • trajectory-based is more userfriendly; control of motion for diverse entities

DEFINE CAMERA MOVEMENT

• LivePhoto RealImageAnimation with Text-guided Motion Control

  • motion-related textual instructions: actions, camera movements, new contents

  • motion intensity estimation module(control signal)

• MotionCtrl A Unifiedand FlexibleMotionController for Video Generation

  • independently control camera and object motion, determined by camera poses and trajectories

  • using drawn lines

  • motionctrl for svd comfy

• Icon Direct-a-Video Customized Video Generation with User-Directed Camera Movement and Object Motion

  • define camera movement and then object motion using bounding box

REFERENENET

• Animate Anyone ConsistentandControllable Image-to-Video Synthesis for Character Animation

  • ReferenceNet(controlnet), to merge detail features via spatial attention (temporal modeling for inter-frame transitions between video frames)

  • Moore-AnimateAnyone(over sd 1.5)

LONG VIDEO

• NUWA-XL Diffusion over Diffusion for eXtremely Long Video Generation

  • coarse-to-fine process, iteratively complete the middle frames

• sparseformer

  • Towards End-to-EndGenerativeModeling of Long Videos with Memory-Efficient Bidirectional Transformers

    • autorregresive with patches

• FusionFrames Efficient Architectural Aspects for Text-to-Video Generation Pipeline

  • keyframes synthesis to figure the storyline of a video, then interpolation

• SEINE

GENERATED VIDEO ENHANCEMENT

• optical flow background removal

TRICKS

• script cinema https://xanthius.itch.io/multi-frame-rendering-for-stablediffusion \[\[https://www.reddit.com/r/StableDiffusion/comments/11mlleh/custom_animation_script_for_automatic1111_in_beta/\]\[reddit\]\inspired]]

• gridof frames

USING MODEL

• MS-Vid2Vid

  • enhance the resolution and spatiotemporal continuity of text-generated videos and image-generated videos

OTHERS EDITING VIDEO

• VIVE3D Viewpoint-IndependentVideo Editing using 3D-Aware GANs

• MoonShot Towards Controllable Video Generation and Editing with Multimodal Conditions

  • zero-shot subject customized, controlnet only, video transformation

• ActAnywhere Subject-Aware Video Background Generation

  • input: segmented subject and contextual image input

• STABLEIDENTITYinserting identity

VIDEO INPAINT

• Anything inAny Scene Photorealistic Video Object Insertion (realism, lighting realism, and photorealism)

• InteractiveVideo User-Centric Controllable Video Generation with Synergistic Multimodal Instructions

  • use human-painting, drag and drop, as prior to inpainting generation, dynamic interaction,

• Place Anythinginto Any Video

  • using just a photograph of the object, looks like enhanced VR

• Videoshop Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion

  • add or remove objects, semantically change objects, insert stock photos into videos

OUTPAINTER

• Be-Your-Outpainter Mastering Video Outpainting through Input-Specific Adaptation ==best==

  • input-specific adaptation and pattern-aware outpainting

VIDEO EXCHANGE

• VideoSwap Customized Video Subject Swapping with Interactive Semantic Point Correspondence

  • exploits semantic point correspondences,

  • only a small number of semantic points are necessary to align the subject's motion trajectory and modify its shape

CONTROLNET VIDEO

• StableVideo Diffusion Temporal Controlnet

FRAME INTERPOLATION

• MA-VFI: Motion-AwareVideo Frame Interpolation

• BlazeBVD Make Scale-Time Equalization Great Again for Blind Video Deflickering

  • illumination histograms that precisely capture flickering and local exposure variation

  • to restore faithful and consistent texture affected by lighting changes; 10 times faster

• https://www.twitch.tv/videos/611050187]
• about John Seely Brown, Design Unbound and Insight through making about design
• starts a daily note, hierarchical bullets but also normal text. Making new pages and clicking on them. Bear
• taking notes from the book - not clear if he has already taken notes while reading the book, or if he is just talking from memory and flipping through the book while taking notes
• Initially adding a bunch of concepts, and then indenting under, saying "I will separate these out into different notes later"
• [[types of reasoning]] / thinking
  • synthetic
    • syntopic
  • abductive
    • Abductive reasoning (also called abduction, abductive inference, or retroduction) is a form of logical inference that starts with an observation or set of observations and then seeks to find the simplest and most likely conclusion from the observations. #**Occam’s razor**? designerly thinking?
    • Despite many possible explanations for any physical process that we observe, we tend to abduce a single explanation (or a few explanations) for this process in the expectation that we can better orient ourselves in our surroundings and disregard some possibilities. Properly used, abductive reasoning can be a useful source of Bayesian priors
      • This process, unlike deductive reasoning, yields a plausible conclusion but does not positively verify it - inference to the best explanation
  • inductive
  • Deductive
    • a valid deduction guarantees the truth of the conclusion
• curious about how Andy thinks about mapping, would any of this thinking have been better done visually?
• creates a note for the book, literature note
  • not evergreen, jumping off point, gets bibliography metadata from Zotero (manually)
    • wonder if he automatically pulls in these for citations
  • named autors comma title
  • writes short summary - not copying from his daily note. Grounds his thinking, provides feedback about how well he understands the bookj.
• coinage-oriented note, named after the authors - doesn’t like this, means he hasn’t properly digested it yet. Not sure if this concept has transcended the book yet (in wider usage)
  • trying to put it in his own words, instead of copying
• Example of using search, and working across a large number of notes
  • 

    ⥅ [[https://firebasestorage.googleapis.com/v0/b/firescript-577a2.appspot.com/o/imgs%2Fapp%2Fstian%2FxmpISAZMqd.png?alt=media&token=81b5d8a7-69d7-432b-8d69-acc0f3625f56]]

• [[flancian]] https://social.coop/@flancian/108438491804983015 pull

• [[2022-11-27 04:37:37+00:00]] @[[metamitya]]: https://twitter.com/metamitya/status/1596724902786043904 pull

• [[smallcircles@social.coop]] https://social.coop/@smallcircles/109659133816511752 pull
• [[smallcircles@social.coop]] https://social.coop/@smallcircles/110281011297440641 pull

• [[go]] https://drive.google.com/file/d/1MqN2MY39BhVxFRquDTVjy6779BDS4Xjg/view?usp=sharing pull

• [[flancian]] https://twitter.com/flancian/status/1575585012002754560 pull

• [[flancian]] https://twitter.com/flancian/status/1575585012002754560 pull
  • @an_agora [[make a video]] https://t.co/bR01sZAiB8

{{< youtube _IbTZBMHiY4 >}}

[[P- Yochai Bankler]] makes the argument in [[R- Wealth of Networks]] that people have a wealth of idle attention. [[Open source]], [[Wikipedia]], etc. has some architecture designed for many people to contribute in modular ways without needing to worry about the contributions of others. 40:30

When you look at these systems, what is it that you construct in these systems that makes these systems, in the aggregate, have more knowledge in them, behave more intelligently, produce more innovations, or be more productive in general.

If you’re into the world of UI design, what do I do at the level of individuals and individual user interfaces that have properties that percolate upward into the aggregate social architecture?

[[Q- How do people acquire and transfer knowledge and expertise in a decentralized discourse graph]]

Around 43:50: StumbleUpon had a user rating system that went from rating content on a spectrum from 1-10, then 1-5, then upvote/downvote. Each time they reduced the range of the scale, they would get more participation. This is used to justify the claim that reducing the effort to participate increases participation.

Starting from 49:00, they are talking about research in increasing participation rates in social tagging. Primarily trying to solve

• Increase participation through lower effort tag production
• Increase individual tag production rates
• Increase learning and memory
• Reduce effects of "tag noise"
• Support the transfer of expertise

Around 1:00:00, they introduced Mr. Taggy. This is similar to [[R- Interactive Intent Modeling for Exploratory Search]] insofar as it shows you a bunch of socially generated tags, allows you to narrow it down by filtering tags in or out of the search results (like [[Roam’s]] backlinks filters). This allowed people without prior background knowledge to learn a new domain about as well as those who came in with prior knowledge. This addresses a core problem in exploratory search with new domains: often you can’t find what you are looking for because you don’t know the vocabulary of the field. You enter the space searching for answers with a vocabulary that doesn’t match the space. By seeing the most frequent tags in order from top to bottom, you are able to pick up on the vocabulary more quickly. Since these social tags were generated by experts, this facilitates the transfer of expertise to newcomers. [[Waldo]] has a similar interface

⥅ [[Pasted image 20210920171835.png]]

⥅ [[Pasted image 20210920171911.png]]

Rel8 (and OGM) Vision Video§

Brainstorming a script:

• Because extreme voices are so loud and visible, we aren’t hearing from moderates.
• Breaking the ideas apart might help us share what we have in common and make progress.

Narrative Arc:

• So we don’t all keep duplicating effort (reinventing the wheel)
• Collaborations
• Preserve personal preferences and points of view
• Encourage "crystalization" of points of view
• Bridge and changing disciplines
• Governance: how?
• Financing
• Survey of other vision videos
• Create this video using our own tools and concepts (woven, linky, shared)

• [[2026-03-15T06:21:51.151Z]] @[[mmr3cords.bsky.social]]: https://bsky.app/profile/did:plc:aw3bwefrkzhpt2enywvfzurm/post/3mh3ciyz3a226 pull

• https://www.youtube.com/watch?v=Zvo4NzlG2Qc