10 ms·
The emergence of full-body Gaussian Splat deepfake humans
- bondarchuk 3y ago>Since the advent of Gaussian Splats, in August of this year, the image synthesis research community has clearly embraced this innovative approach to neural recreation of people, things and scenery. I thought splats were explicitly not neural?
- danbruc 3y agoI also find their usage of the word neural very confusing but this is how they explain it. A Gaussian Splat, instead, is a neural representation unit that is not limited in this way – not only can it be assigned anywhere in XYZ/3D space, but it can as necessary multiply and subdivide into additional splats, as coverage requires.* And the asterisk leads to the following explaination. Technically it’s a rasterization unit rather than a neural unit, but in all current Splat implementations that are of any power or interest to the synthesis community, it ends up as a neural unit, passed through standard training processes. Amended Friday, December 15, 2023 13:33:47 EET to clarify ‘neural unit’ But even with this I am not sure that I understand what they want to say.
- lainga 3y ago> But even with this I am not sure that I understand what they want to say. Look at the domain name of TFA. They are saying "neural neural PageRank look at meeee"
- sorenjan 3y agoWhat does neural mean nowadays anyway? I thought neural nets got their name from the perceptron's resemblance to interconnected neurons, but what about CNN's or transformers? From what I can tell neural just means "plenty of parameters optimized using a learning process", in which case gaussian splats fits the description. I also think gaussian splats encode the same information as NeRFs, which have neural in the name.
- throwup238 3y agoI thought neural meant df/dx optimization applied to a giant function.
- deleted 3y ago[deleted]
- bondarchuk 3y ago>but what about CNN's or transformers? They also consist (in the abstract, at least) of interconnected neurons with activation functions.
- numpad0 3y agoAnything that require cuDNN installed in dev environment.
- deleted 3y ago[deleted]
- danbruc 3y ago[...] plenty of parameters optimized using a learning process [...] Would that not make fitting a degree 1,000,000 polynomial a neural method? I also think gaussian splats encode the same information as NeRFs, which have neural in the name. The collection of source images also encodes the same information but I don't think we should call them neural images. I would argue we should stick with the original meaning, inspired by biological neural networks, a network of artificial neurons. Calling everything neural because of the hype will benefit no one in the long run as the term will just lose any meaning.
- hansvm 3y agoPractically speaking, it's a history-dependent definition. Some set of things are considered "neural", somebody makes an innovation that's based on the existing set of "neural" tools, and now the set of "neural" things is expanded to include the new innovation. For this post, they have more traditional neural networks embedded in the pipeline (search TFA for "decoder"), and they're thus calling the composite technique a "neural" technique. As to the CNN/Transformer/... question, those are still very similar to a perceptron. A multi-layer perceptron has a sequence of alternating matmuls and activations. The CCN has a particular choice of sparse matmul and activation (and a couple choices of fast implementations to operate on that sparse matmul). A Transformer is one way to wire up many such MLP predictions (recurrent networks being another) to work with long/uneven inputs. There's more going on, but most of it is fluff to make the thing numerically stable and do something interesting. The core structure is still a dressed-up perceptron.
- elif 3y agoI'm sure the primary use case of this technology is in film. (e.g. Disney populating background extras) But I would LOVE to see this transformed into a game engine/game dev IDE. Imagine if anyone with a smartphone could create photorealistic character models in under a minute. Hell, players could even provide themselves as character models ad-hoc without involving any devs. It would be a renaissance moment for game development speed and accessibility, and make things like 'asset store games' irrelevant. Imagine an LLM built to understand a base game engine template code, and also able to perform gaussian splat transformation on frames of input video. It also seems like it could be the missing link for AR/VR crossing the uncanny valley.
- sorenjan 3y agoLets hope the primary uses are entertainment and maybe communication. It will soon be possible to create a realistic video of anyone doing anything.
- deleted 3y ago[deleted]
- numbsafari 3y agoWhich is why we need laws that make doing so without very explicit consent for every generated work a felony offense with social and penal consequences equivalent to rape.
- sorenjan 3y agoThat won't help against nation states spreading propaganda or manufacturing false evidence. If you thought Russian Twitter trolls were bad in 2016, wait until you see the TikTok trolls of tomorrow. This won't be regulated away, it will happen and everyone should be educated and prepared. That's just an unrealistic dream though, enough people will get fooled (I will probably be among them at least some of the time) that it will have consequences. Deepfakes have been used for scams for years already: https://www.youtube.com/watch?v=vqr0oER03SE https://www.youtube.com/watch?v=vqr0oER03SE
- Podgajski 3y agoIf kids are depressed from looking at the fakeness of people on Instagram, how are they going to feel put up against the possibility of these AI created perfected people?
- readyplayernull 3y agoSame as every time in history when cultures have gone morally bankrupt and nobody can be trusted, they'll keep isolating.
- Podgajski 3y agoHow do you become someone people can visibly see as a person of trust in a society like this? I know religion doesn’t do it, that’s for sure. The only thing I can think of is some kind of authenticity.
- mortb 3y agoNew innovations - gives you PTSD every day. Computeres and technology used to be fun. It hasn't been for years. We're building ourselves a horrible future. If it wasn't my main income I would have stopped using them and moved into the woods long ago. I've seen this comming. It's no wonder that birth rates are plummeting world wide. The only help I can think of is divine intervention or if someone has a very well hidden plan to work it out. For authenticity to work it needs to be universal and I'm not sure that is the case.
- Podgajski 3y agoYes, same story for me. But I got out of it in 1999. I seem to have mentally foreseen the crash coming and it portrayed itself as a mood disorder. I know many of you will disagree with me on this, but I also think a large problem is the increase in both extremely low frequency, and radio frequency, electromagnetic fields. No one can tell me that the bloody nose I get whenever I use laptops that emit high levels of low frequency electromagnetic radiation is psychosomatic.
- 3y ago
- skc 3y agoThe "Mandela Effect" is one day going to be a completely understandable and common phenomenon
- DrNosferatu 3y agoSeems we have finally crossed the uncanny valley… Hope the benefits will outweigh the dangers!
- kridsdale3 3y agoThe benefits will be to Disney shareholders.
- DrNosferatu 3y agoDon’t forget porn shareholders!