No, it’s not over yet: AI tools’ ability to manipulate images continues to grow. The latest example is just a research paper for now, but a very impressive one, allowing users to simply drag and drop elements of an image to change their appearance.
This doesn’t sound too exciting at first glance, but take a look at the examples below to get an idea of what this system can do.
Not only can you resize a car or turn a smile into a frown with a simple click and drag, but you can also rotate the subject of a photo as if it were a 3D model, for example changing the direction someone is looking. change. One demo even shows the user adjusting the reflections on a lake and the height of a mountain range with a few clicks.
Here is an overview of various topics:
Here’s a closer look at landscape manipulation:
And just for fun, messing around with lions:
These videos are from the research team’s homepage, although it crashes due to the amount of traffic sent to the site by Twitter (mainly by user @_akhaliq, who does a fantastic job of highlighting interesting AI papers and is well worth following if that interests you). You can also read the research article on arXiv here.
As the team leader points out, what’s really interesting about this work isn’t necessarily the image manipulation necessarily, but the user interface. We’ve been able to use AI tools like GANs to generate realistic images for some time now, but most of the methods lack flexibility and precision. You can tell an AI image generator to “take a picture of a lion prowling the savannah” and you’ll get one, but it may not be the precisely pose you want or need.
This model, called DragGAN, offers a clear solution for this. The interface is exactly the same as traditional image warping, but instead of simply smearing and mushing existing pixels, the model regenerates the subject. As the researchers write: “[O]Our approach can hallucinate hidden contents, like the teeth in a lion’s mouth, and can deform due to the object’s rigidity, like bending a horse’s leg.
Obviously this is just a demo for now and it is impossible to fully evaluate the technology. (For example, how realistic are the final images? It’s hard to say based on the low-resolution videos available.) But it’s another example of making image manipulation more accessible.