How Deep Learning Can Enhance Digital Artist's Creative Workflows

 This talk by Anastasia Opara on 'Proceduralism and Deep Learning' is very thought provoking.  It covers some specific topics HTC is interested in. Specifically, how can we enhance digital artist's workflows by using proceduralism and Deep Learning to enhance creativity and reduce the amount of 'busy work tedious craft' required to get from point A to point B in your artistic journey.

The talk focuses on enhancing workflows for game artists.  But i think the same principals also applies to more general digital art workflows.  How can we use 'software 2.0' principals to train a system by showing it examples.



Observations

1.  The GAN synthesis example was interesting. Note that she pointed out that it doesn't always connect the lines completely.

2.  I think the 'draw a line and make a 3D mountain' demo could be directly implemented using Studio Artist Geodesic or Chamfer distance interpolation approaches. So in my mind using the Pix2Pix neural net architecture for this is maybe a distraction?

3.  HTC has specific courses on deep learning in our HTC Education Series if you are interested in learning more.

4.  Good question at the end about how to incorporate this into a typical digital artists's workflow (someone who just wants to get artistic work done and doesn't want to fool around with writing neural network code and may find this whole notion alien to how they have traditionally worked).

Note that her comments about this approach 'democratizing artistic design' mesh with HTC's views on this topic.  The goal is to make workflow more creative by reducing tedium (let the computer do your busy work so you can concentrate on being creative).  And to allow a pathway for people who are interested in creating art but have not spent years learning 'craft' to actually generate compelling output.


Here's another related more recent talk by Anastasia on 'Example-based Procedural Placement in a Dialog System'.


Observations

1.  Note that Wasserstein distance is important in GAN research.  Also note that the graphical explanation of it is way easier to understand than if we just looked at math equations for it's definition.

Sliced Wasserstein was new for me.

Wasserstein is a fancy name for 'earth movers distance'.

2.  Wouldn't SGD find that multi-dimensional minimum?

In the same train of thought, GAN systems have been trained for this kind of thing.

Comments

Popular posts from this blog

CycleGAN: a GAN architecture for learning unpaired image to image transformations

Pix2Pix: a GAN architecture for image to image transformation

Smart Fabrics