Yesterday I did my first dive into the practical usage of VQGAN+CLIP. I've spent a few months exploring STYLEGAN models, and have even trained a few of my own on RunwayML. They've been fun, and will definitly be sticking around as a fun curiosity for awhile, but there is something about VQGAN+CLIP that is extra sticky. Generating images and video from text is just cool.
I ultimately upgraded to Google Colab+ for access to faster GPUs. I'll be canceling my membership after my curiosity has been sated. They really were not kidding about the faster GPUs—the ~5 hours of processing for my first run through was cut down to ~1 hours after upgrading. More time for projects!
So, with that, I re-ran the model using almost all of the same settings. There was only one difference—instead of using the S-FLCKR model which is comprised of photographs, I used the WikiArt 16384 model which is build off of paintings from 195 different artists.
Far over the misty mountains cold || To dungeons deep and caverns old || We must away ere break of day || To seek the pale enchanted gold. || The dwarves of yore made mighty spells, || While hammers fell like ringing bells || In places deep, where dark things sleep, || In hollow halls beneath the fells. || For ancient king and elvish lord || There many a gleaming golden hoard || They shaped and wrought, and light they caught || To hide in gems on hilt of sword. || On silver necklaces they strung || The flowering stars, on crowns they hung || The dragon-fire, in twisted wire || They meshed the light of moon and sun. || Far over the misty mountains cold || To dungeons deep and caverns old || We must away, ere break of day, || To claim our long-forgotten gold. || Goblets they carved there for themselves || And harps of gold; where no man delves || There lay they long, and many a song || Was sung unheard by men or elves. || The pines were roaring on the height, || The winds were moaning in the night. || The fire was red, it flaming spread; || The trees like torches blazed with light. || The bells were ringing in the dale || And men looked up with faces pale; || The dragons ire more fierce than fire || Laid low their towers and houses frail. || The mountain smoked beneath the moon; || The dwarves, they heard the tramp of doom. || They fled their hall to dying fall || Beneath his feet, beneath the moon. || Far over the misty mountains grim || To dungeons deep and caverns dim || We must away, ere break of day, || To win our harps and gold from him!