VQGAN + CLIP pt 3

While continuing to play around with VQGAN+CLIP, I started passing in additional parameters. The first time around I treated the entire poem as a mix of texts, with the addition of in the style of tolkien. The goal was to call upon the vast trove of art available in the WikiArt 16384 to channel something tolkien specific.

Far over the misty mountains cold | To dungeons deep and caverns old | We must away ere break of day | To seek the pale enchanted gold. | The dwarves of yore made mighty spells, | While hammers fell like ringing bells | In places deep, where dark things sleep, | In hollow halls beneath the fells. | For ancient king and elvish lord | There many a gleaming golden hoard | They shaped and wrought, and light they caught | To hide in gems on hilt of sword. | On silver necklaces they strung | The flowering stars, on crowns they hung | The dragon-fire, in twisted wire | They meshed the light of moon and sun. | Far over the misty mountains cold | To dungeons deep and caverns old | We must away, ere break of day, | To claim our long-forgotten gold. | Goblets they carved there for themselves | And harps of gold; where no man delves | There lay they long, and many a song | Was sung unheard by men or elves. | The pines were roaring on the height, | The winds were moaning in the night. | The fire was red, it flaming spread; | The trees like torches blazed with light. | The bells were ringing in the dale | And men looked up with faces pale; | The dragons ire more fierce than fire | Laid low their towers and houses frail. | The mountain smoked beneath the moon; | The dwarves, they heard the tramp of doom. | They fled their hall to dying fall | Beneath his feet, beneath the moon. | Far over the misty mountains grim | To dungeons deep and caverns dim | We must away, ere break of day, | To win our harps and gold from him! | in the style of tolkien

I made two mistakes here.

1. I ran this on the Google Colab CPU, so it took forever.
2. It was set to only produce 72 images.

Moving on.

I then re-ran the model with a configuration slightly more in-line with what I had done in previously for VQGAN+CLIP pt 2. This time I kept the seperate strings, and added in the style of tolkien to each one as a param.

Far over the misty mountains cold | in the style of tolkien || To dungeons deep and caverns old | in the style of tolkien || We must away ere break of day | in the style of tolkien || To seek the pale enchanted gold. | in the style of tolkien || The dwarves of yore made mighty spells, | in the style of tolkien || While hammers fell like ringing bells | in the style of tolkien || In places deep, where dark things sleep, | in the style of tolkien || In hollow halls beneath the fells. | in the style of tolkien || For ancient king and elvish lord | in the style of tolkien || There many a gleaming golden hoard | in the style of tolkien || They shaped and wrought, and light they caught | in the style of tolkien || To hide in gems on hilt of sword. | in the style of tolkien || On silver necklaces they strung | in the style of tolkien || The flowering stars, on crowns they hung | in the style of tolkien || The dragon-fire, in twisted wire | in the style of tolkien || They meshed the light of moon and sun. | in the style of tolkien || Far over the misty mountains cold | in the style of tolkien || To dungeons deep and caverns old | in the style of tolkien || We must away, ere break of day, | in the style of tolkien || To claim our long-forgotten gold. | in the style of tolkien || Goblets they carved there for themselves | in the style of tolkien || And harps of gold; where no man delves | in the style of tolkien || There lay they long, and many a song | in the style of tolkien || Was sung unheard by men or elves. | in the style of tolkien || The pines were roaring on the height, | in the style of tolkien || The winds were moaning in the night. | in the style of tolkien || The fire was red, it flaming spread; | in the style of tolkien || The trees like torches blazed with light. | in the style of tolkien || The bells were ringing in the dale | in the style of tolkien || And men looked up with faces pale; | in the style of tolkien || The dragons ire more fierce than fire | in the style of tolkien || Laid low their towers and houses frail. | in the style of tolkien || The mountain smoked beneath the moon; | in the style of tolkien || The dwarves, they heard the tramp of doom. | in the style of tolkien || They fled their hall to dying fall | in the style of tolkien || Beneath his feet, beneath the moon. | in the style of tolkien || Far over the misty mountains grim | in the style of tolkien || To dungeons deep and caverns dim | in the style of tolkien || We must away, ere break of day, | in the style of tolkien || To win our harps and gold from him! | in the style of tolkien

That is more like it. Now, lets revisit that original idea. I matched the step count to the final output of the separate string model, 2920, and started it.