In the spring of 2020, as we reluctantly entered a world filtered through the lens of technology, I rewrote my graduating thesis. Grappling with the notions of humanity, I set out to explore how human nature could engage with the futures of technology.

Arc is a functioning AI model and speculative experience that enables people to generate entirely original films through collaboration with artificial intelligence.

Collaborate with Arc

The craziest data viz I've ever built: An input to generate stories.

Built on the premise that each story possesses a distinct shape, this technology allows users to effortlessly guide the narrative arc of their film by manipulating its mood. Through analyzing the nuanced ambiance and atmosphere within a movie clip, arc intelligently maps an extensive film library to follow your chosen shape, resulting in an entirely unique story.

Loading screens become a valuable medium to help explain what is happening in the background.

Research

So many papers, emerging tools, essays on film theory, colab notebooks, prototypes, animations...

Building the Model

Constructing and training a computer vision model is an arduous and intricate endeavor that involves careful consideration. The inputs and training data directly shape the model’s characteristics. If biased or inadequate data is used during training, the AI will inevitably mirror these shortcomings. The initial training of arc proved disastrous. Weeks of exhaustive image analysis and labeling left me confused and exhausted, resulting in a model that achieved only 50% accuracy in mood classification. Why? My classifiers lacked the necessary distinctiveness and thoughtful curation.

After over 6,000 images and two attempts, arc achieved 90% accuracy in mood classification!

Following thorough reflection, immersing myself in film theory, and redefining the essential elements of mood, I felt prepared to make another attempt. Equipped with a clear vision for the model, I meticulously constructed a set of rules for five distinct mood classifiers and curated an extensive dataset of nearly 3,000 images. The dataset was then fed through a computer vision pipeline for training. It took about two weeks to compile the entire dataset by hand, and only ~12 minutes to run the training program.

Forming your own opinion

My exploration of future technologies expanded my understanding, emphasizing the need for collaborative and reflective spaces to engage with seemingly complex systems. This project sparked new questions and ideas about AI’s future, emphasizing the significance of addressing biases and perspectives in the development of technologies and systems.

To foster dialogue on future technology and the collaborative relationship between humans and technology, I created a reflective space, encouraging users to consider their role in the system.

Ultimately, I believe that collaboration between people and AI is an exercise in self-awareness and reflection. Our experiences and perspectives serve as the foundation upon which the AI learns, underscoring the importance of considering how we will inform the future of technology.

and the credits roll...

What’s next

This project has ignited my passion for generative AI and filmmaking, informing my ongoing exploration. Future updates to arc, such as dynamic shapes, sharable films, and customizable inputs, are just the beginning. I am excited to continue pushing the boundaries, working on more intricate AI + film tools.

This technology will reshape the way we create films. If you’re interested in helping, reach out. Exciting stuff is happening.