Runway Chronicles: Melding Art and AI in a Transient Tech Terrain
Watch Runway's evolution, learn tech behind Stable Diffusion, as well as legal challenges, financials, and company's mission to democratize generative AI
There is almost nothing less stable than the landscape of Generative AI (GenAI). The last week's changes include a sudden $0.3 billion drop by Jasper. It’s not a surprise in a world where apps are built on someone else’s model/technology. But we will be talking about something ‘stable’ today: the Stable Diffusion model and Runway – whose researchers are behind the model development. It’s interesting that Stable Diffusion has also become a bedrock for another unicorn from our chart – Stability AI. But today we concentrate on Runway, number 7 in the unicorn family (so far).
The starting point of Runway
Financial situation
Open-source and copyright tensions
Mission of Runway
Founder’s thoughts about AGI
Way to Runway and Stable Diffusion success (linked to the important research papers)
Flagship product (and other interesting projects)
Tech behind Runway’s latest Gen 2 model
How does Runway make money?
Little-known facts about Runway
Bonus: All important links about the founders
The starting point of Runway
Four years before Stable Diffusion sparked the text-to-image excitement wave in the GenAI world, the idea of melding art and tech was brewing in the minds of three young visionaries in New York University’s Interactive Telecommunications Program (ITP), a grad program at the intersection of technology and the arts. Meet Cristóbal Valenzuela, Anastasis Germanidis, and Alejandro Matamala-Ortiz.
The idea of the name for Runway traces back to Valenzuela’s thesis project at NYU’s ITP where he collaborated with Dan Shiffman (Board of Directors at ITP) and met his future co-founders while understanding the applications of machine learning (ML) in the creative domains. In the early stages of research, Valenzuela sought a concise name to discuss the project with his advisors. During his brainstorming sessions, he recognized that the concept of "a platform for models" already had a fitting moniker: a runway.
In August 2018, while TikTok was gaining prominence followed by its merger with Musical.ly, the trio decided to establish Runway – a startup dedicated to developing tools for video creators supercharged by ML; while also crafting the tech behind these tools.
Initially launched in 2019 as a model directory, Runway lets users deploy and run open-source models for various artistic use cases. Over the years as the user base expanded, the startup pivoted towards developing ML-based video editing tools. It was in the founders’ initial concept to be open-minded about how people might be using their platform.
Today, Runway has tapped into multi-billion dollar software markets.
Financial and legal situation
We’ve put a comprehensive table for your convenience, available only for Premium subscribers →subscribe for only $6/month
Total funds raised soar to $237 million, tripling its valuation to a whopping $1.5 billion in 2023. But there are some tensions in the GenAI world…
Open-source tension
While the company has managed to keep its legal slate clean during all these years, last year, the company got in a tussle with Stability AI due to which the companies have turned from friends to foes.
The discord between the companies arose from the release of the Stable Diffusion model, a collaborative initiative led by Patrick Esser from Runway and Robin Rombach from LMU Munich (also, Stability AI). The model's code was open-sourced under the CreativeML Open RAIL M License, thanks to a trilateral effort involving Runway, Stability AI, and LMU Munich's CompVis group. However, tensions flared when Stability AI lodged a takedown request against Runway citing an IP leak with Runway ML's SD 1.5 model, an extended version of Stable Diffusion. Runway's CEO, Cris, refuted the IP breach claim in a Hugging Face discussion, appreciating Stability AI for a compute donation aiding the retraining of the original model. The takedown request was later withdrawn by Stability AI, suggesting a resolution to the disagreement.
Such hiccups in collaborative AI projects highlight the importance of balance promoting an open-source ethos for communal growth, and upholding IP rights to ensure fair recognition and protect innovations.
Copyright tension
Furthermore, shortly, it is highly unlikely that the company akin to Midjourney.Inc and Stability will be able to dodge legal issues arising from the unconsented use of copyrighted material in their model training processes. In May, Runway appointed its first general counsel, hinting towards the fact that the startup is gearing up for the incoming matters of law.
In a recent discussion with The Information, Valenzuela recollected how bootleg DVDs were common, peddled by street vendors when he was a child. He noted that eventually, norms and regulations were established to curb such practices, pointing toward the current situation surrounding AI and copyrighted content. “No one ever thought about, like, restricting the technology or the progress of cameras,” he said.
Mission of Runway
Hailing from Chile (Valenzuela and Matamala-Ortiz) and Greece (Germanidis), these immigrant entrepreneurs wanted to democratize generative AI, making capabilities ranging from crafting realistic copy to generating art accessible to people without coding backgrounds, akin to Adobe Creative Suite's approach.
For him, the primary mission isn’t catering to pros; it’s expanding the borders of the AI art world, Valenzuela stated in 2019.
Unlike the tech goliaths, Runway has built its model with a customer-centric approach in mind. “This is one of the first models to be developed really closely with a community of video makers,” the CEO had told MIT Technology Review. “It comes with years of insight about how filmmakers and VFX editors actually work on post-production.”
Founder’s thoughts about AGI
Valenzuela is certainly from the anti-AI doom camp. “First of all, not everyone in the industry is trying to build God,” he told The Information. “We’re trying to build tools. The keywords are not replacement, but augmentation, right? Enhancement.”
To illustrate his point, he drew parallels between the current AI discourse and the early days of cinema when the first film was screened. “Legend has it that when the film was first screened, the audience panicked and fled the theater, convinced that the train in the film was going to burst through the screen and run them over. AI is not going to burst through the screen and run us over.”