How does AI redefine creative tools and mediums?

Editor’s Note:

Some people are happy and some are disappointed when TRUMP issues coins. Putting aside the Fomo sentiment, Meme is just the entrance. AI is the future of the spring on the chain. Grasp the most critical trend and the world is in our hands.

Today I would like to share with you an article titled Neural Media from crypto VC @baincapcrypto. The author @natalie reflects on the impact of generative artificial intelligence and cryptocurrency on creative production. I hope it can help you get some inspiration when looking for the next opportunity.

🎯 Key Highlights

1. Generative AI is causing a profound change in the field of creative production, and its impact can be compared to the Napster moment when the cost of media distribution dropped to zero in the Internet era:

• The core of this transformation is that the cost of creative production has been reduced to zero, striking directly at the core of human creativity.

• Under the new paradigm, humans should shift their focus from the final output to the system and process, that is, teaching neural networks to think at the programming level.

2. Through programming, we can create a unique software brain to generate unique ideas and works. Application scenarios include:

• Agent-based media: The model simulates a human partner, interacts through text conversations, and can perform operations such as financial transactions.

• Real-time game engine: Model simulation game engine that generates game frames based on user actions and achieves real-time rendering.

• Multiverse Generator: The model generates infinite variations, expanding the user’s original ideas and exploring the space of possibilities.

3. A trend in the future may be:

• Tooling of creation: Prompting is being embedded in more interfaces to stimulate end-user creativity. Most prompts will be abstracted into controls, but creative vision, precision, taste and skills will be more important.

• Evolution of media business models: From corporate media to user-generated media, and then to machine-generated media. The future consumer media business model will be built around agent-generated media (innovative scenarios include chatbots such as Character.ai, interface generation such as WebSim, user-generated currency such as Pump.fun, etc.)

• Intellectual property challenges: Machine learning enables programs to “learn” the aesthetic style of human creators, reducing the cost of creative production and aesthetic imitation to zero. The value and significance of intellectual property rights need to be re-examined.

4. The roles that cryptocurrencies can play include:

• The intersection of on-chain markets and agent-generated media (such as the recent DeFAI);

• Act as an IP incentive layer;

• Media monetization and access control, such as Minting becoming a new business model; NFT can serve as the infrastructure for personal programs and user-generated software;

• As an economic coordination layer between human and machine social interaction, explore new paradigms of community operation and agent interaction.

All in all, this is an article that may be difficult to read but is worth thinking about. AI enables human creativity to be more reflected in the design of systems and processes, and cryptocurrency provides a new economic and social coordination mechanism for this change. In the next media era, what new opportunities and trends can be created by combining the two? Let us wait and see.

►►►Main text

▎“All media are extensions of some human capacity—mental or physical.” ~Marshall McLuhan

For much of 2024, I’ve spent a fair amount of time trying to understand what we now call “Generative AI” and its implications for me personally and society at large. I’ve been fascinated by the possibilities of AI as a creative tool and have been using these new products heavily in my workflow, especially in creative writing and music composition.

However, as a crypto investor focused on consumer media and user-facing applications, AI increasingly feels like a blind spot to me. When we talk about the most successful consumer media businesses of the internet era, we don’t discuss them in terms of technology silos because they are not built that way, just like Facebook’s success is inseparable from technological innovation, but we don’t think of Facebook purely as a “mobile app” or “AI app”, instead, we recognize that it is the convergence of many different innovations that makes apps like Facebook possible.

Against this backdrop, this article aims to consolidate and refine my personal findings and insights from my exploration of AI over the past year. I share these in the hope that they will resonate or be helpful to others (especially my fellow crypto enthusiasts).

Part 1 Another Napster Moment

Today, much of the discussion around AI-generated media focuses on: (1) the ethics of model training and data scraping, (2) whether “AI art” is real art, and (3) the dystopian prospects of deepfakes. These discussions are all very interesting and worth listening to, but I think they miss the forest for the trees in some important ways.

I find the most useful framework for understanding the rise of generative AI is to think of it as intellectual property experiencing another “Napster moment” (Napster was the first widely used peer-to-peer music sharing service, which had a huge impact on the way people, especially college students, use the internet), but this time it’s a production moment rather than a distribution moment.

The rise of the internet and the subsequent drop in the cost of media distribution to zero was a “nothing to something” moment. The suddenness of this shift is brilliantly captured in the documentary How the Music Got Free, which tells the story of how a CD factory worker and a group of teenage hackers brought the entire music industry to its knees overnight.

Before the advent of Napster and the rise of digital file sharing more broadly, the entire corporate media-industrial complex (and the livelihoods of artists) depended on the technological realities of expensive, high-friction, and centralized media distribution. In just a few years after its launch, the major record labels went from record sales to begging the federal government to save them through legal intervention. The industry faced an extremely difficult reality: the economic system that underpinned its business had fundamentally and irreversibly changed, and the era of buying music was over.

Today, I think generative AI presents us with a more difficult reality, and the impact of the cost of creative production falling to zero is in many ways more difficult to deal with because it goes directly to the core of what many believe makes us human : our creativity. This existential dread does not change the fact that media generation (especially “style transfer” or aesthetic mimicry) is free, including now all media types we care about (text, images, video, audio, software) - this is another “something from nothing” moment.

However, the most important difference between today and the early 2000s is that in the fight between Napster and the media companies, the government sided with the companies and ultimately criminalized file sharing as “piracy.” (This is why we often refer to corporate media/intellectual property as “legal media.”) This decision, along with Steve Jobs launching the iPod to promote what became iTunes and eventually “streaming,” saved the industry from total collapse. Unfortunately, I think creators who are counting on the government to step in and take action here are comforting themselves at best and fooling themselves at worst.

I think we may find that the IP system is primarily designed to protect corporations and their legal media, and no one is coming to save us. Traditional media companies learned their lesson the hard way last time, so they are proactively entering into licensing deals with AI companies and are compensated to some extent. New media companies are also leveraging model training on user-generated content shared on their platforms, even if they claim not to be doing so. However, independent creatives are largely being left behind.

Part 2 Computing: The medium of our time

It’s easy to understand why many creators feel that generative AI undermines their abilities, and I think that concern is largely valid. However, I also think there’s an opportunity to think about computing in a new way that requires us to think about it not just as a communication medium, but as a creative medium.

The idea of computation as a creative medium is not new to anyone who has created video games or generative art. Yet today, it is still not really understood by many. Software was the first digitally native media category, and most people understand it primarily from the perspective of “service,” “utility,” and “optimization,” and not necessarily from the perspective of creative expression. Now, generative AI is driving this view in a very direct way, by driving the production costs of almost every other medium to zero . This seems to raise an existential question: “So where is human creativity? And where is the value of manual craft?”

My answer may not come as a surprise: “It’s programmable.” Before we delve into what I mean by this, we need to understand a few important technical concepts.

2.1 Neural Networks 101 (for beginners)

Training is the process of essentially teaching a model how to accomplish a task by providing it with lots of examples of how it accomplishes that task, then letting it find patterns, make predictions based on new inputs, and self-correct when it makes mistakes. Conceptually, this is similar to how we learn to draw: we start by imitating shapes until we are able to create original works, while continually improving our skills using feedback from peers and teachers. There is, of course, a key difference: a text generation model, for example, doesnt learn to write like you and I do, but rather learns to simulate writing with extreme accuracy. This is one of the many reasons why Im increasingly convinced that simulators rather than agents are a better mental model for neural networks.

Latent Space , or what I prefer to call high-dimensional possibility space, is a representation space in a neural network where what is learned during training is presented in a compressed form. To put it in a metaphor, this is similar to the internal world model that the model builds as it learns to understand the complex relationships between various detectable features in the training data. Understanding the concept of latent space is key to understanding neural networks as a creative tool and medium.

Latent Space Visualization #1 — Interpolating Between Known Embeddings

Latent Space Visualization #2 — Representation of Multi-dimensional Attributes Relations in Different Embeddings

Embeddings : Embeddings can be thought of as the process of mapping an input to a specific point in the latent space. This is the process where the prompt is essentially translated into the model’s “thinking language”. In this way, we can understand “prompts” as a way to explore and navigate the model’s latent space - meaning that mastering the prompts is about forming an intuition about the shape of the model’s latent space, which can guide the model to generate specific, expected outputs.

Part of the fun of playing with neural networks is that their deep inner workings are still a mystery to us. However, I think these basic concepts provide the necessary context for thinking about neural networks as creative tools.

Part.3 Neural Networks: A New Innovation Paradigm

A core point about computational media is that it requires us to shift our focus from the final output (song, image, video, text) to systems and processes. In the case of neural networks, this means we need to think of them as programmable media generation engines, rather than simply tools for generating a particular type of media. Through this lens, I see the answer to the question above about where the value of human creativity and craftsmanship lies: it lies in the design of the training process and model architecture — this is what I mean by “at the programming level.”

xhairymutantx is a collaboration between Holly Herndon and Mat Dryhurt - the model was trained strictly on Hollys photos, and will generate photos inspired by her appearance regardless of the input prompt.

If you think of neural networks as an attempt to abstract human cognitive functions in software, then it becomes clear that training and designing the model is tantamount to teaching it how to think.

You can imagine giving all your friends a single instruction (“prompt”): “Recall a childhood memory.” Everyone’s response will obviously be different, because what they generate will depend on their personal background and imagination (i.e., the “training data”). After multiple prompts, you might also find that some friends consistently generate more beautiful or creative responses, perhaps even displaying a particular personal style. So what if you could do this exercise with every human brain that has ever lived? What if you could single out particularly unique human brains, like Picasso’s or Kanye West’s?

This is essentially the creative superpower that neural networks give us - the ability to use other minds as creative tools. What I think is really compelling here is not the specific output of a model, but the opportunity to creatively program a software brain that can produce unique ideas and unique works.

Arcade.ai is a “tip to product” marketplace that allows users to design their own jewelry products. They have specially tuned a model to generate high-fidelity jewelry images, and these images only use materials that the end user can use for manufacturing.

Furthering the idea that “the system is more important than the output,” another notable feature of interacting with neural networks is participating in a continuous feedback loop of prompts and responses — an experience I’ve heard some people compare to the feedback loop of reading and writing. I’ve personally noticed that I rarely end my interaction with the model after sending a prompt to it and receiving an output. Almost every interaction with the model brings me into this interactive feedback loop, allowing me to iterate, reflect, and explore. This may seem subtle, but it is a key to understanding the type of media generated by neural networks:

3.1 Agent-based media

I briefly touched on this concept in a previous post, and the core idea is pretty simple - here the model simulates the role of some kind of human companion, interacting with us through text conversations, while it can also understand and respond to other forms of media. We also see some models here that are able to take actions on behalf of others or itself (such as performing financial transactions). Typical examples include chatbots, AI companions, NPCs (non-player characters) in games, or any other anthropomorphic user experience. For example, Andy Ayreys creative experiment Infinite Backrooms, which is to set up multiple Claude instances to communicate without human intervention, is a particularly interesting case.

3.2 Real-time Game Engine

Here, the model simulates a game engine (or more specifically, a game state transition function) that receives user actions in the game as prompts and generates the next frame of response output in the game . If the speed is fast enough, this experience should be similar to navigating in a virtual world that is rendered in real time based on your actions. This is the ultimate expression of immersive and interactive media.

DOOM game frames are generated by GameNGen, a game engine driven entirely by neural models, as described in Google’s paper “Diffusion Models as Real-Time Game Engines”.

3.3 Multiverse Generator

In this scenario, the model acts as a creative oracle, helping us expand on the original idea by generating infinite variations , each of which can be further explored and manipulated. This allows us to start from any idea or concept and explore the possibility space around it. For example, AI Dungeon (a text-based choose your own adventure game) is a great example of this.

User interface view of Loom , a tree-based writing interface for language models like Chat GPT, courtesy of @repligate.

3.4 Latent Space as a Creative Tool

I am increasingly convinced that this idea of “exploring the possibility space” is central to understanding neural networks as a creative tool and medium . As I use tools like Midjourney, Suno, Websim, Claude, etc., I’ve noticed that most of my workflows can be boiled down to the following pattern:

Prompt → Generate a variant of a specific output → Use the variant as a prompt for a new output → Generate a specific variant again → Repeat this cycle...

For example, when using the AI-powered music generation tool Suno, I typically provide the model with a 60-second solo singing example and some written lyrics as a prompt. I then use the Cover function to generate an output, then generate more than 10 variations of that output, and pick out the parts I like from these variations as input for further prompts.

Essentially, I’m exploring the space of possibilities around my own examples in the model’s latent space — discovering variations on my original work that I wouldn’t have come up with on my own, or that I couldn’t have done in a reasonable amount of time. I think this approach unlocks an unprecedented process of rapid prototyping and creative testing, and will lead to the emergence of “100x creators,” similar to the “AI-powered 100x engineers” discussed in the software space.

It became clear to me that latent space is a creative tool. Leveraging AI for creative production is not just about training powerful models, but also about designing interfaces that enable users to explore and manipulate these vast spaces of latent possibilities with greater precision and granularity.

Part.4 Consumer Behavior and Cultural Influence

Here are three predictions I have about how this technology will change consumer behavior and what new business opportunities it will create:

4.1 Will become a creative tool

Prompting — whether text-based, image-based, or other forms — is an interaction method that is gradually being embedded in more and more interfaces and experiences, bringing end-user creativity into areas that have never been explored in the past. Scott Belsky pointed out that the early era of GenAIs prompt-based text-to-image generation weakened creativity, while the era of controls has unleashed human creativity in unimaginable ways. Tools continue to evolve, but creative vision, precision, taste, and skills will be more important than ever. I agree with this view, and most prompts will eventually be abstracted into controls (controls: components with user interfaces), allowing users to operate without noticing. But more importantly, I think this trend fundamentally changes the way we think about interface design.

4.2 Corporate Media → User-Generated Media → Machine-Generated Media

The last major shift in media business models was the move away from corporate-generated media to entirely user-generated media. Now it looks like the next major consumer media business model will be built around the ubiquity of machine-generated media . However, it’s still unclear what the “winner” will look like. Will it be a generalized model like Midjourney? More specialized authoring tools? Social experiences built on top of these technologies? Or some more obscure third option?

Regardless, if you’re a founder or independent creator in today’s consumer media landscape, you may need to strategize about how you can leverage these tools to enhance value and drive growth for your business.

In addition, I think another area worth paying attention to is how to make AI-driven experiences more social and multi-user collaborative. Taking my personal experience as an example, most AI applications today seem very anti-social because you are mainly interacting with the model rather than with other people. There may be many opportunities and design spaces in this area , such as building human-centered collaborative creation experiences, or creating new ways for humans and robots to achieve more meaningful social interactions.

4.3 Impact on Intellectual Property

It’s not just the cost of creative production that is falling to zero, but especially the cost of aesthetic imitation. I can take a picture of a person’s outfit and feed it into Midjourney as a prompt to design a sofa in the same style. I can also do similar style transfers of that person’s voice, writing style, etc. What is the value and significance of intellectual property in this new paradigm?

I haven’t found the answers yet, but it’s clear that most of my previous assumptions and mental models no longer apply.

Part.5 The role and summary of cryptocurrency

If you’ve made it this far – thank you for your patience!

I’ll dive deeper into the implications of this for cryptocurrencies in future posts, but here’s a preview of a few areas I’ll be focusing on next:

• Opportunities for crypto companies to build around new media

  Exploring the potential at the intersection of on-chain marketplaces and machine-generated media.

• Crypto as an Incentive Layer for Intellectual Property

  Go beyond attribution and traceability, and think about building incentive mechanisms and networks around media.

• Encryption as a monetization and access control layer for media

  Especially in the field of user-generated software, rethink the architecture of the web; use minting as a business model for small models; and use NFT as the infrastructure for personal programs and user-generated software.

• Crypto as a social and economic coordination layer between humans and machines

  Supporting human and AI collaboration in identifying, funding, and solving a wide range of problems; exploring community owned and operated models.

Original link:

https://paragraph.xyz/@eclecticcapital.eth/neural-media

Author:natalie

*All content on the Coinspire platform is for reference only and does not constitute an offer or recommendation of any investment strategy. Any personal decision made based on the content of this article is the responsibility of the investor, and Coinspire is not responsible for any gains or losses arising therefrom. Investment is risky, so make decisions carefully!