The key to success in AI projects is to iterate quickly.

In the past, I’ve made the mistake of overengineering and overthinking a lot. Especially in the early phases of the project, it’s tempting to address all problems at once. But this approach leads to project delays, frustrating all stakeholders.

Over time, I adopted a different approach to running my projects. It is primarily based on the Lean Startup methodology.

Let’s dive right in.

Learn more and ship quicker with fast iterations

The Lean Startup methodology says entrepreneurs should focus on running many small iterations to find product-market fit. You build an early MVP based on your business model, evaluate it with prospective customers, and adjust your business model and/or product accordingly.

We can apply this to AI projects as well. In our context, the MVP is a complete ML pipeline, and an iteration equals an experiment.

This approach has many advantages:

• You will learn more in a shorter amount of time.

• You will avoid overthinking and over-engineering.

• You will be able to demo your current model at all times.

• You will not run the same experiment twice.

Three simple steps are required to enable running more experiments in less amount of time.

Let’s go through them one by one.

Step #1: Create a reproducible environment

Nothing creates more friction than having to install specific package versions over and over again, just to run your preprocessing notebook.

A reproducible environment will enable you to run experiments quickly without having to set up the required infrastructure over and over again.

What kind of environment you choose depends on the use case.

In most cases, a simple Python virtual environment will be sufficient. I prefer a combination of pyenv (for managing Python versions) and venv (for managing package versions) for most of my projects.

If you’re switching between platforms a lot, or have multiple people working on the same project, containerized environments like VSCode Dev Containers might be a more suitable choice.

Step #2: Establish an end-to-end pipeline early on

You should be able to go from raw data to performance metrics on your test set at all times during the project.

This way, you will always have up-to-date metrics to share with major project stakeholders.

Don’t get me wrong. This does not have to be a completely automated Kubeflow pipeline.

It can be as simple as a collection of notebooks and scripts, plus a configuration and readme file.

Typical pipelines in my projects look like this:

• An EDA notebook for examining large amounts of data and descriptive statistics

• A data preparation script that (optionally) downloads the raw data and converts it to the required format

• A model training script that takes the preprocessed data, trains a model, tracks the experiment, and stores the resulting artifacts in the artifact store

• An inference script applying the trained model to the test set

• A performance evaluation notebook that calculates all relevant metrics on the test set and enables qualitative evaluation

This is all that’s needed in the early stages of a project.

Step #3: Document your findings

I know, I know. Nobody wants to do documentation. But hear me out.

Documenting the main findings of your experiment will save you lots of time down the road.

You don’t need to write pages of text. Just summarize a few key points of each experiment for your future self (and your teammates):

• The primary experiment variables

• Link to the tracking run for this experiment (e.g., in Weights & Biases)

• Short description of the results (with some key visuals)

• Proposals for the next experiments based on your findings

You can do this in markdown pages in your repo, or a dedicated solution such as a Weights & Biases report. It only matters that you do it.

That's it.

If you focus on these three factors, you will run two iterations where you previously ran one.

As a result, your AI-powered product will improve faster.

And that's what we're all after, right? 😉

If you find value in these posts, consider forwarding them to your friends/colleagues (if you’re receiving the email), or subscribing to my newsletter (if you found this via the blog). Also, follow me on LinkedIn for daily tips on building AI-powered products.

I stumbled upon this tweet by Alex Ratner, the CEO of the data-centric AI platform Snorkel, a while ago.

Twitter tweet

tl;dr: Generative AI companies can launch their products quicker, but most high-value AI use cases are driven by predictive AI.

Especially the first point resonated with me, based on my recent experience.

While I’ve been working on predictive AI systems for years, I recently got the chance to work on an app powered by DreamBooth-customized Stable Diffusion models.

In fact, we were able to launch a working prototype fairly quickly, at least much quicker than we usually do, e.g., when working on visual inspection systems.

Naturally, I wanted to dive in deeper.

Here’s my attempt to compare predictive and generative AI systems from a technical perspective.

The comparison framework

I will compare both approaches along the typical tech stack for AI-powered applications.

Here’s what it looks like:

It’s largely inspired by this great blog post from Cohere’s Jay Alammar.

All AI companies rely on ML models which are trained using data and MLOps tooling.

Don’t get me wrong. I’m referring to MLOps tooling as the software which is required for training the models.

It can be as simple as a collection of Jupyter notebooks, and as complex as a KubeFlow pipeline for automatic retraining.

The model training is typically run on cloud platforms which allow for flexible computing.

Finally, trained models are exposed to the end user through the application layer, for example a web application.

Why is building predictive AI systems so hard?

As described in my previous post, predictive AI systems typically have very high performance requirements.

Let’s say we want to develop an AI-powered application for supporting the detection of lung tumors from chest CT scans.

Clinical application of such as system will require precision and recall metrics beyond 95%.

To achieve these metrics, we will need high-quality data for training our models which covers a lot of edge cases.

How do we accomplish that?

Sure, we can download a publicly accessible dataset like LIDC-IDRI and start from there.

But it won’t be enough. Why?

• The dataset contains a little over 1k scans. This won’t be enough to cover the natural variance in the population.

• The annotations are not optimal. Some tumors were detected by only one annotator and some by five annotators. In addition, the annotators often don’t agree on the severity of the tumor.

Thus, we need to build a data engine to derive additional edge cases and high-quality consensus annotations.

I will go into the details of a data engine in a future post.

For now, it’s sufficient to understand that it involves many iterations of sourcing data, deriving and evaluating labels, and performing detailed error analysis to inform the next steps.

On the model side, we will most likely use a proven neural network architecture (maybe even pre-trained weights), but it will require fine-tuning on our custom dataset.

At this point, you can see that the development process for predictive AI systems is experimentation-heavy.

It requires an iterative and more scientific approach.

As a result, predictive AI companies typically own large parts of their tech stack, typically everything besides compute (cloud platform) and some parts of their MLOps tooling (e.g., an experimentation platform).

Now, what about generative AI?

Why can generative AI apps launch so quickly?

As you may have noticed, generative AI apps are spawning left and right.

I wanted to post a recent market map here, but by the time you’re reading this, it is most likely outdated already.

Why is this the case?

As described in my previous post, generative AI systems are built on top of foundation models.

These foundation models can be open (e.g., Stable Diffusion on HuggingFace) or closed source (e.g., ChatGPT by OpenAI).

Let’s say we want to launch an assistant for creating meeting notes. It automatically creates notes for our Zoom meetings and lets users interact with the notes using a chatbot.

We can easily create this using the OpenAI API endpoints:

• We can transcribe the Zoom recording using the Whisper model.

• Summarize the meeting transcript using the GPT-4 endpoint.

• Our chatbot can be powered by the ChatGPT endpoint.

It is not required to train custom models or collect proprietary datasets for launching this app.

You’re mostly playing at the application layer. The development process is more engineering-heavy and requires traditional software engineering techniques.

Our main tasks will involve building a web application and developing middleware to keep track of the various API calls.

To conclude, generative AI apps are less vertically integrated at launch, because they only require the development of the application layer.

Tech stack of predictive vs. generative AI companies at launch (blue = owned, yellow = externally sourced, green = part owned, part externally sourced)

How will this play out over time?

At this point, you might ask yourself: How defensible is our product? What keeps other startups or incumbents like Microsoft or Zoom from replicating our meeting notes app?

Good question.

Right now, a superb user experience and moving quickly seem to be the only moats for our product, and they’re not defensible over time.

Incumbents are moving quickly these days, as exemplified by the daily product announcements from big tech companies.

What can we do about this? Here are a few options:

• Use the chatbot interactions and user feedback to train even better, custom models which create an enhanced user experience.

• Connect additional data sources to enrich the meeting notes, e.g., a company’s Notion workspace.

• Deploy our own transcription and summarization models (e.g., on Huggingface) to save costs and create a pricing moat.

All of these approaches require us to own deeper parts of the tech stack.

Prediction: The few big winners in generative AI outside of big tech companies will likely be vertically integrated.

To justify this prediction, let’s look at the most established startups in this space:

• Runway trains custom models for its video and image editing platform, going as far as publishing its own research on video generation models.

• Jasper for Business allows companies to fine-tune writing assistants to match their brand voice.

Let’s see how this prediction ages. At least a16z tends to agree with me.

What do you think about this topic? Please let me know by answering to this email or DM’ing me on LinkedIn or Twitter.

If you came across this article on my website, please consider subscribing to my newsletter to receive these kinds of insights straight in your inbox. You can find the archive of previous issues here.

Everybody is talking about Generative AI these days.

Even relatives and friends who usually don't care about my job suddenly ask me about things like ChatGPT and Midjourney.

However, generative models can't solve all problems we typically address with AI.

This is where predictive AI comes in.

In this issue, I want to address the differences between these fundamental approaches, and why they are both needed.

Let's dive in.

Note: If you're an AI expert, this issue might not contain any new information. Feel free to skip this one, or stick around if you want to review some fundamentals.

Predictive AI: The Veteran

Predictive AI is concerned with labeling a data point.

The label can be a categorical (as in classification tasks) or continuous value (as in regression tasks), or a collection of these values (e.g., a bounding box consisting of four coordinates and an object category).

We typically train predictive AI systems using supervised learning.

That means we pass data points that were previously labeled by human experts to a machine learning algorithm with the task of detecting patterns.

These patterns are stored in the form of models (e.g., deep neural networks or decision trees) and can be used to make predictions on unseen data points.

Examples of predictive AI systems I've worked on in the past are:

• Supporting lung cancer diagnosis by detecting tumors in chest CTs

• Detecting defects as part of automated quality inspection in manufacturing lines

• Fraud detection in credit card transactions

• Sales and utilization forecasting for public transport

What do all of these use cases have in common?

They try to automate or augment a process that was previously handled by a human.

In these scenarios, we have well-defined and very high performance requirements.

For tumor diagnosis and quality inspection, these are usually in the 95%+ range for metrics like precision and recall.

This can only be achieved by collecting large amounts of data, carefully labeled by experts like physicians.

That's because these systems are very hard to build in practice.

While predictive AI research has been around for a long time, it took off with the beginning of the deep learning era in 2012.

The emergence of deep learning allowed a larger variety of problems to be tackled, especially those involving unstructured data like text and images.

Now, let's contrast this with generative AI.

Generative AI: The New Kid on the Block

Instead of labeling data points, generative AI is concerned with creating entirely new data points.

I know this sounds abstract. Let me illustrate this with recent examples:

• ChatGPT generates text based on a given user query (usually also text-based)

• Midjourney generates images based on a text-based user prompt

• Whisper generates transcripts for audio files

In the past, these kinds of models were thought of as being harder to train than predictive AI systems.

Why?

Because the model needs to learn the underlying data distribution instead of focusing on the most important patterns (as in predictive AI).

Wait, but why are generative models so much more popular than their predictive counterparts?

Good question. To answer it, we have to look at two things: the training mechanism and applications of generative AI.

Let's start with the training mechanism.

Generative AI models are typically trained in a self-supervised fashion on publicly available data.

Take GPT-3 as an example, the underlying technology of the first ChatGPT version.

GPT-3 was trained by scraping all available text on the internet, and then training an autoregressive language model on this text corpus.

Autoregressive language models aim to predict the word (or word piece, to be precise) given the previous words.

Of course, I'm oversimplifying here, but you get the point.

This kind of training mechanism is much more scalable than supervised learning.

It has resulted in the creation of large-scale foundation models such as GPT-4 and Stable Diffusion which form the basis for most generative AI apps.

The primary limiting resource was (and still is) computing power.

Given the enormous progress and availability of accelerator chips, this resource is more available than human expertise (as required for predictive AI).

Now, let's look at the typical applications of generative AI.

As of today, we have two main use cases (read more in this post):

• Autocomplete for everything (e.g., writing assistants, code auto-completion)

• Answer engines (e.g., web search, chat-your-data apps)

What these use cases have in common is that they increase human productivity in tasks like writing, research, design, or programming.

These tasks are more subjective and typically have less well-defined performance requirements.

Thus, you can launch products with 80%+ accuracy levels because a human will always be in the loop.

Case in point: ChatGPT still hallucinates a lot of facts, but still achieved 100M users faster than applications before.

Bottom line: Why has Generative Overtaken Predictive AI?

To summarize, we have a technology that has a more readily available limiting factor (compute vs. human expertise) and lower performance requirements.

It's no wonder that generative AI achieved widespread adoption quicker than predictive AI.

Still, both approaches have their raison d'être for now.

As of today, predictive AI is more suitable for problems where proprietary data and deep human expertise are required.

As AI practitioners or executives, we need to understand when each approach is more suitable for solving a specific business problem.

What do you think about this topic? Please let me know by answering to this email or DM’ing me on LinkedIn or Twitter.

If you came across this article on my website, please consider subscribing to my newsletter to receive these kinds of insights straight in your inbox. You can find the archive of previous issues here.

In a previous issue, I introduced the four pillars of successful AI development: a clearly defined business case, radical end-user focus, a data-centric approach to model development, and appropriate operationalization.

I firmly believe that AI practitioners, including data scientists, should strive to be full-stack developers.

To me, being full-stack means covering the complete process of developing AI-powered applications, rather than simply knowing various technologies. While end-to-end is an alternative term, it doesn't make for a good newsletter name.

Being full-stack is especially important for practitioners in small AI teams, such as those in startups or enterprises just getting started in AI.

This may go against common practices in the tech industry, where professionals are often hired for specialized roles, such as ML researcher, ML engineer, data scientist, or AI product manager.

In this issue, I want to present the benefits of being a full-stack AI developer, the inherent trade-offs, and how to become one.

Let’s dive in.

Benefits of being a full-stack AI developer

In a brilliant post, Eugene Yan introduced the advantages of being an end-to-end data scientist. Here, I will keep it brief.

Increased context: With end-to-end knowledge, you will have a better understanding of the entire development process and the underlying business case. This leads to...

Decreased coordination overhead: You will require less syncing with your co-workers, giving you more time for deep work and quicker iteration.

Clear ownership and accountability: As the project owner, you will be responsible for the outcome. No one else can be blamed if something is not working.

Skill transferability: Building a skill set that can transfer well between organizations and clients will make you an effective individual contributor. It also prepares you to become a freelancer if that interests you.

Preparation for leadership roles: Understanding the end-to-end process of developing AI products is essential for leadership, whether you aim for tech lead or management roles.

Trade-offs you need to be aware of

This is all fun and games, but becoming full-stack inherently comes with some trade-offs.

Your skill set will be T-shaped, meaning that while you can develop basic skills for each of the four pillars, you won’t be an expert in all of them. As full-stack AI developers, we embrace breadth over depth of knowledge.

If you decide to pursue this path, there are a few truths that you must accept:

• Acknowledge that it's impossible to be an expert in everything.

• Realize that even specialists can't keep up with every new paper, given the rapid pace of developments in the field.

• Avoid getting sidetracked by the latest news and tools.

Instead, focus on building a strong foundation of skills in each of the four pillars of AI development. Then, go after the metagame.

Let’s go into more detail.

First, build the fundamentals

Mastering the fundamentals is key to success in any field. For example, NBA players spend hours every day working on their jump shots, while musicians practice their scales.

To become a full-stack AI developer, you need to develop basic skills for each of the four pillars. Here are some examples:

• For defining a clear business case around your AI project, you need to understand how businesses work, what the underlying value drivers of AI are, and how to conduct a feasibility study.

• Having an end-user focus requires you to know the fundamentals of prototyping and UX design.

• To develop ML models in a data-centric manner, you need to know how to handle and validate different forms of data and perform error analysis.

• Putting your model into production is only possible if you understand the basic building blocks of an AI-powered application.

I’m by no means an expert in any of these areas, but we will dive into each of them in upcoming issues, and I will point you to good resources that have helped me on my journey.

Then, go after the metagame

I first stumbled over this term in an article by Cedric Chin. He calls it “the game about the game”.

I know that’s abstract, but bear with me for a second.

In business, the metagame is about observing how the rules and best practices of an industry change over time, and acting accordingly to stay ahead.

In a fast-moving field like AI, playing the metagame means applying proven mental models to distill the signal from the noise.

In essence, this allows you to look beyond the hype and identify:

1. the sources of actual business value in new technologies.

2. the key trends that will shape the industry for years to come.

Let me give you an example of each.

• Generative AI is hyped right now. New research papers and tools pop up seemingly every hour, promising to make you more productive. However, all of the current tools revolve around two use cases: autocomplete for everything and answer engines. Now that you are aware of that you can ask yourself: where can I build around these use cases to create business value for me or my organization?

• Most research papers increase the state-of-the-art on some random benchmark by 0.1%. You probably will not derive business value from these incremental improvements. But every once in a while, trends pop up that will have a massive impact. The Transformer paper sparked an entire industry around NLP tooling, e.g., Huggingface. Currently, we’re shifting from a model-centric to a data-centric way of approaching AI problems. Being aware of these overarching trends allows you to strategically extend your skill set.

My goal with this newsletter is to share my journey of becoming a full-stack AI developer. Thus, upcoming issues will be related to both the fundamentals and the metagame of AI development.

That’s it for this week’s edition of the Full Stack AI newsletter.

See you next week!

If you like what you see, please spread the word by forwarding this message to your friends and colleagues.

Let me know your thoughts by replying to this email or hitting me up on Twitter or LinkedIn.

If you came across this article on my blog, please consider subscribing to my newsletter in which I share insights like these every week.

I’ve worked on AI projects in both academia and practice.

There are significant differences between these two contexts. In academia, the focus is often on developing new models and algorithms, with less emphasis on deploying them in real-world scenarios. In practice, however, the focus is on building AI systems that can solve real-world problems and deliver business value.

In my experience, the success of AI projects in practice depends on four key pillars. Whether you are a product manager or an ML engineer, everybody on the project team needs to have an understanding of these factors. It is even more critical if you are a full-stack AI practitioner, who has responsibility for all project phases and parts of the tech stack.

Now, let’s dive in.

Pillar #1: Well-Defined Business Case

To ensure the success of an AI project, it's crucial to align the expectations of developers and all stakeholders by defining the underlying business case. This process should be executed for each AI project, whether it's an AI-powered product or feature.

At a high level, defining the business involves the following steps:

1. Generate potential problems that can be solved (better) with AI.

2. Conduct a feasibility study (research only, no coding!) to draft an AI-based solution for the problem.

3. Determine the ROI for each project based on the desired change in the primary business metric. It's vital to assign monetary values here.

4. Select the project with the highest ROI and complete the ML canvas for it. This forces you to specify both the business and technical aspects of the project.

By the end of this process, all stakeholders should have a shared understanding of what to expect in the given project.

Pillar #2: End-User Focus

In my opinion, a lack of end-user focus is the main failure point in AI projects, period. I make this mistake all the time.

AI as a technology has unique characteristics which need to be accounted for in UX design:

• Probabilistic nature: AI is primarily built by training statistical machine learning algorithms on noisy and biased datasets. This can result in error-prone behavior and undesirable outputs.

• Complexity: Due to huge datasets and model sizes, understanding model outputs is very difficult. This creates additional cognitive effort for the user.

• Capabilities: AI can take over tasks that were previously reserved for humans (e.g., generate images). Thus, new design patterns for user interfaces might emerge.

To ensure that the AI user experience is properly designed, it's important to start the design process early by conducting end-user studies with low-fidelity prototypes. These prototypes could be mockups or simple web apps. The AI can be mocked by fixed rules or a naive baseline model. By doing this, AI practitioners can ensure that the system is properly designed to meet the needs of users and that any potential issues are identified early in the process.

Pillar #3: Data-Centric Approach

Data-centric AI is a term coined by AI pioneer Andrew Ng. The discipline aims to "systematically engineer data to build an AI system". Therefore, in all phases of an AI project, from prototype to deployment, you should prioritize data above all else. The ultimate goal is to build a machine that can learn continuously, even when the engineers are on vacation.

Data-centric AI revolves around several key tasks:

• Data labeling: Focus on creating diverse and high-quality datasets by evaluating label quality.

• Error analysis: Identify common failure cases of your model and update your datasets accordingly.

• Data monitoring: Continuously monitor the quality and distribution of incoming data.

Spend most of your time creating high-quality datasets, and reduce the time spent on model tuning. Use AutoML solutions to derive baselines quickly since ML models have become commoditized these days.

Pillar #4: Appropriate Operationalization

Once you have built a prototype based on a well-performing model and a user-friendly interface, it is time to put your product into production.

Fortunately, we can rely on proven software engineering principles. The field of MLOps is all about applying reproducibility, automation, versioning, and collaboration to ML-specific artifacts, namely data, and models. While best practices are still emerging, there is a wealth of information available to build on.

Warning: As engineers, we often fall into the trap of over-engineering. The amount of required automation for an AI project depends heavily on the use case. If your input data distribution does not change often, you probably do not need an automated retraining pipeline. Manual retraining once a month will most likely suffice.

As a general guideline, it is best to focus primarily on setting up appropriate monitoring. This will enable you to detect issues quickly and collect additional data for improving your product.

Conclusion

If you focus on these four pillars, you will drastically increase your chances of success when building AI-powered products.

We will dive deeper into each of these pillars in the upcoming issues.

If you like what you see, please spread the word by forwarding this message to your friends and colleagues. If you came across this article on my blog, please consider subscribing to my newsletter in which I share insights like these every week. Make sure to follow me on Twitter and LinkedIn where I post short-form content daily.

The Generative AI industry is on fire:

• OpenAI's ChatGPT reached 100 million users within two months, faster than any other app before.

• Generative AI companies received over $2.5 billion in funding in 2022.

• Stable Diffusion models have been downloaded from Huggingface almost 10 million times in the last month alone.

However, what is the long-term business value that can be derived from these emerging technologies? Let's explore both current and future use cases.

Today

Generative AI tools can currently be grouped into two main use cases.

Use case 1: Autocomplete for everything

This term was coined by Noah Smith to describe the potential of generative AI models in supporting us with creative endeavors. These models allow humans to enter their idea in the form of a prompt, and the AI system generates a first draft based on the prompt. Humans can then edit and refine the output, resulting in a much more polished final product. In fact, I used this technique to write this post using Notion AI. This approach has already been applied in several use cases.

• Jasper.ai is an AI writing assistant that can help with creative writing,

• Github Copilot is a tool that uses large language models to assist with programming.

• Text-to-image models like Stable Diffusion or Midjourney can be used by creatives to prototype assets quickly, e.g., app icons or interior design.

Use case 2: Answer engines

Although GPT-3 has been available for some time now, it was ChatGPT that popularized generative question answering. This technology has the potential to revolutionize search engines, as demonstrated by Bing, You.com, and Neeva, who have incorporated it into their systems. Instead of receiving links based on the user’s query, these enhanced search engines return answers.

Another exciting use of similar technology is called "chat-your-data". These systems enable users to ask custom chatbots questions about a given piece of data. The data can take any form that can be represented as text, such as meeting notes, software code, or podcast transcripts. The user receives an answer directly, without needing to scan the entire text. Example use cases are:

• Context is a tool that allows users to search through YouTube playlists (e.g. of popular podcasts like The Tim Ferriss Show) based on keywords or topics of interest.

• Elicit is another tool that utilizes GPT-3 technology to summarize research findings based on the user's interests. This tool can save researchers time and effort by automating the literature review process.

In the (near) future

With the available technology and current developments in the field, I predict that there will be at least two main use cases for even more powerful generative AI tools in the not-so-distant future.

Use case 1: Personal assistant

Hypothesis: Future personal assistants will assist users with a wide range of tasks, from answering simple questions to performing complex actions.

Creating Ironman-like personal assistants will require solving the following problems:

The next generation of chatbots has to be multimodal by nature, capable of understanding various forms of input, such as speech, text, images, and data tables. They will also be able to choose the most appropriate representation of results for a given task. Initial research in this field has experimented with multimodal chatbots, and YouChat is already capable of outputting a handful of formats depending on the input query.

Personal assistants will need to rely heavily on external knowledge retrieval to output faithful information. Knowledge can be retrieved from sources such as knowledge graphs (e.g., biomedical ontologies), tools (e.g., Wolfram Alpha), and APIs (e.g., Crunchbase).Models like RETRO from DeepMind can already incorporate external knowledge into their outputs, and initial experiments show that LLMs can teach themselves how to use external tools. You can test this out for yourself in this cool demo by James Weaver.

Moreover, personal assistants must be extensible and capable of providing end-to-end process support. For example, they should be able to set up background processes to monitor prices, and they perform actions such as booking flights or purchasing items without the user having to interface with a website. Some tools on the market are already going in this direction. Although still in private beta, Fixie.ai seems to position itself as a platform for building agents capable of solving complex problems. Additionally, HTTPie’s AI could potentially be useful for interfacing with APIs.

Use case 2: Content creator

Hypothesis: Powerful generative AI models beyond just image generation will speed up all content creation and design processes.

However, in order to fully realize the potential of generative AI in this area, the following technical challenges must be addressed:

Creating long-form multimedia content is essential for supporting creatives in a wide range of fields. Whether you are working on a film, designing a video game, or writing a book, modeling temporal dependencies is necessary to generate reasonable outputs. Existing tools that support with creating long-ish content include Tome (for presentations), Runway (for video editing), and Luma (for 3D rendering).

Content creation systems will not just be limited to text-to-X functionality, but will also provide intuitive interaction features. Creators will be able to input rough sketches in different formats instead of just text prompts. Furthermore, the AI outputs will be editable through text-based interfaces as well as gestures. Recent research already provides better guidance approaches for text-to-image models, such as Stable Diffusion. For example, ControlNet (demo) allows users to control Stable Diffusion generation better by providing sketches or example images. In addition, tools like remove.bg have established what good UX related to image editing should look like.

In addition, future content will be easily customizable. For example, it will be possible to automatically personalize content for the user. Children's books will feature the reader and their loved ones, and it will be possible to insert oneself into any video game with the click of a button. Personalization is already being used in text-to-image models. The most established personalization method is DreamBooth (which is used to generate avatars in Lensa). Additional approaches, such as Custom Diffusion by Adobe Research or LoRA, are being developed to overcome DreamBooth's limitations, such as large checkpoint file sizes and limited combination possibilities. Personalization features also seem to be part of Runway's impressive Gen-1 text-to-video model.

Conclusion

Beyond the current hype around models like ChatGPT, the potential for generative AI is real. Even contemporary approaches allow professionals to become more productive by supporting tasks like writing or programming. It's difficult to say how long it will take to realize the outlined future use cases. However, it should be clear that chatbots and other virtual assistants might change the way we interact with the web. Thus, organizations and individuals should prepare for the foreseeable future by asking themselves the following questions:

• How can we use the existing tooling today to increase our productivity?

• Which of our processes will most strongly be influenced by the future developments in this field?

• How will users interact with our services in a world where the classical search engine-to-website route is not the main entrypoint to our business?