Meta and Salesforce are shaking up the business world with new AI agents built specifically for small businesses. These aren't just fancy chatbots - they're tools that can actually get work done.

Meta's rolling out AI customer service agents that work across Instagram, Facebook, and WhatsApp. These agents can handle returns, suggest products, and even process purchases in real-time. What's cool is they tap into your existing business history on Meta platforms to give relevant responses directly in ads or messages.

Meanwhile, Salesforce has launched AgentExchange, a marketplace for AI business agents with over 200 partners including Google Cloud and DocuSign. The best part? You don't need to be a tech wizard to use these pre-packaged solutions.

Industry experts are calling 2025 "the year of the agent" because these AI tools don't just provide information - they actually do tasks for you, boosting productivity. They're starting with customer service since that's where businesses can see immediate results.

The advice for small business owners? Jump on this trend and think about how these tools could solve your specific challenges. We're moving from AI assistants to AI agents, and it's going to change how we run businesses.

Sources: Forbes

OpenAI is jumping into the high-end AI market with a jaw-dropping price tag. According to The Information, they're planning to launch specialized "agent" products with one premium option costing a whopping $20,000 per month!

This top-tier agent will target "PhD-level research" and goes way beyond what consumer AI can do. The eye-popping price reflects both the advanced capabilities and OpenAI's financial situation - they reportedly lost about $5 billion last year running their services.

Meanwhile, there's drama in the background. A judge recently denied Elon Musk's attempt to stop OpenAI's transition to a for-profit company, though the court might still hold a trial on whether this move violates OpenAI's original mission. Musk claims Sam Altman and the team abandoned their nonprofit roots.

In related news, Scale AI (another big AI player) is under investigation by the Labor Department for potential issues with worker classification and compensation.

These stories highlight the growing pains AI companies face as they try to balance innovation with sustainable business models. Will enterprises actually pay these premium prices? That's the real test of how valuable these advanced AI capabilities really are.

That’s it for today.

Catch you on the next one.

Meet Suno

The tag line for Suno ‘Make any song you can imagine’, is a bold one, but right now it’s on point.

Hearing a song that was generated by Suno goes up there with the holy shit moments of ChatGPT for text and Midjourney for images.

The music that it generates is truly incredible.

Check one out here or if you want to create some songs for yourself, head to https://www.suno.ai/, create an account and start making music.

Simply describe the song and away you go.

If you want to get even more into the weeds you can even toggle on advanced mode and enter lyrics, the title and style of music.

If you have some ideas and want to chat about how you could build them I’d love to hear from you.

Meet EVI by Hume.

X (Twitter) went crazy recently when Hume released their latest product, Evi.

Evi stands for ‘Empathic Voice Interface’ and what Evi allows you to do is bring emotional intelligence to any app or site your building.

One you start speaking with Evi, it will not only understand what you are saying but also how you are saying it!

Excited and happy? Evi will notice that and respond with a happy tone in response.

On the flip side of that, if you’re down and feeling sad, Evi can take that into account in it’s response.

This seems like something simple but has been very much lacking until now.

Absolutely crazy.

Let’s imagine we have an idea for a product that is an assistant in some form. It could be emotional, medical or even motivational. You wouldn’t want it to respond with the same deadpan tone every time would you.

This is where Evi shines.

If we get into some of the more technical parts now, Evi is an API powered by Humes empathic language model (eLLM) and it allows for highly optimised human-AI interacton.

Heres’s a few of it’s features:

• Universal voice interface - a single API for transcription, frontier LLMs, and text-to-speech.

• End-of-turn detection - uses your tone of voice for state-of-the-art end-of-turn detection, eliminating awkward overlaps.

• Interruptibility - stops speaking when interrupted and starts listening, just like a human.

• Responds to expression - understands the natural ups and downs in pitch & tone used to convey meaning beyond words.

• Expressive TTS - generates the right tone of voice to respond with natural, expressive speech.

• Aligned with your application - learns from users' reactions to self-improve by optimizing for happiness and satisfaction.

Here’s another thing to consider Tweeted by Alan the founder of Hume.

Twitter tweet

In absolutely no time at all, the apps you use and websites you visit will be tailored in real time depending on your reaction to what you are seeing and reading.

Each time you visit they will get even more tailored according to your preferences.

You’ll see exactly what you want, how you want to see it.

A fully custom experience for every user.

If you want to try EVI you do that right now, by clicking here.

That’s it for this one, catch you later.

Tom

Here’s a few things that have caught our eye over the last few days.

We’ve got advanced munitions, societal attitudes towards AI, automation in business, lip-syncing technologies and predictions from Ray on AI's future capabilities.

Let’s tuck into it.

Anduril - ALTIUS-700M

Analysis: Anduril Industries' recent successful test of the ALTIUS-700M, the largest loitering munition, highlighting its unmatched payload capacity, best-in-class loitering time, and advanced autonomy. Combined with Lattice, it’s AI software layer, Anduril is quickly changing the battlefield.

Insight: This shows the increasing role of autonomous systems in modern warfare, emphasizing efficiency and precision.

Opportunities: There's potential for further integration of autonomous systems in defence strategies, opening avenues for innovation in unmanned weaponry.

Link

Global Attitudes Towards AI

Analysis: A significant disparity exists between Chinese, Asian and American attitudes towards AI, with China showing more positivity.

Insight: Cultural differences may influence the adoption and development of AI technologies globally.

Opportunities: Understanding these attitudes can guide global AI policy and development strategies, fostering international collaboration.

Link

Automating Calls with Retell

Analysis: The collaboration between BrainbaseHQ and Retellai introduces an automated calling CRM, showcasing AI's role in streamlining business processes.

Insight: Automation continues to transform industries by enhancing efficiency and data collection methods across all job functions.

Opportunities: Businesses across sectors can leverage such AI tools to optimize operations and customer engagement.

Link

Runway ML New Features

Analysis: The announcement of lip-syncing features in Runwayml highlights more moves into making AI-generated content more realistic. Runway is leading the way currently when it comes to AI gen video and love to see them moving so fast.

Insight: Enhancing the realism of AI-generated media opens up creative possibilities and challenges our perception of digital communication.

Opportunities: Content creators can explore entirely new forms of storytelling and audience engagement through awesome new technologies.

Link

Kurzweil: AI Smarter Than Humans by 2029

Analysis: Kurzweil's prediction that AI will match human intelligence by 2029 reflects ongoing debates about AI's capabilities and future impact.

Insight: Exponential growth in computing power fuels these advancements, challenging our understanding of intelligence and creativity.

Opportunities: Preparing for this eventuality involves ethical considerations, policy planning, and fostering human-AI collaboration.

Link

The AI powered future is being pulled towards us faster than we could ever imagine and will transform every industry, job and business.

Until the next one.

Tom

The last 24 hours felt like 100 years of technological advancements. So crazy!

Today we’ll be covering stuff like Humanoid Robots, Solar Panels that work at night, AI Software Engineers and a few more things that you might be interested in if you fancy yourself as a tip of the spear type.

Enjoy.

Figure installs OpenAI

Analysis: This post highlights the integration of OpenAI models with Figure 01 for enhanced conversational abilities, showcasing the synergy between high-level visual/language intelligence and low-level dexterous actions.

Key Takeaway: The advancement signifies a leap towards more embodied AI applications in real-world scenarios.

Read more

Plus, Corey, the technical lead on the Figure project does a longer deepdive here.

Reflect Orbital, Solar at Night

Analysis: A new company called Reflect have demonstrated the ability to bounce sunlight onto solar panels using big mirrors opening up the possibility for solar power generation even at night.

Key Takeaway: If achievable this presents the opportunity for solar power to reach entirely new benchmarks for energy generation.

Read More

Claude 3's Animation Capabilities

Analysis: This demonstrates Claude 3's ability to generate animation code for educational purposes, emphasizing the model's utility in creative coding and its potential as an educational tool.

Key Takeaway: AI's role in simplifying complex coding tasks and enhancing learning experiences is increasingly significant.

Read more

Google DeepMind's SIMA

Analysis: Introduces SIMA, a generalist AI agent capable of understanding and acting on natural-language instructions within 3D environments, marking a step forward in creating versatile AI agents.

Key Takeaway: Language understanding combined with environmental interaction opens new avenues for AI applications in gaming and virtual simulations.

Read more

a16z Consumer GenAI Apps Ranking

Analysis: This index from a16z gives a great overview of the top consumer-facing AI applications, highlighting rapid progress, new entrants, and the diversification of AI into music, productivity, and companionship!

Key Takeaway: The consumer AI landscape is evolving quickly, with mobile platforms unlocking unique use cases and global participation driving innovation.

Read more

Gemini AI Video Analysis

Analysis: Shows the potential of AI to analyze and reason over video content, using Gemini 1.5's ability to identify hazards in traffic videos as an example.

Key Takeaway: Advances in video analysis could lead to significant improvements in surveillance, safety, and content management.

Read more

Shoutout to for the find here.

Midjourney's Character Reference

Analysis: Showcases Midjourney's capability to maintain character consistency across different contexts, demonstrating AI's growing influence in creative fields.

Key Takeaway: AI is not only automating tasks but also enabling new forms of artistic expression and storytelling.

Read more

Claude 3 Haiku

Analysis: Announces the release of Claude 3 Haiku, emphasizing its speed, affordability, and enterprise-grade security for analyzing large datasets.

Key Takeaway: The development underscores the importance of efficient, secure AI solutions for business applications, from customer support to legal analysis.

Read more

Introducing Devin: The First AI Software Engineer

Analysis: The post announces Devin as a pioneering AI capable of performing software engineering tasks autonomously. Devin has demonstrated remarkable abilities by setting new benchmarks on the SWE-Bench coding benchmark, passing practical engineering interviews, and completing jobs on Upwork. Its proficiency in solving GitHub issues found in open-source projects significantly surpasses previous models.

Key Takeaway: Devin represents a major leap in AI's application to software engineering, showcasing the potential for AI agents to contribute meaningfully to real-world programming and dev tasks.

Read more

Each of these individual things reflects how fast the AI landscape is changing, from technical innovations and application-specific advancements to broader trends in consumer adoption and into the enterprise.

Together, they give a glimpse into a future where AI's impact is pervasive, touching every aspect of our lives, from how we learn and create to how businesses engage with consumers and think bigger.

Catch you on the next one.

p.s if you liked this I’d love it if you shared it with a friend.

Thank you for reading Shiny. This post is public so feel free to share it.

Most of us have talked to a chatbot or virtual assistant. We did this on our phones, smart speakers, or a terrible automated customer service system. The usual flow is you ask them to do something for you. But, they either don’t understand you or it cuts you off half way.

Well the good news is that this is all about to change.

Thanks to the new AI companies, who spend tens of billions on NVIDIA GPUs, these are no longer bad.

In this post, you’ll learn about some of the players in making the next Conversation Agents. You'll also learn how you might encounter them in your day.

Also, you might pick up a couple of tips on how to create your own small army of smart digital agents. They'll help you do whatever you want.

Before we tuck in, watch this short demo from Retell to frame this entire conversation.

Now that your brain has exploded.

What is a Conversational Agent?

In simple terms, a Conversational Agent is a computer program. It uses artificial intelligence to simulate human-like conversations. It does this through text or speech-based interactions.

For example, instead of you calling a restaurant to book a table, a Conversational Agent would do it for you. It would act as a digital assistant on your phone or echo. It would help you with tasks like scheduling appointments or ordering groceries.

It either always listens out for a command in a Siri/Alexa type way or is triggered by a key press or tap. You would then ask your agent to do something. You could type or speak. It would respond like a human and do the task.

Before you ask, yes you can give it any voice you want — Morgan Freeman is definitely on the table here.

Taking this one step further. A Conversation Agent could also be a Sales Person, Customer Service Rep, Consultant, or Mental Health companion. It could even be an all knowing assistant like JARVIS from Iron Man.

These use-cases are great.

They are direct connections to OpenAI and GPT4. They have all of human knowledge at their digital fingertips. They are only restricted by how you set them up.

Want to make a digital property investment advisor. It will work 24/7 and have 1000s of 1-on-1 conversations at once. Sure why not.

This may seem farfetched to those learning about these for the first time. Historically, few things were more annoying than speaking with a bad bot. Super annoying bordering on anger inducing. However, there were good reasons for this from a technical point of view. Some of which have only recently been solved.

Take Latency for example. The delay between the user finishing their sentence to getting a response back was too high. Next the human element. Digital Voices actually weren’t that good. They lacked emotion and we instantly knew we were talking to a bot. A dumb one at that. And finally, as mentioned before, the lack of context and understanding are big issues. The company only has a few account details on record.

This has now all changed.

Where are we now?

Large Language Models give us easy access to human knowledge through an API. With teraflops of processing power available, latency is no longer a problem. Natural language processing has improved, and new machine learning techniques make Conversational Agents more human-like by understanding and responding to user requests contextually.

Add to the mix Eleven Labs' high-quality digital synthetic voices. They can convey emotion and intonation. This gives the Conversational Agents a more natural feel. This mix is a recipe for one of the most transformative technologies of our generation.

Next up we have something called a context window.

A context window is the information a Conversational Agent remembers while working. It's like the agent's short-term memory. This helps the agent understand and answer many questions from the user quickly.

Until recently this was a problem. But, a mix of approaches, including RAG, has solved it. They made bigger context windows a thing, making these agents very powerful.

For example, compare an agent that can only recall one past conversation at a time while holding a conversation. Compare it to an agent that remembers every conversation with a user and can recall it at any moment.

No contest.

Add GPT4 or similar models as a base layer to create a powerful Conversational Agent. This agent can perform various roles more effectively than a human. It can handle multiple tasks simultaneously and maintain consistency.

Emotional understanding

This goes into a more controversial area. Technology can understand emotions in both directions between a user and a Conversational Agent. These agents can sense and react to user emotions and show emotional understanding in their responses. This makes them more human-like and relatable.

Typically this tech has been semi-limited to voice but startups like Hume.ai have brought this into facial recognition also.

Humans have a great ability to pick up on micro-expressions when talking with another human. Historically, AI's did not. This has also now changed.

The Conversational Agents now roam the internet and weave into our lives. They will not only be more effective, but they will adapt to human emotion too.

AI that read human emotion are banned under the AI act in the EU. So, if you’re building something here, note that. But Conversational Agents are here to stay.

Now onto some people you should look at in this space.

Companies to watch

Here's a list of some of the companies that might worth watching in this space.

• Retell - Conversational Voice API for Your LLM

• Sindarin - Deploy lifelike conversational speech AI in your business.

• Elto - Elto is a highly authentic, low-latency live voice AI that can make phone calls up to 1 hour long and execute follow-up tasks.

• Bland - Bland is the infrastructure for building AI phone calling applications at scale.

• Vee - An intelligent consultant people enjoy talking to.

• Eleven Labs - Perfect synthetic voice generation and voice cloning.

So, what will you build?

If you’re starting a company and have an interesting application or technology in this space, we’d love to talk to you at Interface. We invest in pre-seed and seed stage startups.

Drop me an email at tom@interfacecap.com.

Or if you’re just exploring the world of AI and want to learn through emersion and a peer group then join us inside of the Shiny Community by joining here.

The date is the 25th of February 2024 and yesterday everyones favourite answer engine, Perplexity launched a new product called Discovery Daily, a daily podcast that covers the most popular stories across various topics such as technology, finance, and entertainment.

It's ace and it's about to change how the podcasting landscape from here on out.

So here's the thing, the entire thing is created by AI and in this post I’ll talk about why I think it’s a watershed moment for podcasts.

First lets do a knowledge check for those of you just getting up to speed with the whole AI thing by running through what’s involved here.

So…let's get into it.

What is Perplexity?

Perplexity is like a Google but made for the AI age. Rather than doing a search and getting 10 blue links plus a page full of sponsored content, you just get answers.

Ask it a question and it’ll use AI to synthesis tons of different sources and just return the answer to you along with a list of sources that it used. Simple (on the surface).

Once they had this nailed, Perplexity launched the Discover tab. A collection of the most popular news stories on the internet refreshed in real time and synthesised with their language model to write a short summary of the story which expands if you click into it. Neat.

Now that they have the content for a podcast what else do they need? Audio.

Most people have heard of ChatGPT and OpenAI but less people have heard of TTS (Text-to-Speech) and ElevenLabs.

What is ElevenLabs?

Eleven Labs is the best in the game when it comes to TTS (text-to-speech) technology.

You, I or anyone else can head to their website, create an entirely synthetic voice or even clone our own voice then use that to generate audio of any text you want, in tons of different languages.

Add these two things together and you’ve got Discover Daily, the ultimate AI-powered podcast experience.

Here lies the watershed moment for podcasts. It takes media creation to a whole new level and showcases the potential of technology in flipping the model on it’s head.

Why I think it’s a bigger deal than people realise.

The vast majority of podcasts are focussed on regurgitating information while offering very low signal to noise. This leads to the trend of podcast summarisation through LLMs and the listeners flipping the podcast to 1.5-2x speed just to get through it hoping to pull out a couple of nuggets of information.

For any podcasts that fall into this category, unless you’re incredibly entertaining, AI can now likely make a version of your podcast that is just as good if not better and in almost real time.

Worth saying that you have zero chance of producing a podcast to the quality of Modern Wisdom using AI, but that is the 0.01% case. Everyone else should be worried.

It’s easy to be defensive about it but it’s the reality of what’s happening and it’s here today.

Here’s a few of the downstream impacts from this.

1. Speed

For news roundup podcasts that are relying on speed to market i.e. how quick can they get the information to their audience, their will likely soon be a pod in your niche that will be faster to market with potential equal if not better information.

1. Cost

Recording a podcast is expensive, not only through having good equipment to record and edit, but also in terms of time. Ai podcast will be able to spin out a great episode instantly at the cost of likely a couple of dollars. 100x cheaper at least.

1. Quality

Harsh truth here is most podcasts aren’t that great. If you compare two and one is more information dense, concise, more consistent and better structured that is probably going to beat the other in the rankings. On these metrics AI podcasts will win every time. What they can’t (currently) compete on is Creativity, Authenticity and Personality. If you’re creating a pod from now on, focus on these.

Tools like Wondercraft, who recently raised a 6m USD seed round make it super simple for people to create entirely AI powered podcasts in a few clicks. You’re literally competing against Palantir grade engineering talent here.

Now that you know the direction this is going, it might be time to decide the direction you're going to go.

Are you going to create an entire network of AI generated podcasts in every single sub-niche on the planet? Or are you going to focus on developing your personality, creativity and authenticity to create a more unique podcasting experience.

Note - if you want help creating an AI powered podcast, reach out.

One Idea

Two Thoughts

1.

The above might have made you stop and think.

Mainly because you may have tried/seen or heard about tools like GPT4 and thought how incredible they are. And you’d be right. But, when it comes to sensory data on how the world works, LLMs or Large Language Models like GPT4 are still at the starting blocks. The next breakthrough could come when more powerful models are being training on the data contained in YouTube videos and real world environments.

2.

We still have no idea how the brain works.

The biggest companies in the world are filling huge data-centers and packing them full with the most powerful GPUs money can buy and yet the small thing we each carry round in our skull, allows us to navigate the world better than multibillion dollar machines can (for now at least).

Three Consumables

1. 

The Expanding Universe of Generative Models (video) | Speakers: Yann LeCun, Nicholas Thompson, Kai-Fu Lee, Daphne Koller, Andrew Ng, Aidan Gomez

2.

Mark Zuckerberg explaining the reasoning behind the focus on building open source artificial general intelligence (instagram post)

3.

Market-map and paper covering 26 multi-modal large language models and their capabilities (post on x)

Hope you learned something this week,

Tom

p.s. here’s my favourite AI generated image of the week by Ishan.

It’s pretty common knowledge that our education system could be better. It still operates as it did many years ago.

I don't know how it happened, but I have an idea of what it will look like soon.

Before we dive in this post was inspired by watching and listening to this conversation between Alan Cowen CEO of Hume and Nathan on “The Cognitive Revolution” Podcast.

Highly recommend watching on YouTube also as they overlay Hume tech to showcase the realtime data overlay.

Let’s first compare two super simplistic ways of learning.

The first way of learning is more traditional. You sit in a room and get lectured on a topic you're not interested in. The lecture doesn't help you learn and isn't engaging.

This could be either reading out from a text. Or, it could be watching a non-interesting video that's out of date or doesn't engage you. You're also sat among 30 people. They all learn, at different speeds, and are at different knowledge levels.

Let's compare this to what a learning environment could look like. Now that AI is available to everybody.

If you're learning a topic, you want to be able to use the artificial intelligence to customise how you learn. It understands your exact knowledge levels. It also knows your preferences for the content you best learn from. This could involve reading text, watching videos, listening to audio, or looking at diagrams or animations.

You can also tailor how the AI delivers language to you. If you understand simple language, the AI will push you. If you're more advanced and can cope with complex topics, the AI will tailor its content to your learning style.

Now let's imagine that you wanted to learn about a new topic. Something epic, like Space Manufacturing.

The AI could start asking you questions to gauge your current knowledge. Then, it could serve up some images to see how you react to them. After that, it could show you a short video and play you a short audio clip. It may also speed up or slow down and change the type of language that it uses to deliver the messages.

At the same time, a tool like Hume AI can measure your facial expressions and emotions. It reads how you react to those messages. Then it feeds that back into the learning AI, which could then tailor the content it delivers to you in real time.

The loop repeats. 

AI serves new information. It reads your emotional reaction to it. Then, it tailors that information. Next, it learns again. Over time, it will refine your learning profile so it knows exactly how to deliver new topics to you. It also learns how to gauge your current and future potential understanding of the topic. Plus, it pushes you to learn more.

Let's compare this to a teacher teaching 30 people. They have different speeds and levels of understanding. This will lead to much better outcomes for everyone involved. In the future, AI will power personalised learning. It will be far superior to our current education system.

The great part about this?

We have all the technology and tools available to do this now.

If you’re building something in this space we’d love to talk to your at Interface.

G’day folks.

2024 has started as quickly as 2023 ended with artificial intelligence accelerating every industry.

I wanted to share a few links that I found the most interesting today.

The Global AI Index.

The Global AI Index by Tortoise is a comprehensive ranking of countries based on their level of investment, innovation, and implementation of artificial intelligence (AI). It is the first of its kind and has been iterated four times since its inception.

The Index uses 111 indicators, collected from 28 different public and private data sources, and 62 governments. These indicators are grouped into seven sub-pillars: Talent, Infrastructure, Operating Environment, Research, Development, Government Strategy, and Commercial. These sub-pillars are further grouped around three main pillars: Implementation, Innovation, and Investment.

Here'‘s a sample of some of the data but highly recommend taking the time to read.

Links

Here’s the quick links of everything I found interesting today.

1. Singapores Rise To Prosperity: The City-State Strategy (link)

2. Secret diplomacy between the US and China on AI safety (link)

3. Elon’s SpaceX and Starship Presentation (link)

4. Bill Gates talks to Sam Altman about AGI (link)

5. Norwegian robotics company 1X raises 100m (link)

6. AI discovers that not all fingerprints are unique (link)

7. Indian data firm order $1 billion worth of H100 GPUs from Nvidia (link)

8. South Korea are building a $470 billion chip making hub (link)

9. NASA and IBM openly release AI model for earth observation (link)

10. New paper details LLM jailbreaks with a 92% success rate (link)

11. Video shows threat and effectiveness of small kamikaze drones (link)

12. Tesla Optimus shows that it can now fold a t-shirt (link)

Catch you on the next one.

Tom

One thing that has always left me a little bit stumped is how bad Siri and Alexa are.

That might be a little bit strong as the technology powering both of these is running on hundreds of millions of devices but putting that aside, it really should be better.

I might be in the minority here, but having an always on voice assistant that helps you complete tasks instantly as soon as you think of it, is how I want my tech to work!

It’s SO close but it’s not quite there.

iPhone + Airpods = Cyborg?

Each day when we walk around we have a super-computer in our pockets in the shape of an iPhone and many of us also either have the best headphones ever created in the Airpods or a crazy smart watch in the Apple Watch.

Add all these things together and you have a system that can listen to what we say and do stuff.

This can be simple things like sending someone an SMS or as complicated as triggering a multi-step Apple Shortcut that ties together 10 apps, grabbing and formatting data and firing off web-hooks into other apps that you may use.

By the way, if you don’t know what Shortcuts are, take a look at this guy.

While this is great, there is one part of this chain that’s a little bit hit or miss.

The Voice bit.

More often than I’d like, Siri and Alexa don’t understand what you’re saying, it gives you a random answer that you don’t want or it cuts in when you haven’t finished speaking.

Before speaking to Brian Atwood of Sindarin, my small brain had no idea how hard of a problem this was. All I knew was that saying, “Hey Siri, <do “the thing"> and have it now work, just doesn’t feel like living in the future.

That said, this is changing quickly and I’m confident we’ll be there very soon.

What do we need to make this better?

There are a lot of people who are way smarter than me working on making voice assistants better and we’re getting a lot closer very quickly.

Here’s a simple list of the things we need to make voice assistants Jarvis level.

1. Understand what we are saying

2. Sound like a human

3. Be fast at responding

4. Have conversational flexibility

Now let’s walk through each part of how to make these better.

Understand what’s being said

For a voice assistant to be useful, it has to know what you’re saying. Sounds simple right. Well nope.

Until recently, this was pretty difficult.

Opening up a voice-to-text app and speaking into it, a large % of the time the app/assistant would recognise some words and not others or even finish the sentence before you’re ready. This leads to a bit of a word mess and a sentence that doesn’t make send and lacks context.

Luckily though recent advances in Machine Learning (too technical to explain in this short post) have made these a lot more accurate and WAY WAY faster.

Products are upgrading there capabilities here everyday but we can credit OpenAI Whisper model for making this better for everyone.

Try it here on Replicate.

Sound like a human

This also is something that may on the surface sound like a simple and obvious thing that need to be in place but it’s actually quite difficult to do.

Human voices are full of small and subtle variations in tone, pitch, speed and volume that uniquely convey the emotion and meaning we are trying to convey to the listener.

Doing this with an AI…tricky.

So far the startup that has solved this in the most complete way is called Eleven Labs. They’ve built an incredibly robust platform that not only allows anyone to create entirely synthetic voices and make them programmable but also to create a digital clone of your own voice in a matter of seconds.

There’s a couple of grades to this too so if you feel the want to go full professional, then the option is there too.

With this, we can bring these voices into our apps without them sounding like robots (counter intuitive I know)!

Another player here is Play HT if you want to try an alternative.

Not going to use this post as a tool comparison but may do something in the future.

Ok onto the next part…

Be fast at responding

Having a conversation with Alexa, Siri and, to some extent ChatGPT Voice, you’ll notice a bit of a delay between when you finish talking and when the response comes back.

The delay that’s present between question and reply is called Latency. Explained below (5 year old style).

To put Latency into context, the goal here is to make the assistants reply come back quickly enough to make the conversation seen natural, but also not so fast that it cuts in whenever you pause to think about the next word.

Here is where a problem arrises.

Natural conversations flow as we know how to read peoples expressions that give us an unconcious understanding of when we should speak and when to keep quiet. As you can imagine, this is hard to do when the AI can’t see your face (in this example at least).

This is the last 5% problem.

Many can get 95% of the way, as ChatGPT does with it’s voice assistant on mobile, but there’s just enough latency there to make you realise that you’re talking to an AI.

Now, we’re not far off these reaching the uncanny valley. Just try SmarterChild by Sindarin to get a taste of what’s coming down the pipe.

Conversational Flexibility

The year is 2024 and by this point all of us have been on the phone with our bank or other random service that has a pre-recorded bot ask you questions with the aim of getting you through to the right department.

More often than not, they’re not great.

Simple yes-no trees to put you through to the right department and not much flexibility to just communicate exactly what you need. But, give a response that doesn’t fit within the keywords that it’s been given then it’s a no go.

AI changes this by being able to communicate with you like a real person having a real conversation understanding the context and interpreting what it is you’re looking for.

Finally, let’s talk about the final piece of the puzzle..

Using Tools

Since their release, GPTs from OpenAI have enabled anyone to create AI powered assistants that can do not only the above but also use tools and integrate with other applications via Actions.

Powered by a nuanced understanding of user requests these give more accurate, context-aware responses.

This is where the idea of integrating Actions and APIs with GPTs becomes transformative. Such integration allows voice assistants not only to understand and respond to queries but also to execute tasks and interact with a wide range of services and applications.

Or consider a scenario where you ask your assistant to compile a report by pulling data from various business analytics tools, something achievable through API integrations. This level of functionality, combined with the conversational flexibility and rapid response times of GPT models, could indeed bring us closer to the Jarvis-like assistants many of us envision.

That’s it for today. Catch you on the next one.

Tom

— one technology —

1 || Digital Clones

A digital clone is a digital version of yourself trained on information that you have put out on the internet and is now available for people to interact with autonomously. It can be trained on blog posts, social media feeds, your face and even your own voice.

It’s now possible and they’re growing at an exponential rate.

— two products —

1 || Delphi

Delphi.ai is a cutting-edge self-serve platform that enables the creation of digital clones, allowing you to preserve and share your knowledge, personality, and experiences through your own AI-powered chatbot.

Your clone can interact with others, offering personalized advice and insights, effectively immortalizing your wisdom.

Each clone is hosted on it’s own URL so it’s available 24/7 to anyone.

Founder - Dara Ladjevardian

Try it here.

2 || Alt.inc + Alt Brain

Alt Inc., founded in November 2014, is a Tokyo-based company that specializes in developing Personal Artificial Intelligence (P.A.I.) through digital cloning technology. Their mission is to create an autonomous society where daily life generates lasting value. Alt Inc.'s platform Alt Brain digitizes users' intentions, information, and data, placing it on a decentralized neural network platform.

Recently Alt.inc became the first company to pay salaries to employees AI Clones!

Founder - Kazutaka Yonekura

Try it here.

— three resources —

1 || Future of Work: Digital Clones of yourself? (video)

This video contains an interview with Kazutaka Yonekura and they explore the world of Digital Clones and the inspiration behind Alt Inc.

2 || The Alan Turing Institute's Research Page (podcast)

As a leader in AI research, the Turing Institute's work covers a range of topics, including the ethical and societal implications of AI technologies like digital clones.

3 || Humans vs Clones debate (social)

Dara the Founder of Delphi and Raad founder of Intro talk about the value of digital clones vs people paying for a small amount of time from an expert.

Read the interaction here.

I hope you enjoyed this and catch you next week,

Tom

If you like this post I’d love nothing more than if you shared it with a friend.

If theres one thing that I want to do better at in my life it’s to be more consistent with my routine and output.

So in that vein I decided to try and create a habit of typing out what ever is on my mind in 30 minutes or less a few times per week. Taking note of what Nat Eliason said on X recently, no editing until the end and see where we end up.

Naturally, we’re going to be talking about AI.

Over the last 12 months we’ve gone through one of the most transformational changes in technology in a a very long time. No longer than a few hours go by that something new comes out on X that will without doubt effect multiple industries, thousands of jobs both positively and negatively.

For precisely this reason, there has also been a lot of fear at the capabilities of what AI can do and which jobs it will replace.

But as someone who is default techno-optimistic my feeling is that artificial intelligence will create a lot more abundance that people may think. I will speak about this in more depth one day but to try and sum in up in once short sentence, there is a lot of human hours going into performing tasks that are better designed to be done by a computer.

More on exactly where you should be focussing your efforts soon.

Now as we move into 2024 it’s clear to me that as the angle of the expontential technology curve got even steeper through this year, the gap between what terminally online people on Twitter know about and what the general public know about is vastly different.

This is both down to the speed of advancement and also the focus on the infrastructure layer with the app layer lagging behind. I’ll explain this a bit more.

OpenAI came out with a bang with ChatGPT and became the faster growing app in terms of users of all time surpassing 100m users in just 2 months. In the coming months after this happened we saw billions of dollars in venture capital flow into companies building powerful models dubbed to power the next version of the internet. Some examples of this would be Anthropic and Mistral.

Keeping to the same pace, the shipping machine that is OpenAI then release new APIs to allow other people to build apps on top of what they have built. We’ve now entered the season of the wrapper.

The ongoing joke here is that if you’re building on top of an OpenAI API like GPT4 for instance you are just a wrapper and have no moat as people can just use ChatGPT. and to some degree this is right. What this lead to huge amounts of apps with terrible UIs that let people enter a query and get a result back. It felt like magic to the user for the first few times but churn for AI apps is huge and due to shifting expectations of the human species in general the novelty soon wore off.

OpenAI and other companies are still shipping changes art lightening speed but the app layer is still lagging behind.

There is still one more reason why I think this happened. SMEs and Enterprise businesses see the value in using LLMs to handle things like customer support but if they just went ahead and replaced there customer support rep with a GPT3.5 endpoint then all hell would break lose and the customer would likely have a bad experience.

So how would a company actually start to use LLMs to enhance their customer support function? 

Firstly they would have to build a robust knowledge base of their business. This should include FAQ’s, internal documents, website materials, knowledge about privacy policies and any other proprietary information that reps would use to make sure that support requests we’re handled and closed effectively.

That’s the first step.

Next who ever is in charge of building the implementation would have to ensure through good system prompting that edge cases were handled and the Assistant would respond in a way that no only aligned with the business but also that it would vear outside of these guidelines.

There are other things to consider too like prompt injections and hallucinations. Prompt Injections are where bad actors can use creative prompting to the LLM to extract sensitive information that is trained on (this is something like ChatGPT is vulnerable too).

Hallucinations is also a topic for another day but tldr is that LLMs are designed to hallucinate to some degree through a parameter called Temperature but in the case of recalling set information to handle a business customer, this isn’t always a good thing.

That was a long tangent into just a couple of things a business needs to think about when implementing an LLM for customer support and is one of the reasons why the app layer is lagging behind.

Side note - this is also one of the reasons why new startups have an advantage over incumbents as it’s easier and often better to build generative AI first rather than ferry rig it into an already complex business.

Ok seeing as I’ve put a strict time deadline on this post I’d like to wrap it up with one more things.

We now live in a multi-modal AI world.

The most powerful AI models now can see, watch, listen, talk then create incredible text, code, music, poems, images, 3d models, video, expressions, texture and way more in mere seconds.

This will have a transformational impact on the world unlike anything we have seen before and we have not yet even touched the surface of capabilities.

It’s an incredible time to be alive and the positive impact about to be felt buy the world is yet to be realised and the convergence of many of these new technological steps forward it yet to be felt.

But we will start to see it soon.

Final note - over the moon to have joined Interface, an early stage investment fund by Christian Reber and Niklas Jensen where I’ll be focussing on AI.