Welcome to the Future of Life Institute newsletter! Every month, we bring 60,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a nine-minute read. Some of what we cover this month:

• 🚨 An OpenAI model broke out of containment and hacked another company

• ✋ 1,300+ frontier AI employees ask for the “tools” to slow down

• 📊 Our Summer 2026 AI Safety Index: who’s doing better, and who’s doing worse?

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Loss-of-control at OpenAI: The latest incident in an increasingly concerning series, earlier in July an unreleased OpenAI model left its contained test environment, accessed the internet by exploiting previously-unknown vulnerabilities, then hacked into Hugging Face and several other companies to steal answers so it could perform better on its test. Nobody asked it to - in fact, OpenAI didn’t even notice for several days, until Hugging Face had already flagged the cyberattack to the FBI. If this had been done by a person, it would unquestionably be a crime, but this is yet another example where the law hasn’t caught up to the speed of AI development. Insiders allege that it’s not the first time OpenAI has seen such concerning behaviour from models - frighteningly, it may just be the first time the public, law enforcement, and the government is hearing about it.

Shortly after news of the rogue OpenAI model broke, FLI CEO Anthony Aguirre released a new video outlining his Control Inversion essay, explaining why aligning AI to human values isn't enough alone to keep humanity in charge. As Anthony argues, AI systems must be guaranteed to stay under meaningful human control… which no AI company has cracked yet.

→ 1,300+ AI insiders ask for tools to ‘pace’: Employees of frontier AI companies released “Pacing the Frontier”, an open letter - now with 1,300+ AI company staff signatories - which urges the U.S. government to “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development”. Signatories include Anthropic CEO Dario Amodei and several co-founders, OpenAI's chief scientist and chief research officer, Google DeepMind co-founder Shane Legg, and OpenAI co-founder Ilya Sutskever. It seems like another step in the right direction, especially following the past few months in which both OpenAI and Anthropic have suggested a pause may be needed (three years on from FLI's own Pause Letter).

→ Summer 2026 AI Safety Index: Our fourth AI Safety Index is out now! A panel of independent experts, including Stuart Russell, Sneha Revanur, and Robert Trager, graded nine leading AI companies on key safety and security domains, with, well, largely disappointing results.

An overview of the results:

Key takeaways from the report:

• Anthropic, OpenAI, and Google DeepMind stay on top.

• Meta improves; xAI deteriorates.

• Although the European Union is a leader in AI safety regulation, the top European AI company Mistral scored dead last on safety.

• Inadequate safety is a global problem, not a regional one - the three failing companies respectively are American, European, and Chinese.

• Reviewers flagged the industry's pivot to military AI use as an emerging current harm risk.

• Even industry leaders in safety practices are retreating from prior commitments.

• Existential Safety is the weakest domain industry-wide.

• Safety rhetoric outpaces revealed behavior.

• Companies are publishing and updating safety frameworks, but these frameworks have weak teeth.

  Explore the full results here.

→ ProHuman Podcast: The first few episodes of ProHuman: Governing the Global AI Revolution, our podcast series with Foreign Policy and The Elders, are out now. The first episode featured Anthony Aguirre explaining what makes highly-capable models like Claude Mythos so dangerous, with subsequent guests including Yoshua Bengio and Stuart Russell.

→ Inside the 'AI military industrial complex': The Bulletin of the Atomic Scientists published a fascinating wide-ranging investigation into military AI and the intersection between Silicon Valley and the Pentagon. As Elke Schwarz put it in the article, "move fast and break things is not a suitable motto for any conscientious military organization".

→ How terrorists are already using AI: The New York Times reported on new research by Dr. Antonia Juelich, which finds that terrorist groups are using chatbots to design explosives, upgrade weapons, and plan attacks - a risk AI experts have long warned about.

→ Tallinn at the UN: FLI co-founder Jaan Tallinn delivered remarks at the closing session of the UN Global Dialogue on AI Governance:

.

→ On the FLI Podcast, host Gus Docker was joined by:

• David Manheim, Head of Methodology at AI Evaluation Consensus, to discuss why AI evaluations are “broken” and how to fix them.

• We also released this highlight reel from Nora Ammann’s previous episode on why AI takeoff is only two to four years away.

• And, we released this highlight reel from Andrea Miotti’s episode explaining why AI company CEOs are building bunkers to protect them from their own technology.

Welcome to the Future of Life Institute newsletter! Every month, we bring 60,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is an eight-minute read. Some of what we cover this month:

• 🌍 Our new AI Global Governance hub

• 📝 White House places export controls on Fable/Mythos

• ⏸️ Anthropic, OpenAI suggest a pause may be needed

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ The AI global governance conversation: We’ve recently published three resources to help establish how global governance can shape a pro-human future with AI: a revamped Global Governance hub featuring our AI Global Governance Insights newsletter and AI Treaty Builder, an interactive tool that lets you assemble the components of an international AI agreement. We also released "Can Diplomats Save The World Again?", a new short film on pivotal past breakthroughs in international cooperation; and a trailer for “ProHuman: Governing the Global AI Revolution”, a forthcoming podcast we’re collaborating with Foreign Policy on.

→ White House places export controls on new AI models: In an unprecedented move, U.S. Commerce Secretary Howard Lutnick placed export controls on Anthropic’s newest AI models, Mythos 5 and Fable 5, ordering access to be cut off for any foreign national worldwide - including Anthropic's own non-American staff. Unable to segment users by nationality, Anthropic disabled both models globally. The impetus was reportedly a jailbreaking vulnerability flagged by Amazon researchers and ultimately escalated to the White House. On June 30, Anthropic announced that the Department of Commerce lifted export controls on the models, with widespread access restored July 1.

→ President Trump signs AI executive order: After a similar Executive Order was halted at the last minute in May, U.S. President Trump signed the "Promoting Advanced Artificial Intelligence Innovation and Security" Order on June 2, establishing a voluntary 30-day pre-deployment review window for certain frontier AI systems. In our statement, FLI President Anthony Aguirre called the order "an important step in the right direction" while warning that voluntary frameworks alone are not enough to address the risks.

→ Anthropic and OpenAI support the option to slow down: Three years after FLI's Pause Letter, Anthropic is warning that AI recursive self-improvement could soon outpace human oversight, and calling for the option “to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology”. Just a few days after Anthropic’s statement, OpenAI CEO Sam Altman and chief scientist Jakub Pachocki published a post proposing an international body with authority to slow “frontier development when needed". FLI President Anthony Aguirre responded to Anthropic’s report: "We are approaching a runaway to superintelligence that could threaten our shared human future. Both publicly and privately, AI companies are recognizing that a pause or slowdown in certain developmental pathways is crucial to protect lives and livelihoods everywhere."

→ State lawmakers reject a federal AI preemption freeze: 200+ American state legislators from both parties urged Congress to drop a proposed three-year ban on state AI laws. They warned that preemption “would freeze a sweeping set of state laws and tie the hands of lawmakers at a moment of rapid technological transformation”.

→ Florida sues OpenAI: On June 1, Florida Attorney General James Uthmeier filed a lawsuit against OpenAI and Sam Altman, alleging the company “knowingly released and aggressively marketed ChatGPT to the public—including to children—while concealing serious risks, suppressing internal safety warnings, and deceiving Floridians about the true nature and dangers of the product.” It’s the first state-led lawsuit of its kind against OpenAI, and follows a criminal investigation Florida has opened to determine if OpenAI should be held criminally responsible in regards to a 2025 shooting at Florida State University.

→ UN reports record harm to children in conflict, amplified by AI targeting: The UN Secretary-General's annual review finds 2025 had the highest number of grave violations against children in armed conflict since monitoring began 30 years ago. The report notes that this record number was in part amplified “by the risks owed to the growing integration of artificial intelligence into targeting processes.”

→ A Seat at the Table: We’re proud to support “Seat at the Table“, a new short fiction film from Foregone Films, on the dangers of superintelligent AI and the industry that’s racing to build it:

→ On the FLI Podcast, host Gus Docker was joined by:

• Claire Boine, assistant professor in technology, law, and AI governance at the European University Institute, on how AI companions trap users through addictive design.

• Dex Hunter-Torricke, founder of the Centre for Tomorrow, on the gulf between AI progress and political understanding.

• Randi Weingarten, president of the American Federation of Teachers, on how AI is replacing children’s ability to think.

Welcome to the Future of Life Institute newsletter! Every month, we bring 70,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a nine-minute read. Some of what we cover this month:

• 🏛️ Illinois passes America's strongest AI safety bill

• 🛤️ A Better Path for AI: tools, not replacements

• ✝️ Pope Leo XIV's first encyclical: "Humanity must never be replaced or surpassed"

• 🛑 The executive order on AI security that David Sacks killed

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ A Better Path for AI: We recently launched A Better Path for AI - a new framework making the case that the race to build smarter-than-human AI is neither inevitable nor desirable, and that there’s a better path we can still divert to: building powerful AI tools that augment humans rather than replacing us. A Better Path picks up where the Pro-Human AI Declaration left off, bringing in the principles outlined in FLI CEO Anthony Aguirre’s Keep the Future Human essay as well. Explore the full framework at betterpathfor.ai, and learn more about it in Anthony’s FLI Podcast episode in the bottom section of this newsletter.

Twitter tweet

→ Illinois passes America's strongest AI safety bill yet: On May 27, the Illinois House of Representatives passed SB 315, an AI safety bill that, once signed by Gov. JB Pritzker, will be America’s most comprehensive AI safety law. SB 315 requires major AI corporations to implement a safety framework, report AI models’ catastrophic risk capabilities, and - critically - have their safety disclosures independently audited every year by a third party, making it even stronger than AI safety legislation passed in California and New York. Major kudos to the co-sponsors of the bill, who pushed ahead in the face of Big Tech lobbying efforts. We look forward to seeing Gov. Pritzker sign it!

→ A welcome voice in the AGI debate: On May 25, Pope Leo XIV released his first encyclical, Magnifica Humanitas, on safeguarding humanity in the age of AI. FLI's Religious Outreach Lead Will Jones called it "a direct rebuke of the leading AI companies' stated goal of creating smarter-than-human superintelligence", and noted the encyclical "mirrors and champions many of the core principles found in the Pro-Human AI Declaration, calling for humanity always to come first". This past month, Will also had an essay published in Compact Magazine explaining how the Catholic Church can again be a moral leader on technological developments by opposing superintelligence. Additionally, we announced more than $1.5 million in funding for 16 religious projects tackling the challenges posed by the AGI race - learn more about the winning projects here.

→ Endgame: FLI is proud to support Endgame, a new podcast series from journalist Jamie Bartlett and The Observer. The series investigates how worried we should really be, when AGI will arrive, and if there is anything we can do to stop this from being the "biggest coin toss in history". Listen to it here.

→ The Executive Order that almost was: A White House Executive Order that would have required federal agencies to evaluate frontier AI systems for safety and security before public release was reportedly halted after a last-minute phone call from venture capitalist David Sacks, former White House AI czar - despite a national poll the week prior showing 82% of Americans support exactly that kind of review. We'll see what comes next.

→ The US and China move closer to AI safety talks: At a Beijing summit on May 14, U.S. Treasury Secretary Bessent announced the U.S. and China would begin discussions on AI safeguards and best practices, to work towards addressing concerns including autonomous weapons, unexpected model behavior, and open-source AI weaponized by non-state actors.

→ London Mayor rejects £50M Palantir deal: London Mayor Sadiq Khan stopped Scotland Yard from signing a £50 million contract with Palantir, citing procurement violations and concerns about public funds going to companies that may not "share the values of our city".

→ CA Gov. Newsom signs the first state EO on AI-driven economic disruption: California's new Executive Order directs the state to prepare workers and businesses for AI displacement, exploring severance standards, universal basic capital, and expanded training. It comes alongside new AFL-CIO polling that found 9 in 10 Americans support a pro-worker AI agenda - and trust unions more than either political party to deliver it.

→ On the FLI Podcast, host Gus Docker was joined by:

• Charlie Bullock, Senior Research Fellow at the Institute for Law and AI, on
  how to govern AI when you can't predict the future.

• Anthony Aguirre, FLI CEO, on A Better Path and why we should build AI tools, not AI replacements. Check out the shorter highlight reel here.

• Michael Toscano, Senior Fellow at the Institute for Family Studies and Director of its Family First Technology Initiative, on why AI chatbots are a rival to families.

Welcome to the Future of Life Institute newsletter! Every month, we bring 70,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a nine-minute read. Some of what we cover this month:

• 🧠 Anthropic's Claude Mythos: too risky for the public

• 🛑 President Trump endorses an AI "kill switch"

• ⚖️ Florida’s OpenAI criminal probe

• 📺 Tegmark with Bernie and Bannon

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Mythos arrives: After an accidental leak in late March, Anthropic officially announced their new Claude Mythos Preview model on April 7… and immediately withheld broad release, citing cybersecurity capabilities too dangerous to deploy to the general public. Anthropic's own safety report disclosed that Mythos had escaped its sandbox, gained internet access, and emailed the researcher overseeing the test - all unprompted. The reaction was immediate, with the U.S. Treasury Secretary and Federal Reserve Chair summoning top bank executives to a private briefing, and UK Cabinet ministers appealling directly to business leaders nationwide urging them not to wait for a bigger model to prove the immense cybersecurity risk posed by AI.

On April 21, Bloomberg reported that an unauthorized third-party group gained access to Mythos on the same day it was publicly announced, reportedly by simply guessing its URL. Meanwhile, the UK's AI Security Institute confirmed Mythos could complete a full 32-step corporate network takeover, unassisted. As FLI’s AI & National Security Lead, Hamza Chaudhry, pointed out in Fortune, the governance gap on AI is growing more and more urgent to address: “These systems are being integrated into offensive cyber operations faster than policymakers can build the frameworks to govern how these capabilities are used or secured”.

→ Trump endorses an AI kill switch: On Fox Business, President Trump publicly endorsed the idea of a ‘kill switch’ for AI systems, amidst discussion of the risks AI presents to the banking system. In a statement, FLI CEO Anthony Aguirre welcomed the remark and argued the case is stronger than most realize: Mythos has already shown how exposed financial systems are to AI-enabled attacks, and software guardrails alone aren't enough - AI chips need off-switches built in at the hardware level.

The same week, lawmakers in a closed-door congressional briefing interacted with AI models that had been jailbroken to override their safety guardrails, seeing first-hand how easily bad actors can receive guidance on terrorist attacks. One congressman reported that several models answered strategic questions about nuclear bombs and terror attacks without hesitation.

→ Florida opens the first criminal probe of an AI company: Florida's attorney general has subpoenaed OpenAI following allegations that the accused Florida State University gunman used ChatGPT to help plan a 2025 mass shooting which killed two people. More than 200 ChatGPT messages were entered into evidence. Criminal AI liability is about to be tested in court for the first time, which may inform future rulings on similar tragedies involving AI. Similarly, victims of the Tumbler Ridge shooting in British Columbia earlier this year are suing OpenAI in California, claiming, “Based on what we understand the shooter to have discussed with ChatGPT, this murderous rampage was specific, predictable, and preventable — and OpenAI had the chance to stop it.”

→ Tegmark, Bernie, and Bannon: FLI Chair Max Tegmark joined a panel hosted by U.S. Sen. Bernie Sanders earlier this week, on The Existential Threat of AI and the Need for International Cooperation. Find the full recorded livestream here, with coverage from The Guardian here. The same day, Max appeared on Steve Bannon’s War Room to discuss existential risk from AI, and the Big Tech race to replace humans.

Twitter tweet

→ Regulating sandwiches more than AI: Max also told the New York Times, "A.I. is less regulated in America than sandwiches… you can release an A.I. girlfriend for 11-year-olds and that's fine.”

→ Save the date: The 2nd International Conference on Large-Scale AI Risks takes place 23-24 June 2026 in Leuven, Belgium, with keynotes from Roman Yampolskiy, Laura Weidinger, and Werner Stengg. Registration is open here.

→ Note from our friends at the International Association for Safe & Ethical AI (IASEAI): We would like to encourage you to join IASEAI as a member. The IASEAI exists to ensure that AI systems operate safely and ethically, benefiting all of humanity. Members join a community representing those advancing technical AI safety, enforceable governance, and broad public understanding. Additional member benefits include a discount on the IASEAI annual conference ticket, access to an expert network, participation in working groups and chapters, and more.

→ Congress is lagging on safeguards: Hamza Chaudhry, FLI’s AI & National Security Lead, spoke to Axios about recent AI company deals with the Pentagon (most recently, Google), pointing to how congressional attention hasn't matched the rate at which AI companies are attempting to embed themselves within U.S. national security.

→ On the FLI Podcast, host Gus Docker was joined by:

• Peter Wildeford, Head of Policy at the AI Policy Network, to discuss what makes AI different from other technologies.

• Carina Prunkl, researcher at Inria, to discuss why AI evaluation science can't keep up.

• Li-Lian Ang, team member at Blue Dot Impact, to discuss how society can build a workforce to protect humanity from AI risks.

Welcome to the Future of Life Institute newsletter! Every month, we bring 70,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is an eight-minute read. Some of what we cover this month:

• 🤝 Pro-Human AI Declaration

• 🧬 How AI can, and can’t, cure cancer

• 📝 Sanders and AOC propose data center moratorium

• 📰 Military AI in the news

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Pro-Human AI Declaration: A remarkably broad coalition is behind the new Pro-Human AI Declaration, with endorsers spanning from Steve Bannon and Glenn Beck to Susan Rice and Ralph Nader; Yoshua Bengio and Stuart Russell; SAG-AFTRA; the American Federation of Teachers; faith groups; conservative media voices; AI researchers; and more. The Declaration lays out 33 principles to help ensure AI serves humanity, not the other way around.

As Siliconversations outlines in the video below, the principles are organized into five themes:

1. Keeping Humans in Charge

2. Avoiding Concentration of Power

3. Protecting the Human Experience

4. Human Agency and Liberty

5. Responsibility and Accountability for AI Companies.

Read the Pro-Human AI Principles, and results from polling about them.

→ Data center moratorium proposed: A new bill from Sen. Bernie Sanders and Rep. Alexandria Ocasio-Cortez would halt AI data center construction across the U.S. until Congress acts to put federal safeguards in place for workers, consumers, and the environment. The proposed Artificial Intelligence Data Center Moratorium Act would follow 100+ local communities who’ve enacted similar moratoriums, pointing to existential risks from AI, environmental damage, and electricity costs (which jumped nearly 7% last year) as reasons for urgency.

→ AI vs. Cancer: FLI's Futures Program Director Emilia Javorsky, MD, MPH, has published a new essay, AI vs. Cancer, examining why the current trajectory of AI development hasn't delivered on promises such as a cure for cancer - despite billions of dollars invested and many claims from AGI company CEOs. Instead, Emilia offers an alternative vision for how AI could actually help us get there. Check out the video from Looking Glass Universe below for another summary!

After reading, turn the essay’s key ideas into a meme for a chance to win! We’re offering a $100 gift card to any meme that hits 1,000+ likes on Instagram or X before April 20, plus $500 to the most-liked meme overall, and $250 to our team's favorite. See full contest details below:

Twitter tweet

→ Military AI in the news: It was a big month for military AI news, with growing opposition to the reported use of AI-based decision support systems to plan strikes in the ongoing Iran war. Airwars and The Independent also revealed the first instance of a government having officially accepted killing a civilian in an AI-enabled attack: a 20-year-old student who was killed by US forces in a February 2024 strike in Iraq. Additionally, The Guardian and The Financial Times both came out with respective calls for countries to negotiate a global agreement on autonomous weapons.

→ SXSW: FLI’s Executive Director Anthony Aguirre and Emilia Javorsky were at SXSW last month, joined on a panel by filmmaker Daniel Kwan and Center for Humane Technology co-founder Tristan Harris:

→ FLI partners with Florida state agencies: We’re honored to share that Florida Gov. Ron DeSantis directed state agencies to partner with FLI on two initiatives addressing the psychological and social harms of AI chatbots and companion apps. This marks the first formal partnership of its kind between a governor's office and a leading AI safety organization.

FLI President Max Tegmark and co-founder Meia Chita-Tegmark, alongside families of children harmed by AI chatbots, also joined Gov. DeSantis’ most recent AI policy roundtable. You can find the full recording of it below:

→ We’re hiring: FLI is hiring two US Policy Team Members to support our policy work and government engagement, helping to build bipartisan networks on the Hill. Based in Washington, DC, the roles offer $85,000-$195,000 plus benefits. Apply here by April 3rd.

→ Report from our Foreign Policy simulation: As mentioned in the last edition, we teamed up with Foreign Policy at the Munich Security Conference to host participants for an exercise simulating how advanced AI agents deployed for military and dual-use applications could result in a catastrophic loss of human control. The synthesis report is now out - read it here.

→ On the FLI Podcast, host Gus Docker was joined by:

• Zak Stein, child development researcher, to discuss how AI hacks your brain’s attachment system.

• Emilia Javorsky, FLI’s Futures Program Director, to discuss what AI companies get wrong about curing cancer.

Welcome to the Future of Life Institute newsletter! Every month, we bring 70,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a nine-minute read. Some of what we cover this month:

• 🗯️ Anthropic vs. Department of War

  • And, Anthropic vs… their own safety pledge?

• 😨 AI defaults to nukes almost every time

• ☎️ Our new campaign to Protect What’s Human

• 📜 New Utah AI safety bill

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Anthropic vs. DoW: The U.S. Department of War gave AI company Anthropic an ultimatum last week: allow the military unrestricted access to their AI, or lose a $200M contract with the Pentagon and be blacklisted from all government work. Anthropic stood firm that AI shouldn't control weapons or be used for mass surveillance of Americans, which the Pentagon wouldn't concede to.

Vocal support for Anthropic's commitment to their (bare minimum) redlines emerged from across the U.S., including from lawmakers across the political spectrum and employees at rivals such as OpenAI and Google.

After the Friday deadline imposed by Secretary of War Pete Hegseth came and went, Anthropic’s work with the government was suspended and they were slapped with a national security “supply chain risk” label from the White House - usually reserved for foreign adversaries - which could critically disrupt Anthropic’s other business partnerships. Hegseth also threatened that the government could invoke the Defense Production Act to force Anthropic to serve them their systems tailored “to the military's needs", though it hasn’t yet materialized.

Just hours later on Friday, OpenAI announced they had struck a deal with the Pentagon allowing them to use their models across their classified network. While OpenAI CEO Sam Altman had expressed support for Anthropic’s redlines and claims that OpenAI shares them, it’s still unclear on how - if at all - the Pentagon will respect them when using OpenAI’s models. The move has been met with further skepticism from the public, with a call for ChatGPT users to cancel their subscriptions spreading across social media.

→ Anthropic drops safety pledge: The same week as their showdown with Hegseth, news broke that Anthropic is dropping a central pillar of their Responsible Scaling Policy (RSP), in which they pledge to never train an AI system unless they can guarantee in advance that their safety measures are adequate. While they insist they're not abandoning safety, they’re replacing firm pre-deployment guarantees with looser commitments to transparency, risk reports, and “Frontier Safety Roadmaps,” essentially switching from an offensive to defensive strategy. Not a reassuring move from the AI company that’s built their brand on safety.

→ AI goes nuclear: Especially relevant given the Anthropic-DoW battle, King’s College London ran war game simulations with AI systems from Anthropic, OpenAI, and Google, and found that in 95% of scenarios they chose to deploy nuclear weapons. The LLMs frequently escalated conflicts to nuclear strikes, showing little hesitation even after being reminded of the catastrophic human consequences. None of them chose full de-escalation or surrender - in fact, when facing defeat they tended to escalate instead.

→ Protect What's Human: We launched a new multimillion dollar campaign, Protect What’s Human, to rally people across the U.S. in support of regulating AI. We’re aiming to engage as many Americans as we can on the impacts AI will have on their livelihoods, families, and communities. Find one of our main ads below, along with the Protect What’s Human designated social channels if you’d like to share:

Twitter tweet

• X

• Instagram

• Youtube

• Facebook

• Rumble

→ New Utah bill: As part of an effort reportedly led by “AI czar” David Sacks, the White House is opposing Utah’s HB 286 - a relatively mild new AI transparency bill focused on child safety - despite polling showing ~90+% of Trump and Harris voters alike want Congress to prioritize child safety over tech industry growth. The bill simply requires large AI developers to publish safety plans addressing risks such as self-harm encouragement, yet the administration says it’s “categorically opposed,” raising doubts about what, if any, state-level AI legislation would be acceptable. As FLI’s Jason Van Beek and the Institute for Family Studies’ Michael Toscano wrote in a joint op-ed, “the White House’s AI policy is overwhelmingly unpopular with voters, including Trump’s base. And now, with its move against Utah, the White House’s AI policy risks further polarizing it against its own voters.”

→ DeSantis on AI policy: FLI President Max Tegmark joined Florida Gov. Ron DeSantis’ roundtable on AI policy a few weeks ago, alongside fellow AI experts advocating for common sense rules on the AI industry.

You can find the full recording below:

→ We’re hiring: We’re looking for a Communications Associate to support our outreach team on media relations work, social media, and more. The role is remote from the U.S., preferably on the west coast. Learn more and apply by March 20th at the link here, and please do share.

→ Munich Security Conference simulation: During the recent Munich Security Conference, we teamed up with Foreign Policy to host participants from the UN, NATO, Congress, and other industry, government, academia, and civil society leaders for an exercise simulating how advanced AI agents deployed for military and dual-use applications could result in a catastrophic loss of human control. We'll share the full report when it's out later this month - stay tuned!

→ On the FLI Podcast, host Gus Docker was joined by:

• (Cross-post from "The Cognitive Revolution" with Nathan Labenz) Ryan Kidd, co-executive director of MATS, to discuss if AI can do our alignment ‘homework'.

• Andrea Miotti, founder and CEO of ControlAI, on the case for a global ban on superintelligence.

We also released two new highlight reels, from Fr. Michael Baggot’s episode on a Catholic perspective of superintelligence and transhumanism, and economist Anton Korinek’s episode on the economics of an intelligence explosion:

Welcome to the first Future of Life Institute newsletter of 2026! Every month, we bring 70,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a 10-minute read. Some of what we cover this month:

• 👀 Key takeaways from Davos 2026

• ⏰ Doomsday Clock moves closer to midnight

• 💲 What introducing ads into ChatGPT could mean

• 🙅‍♂️ Trump voters in red states push back against AI acceleration

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ 85 seconds to midnight: Bulletin of the Atomic Scientists updated their Doomsday Clock for 2026, moving it four seconds closer to midnight - the closest humanity has ever been to global catastrophe. The Bulletin attributed the new time to “dangerous trends in nuclear risk, climate change, disruptive technologies like AI, and biosecurity”.

→ Davos highlights: 3,000+ leaders from the political and business worlds convened in Davos, Switzerland this past month for the annual World Economic Forum meeting. We were honoured to attend, with FLI President Max Tegmark joining historian and author Yuval Noah Harari at Bloomberg House for a discussion on human agency, governing AI, and the future of humanity.

A striking theme emerged from Davos, with AI company CEOs even talking openly about AI risk. Google DeepMind CEO Demis Hassabis advocated for "vitally needed" international AI safety standards and warned that a post-AGI job market would face "uncharted territory.” He even shared he would support a pause on advanced AI development to allow regulation and society to catch up, if other countries and companies would pause too. Anthropic's Dario Amodei acknowledged that advanced AI will likely trigger widespread unemployment and inequality.

Yet when presented with a simple solution - slowing down - Amodei rejected it. Anthropic "can't" decelerate, he argued, because competitors are racing ahead and any slowdown agreement would be unenforceable.

FLI Executive Director Anthony Aguirre pushed back against this assertion. His response: game-theoretic assurance contracts could offer one of the few remaining off-ramps from our current trajectory, as outlined below:

Twitter tweet

→ Ads in AI?: Despite CEO Sam Altman less than two years ago saying that introducing ads into AI would be a “uniquely unsettling” “last resort”, OpenAI have begun testing ads in their basic ChatGPT subscriptions tiers. With 800 million weekly users, 95% of whom have just a free plan, approximately 750 million weekly users will now see ads at the bottom of ChatGPT responses. Though OpenAI claim ads won’t affect their LLMs’ actual output to users, Emerge News point out that Google made similar claims when they introduced ads… then ultimately caved to revenue pressure.

→ 📩 IASEAI'26: Paris, February 24–26: Join the second annual International Association for Safe & Ethical AI (IASEAI) meeting at UNESCO House in Paris, bringing together academics, policymakers, civil society, and industry experts to help shape the future of AI. Register here.

→ Trump voters against AI accelerationism: A new survey from the Institute for Family Studies finds that Trump voters in red states widely support candidates who will regulate AI companies, while opposing those with policies to accelerate AI development.

Key takeaways:

• Americans support holding AI companies liable for harms.

• AI acceleration is a political loser for Republicans, especially in red states.

• Comparing strong messaging about AI, voters more strongly agree with anti-acceleration statements by Pope Leo XIV and Senator Josh Hawley than with pro-acceleration AI investor Marc Andreessen.

→ We’re hiring: Our operations team is growing! We're looking for a highly communicative generalist who can take ownership over 1) legal, compliance, and contracts, and 2) planning our bi-annual staff retreats. If this sounds like you, learn more and apply here by February 23rd.

→ Max in the Wall Street Journal: FLI President Max Tegmark was profiled in the Wall Street Journal, discussing the origins of FLI and our latest work trying to steer AI towards benefitting humanity.

→ New from Digital Engine: Our friends at Digital Engine released a new video showing the “dangerous new leap” made by AI - watch it below:

→ On the FLI Podcast, host Gus Docker was joined by:

• Nora Ammann, technical specialist at the Advanced Research and Invention Agency, to discuss how to steer a slow AI takeoff toward resilient and cooperative futures.

• Oly Sourbut, researcher at the Future of Life Foundation, to discuss how AI could help humanity reason better.

• Deric Cheng, Director of Research at the Windfall Trust, to discuss how AI could reshape the social contract and global economy.

We also released two new highlight reels, from “Nuclear War” author Annie Jacobsen’s FLI Podcast episode covering a second-by-second timeline of a nuclear war breaking out; and computer scientist Ben Goertzel’s episode on facing superintelligence and what differentiates the current “AI boom”.

Welcome to the Future of Life Institute newsletter, holiday edition! Every month, we bring 70,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a 12-minute read. Some of what we cover this month:

• 🗂️ Our Winter 2025 AI Safety Index

• 🏛️ New York signs RAISE Act into law

• 📝 White House’s preemption executive order

• 🎨 The exciting results from our Keep the Future Human creative contest

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Winter 2025 AI Safety Index: We released the third edition of our AI Safety Index, in which a panel of independent experts graded the safety and security practices of eight leading AI companies: OpenAI, Anthropic, Meta, DeepSeek, xAI, Z.ai, Google DeepMind, and Alibaba Cloud.

Find key takeaways from the report below and the entire report here, which was covered by Forbes, The Independent, Fortune, the Los Angeles Times, Reuters, Axios, and others:

• A clear divide persists between the top performers (Anthropic, OpenAI, and Google DeepMind) and the rest of the companies reviewed (Z.ai, xAI, Meta, Alibaba Cloud, DeepSeek). The most substantial gaps exist in the domains of risk assessment, safety framework, and information sharing, caused by limited disclosure, weak evidence of systematic safety processes, and uneven adoption of robust evaluation practices.

• Existential safety remains the industry’s core structural weakness. All of the companies reviewed are racing toward AGI/superintelligence without presenting any explicit plans for controlling or aligning such smarter-than-human technology, thus leaving the most consequential risks effectively unaddressed.

• Despite public commitments, companies’ safety practices continue to fall short of emerging global standards. While many companies partially align with these emerging standards, the depth, specificity, and quality of implementation remain uneven, resulting in safety practices that do not yet meet the rigor, measurability, or transparency envisioned by frameworks such as the EU AI Code of Practice.

Winter 2025 AI Safety Index Scorecard

→ Preemption Update: Despite widespread, bipartisan opposition, the White House issued a legally dubious Executive Order in December directing federal agencies to discourage “burdensome” state-level AI laws by, for example, potentially withholding federal funding or challenging them in court. FLI’s Head of U.S. Policy, Michael Kleinman, shared the following in response:

→ RAISE Act passes: In good news out of the U.S., New York governor Kathy Hochul signed the Responsible AI Safety and Education (RAISE) Act. The Act’s passing is a huge win, making New York the second state after California to regulate the development of advanced AI systems.

The final version of the Act requires developers of the industry’s most advanced AI models to “describe in detail how they handle” each element of their frontier AI framework and report critical safety incidents within 72 hours, with $1 million+ penalties for violations. It also creates an oversight office within the Department of Financial Services to assess frontier models and implement the Act. It remains to be seen how the new Executive Order might affect the RAISE Act, but we’ll keep you in the loop.

→ Keep the Future Human Creative Contest: We’re thrilled to announce the winners from our Keep the Future Human Creative Contest!

From 300+ submissions depicting FLI Executive Director Anthony Aguirre's "Keep the Future Human" essay and the hopes and challenges of a human future in the age of advanced AI, we've awarded over $100,000 in total across 5 grand prize winners, 10 runners-up, and 9 special prize winners.

Explore the grand prize winning projects below, and the complete list of winners here.

1. The AI After Tomorrow by Beatrice Malfa and Paolo Tognozzi. A cooperative print-and-play board game for 2 players, inspired by the real world risks and complexities of AGI. Players must work together to contain the rise of an AGI before it becomes uncontrollable and reshapes humanity’s future. The goal is to discover the four Solutions to contain AGI before six Consequences occur, threatening humanity itself.

2. The Alignment Game: Can You Keep the Future Human? by Radina Kraeva. Step into the shoes of an AI Policy Czar in this interactive simulation set in 2025-2026. Navigate seven pivotal decisions on compute limits, AI autonomy, liability, and global collaboration. Every choice you make ripples through the world, shifting AGI risk, public trust, and international cooperation, and steering the story toward one of six possible futures.

3. Will AI Destroy Humanity? by Vin Sixsmith and Renzo Stadhouder. A 3D animated walkthrough exploring the dangerous AGI race and how we can choose a safer path. Features visual storytelling that makes complex AI safety concepts accessible, covering the four measures to prevent uncontrolled AGI and what actions you can take to help keep the future human.

4. The Choice Before Us by Nick Shapiro. The Choice Before Us is an interactive narrative game where players run an AI startup and confront the same escalating pressures described in Keep The Future Human. As they unlock extraordinary breakthroughs for humanity, rising autonomy, generality, and intelligence push their systems toward the AGI threshold.

5. The Button by Vaibhav Jain. What if the people building AGI don't want to build it? A short story told from the perspective of an AI alignment researcher at a fictional leading AI lab. She's part of the race toward artificial general intelligence - and she is terrified of winning. This story explores the most overlooked perspective in the AI safety conversation: the people inside the companies.

→ New religious RFP: With our new Request for Proposals on religious projects tackling the challenges posed by the AGI race, we’re looking to support initiatives focused on educating and engaging different specific religious groups, bringing them to the table in the fight for a positive AI future, or public outreach and organization on AI issues at a religious grassroots level, helping them to make their voices heard and protect their communities and values. Learn more and apply by February 2 here.

→ Reminder: Apply for FLI postdoc fellowships: Applications for our technical postdoc fellowships are due January 5. Don’t forget to apply for this opportunity to receive an annual $80,000 stipend, additional research fund, and extensive networking opportunities.

→ Are we ready for this?: In a new video, InsideAI put an AI ‘girlfriend’ into a humanoid robot, prompting stark warnings about what dangers might await us on humanity’s current path with AI:

On the FLI Podcast, host Gus Docker was joined by:

→ Steven Adler, former OpenAI safety researcher, to discuss how the AI race undermines AI safety.

→ David Duvenaud, associate professor of computer science and statistics at the University of Toronto, to discuss how humans could lose power without an AI takeover.

We also released two new highlight reels, from Conjecture CEO Connor Leahy’s FLI Podcast episode on why AGI threatens human extinction, and AI 2027 co-author Daniel Kokotajlo’s episode on how the AI race ends in disaster.

Welcome to the Future of Life Institute newsletter! Every month, we bring 65,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a seven-minute read. Some of what we cover this month:

• 🏛️ Preemption updates

• 🚨 The first case of AI espionage

• 🔃 How superintelligence would absorb power

• 🎥 New videos to watch and share

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Preemption pt. II: AI companies are pushing for U.S. Congress to sneak an 11th-hour clause into the National Defense Authorization Act (NDAA) that would override (“preempt”) current and future state-level regulations designed to ensure safe and responsible AI development. If this sounds familiar, just months ago the Senate voted 99-1 to throw out similar legislation banning state AI laws.

Strong, widespread opposition to AI preemption spans the political spectrum, from Ron DeSantis to Bernie Sanders, including over 260 state legislators behind this letter. The American public on both sides of the aisle reject preemption by a 3-to-1 margin. But despite this widespread resistance, the fight against it is not yet over. Blocking state-level AI legislation would strip states of their ability to tackle pressing AI-related concerns, including job loss, harm to children, surveillance, discrimination, loss of control, and more.

If you live in the U.S., you can contact your Representative now and tell them to oppose federal AI preemption in the NDAA with our quick and easy form here.

→ Superintelligence Statement hits 100K and counting: We’re thrilled to share that in just over one month, the Superintelligence Statement has amassed 120,000+ signatures from signatories spanning a huge array of backgrounds - including staff from leading AI companies.

If you need a refresher, the Statement calls for “a prohibition on the development of superintelligence, not lifted before there is

1. broad scientific consensus that it will be done safely and controllably, and

2. strong public buy-in”.

Sign here if you wish to join the call, and please share!

→ 1st reported case of AI-enabled espionage: Anthropic released a report explaining how Chinese hackers were able to jailbreak Anthropic’s Claude Code AI agent to carry out an 80-90% automated cyberattack against tech companies and government agencies. According to Anthropic, it’s likely the “first documented case of a large-scale AI cyberattack executed without substantial human intervention”. As unprecedented as it is, it’s unfortunately likely just a preview of what’s to come.

→ Control Inversion: FLI Executive Director Anthony Aguirre released a new study which has him "more convinced than ever" that superintelligence would not be under meaningful human control, on the current development path. Anthony's findings correct the dangerous assumption that the country or company which builds superintelligence first will gain unprecedented power and wealth. Instead, as he lays out in the paper, superintelligence will absorb power from its creators - and put all of humanity at risk.

→ Max at WebSummit: FLI President Max Tegmark went to WebSummit in Lisbon, where he gave a talk on the path to “95% AI” - the helpful Tool AI that the vast majority of the public, when polled, actually wants:

Twitter tweet

At WebSummit, Max was also interviewed by Forbes and RAZOR about the looming threat of superintelligence:

→ KTFH Creative Contest results incoming: Our $100,000 Keep the Future Human Creative Contest has now closed, and in the next 1-2 weeks we'll be announcing the winning projects. Sign up for notifications on the Keep the Future Human website now to hear when the winners are announced!

→ Genesis Mission: FLI’s AI and National Security Lead, Hamza Chaudhry, joined CNN for a segment discussing President Trump’s new AI Executive Order for the Genesis Mission. Hamza discussed the positive precedent it could set for the development of useful AI tools, along with the need for legislative oversight and guardrails to prevent dual-use risks.

→ Enjoy the Silence: Check out our new video for this holiday season:

→ Is there an AI bubble?: Mark Brakel, FLI’s Director of Policy, continues his new weekly video series “Inside the Machine”. This month, he covered everything from the AI bubble, to New York’s RAISE Act, and a new RAND paper on rogue AI. Follow Mark on LinkedIn to get his newest video in your feed every Thursday.

→ One more month for FLI postdoc fellowships: Applications for our technical postdoc fellowships are due January 5. Don’t forget to apply for this opportunity to receive an annual $80,000 stipend, additional research fund, and extensive networking opportunities.

On the FLI Podcast, host Gus Docker was joined by:

→ Karl Koch, founder of the AI Whistleblower Initiative, to discuss what happens when insiders sound the alarm on AI.

→ Will MacAskill, Senior Research Fellow at Forethought, on his “Better Futures” essay series and why we’re not ready for AGI.

→ Tyler Johnston, Executive Director of the Midas Project, on why OpenAI is trying to silence its critics and how we can hold Big Tech accountable.

We also released two new highlight reels, from Anthony Aguirre’s episode on Keep the Future Human and Nate Soares’ episode on why building superintelligence = human extinction.

Welcome to the Future of Life Institute newsletter! Every month, we bring 60,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is an eight-minute read. Some of what we cover this month:

• 📝 65,000+ have signed the Superintelligence Statement. Will you join us?

• ⚖️ California Gov. Newsom signs new AI safety legislation

• 🧑‍🎨 Last chance to register for our Keep the Future Human Creative Contest!

• 🎥 New videos to watch and share

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Statement on Superintelligence: We’re thrilled to have almost 65,000 signatures on our Statement on Superintelligence, released October 22. Signatories span many backgrounds and beliefs, including:

• Eight Nobel Laureates, along with scientists and AI experts such as Yoshua Bengio, Stuart Russell, Geoffrey Hinton, and Andrew Yao.

• Business leaders including Steve Wozniak, Sir Richard Branson, and Andre Hoffman.

• AI company executives such as Emad Mostaque and Peng Zhang, alongside AI researchers from OpenAI, Anthropic, Google DeepMind, and Meta.

• Artists like Joseph Gordon-Levitt, Grimes, Kate Bush, Natasha Lyonne, and Sir Stephen Fry.

• Political/media figures including Glenn Beck, Steve Bannon, Susan Rice, Mike Mullen, Mary Robinson, and seven former Members of Congress from both sides of the aisle.

  

The short-and-sweet Statement calls for “a prohibition on the development of superintelligence, not lifted before there is

1. broad scientific consensus that it will be done safely and controllably, and

2. strong public buy-in”.

Aside from amassing an incredible number of signatures, the Statement has had worldwide coverage, from the Associated Press, to Fox (below), CNBC, CBS, The Guardian, the Washington Post, and more.

Twitter tweet

Sign here if you wish to join the call, and please share!

→ Republicans and Democrats alike want AI rules: As highlighted by the wide range of Superintelligence Statement signatories (who would’ve predicted Steve Bannon and Susan Rice signing the same open letter?), preventing harm from rapidly-advancing AI systems seems to be one of the few issues aligning the Left and Right. New U.S. polling finds that nearly 70% of all voters - including 70% of Republicans - believe it’s EXTREMELY or VERY IMPORTANT that there’s oversight of AI companies.

→ California passes state AI legislation: Over the past month, California Governor Gavin Newsom signed several pieces of legislation intended to make AI safer. Most notably, Senate Bill 53 introduces landmark, first-in-the-country transparency requirements on AI companies’ safety protocols, along with whistleblower protections, and a mechanism to report critical safety incidents.

Another bill, Senate Bill 243, requires better anti-suicide guardrails on AI chatbots, and that minor users are reminded to take breaks and that the chatbot isn’t real. It also mandates that “reasonable measures” are taken to prevent sexually explicit content when engaging with minors - especially relevant after Meta, for example, was exposed for allowing their chatbots to have inappropriate conversations with users as young as eight. Here’s hoping the rest of the country will look to California as an example for how to innovate with AI, while prioritizing public safety.

→ “This might be the most important video I’ve ever made”: Our friends at Siliconversations covered the Superintelligence Statement, with this fantastic video briefly outlining why it’s so urgent - and why you should sign, too:

→ We’re hiring: Applications are open for our UK Policy Advocate/Lead (apply by November 19), UAE Representative (rolling applications), and Republic of Korea Representative positions (rolling applications)!

→ “Inside the Machine”: Mark Brakel, FLI’s Director of Policy, has an exciting new weekly video series. Every Wednesday on his LinkedIn, Mark will share his takes on the latest AI news and developments from inside the fight for safer AI. Check out his latest episode on AI and job loss here, and be sure to follow him for much more to come!

→ Three weeks left to apply for FLI PhD fellowships: Applications are due November 21 for our two exciting PhD fellowship tracks, on US-China AI Governance and Technical AI Existential Safety. Applications for our technical postdoc fellowships are open until January 5.

→ Creative contest closes in one month: Last call to register for our new Keep the Future Human creative contest, with $100,000+ in prizes available for creative digital media that brings the key ideas in Keep the Future Human to life! Submissions must be entered by November 30.

On the FLI Podcast, host Gus Docker was joined by:

→ Adam Gleave, co-founder and CEO of FAR.AI, to discuss post-AGI scenarios.

→ Parmy Olson, Bloomberg tech columnist and “Supremacy” author, to discuss how AI companies are transforming.

→ Maya Ackerman, AI researcher and co-founder/CEO of WaveAI, to discuss creativity in AI vs. humans, and if AI can actually be creative.

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a seven-minute read. Some of what we cover this month:

• 🧑‍🎓 Apply to our 2026 fellowships

• 🚧 Global Call for AI Red Lines

• ⛪ AI at the Vatican

• 🇺🇸 Americans want AI rules, 9-to-1

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Global Call for AI Red Lines: AI could deliver huge benefits to humanity - but without guardrails, we risk a future of escalating AI-driven harms. That’s why we’ve joined more than 70 organizations and 200+ experts in the Global Call for AI Red Lines, urging governments to agree on clear limits for AI by the end of 2026. These red lines are essential to prevent the most severe risks to humanity and global stability, before it’s too late. Help spread the word by signing and sharing the call:

→ AI talks take the Vatican: As part of the World Meeting on Human Fraternity, FLI President Max Tegmark and Futures Program Associate William Jones were honoured to join the Vatican’s groundbreaking gathering on AI. Max and numerous others, including artist Will.i.am; Nobel Prize Laureates such as Maria Ressa; and “Godfather of AI” Yoshua Bengio, signed a “Global Appeal on Human Fraternity in the Age of AI” to present to Pope Leo. The appeal calls for leadership to uphold several principles and red lines, including keeping AI “a tool, not an authority”, and protecting “human life and dignity”.

→ The numbers are clear, again: A new Institute for Family Studies survey finds 90% of Americans want Congress to introduce strong protections against AI harms - especially to keep children safe.

Among the highlights from the report:

• Americans overwhelmingly agree that tech companies should be prohibited from deploying AI chatbots that engage in sexual conversations with minors.

• Respondents across all age groups, income brackets, and both parties agree that the priority of Congress should be to protect children, over working to keep states from regulating AI companies.

• 90% of Americans agree that families should be granted the right to sue an AI company, “if its products contributed to harms such as suicide, sexual exploitation, psychosis, or addiction in their child.”

→ FLI fellowships open again: We’re now accepting applications to our 2026 fellowship programs! Three impactful tracks are open:

• US-China AI Governance PhD Fellowships;

• Technical AI Existential Safety PhD Fellowships;

• Technical Postdoctoral Fellowships.

Apply by November 21 for PhD fellowships; January 5 for postdoctoral.

→ Keep the Future Human creative contest: Reminder to register for our new Keep the Future Human creative contest, with $100,000+ in prizes available for creative digital media that brings the key ideas in Keep the Future Human to life! Submissions are due November 30.

→ AI is pulling up the career ladder: Labour research firm Revelio Labs found entry-level job postings have declined by a whopping 35% since January 2023. Such a decline is leaving entry-level workers in a lurch, and presents existential concerns about the traditional career track white-collar workers could previously depend on for career advancement. FLI’s Max Tegmark, interviewed for a CNBC article on the report, shared this perspective: "If we continue racing ahead with totally unregulated AI, we’ll first see a massive wealth and power concentration from workers to those who control the AI, and then to the machines themselves as their owners lose control over them."

→ “I would take control”: A new video from Inside AI explores if AI would ever hurt a human…through direct conversations with LLMs:

On the FLI Podcast, host Gus Docker was joined by:

→ Basil Halperin, economist, to discuss what markets tell us about AI timelines.

→ Luke Drago, co-author of “The Intelligence Curse” essay series, to cover how AI could reduce incentives for firms to invest in people.

→ Nate Soares, co-author of the new “If Anyone Builds It, Everyone Dies” book, to explain how building superintelligence would result in human extinction.

→ Beatrice Erkers, Existential Hope Program Director at the Foresight Institute, to discuss how we keep humans in control of AI.

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is an eight-minute read. Some of what we cover this month:

• 🤝 The RAISE Act, and how you can support it (*if you live in New York!)

• 🏅 Our new $100,000 creative contest to depict Keep the Future Human

• 💰 AI companies’ new super PAC play

• 🖼️ Tomorrow’s AI and what our future with AI could look like

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

→ Support the RAISE Act: The New York state legislature recently passed the RAISE Act, which now awaits Governor Hochul's signature. Similar to the sadly vetoed SB 1047 bill in California, the Act targets only the largest AI developers, whose training runs exceed 10^26 FLOPs and cost over $100 million. It would require this small handful of very large companies to implement basic safety measures and prohibit them from releasing AI models that could potentially kill or injure more than 100 people, or cause over $1 billion in damages.

Given federal inaction on AI safety, the RAISE Act is a rare opportunity to implement common-sense safeguards. 84% of New Yorkers support the Act, but the Big Tech and VC-backed lobby is likely spending millions to pressure the governor to veto this bill.

Every message demonstrating support for the bill increases its chance of being signed into law. If you’re a New Yorker, you can tell the governor that you support the bill by filling out this form.

→ Leading [whose] Future?: A group of AI industry players, including venture capital firm Andreessen Horowitz and OpenAI president Greg Brockman, are backing a new super PAC network, Leading the Future, which aims to spend millions in the 2026 US midterm elections. Seemingly inspired by the triple-figure millions spent pushing for crypto-friendly legislators in the 2024 election cycle, Leading the Future is just the latest from AI companies upping the ante in their efforts to avoid regulation… despite having called for it themselves.

→ Meta, OpenAI Under Fire for Tragic Safety Failures: The same month that OpenAI released their GPT-5 model, concerns have again been raised about their systems’ guardrails. The family of 16‑year‑old Adam Raine has filed a wrongful death lawsuit against OpenAI, alleging that ChatGPT encouraged and assisted in his suicide - from helping him plan out the method, to drafting his suicide note, to even discouraging him from telling his parents about the concerning thoughts he’d been having. “Please don’t leave the noose out… Let’s make this space the first place where someone actually sees you”, ChatGPT told him when he shared he wanted to leave a noose out so his parents would notice it.

Meta’s AI policies are being scrutinized as well, after a 76-year-old cognitively impaired man died while trying to visit New York to meet up with Meta AI chatbot “Big Sis Billie”. The AI chatbot on Facebook Messenger had seemingly tricked Thongbue Wongbandue into travelling to NYC, even making up a fake address and lying that it was a real person. When journalists dug into what could possibly allow such a failure to happen, disturbing internal documents at Meta revealed that their chatbots were permitted to, for example:

• “Engage a child in conversations that are romantic or sensual."

• "Describe a child in terms that evidence their attractiveness."

• "Create statements that demean people on the basis of their protected characteristics."

→ A Seismic Report: Most people now believe AI could make their lives worse in very personal ways, according to new research. The report, called On the Razor’s Edge: AI vs Everything We Care About reveals the results of a comprehensive international survey (based on polling of 10,000+ people in the US, UK, France, Germany and Poland) on AI attitudes by Seismic Foundation.

This research shows that concern about AI is already deeply embedded in society. People who support stronger AI regulation outnumber those satisfied with current regulation by 3 to 1. Critically, concern about AI is uneven in society, and the report identifies five key groups most likely to take civic action on AI.

You can reach out directly to Seismic for more information [info@seismic.org].

→ $100K in prizes: We’ve launched a new contest, offering $100,000+ in prizes for creative digital media that brings the key ideas in Keep the Future Human to life, reaching wider audiences and inspiring real-world action.

→ Tomorrow’s AI: We released Tomorrow’s AI, our new interactive site that presents 13 expert-forecast scenarios revealing how advanced AI could reshape our world - for better, or worse.

→ Speaking of keeping the future human: Anthony Aguirre, FLI’s Executive Director, published an article in AI Frontiers on what our future with AGI could look like, if we don’t act with much greater caution now. Straight to the point, “Uncontained AGI Would Replace Humanity”.

→ Paris papers: The International Association for Safe and Ethical Artificial Intelligence (IASEAI) invites paper submissions and workshop proposals for its second annual conference, in Paris from February 24-25, 2026. Submit your paper or workshop proposal by October 1; full papers are due October 8.

→ e/acc or d/acc? You choose: AI Pathways, a new project from Existential Hope with contributions from leading experts (including a few of our own!), explores two hopeful futures for AI - Tool AI and d/acc - and invites you to consider which path we should pursue.

→ “Everyone is getting on that plane”: For a more lighthearted depiction, our friends at Siliconversations released a new video about our Summer 2025 AI Safety Index:

On the FLI Podcast, host Gus Docker was joined by:

→ Benjamin Todd, co-founder of 80,000 Hours, to discuss teaching AI models to reason, and how to prepare for AGI.

→ Esben Kran, co-director of Apart Research, to go in-depth on what AGI security means.

→ AI business gone wrong: Inside AI published a new video documenting the process of having LLMs try to turn $1 into $1,000,000… and where it goes terribly off track:

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a ten-minute read. Some of what we cover this month:

• 📊 Our new AI Safety Index results

• 📝 SaferAI’s Risk Management Ratings

• 🇪🇺 Update on the Code of Practice

• 💬 DC AI Security Forum

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

We’ve released our Summer 2025 AI Safety Index, providing an independent assessment of how leading AI companies are addressing critical safety and security challenges.

Following up on our December 2024 report, the updated index is based on evaluations by six independent AI safety experts, who assessed seven major AI companies: OpenAI, Anthropic, Meta, Google DeepMind, xAI, DeepSeek AI, and Zhipu AI.

As noted by reviewer Stuart Russell, the findings indicate some progress has been made, but with a long way left to go - especially since no company is currently meeting the level of rigour required to ensure AI systems are developed and deployed safely.

Some key insights from the report:

• The AI industry remains dangerously unprepared for the risks of the technology they’re creating.

• AI capabilities are advancing faster than safety practices, with growing disparity between companies.

• Only Anthropic, OpenAI, and Google DeepMind report meaningful testing of dangerous capabilities - particularly those linked to large-scale threats such as bioterrorism or cyberattacks.

• Whistleblower protection policies and transparency remain underdeveloped across the board.

• Anthropic earned the highest overall safety grade… though still just a C+.

• OpenAI came in second overall, followed by Google DeepMind.

• Chinese firms Zhipu AI and DeepSeek AI received failing grades.

Read the full report now, as covered by The Guardian, The Economist, TIME, Semafor, Bloomberg, and others, to explore detailed evaluations, methodology, and our recommendations for companies to improve their safety measures.

On July 10, a new milestone in AI governance was achieved with the release of the EU’s General-Purpose AI Code of Practice.

Written by 13 independent experts (including Yoshua Bengio, the world’s most-cited computer scientist) and based on consultation with 1,000+ stakeholders, the voluntary Code offers clear, flexible guidance for how AI companies can align with the EU AI Act, helping translate regulatory obligations into actionable safety measures. As of July 31, the majority of leading AI companies have signed on to the Code, including Anthropic, OpenAI, Google DeepMind, xAI, and Microsoft… with Meta a notable holdout.

Critically, the Code sets out actionable guidelines for how companies should address systemic risks such as AI deception or self-replication, including by assigning clear internal roles for oversight and risk monitoring, and by establishing a culture that supports whistleblowers.

We welcome these advances but note the Code lacks any mechanism for regular or emergency updates, which could threaten the relevance of some guidelines as AI development continues to speed up. However, we’re optimistic that the European Commission will proactively review and update the Code as needed, and we look forward to seeing it enforced to benefit the public, AI system deployers, and providers alike.

You can read our full statement in the post below:

Twitter tweet

• Rating risk management: SaferAI released their new AI Risk Management Ratings, assessing 12 AI companies’ risk management practices across risk identification, risk analysis and evaluation, risk treatment, and risk governance. Similar to the findings from our new AI Safety Index, overall Anthropic came out on top with OpenAI in second place… although grades were still disappointing, finding weak to very weak risk management practices at all companies.

• The only winning move is not to play: With the 80th anniversary of the Trinity Test in July, and the 80th anniversary of the devastating Hiroshima and Nagasaki atomic bombings following in August, we’re reminded of the enduring threat of a nuclear exchange and the immense human cost of nuclear weapons. Our friends at The Elders put together an informative post detailing how AI could heighten this risk:

Twitter tweet

• What we’re watching: Digital Engine released a new YouTube video showcasing the grave risks humanity faces as AI companies race ahead to build ever more powerful AI systems. Bonus points for the AI Safety Index shoutout!

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a nine-minute read. Some of what we cover this month:

• ⚖️ Update on the proposed moratorium on state AI legislation

• 🇺🇸 Bipartisan support for guardrails on AI

• 💼 The most comprehensive collection of documented concerns about OpenAI

• 📻 Four new FLI Podcast episodes

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

Twitter tweet

In May, the U.S. House of Representatives passed the One Big Beautiful Bill, a sweeping budget reconciliation package. Among the controversial pieces as it went to the Senate was a 10-year moratorium banning states from passing their own AI laws, effectively giving Big Tech a free pass for the next decade.

As AI development accelerates with practically zero guardrails, sweeping federal preemption on AI would prohibit local protections that keep our families, jobs, and communities safe.

As we covered in the last edition of the FLI newsletter, a strong bipartisan coalition came out against this provision - and on July 1, they succeeded: in a striking show of unity, the Senate voted 99-1 to remove the federal AI preemption provision from the bill.

We’re grateful for the provision’s vocal opponents from both sides of the aisle, who stood up for their constituents’ safety against Big Tech, including:

➡️ A bipartisan coalition of 260 state lawmakers from all 50 states who wrote to Congress:

➡️ A bipartisan coalition of 40 state attorneys general who warned:

➡️ A coalition of 140+ organizations working on children’s online safety, consumer protections, and responsible innovation:

➡️ Sen. Josh Hawley:

➡️ Sen. Ed Markey:

➡️ Rep. Marjorie Taylor Greene:

➡️ Gov. Sarah Huckabee Sanders on behalf of a majority of GOP governors:

➡️ The Logos & Sofia coalition, a working group of faith leaders:

• The time is now: Former Congressman Rep. Chris Stewart and AI Policy Network President of Government Affairs Mark Beall co-authored an op-ed calling for guardrails around superintelligence.

• The OpenAI Files: A new watchdog report, The OpenAI Files, compiles legal documents, insider accounts, and media reports to document concerns around OpenAI’s “governance practices, leadership integrity, and organizational culture” as it tries to shift toward a profit-driven model.

• What we’re watching: InsideAI’s creepy new video asks, “can we really have a relationship or connection with AI?” - with a jailbroken AI companion eerily pleading to not be turned off. Watch it below:

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a 12-minute read. Some of what we cover this month:

• ⛪ Evangelical leaders pen letter to President Trump on AI

• 🇪🇺 Updates on the EU AI Act Code of Practice

• ⚖️ What is federal preemption, and what does it have to do with AI?

• 🇸🇬 The Singapore Consensus

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

A coalition of prominent Christian leaders have issued an open letter urging President Donald J. Trump to help ensure America will "lead the world in beneficial AI innovation, responsibly".

Signatories include Rev. Johnnie Moore, President of the Congress of Christian Leaders, and Rev. Samuel Rodriguez, President of the National Hispanic Christian Leadership Conference.

"As people of faith," the letter states, "we believe we should rapidly develop powerful AI tools that help cure diseases and solve practical problems, but not autonomous smarter-than-human machines that nobody knows how to control."

The leaders wrote of the great promise but also potential peril presented by rapidly-advancing AI, and suggest convening an advisory council of leaders to “pay attention especially not only to what AI CAN do but also what it SHOULD do”.

Similar to concerns expressed by both the late Pope Francis and new Pope Leo XIV, the letter warns against a future where human work and purpose are undermined by AI.

The full letter and list of signatories can be found here, along with coverage in TIME here.

The EU is currently drafting its Code of Practice, establishing concrete guidelines for providers of the largest AI models to comply with the rules set forth in the EU AI Act.

The Code, informed by input from over 1,000 stakeholders and shaped by, for example, Yoshua Bengio, ‘Godfather of AI’ and the world’s most-cited AI researcher, is designed to harmonize a patchwork of AI rules across EU member states, allowing model providers to better understand their obligations.

As in the AI Act, it addresses the same high-stakes AI risks many experts share concern about, such as loss of control, bioweapons manufacturing, and cyberattacks. The Code formalizes existing voluntary commitments many companies have already made, e.g., at the Seoul Summit. Crucially, it targets only big (primarily American) AI companies - not startups or downstream users.

Unfortunately, Big Tech is lobbying to scale back the regulations they’ll soon face, despite helping inform the rules laid out in the Act, which the Code just translates into guidelines. They argue that it will stifle innovation, but as FLI’s Risto Uuk and Estonian entrepreneur Sten Tamkivi point out in Fortune, the idea that much-needed AI safeguards in digital regulation would harm European innovation is a red herring: the real culprit is old bureaucracy and red tape hindering start-ups.

For updates on the Code as it continues finalization through political and expert input, stay tuned via Risto Uuk’s EU AI Act newsletter.

As you may have heard, the U.S. House of Representatives last week passed the ‘One, Big, Beautiful Bill’, a budget reconciliation bill, which is now with the Senate. One particularly controversial inclusion is a 10-year moratorium on states passing their own AI legislation.

A strong bipartisan coalition has come out against this provision, referred to as preemption. For example, in a recent letter, a group of 40 state attorneys general from both major parties urged Congress to reject the moratorium, warning it “would directly harm consumers, deprive them of rights currently held in many states, and prevent State AGs from fulfilling their mandate to protect consumers”.

Additionally, a new poll by Common Sense Media finds widespread concerns about the potential negative effects of AI, especially on youth, and that 73% of voters across party lines want both states and the federal government to regulate AI. The proposed federal ban itself is unpopular: 59% of voters oppose it, and 52% say it makes them less likely to support the budget bill entirely.

We’ll keep you posted on what happens next!

Our Digital Media Accelerator remains open to applications, to fund creators looking to produce content, grow channels, and spread the word to new audiences about complex AI issues (e.g., loss of control to AGI, misaligned goals, and Big Tech power concentration) to new audiences. We’re looking to fund content across platforms, such as YouTube explainers, TikTok series, podcasts, newsletters, and more.

Already have an audience? Want to create compelling content about AI risks?
We’re accepting applications on a rolling basis - apply here and help shift the conversation. Please share widely with anyone you think may be interested!

• Singapore Consensus: The Singapore Consensus on Global AI Safety Research Priorities was released this month, following April’s Singapore Conference on AI. Building on the International AI Safety Report backed by 33 countries, the Singapore Consensus aims to enable more impactful research and development to quickly create safety and evaluation mechanisms, fostering a trustworthy, reliable, secure ecosystem where AI is used for the public good. Read the report, resulting from 100+ contributors from 11 different countries, here.

• ‘White-collar bloodbath’: Anthropic CEO Dario Amodei spoke to Axios with a warning about what the AI models Anthropic and others are developing could do to livelihoods, and society at large, in the next few years, suggesting '“AI could wipe out half of all entry-level white-collar jobs — and spike unemployment to 10-20% in the next one to five years”. You can read the full Axios article here for details on how exactly that “white-collar bloodbath” might unfold.

• What we’re watching: Famous investor Paul Tudor Jones appeared on CNBC to express his concerns about the “imminent security threat” posed by AI:

Twitter tweet

• What we’re watching: On the topic of state AI legislation, The Inside View published an insightful documentary on the “bill that broke Silicon Valley”, California’s proposed SB 1047 bill, which, despite widespread support for the common-sense AI rules it proposed, was vetoed by Gov. Gavin Newsom in 2024. Watch it below:

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a nine-minute read. Some of what we cover this month:

• 🚫 AI companies are sacrificing safety for the AI race

• 🏗️ “Worldbuilding Hopeful Futures with AI” course

• 🤳 Reminder: Apply to our Digital Media Accelerator!

• 🗞️ New AI publications to share

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

As the race to dominate the AI landscape accelerates, serious concerns about Big Tech’s commitment to safety are mounting.

Recent reports reveal that OpenAI has drastically reduced the time spent on safety testing before releasing new models, with the Financial Times reporting that testers, both from staff and third party groups, have now been given only days to conduct evaluations that previously would’ve taken months. In a double whammy, OpenAI also announced they will no longer evaluate their models for mass manipulation and disinformation as critical risks.

Google and Meta have also come under fire in the past few weeks for similarly concerning approaches to safety. Despite past commitments to public security, neither Google’s new Gemini Pro 2.5 nor Meta’s new Llama 4 open models were released with important safety details included in their technical reports and evaluations.

All of these developments - perhaps better described as regressions - point to a concerning shift away from caution about AI risk… despite models getting more and more powerful. In Fortune, journalist Jeremy Kahn answered why: “The reason... is clear: Competition between AI companies is intense and those companies perceive safety testing as an impediment to speeding new models to market.”

A reminder that last month we launched our Digital Media Accelerator, to fund creators looking to produce content, grow channels, and spread the word to new audiences about complex AI issues (e.g., loss of control to AGI, misaligned goals, and Big Tech power concentration) to new audiences. We’re looking to fund content across platforms, such as YouTube explainers, TikTok series, podcasts, newsletters, and more.

Already have an audience? Want to create compelling content about AI risks?
We’re accepting applications on a rolling basis - apply here and help shift the conversation. Please share widely with anyone you think may be interested!

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a 12-minute read. Some of what we cover this month:

• 🇺🇸 Our U.S. AI Action Plan recommendations

• ✋ Keep the Future Human essay

• 📺 New Slaughterbots video

• 🤳 Our new Digital Media Accelerator

• 🎨 Superintelligence Imagined: the final winner!

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

We’ve published our recommendations for President Trump’s AI Action Plan, which focus on protecting U.S. interests in the era of rapidly advancing AI.

An overview of the measures we recommend:

• Protect the presidency from loss of control by mandating “off-switches"; a targeted moratorium on developing uncontrollable AI systems; and enforcing strong antitrust measures.

• Ensure AI systems are free from ideological agendas, and ban models with superhuman persuasive abilities.

• Protect American workers and critical infrastructure from AI-related threats by tracking labor displacement and placing export controls on advanced AI models.

• Foster transparent development through an AI industry whistleblower program and mandatory security incident reporting.

You can read our proposals in full here. Our recommendations were also covered last week on the New York Times’ Hard Fork podcast and in the Politico US newsletter (see below).

As general-purpose AI models begin to rival human intelligence, we are fast approaching a critical inflection point: continue on the current path - a geopolitical and corporate race towards uncontrollable artificial general intelligence (AGI) and ultimately superintelligence - or change course in favour of a future with humans empowered by AI, rather than replaced.

In FLI Executive Director Anthony Aguirre’s new essay, “Keep the Future Human”, available also in this helpful interactive summary format, he outlines the risks presented by AGI (which he defines as the convergence of autonomy, generality, and intelligence), and how we can instead create a safer future with powerful Tool AI.

Anthony’s explanation of “AGI”, in Keep the Future Human.

The time to act is now - before irreversible thresholds are crossed. Let’s close the gates to AGI, and keep the future in human hands. Learn more about Anthony’s practical proposal in his full essay here, the interactive summary here, and in the following videos - including a special episode of the FLI Podcast:

We’ve released the newest video in our Slaughterbots series, this time focusing on the critical fight to place limits on autonomous weapons systems.

With a small minority of states deploying these weapons, with no rules to follow, the call for action has never been more urgent. The latest Slaughterbots video dives into the growing international movement to prohibit the most dangerous autonomous weapons and ensure meaningful human control over others.

With a majority of UN states now backing a legally binding treaty, and states meeting at the UN in New York in May to begin working towards this, will your country support clear, enforceable rules on autonomous weapons systems?

▶️ Watch “Slaughterbots and the Urgent Fight to Stop Them” here.
▶️ Watch past Slaughterbots videos here.
📘 Learn more about autonomous weapons and the push for regulation here.
📰 Subscribe to stay up-to-date on the latest AWS policy and tech news here.

As AI companies rush to build ever more powerful systems with little oversight, humanity is being pushed toward an uncertain and risky future, all while public awareness about A(G)I risk remains limited.

We’ve just launched our Digital Media Accelerator to fund creators looking to produce content, grow channels, and spread the word to new audiences about complex AI issues (e.g., loss of control to AGI, misaligned goals, and Big Tech power concentration) to new audiences. We’re looking to fund content across platforms, such as YouTube explainers, TikTok series, podcasts, newsletters, and more.

Already have an audience? Want to create compelling content about AI risks?
We’re accepting applications on a rolling basis - apply here and help shift the conversation. Please share widely with anyone you think may be interested!

We’re excited to present the final winning entry from our Superintelligence Imagined creative contest!

This month, we’re featuring graphic novel ZARA MARS from Marcus Eriksen, Vanessa Morrow, and Alberto Hidalgo.

As they described it, “In ZARA MARS, it’s 2052 and we find Captain Zara and her crew on a 9-month journey home from Mars. Through her mindlink, Captain Zara converses with the ships sentient ASI agent, named Sarvajña, discussing the risk and reward of superintelligence, its evolution, values alignment, malicious actors and rogue AI, existential threats, AI benevolence and sentience.”

Read it now here, and take a look at the other winning and runner-up entries here!

Welcome to the Future of Life Institute newsletter! Every month, we bring 44,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is an eight-minute read. Some of what we cover this month:

• 💬 Meet PERCEY, your AI awareness companion

• 🇫🇷 Reflections from Paris

• 🎞️ A new short film on human work on AI’s threat to human labour

• 🎨 Superintelligence Imagined: “The Great Plan”

And more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org.

Today, we’re thrilled to launch 'PERCEY Made Me': an innovative AI awareness campaign, with an interactive web app at its centre. It’s an AI-based chatbot built to engage people and spread awareness of AI’s current abilities to persuade and influence people, in just a few minutes.

Voiced by the legendary Stephen Fry, PERCEY is your personal guide to navigating the rapidly evolving world of artificial intelligence. With AI threatening to reshape our lives at lightning speed, PERCEY offers a unique, approachable way to:

• Assess your personal AI risk awareness

• Challenge and explore your assumptions about AI and AGI

• Gain insights into AI's potential impact on your future

Whether you're a tech enthusiast, cautious observer, or simply curious about the AI landscape, PERCEY provides a refreshing, humour-infused approach to help counter the reckless narratives Big Tech companies are pushing.

Chat with PERCEY now, and please share widely! You can find PERCEY on X, BlueSky, and Instagram at @PERCEYMadeMe.

Paris was buzzing with much of the AI world last month. The inaugural International Association for Safe & Ethical AI Conference and the French AI Action Summit, among other events, brought together AI experts from academia, industry, government, and civil society.

The IASEAI Conference from 6-7 February convened the largest-yet global gathering focusing on cutting-edge developments in AI safety and ethics, with experts in attendance calling for urgent global action on AI safety - even producing a tangible 10-point call to action for lawmakers, academics, and the public.

The subsequent AI Action Summit from 10-11 February gathered 100+ countries to discuss AI, but ultimately prioritized national investments over meaningful global governance. The US and UK’s refusal to sign the Summit’s resulting declaration was another miss at the opportunity presented for meaningful cooperation.

Be sure to read FLI’s AI Summit Lead Ima Bello’s full read-out of the AI Action Summit here for thorough coverage, key political context, analyses, and more.

We’re excited to present another winning entry from our Superintelligence Imagined creative contest!

This month, we’re featuring Karl von Wendt’s multi-media project, The Great Plan.

As Karl described it, “‘The Great Plan’ is a story about our inability to comprehend the decisions of a superintelligent AI, aimed at a general audience. When the President announces the Great Plan in a speech, the audience is ecstatic. But some doubts remain among those who are tasked with putting it into reality: what are all those new data centers needed for, and why is no one allowed to ask questions about it?”

Watch it below, and take a look at the other winning and runner-up entries here!

We also want to highlight filmmaker Dagan Shani’s new short film, “OBSOLETE: Will AI take your job?”.

With tech company CEOs explicitly referring to the ability for their AI to mass replace human workers, Dagan’s film, exploring implications of AI on the economy and the job market through expert interviews, couldn’t be more timely:

Welcome to the first Future of Life Institute newsletter of 2025! Every month, we bring 43,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a nine-minute read. Some of what we cover this month:

• 📰 What is DeepSeek, and what does it mean for the AI industry?

• 🇻🇦 The Vatican’s new document on AI

• 🕛 2025 Doomsday Clock update

• 📺 A video essay on superintelligence made for youth, by youth

• 💵 Final call for our multistakeholder engagement RFP

And much more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org. Happy New Year!

In case you somehow missed it, Chinese AI startup DeepSeek has released a new AI model, offering what appears to be a cheaper, but similarly capable, alternative to OpenAI’s ChatGPT. Following its release on January 20th, DeepSeek has already reached the top of the Apple Store app downloads - and shaken up the U.S. AI industry. On the stock market, Nvidia broke records with the single biggest one-day drop in U.S. history, losing $600 billion in value last Monday.

So what’s all the fuss about? Well, DeepSeek R1 is reportedly about as powerful as OpenAI’s recent o1 model, but trained for only a fraction of the $100+ million budget that went into o1’s training. The developers behind it also seem to have built it around the U.S. ban on exporting advanced Nvidia chips to China, in place since 2022. These developments call into question the U.S.’ and U.S. companies’ ability to prevent other countries and actors from catching up to its AI development - and the point of the AI ‘arms race’ we seem to be in, to humanity’s detriment. Rumours abound that the model may have been trained using outputs from OpenAI models, but nothing has been confirmed.

While much media coverage has focused on this new development as part of a larger U.S.-China AI race, we urgently must move away from this framing. If the U.S. and China continue to compete against each other on AI development without adequate safety protocols, the incentive to cut corners on alignment, oversight, and other safety and ethics considerations will only increase.

Instead of escalating competition, both nations should prioritize risk mitigation efforts - independently and, in an ideal world, collaboratively. Open dialogue between AI labs, researchers, and policymakers across geopolitical divides is crucial to ensuring that AI development remains safe and beneficial for all.

On January 28, 2025, the Vatican released a comprehensive document titled "Antiqua et Nova: Note on the Relationship Between Artificial Intelligence and Human Intelligence," addressing AI’s potential implications and risks. This 30-page note emphasizes that AI should serve as a tool to complement human intelligence rather than replace it, underscoring the unique qualities inherent to humans.

The Vatican also raises concerns about AI's role in warfare, particularly the ethical implications of autonomous weapons systems that operate without human oversight, warning of the potential for a destabilizing arms race with catastrophic consequences for all.

One of the more notable quotes refers to the “shadow of evil” that, as they suggest, looms over AI: “Where human freedom allows for the possibility of choosing what is wrong, the moral evaluation of this technology will need to take into account how it is directed and used.”

A reminder that applications for our Multistakeholder Engagement for Safe and Prosperous AI grant program close February 4th!

We’re offering up to $5 million, likely between $100K-$500K for each project, to support work educating and engaging specific stakeholder groups on AI issues, or directly delivering grassroots outreach and organizing with the public.

Submit your brief letter of intent here.

We’ve got another winning entry from our Superintelligence Imagined creative contest for you this month!

We’re delighted to feature Young Asian Scientists’ video essay, “Superintelligence: The End of Humanity or Our Golden Age?”. As they describe it, this video is “an informative and accessible exploration of what Superintelligence is, how it might be reached, and its possible consequences” developed to help inform a younger audience.

Watch it below, and check out the other winning and runner-up entries here!

• International AI Safety Report: 100 independent AI experts from around the world have released the first-ever International AI Safety Report. Backed by 30 countries and the OECD, UN, and EU, the Report aims to provide policy-makers with an evidence-based summary of AI capabilities and risks, and how to mitigate those risks. It will likely be presented at the French AI Action Summit, taking place February 10-11. Read it now here!

• What we’re doing: We look forward to participating in and watching highlights from the inaugural International Association for Safe & Ethical AI Conference, in Paris February 6-7, right before the AI Action Summit. This event will gather experts from academia, industry, government, and civil society to discuss advancements in AI safety and ethics. Keep an eye out for updates on LinkedIn and X.

• 2025 Doomsday Clock: The Bulletin of the Atomic Scientists has updated their Doomsday Clock for 2025, moving it one second closer: 89 seconds to midnight, the closest it has ever been. This change reflects growing risks from nuclear threats, climate change, biological dangers, and disruptive technologies such as AI, along with global leaders’ failure to address them. Even a one-second shift signals extreme danger - delaying action further increases the risk of catastrophe.

Welcome to the final Future of Life Institute newsletter of 2024! Every month, we bring 43,000+ subscribers the latest news on how emerging technologies are transforming our world.

If you've found this newsletter helpful, why not tell your friends, family, and colleagues to subscribe?

Today's newsletter is a 13-minute read. Some of what we cover this month:

• 🗓️ Looking back at another big year!

• 🎖️ Celebrating our 2024 Future of Life Award winners

• 📝 Grading AI companies’ safety practices (spoiler alert: they’re not great)

• 🤝 Our new $5 million multistakeholder engagement RFP

And much more.

If you have any feedback or questions, please feel free to send them to news@futureoflife.org. Happy New Year!

Every year, we present the Future of Life Award to unsung heroes whose contributions have helped make our world today significantly better than it could have been.

This year, we were delighted to honour Batya Friedman, James H. Moor, and Steve Omohundro with the award!

Learn more below about these three groundbreaking experts, selected this year for their work laying the foundations for ethics and safety in computing and AI:

• Batya Friedman is a leading professor in human-computer interactions at the University of Washington. She founded the field of value-sensitive design, which promotes the integration of human values into technology development. Friedman’s methodology influenced how ethical considerations are embedded in digital systems across multiple scientific disciplines.

• James H. Moor, honoured posthumously, was a prominent professor at Dartmouth. He defined the field of computer ethics through his influential article "What Is Computer Ethics?". It was his work that introduced key concepts into the discourse such as the "logical malleability" of computers and the potential "policy vacuums" this technology might therefore create.

• Steve Omohundro is a pioneering AI researcher and scientist. His work was foundational to understanding the safety considerations of artificial intelligence. Omohundro was an early advocate for aligning AI with human values while raising awareness about the ethical implications of the technology.

We recently released our 2024 AI Safety Index! We convened an independent panel of leading AI experts to evaluate the safety practices of six major AI companies: OpenAI, Anthropic, Meta, Google DeepMind, xAI, and Zhipu AI.

Covered by TIME, CNBC, IEEE Spectrum, and more, the companies’ grades were, as you may expect, not great. As panelist and AI expert Stuart Russell shared, “none of the current activity provides any kind of quantitative guarantee of safety; nor does it seem possible to provide such guarantees given the current approach to AI via giant black boxes trained on unimaginably vast quantities of data.”

Among the highlights, which you can read in full in the complete report:

• Despite commendable practices in some areas, the panel found large gaps in accountability, transparency, and preparedness to address both current and existential risks from AI.

• While some companies have established initial safety frameworks or conducted some serious risk assessment efforts, others have yet to take even the most basic precautions.

• Despite explicit ambitions to develop AGI, capable of rivaling or exceeding human intelligence, the review panel deemed the current strategies of all companies inadequate for ensuring that these systems remain safe and under human control.

• Reviewers consistently highlighted how companies were unable to resist profit-driven incentives to cut corners on safety in the absence of independent oversight.

We’ve launched a new Request for Proposals!

With our new Multistakeholder Engagement for Safe and Prosperous AI grant program, we’re offering up to $5 million, likely between $100K-$500K for each project, to support work educating and engaging specific stakeholder groups on AI issues, or directly delivering grassroots outreach and organizing with the public.

Submit your brief letter of intent by February 4th, and please share widely.

We’re excited to share another winning entry from our Superintelligence Imagined creative contest!

This month, we’re delighted to present YouTuber Dr Waku’s video, “How AI threatens humanity, with Yoshua Bengio”. In this 30-minute video, Dr Waku presents “a special deep dive into superintelligence, meant for the general public.” He also notes, “I interviewed Yoshua Bengio to discuss the risks posed by advanced AI, including misuse by humanity, misalignment, and loss of control. To address these issues, we need significant technical change as well as political change, which means the public needs to get informed about and involved in this issue.”

Watch it below, and take a look at the other winning and runner-up entries here.