An updated look at where AI red teaming stands nine months after the original piece, and why the gap between what we test and what actually breaks has only gotten wider.

Last July, I wrote about a Stanford and Georgetown paper that argued AI red teaming was too narrowly focused on individual model vulnerabilities. The authors made the case for two levels of red teaming: micro-level (testing the model itself) and macro-level (testing the entire system lifecycle, including how humans and institutions interact with it). At the time, the argument felt slightly ahead of the conversation. Most organizations were still figuring out prompt injection, and "sociotechnical risk" sounded academic.

Nine months later, the landscape has shifted enough that the paper reads less like a forward-looking proposal and more like a description of problems we are actively failing to solve.

What Changed

Three things happened since July 2025 that make the original argument sharper.

Agentic AI went mainstream. The paper talked about risks that emerge from complex interactions between models, users, and environments. That was somewhat theoretical when most deployments were chatbots and content generators. It is no longer theoretical. Gartner projects that 40% of enterprise applications will embed task-specific AI agents by 2026, up from under 5% in 2025. These agents call APIs, access data stores, execute workflows, and make decisions across multi-step chains. The attack surface is not the model; it is the entire execution environment.

OWASP published the Top 10 for Agentic Applications. Released in late 2025 with input from over 100 practitioners, this framework classifies the risks the original paper was gesturing at. Agent Goal Hijacking (ASI01) and Tool Misuse (ASI02) sit at the top. These are not prompt-level vulnerabilities. They are system-level failures where an agent's mission gets redirected across many execution steps, or where legitimate tool access gets exploited in ways the developers never anticipated. The OWASP list makes it concrete: red teaming agents means testing systems that act, not components that respond.

The US policy environment shifted. Biden's Executive Order 14110 had required red teaming for high-risk AI models and tasked NIST with developing guidelines. Trump's Executive Order 14148 rescinded it in January 2025, replacing the safety-focused framework with one centered on deregulation and innovation leadership. The practical effect is that mandatory red teaming requirements for frontier models evaporated at the federal level. NIST continues its AI Risk Management Framework work and launched an AI Agent Standards Initiative in early 2026, but the regulatory pressure that was pushing companies toward structured red teaming has weakened considerably. Organizations that depended on regulatory mandates to justify their red teaming programs now need to justify them on their own terms.

Where the Original Argument Holds Up

The core thesis still stands: most red teaming focuses on model-level bugs while the serious risks are systemic. If anything, agentic AI has proven the point more aggressively than the authors probably expected.

The paper's recommendation to build multifunctional red teams (ML engineers, social scientists, domain experts, security practitioners) looks even more necessary now. You cannot red-team an agentic system by throwing adversarial prompts at it. You need people who understand the business process the agent is embedded in, the data flows it can access, the authorization boundaries it should respect, and the failure modes that emerge when it chains together a sequence of individually reasonable actions that produce an unreasonable outcome.

The call for continuous red teaming over one-time assessments has also aged well. The best practice emerging in 2026 is integrating adversarial testing into CI/CD pipelines so that model updates, prompt changes, or agent reconfigurations automatically trigger attack suites. Red teaming as a pre-launch checkbox is not viable when your system's behavior changes every time you update a prompt template.

Where the Original Piece Was Too Conservative

Looking back at what I wrote in July, I underestimated a few things.

First, the speed at which agents would move from experimental to production. The paper framed macro-level red teaming as something organizations should prepare for. In practice, many organizations deployed agentic systems before they had any red teaming program at all, let alone a macro-level one. The gap is not "we test models but not systems." For a lot of organizations, the gap is "we do not test."

Second, I did not spend enough time on the supply chain dimension. Agents consume tools, plugins, and external data sources. Each of those is a trust boundary. Indirect prompt injection, where malicious instructions arrive through untrusted external content rather than direct user input, showed up in 73% of production AI deployments in 2025. Multi-agent denial-of-service attacks succeeded in over 80% of tests in one ACL study. These are not edge cases. They are the default attack surface for any system that lets an LLM interact with external data.

Third, I glossed over the organizational incentive problem. The paper recommended transparency and external testing, and I nodded along without pushing hard enough on why that is so difficult. Internal red teams operate under institutional constraints. They may not test scenarios that would embarrass leadership or challenge product decisions. External red teaming and independent disclosure mechanisms are not nice-to-haves; they are structural necessities for surfacing the risks that internal teams are incentivized to overlook.

What Matters Now

If you are building or deploying AI systems in 2026, here is what I would emphasize differently than I did last July:

Test the system, not the model. This was the paper's thesis and it is now the operational reality. If your agent can read emails, query a database, and send Slack messages, your red team needs to test what happens when a poisoned email redirects the agent to exfiltrate the database contents via Slack. Model-level jailbreak testing does not find this.

Adopt the OWASP ASI framework. It did not exist when I wrote the original piece. It does now, and it gives red teams a structured taxonomy for agentic risks. Use it. It covers goal hijacking, tool misuse, delegated trust failures, memory poisoning, and the other failure modes that are specific to autonomous systems.

Do not wait for regulation. The federal regulatory environment for AI safety is weaker than it was a year ago. NIST is still doing good work, but mandatory requirements are not coming soon. If your red teaming program only exists because a regulation says it should, it will not survive the current policy climate. Build the program because the risk is real, not because someone told you to.

Red team continuously, not periodically. Wire adversarial testing into your release process. When a prompt changes, when a tool gets added, when an agent's scope expands, test it. The systems that break in production are the ones that changed since the last time anyone looked.

Bring in outsiders. Your internal team has blind spots shaped by the same incentive structures that built the system. Independent red teaming is not a luxury. It is how you find the things you are not looking for.

The Bigger Picture

The Stanford and Georgetown paper was right about the direction. The field needed to move from model-level adversarial testing to system-level resilience evaluation. Nine months later, the need is more urgent and the tools to address it are starting to materialize. But the gap between where most organizations are and where they need to be has widened, not narrowed.

AI red teaming in 2026 is not about finding clever jailbreaks. It is about understanding how autonomous systems fail when they interact with messy, adversarial, real-world environments, and building the organizational discipline to test for that continuously. The paper gave us the framework. The question now is whether the industry will actually use it.

Matt James is a Product Security practitioner focused on threat modeling, AI security, and building security programs that work in practice. He writes at odnd.com.

Reference:
Sharkey, L., Pasquinelli, M., Cheng, B., Dobbe, R., et al. Operationalizing Red Teaming for AI Systems. arXiv preprint arXiv:2507.05538. July 2025.

Nine months later, the landscape has shifted enough that the paper reads less like a forward-looking proposal and more like a description of problems we are actively failing to solve.

What Changed

Three things happened since July 2025 that make the original argument sharper.

Agentic AI went mainstream. The paper talked about risks that emerge from complex interactions between models, users, and environments. That was somewhat theoretical when most deployments were chatbots and content generators. It is no longer theoretical. Gartner projects that 40% of enterprise applications will embed task-specific AI agents by 2026, up from under 5% in 2025. These agents call APIs, access data stores, execute workflows, and make decisions across multi-step chains. The attack surface is not the model; it is the entire execution environment.

OWASP published the Top 10 for Agentic Applications. Released in late 2025 with input from over 100 practitioners, this framework classifies the risks the original paper was gesturing at. Agent Goal Hijacking (ASI01) and Tool Misuse (ASI02) sit at the top. These are not prompt-level vulnerabilities. They are system-level failures where an agent's mission gets redirected across many execution steps, or where legitimate tool access gets exploited in ways the developers never anticipated. The OWASP list makes it concrete: red teaming agents means testing systems that act, not components that respond.

The US policy environment shifted. Biden's Executive Order 14110 had required red teaming for high-risk AI models and tasked NIST with developing guidelines. Trump's Executive Order 14148 rescinded it in January 2025, replacing the safety-focused framework with one centered on deregulation and innovation leadership. The practical effect is that mandatory red teaming requirements for frontier models evaporated at the federal level. NIST continues its AI Risk Management Framework work and launched an AI Agent Standards Initiative in early 2026, but the regulatory pressure that was pushing companies toward structured red teaming has weakened considerably. Organizations that depended on regulatory mandates to justify their red teaming programs now need to justify them on their own terms.

Where the Original Argument Holds Up

The core thesis still stands: most red teaming focuses on model-level bugs while the serious risks are systemic. If anything, agentic AI has proven the point more aggressively than the authors probably expected.

The paper's recommendation to build multifunctional red teams (ML engineers, social scientists, domain experts, security practitioners) looks even more necessary now. You cannot red-team an agentic system by throwing adversarial prompts at it. You need people who understand the business process the agent is embedded in, the data flows it can access, the authorization boundaries it should respect, and the failure modes that emerge when it chains together a sequence of individually reasonable actions that produce an unreasonable outcome.

The call for continuous red teaming over one-time assessments has also aged well. The best practice emerging in 2026 is integrating adversarial testing into CI/CD pipelines so that model updates, prompt changes, or agent reconfigurations automatically trigger attack suites. Red teaming as a pre-launch checkbox is not viable when your system's behavior changes every time you update a prompt template.

Where the Original Piece Was Too Conservative

Looking back at what I wrote in July, I underestimated a few things.

First, the speed at which agents would move from experimental to production. The paper framed macro-level red teaming as something organizations should prepare for. In practice, many organizations deployed agentic systems before they had any red teaming program at all, let alone a macro-level one. The gap is not "we test models but not systems." For a lot of organizations, the gap is "we do not test."

Second, I did not spend enough time on the supply chain dimension. Agents consume tools, plugins, and external data sources. Each of those is a trust boundary. Indirect prompt injection, where malicious instructions arrive through untrusted external content rather than direct user input, showed up in 73% of production AI deployments in 2025. Multi-agent denial-of-service attacks succeeded in over 80% of tests in one ACL study. These are not edge cases. They are the default attack surface for any system that lets an LLM interact with external data.

Third, I glossed over the organizational incentive problem. The paper recommended transparency and external testing, and I nodded along without pushing hard enough on why that is so difficult. Internal red teams operate under institutional constraints. They may not test scenarios that would embarrass leadership or challenge product decisions. External red teaming and independent disclosure mechanisms are not nice-to-haves; they are structural necessities for surfacing the risks that internal teams are incentivized to overlook.

What Matters Now

If you are building or deploying AI systems in 2026, here is what I would emphasize differently than I did last July:

Test the system, not the model. This was the paper's thesis and it is now the operational reality. If your agent can read emails, query a database, and send Slack messages, your red team needs to test what happens when a poisoned email redirects the agent to exfiltrate the database contents via Slack. Model-level jailbreak testing does not find this.

Adopt the OWASP ASI framework. It did not exist when I wrote the original piece. It does now, and it gives red teams a structured taxonomy for agentic risks. Use it. It covers goal hijacking, tool misuse, delegated trust failures, memory poisoning, and the other failure modes that are specific to autonomous systems.

Do not wait for regulation. The federal regulatory environment for AI safety is weaker than it was a year ago. NIST is still doing good work, but mandatory requirements are not coming soon. If your red teaming program only exists because a regulation says it should, it will not survive the current policy climate. Build the program because the risk is real, not because someone told you to.

Red team continuously, not periodically. Wire adversarial testing into your release process. When a prompt changes, when a tool gets added, when an agent's scope expands, test it. The systems that break in production are the ones that changed since the last time anyone looked.

Bring in outsiders. Your internal team has blind spots shaped by the same incentive structures that built the system. Independent red teaming is not a luxury. It is how you find the things you are not looking for.

The Bigger Picture

The Stanford and Georgetown paper was right about the direction. The field needed to move from model-level adversarial testing to system-level resilience evaluation. Nine months later, the need is more urgent and the tools to address it are starting to materialize. But the gap between where most organizations are and where they need to be has widened, not narrowed.

AI red teaming in 2026 is not about finding clever jailbreaks. It is about understanding how autonomous systems fail when they interact with messy, adversarial, real-world environments, and building the organizational discipline to test for that continuously. The paper gave us the framework. The question now is whether the industry will actually use it.

Matt James is a Product Security practitioner focused on threat modeling, AI security, and building security programs that work in practice. He writes at odnd.com.

Reference:
Sharkey, L., Pasquinelli, M., Cheng, B., Dobbe, R., et al. Operationalizing Red Teaming for AI Systems. arXiv preprint arXiv:2507.05538. July 2025.

In 2024, I wrote about the cybersecurity trends I believed would shape the near future. At the time, the industry was focused on generative AI, ransomware, remote work, and an expanding regulatory landscape. Many of those themes turned out to be directionally correct—but the way they played out surprised even experienced practitioners.

Two years later, it’s worth revisiting those predictions. Some held up well. Others missed the mark. More importantly, a few of the most impactful changes weren’t obvious at all in 2024.

This post reflects on what came true, what didn’t, and what I believe now defines cybersecurity heading into 2026.

AI Didn’t Create New Attacks — It Made Old Ones Relentless

In 2024, I warned that AI-driven cyberattacks would become more sophisticated and adaptive. While that framing wasn’t wrong, it overstated the technical novelty of what actually happened.

AI didn’t introduce fundamentally new attack techniques. Instead, it dramatically lowered the cost and effort of executing existing ones. Phishing became faster, more personalized, and easier to scale. Social engineering improved in quality and volume. Reconnaissance and targeting became trivial.

The real shift wasn’t intelligence—it was economics. Attackers didn’t need to outsmart defenders; they only needed to overwhelm them.

What I underestimated in 2024 was how quickly human-dependent security controls would fail under AI-driven scale.

The Perimeter Didn’t Just Expand — It Disappeared

I argued in 2024 that the traditional security perimeter was dissolving and that zero trust would become increasingly important. That prediction proved correct—but incomplete.

What disappeared wasn’t just the network boundary. The distinction between “inside” and “outside” stopped mattering altogether. Identity became the control plane, and authenticated access became the primary target.

By 2025, session hijacking, token theft, OAuth abuse, and adversary-in-the-middle attacks were no longer edge cases. They were common. MFA was still necessary—but no longer sufficient on its own.

The biggest realization for many organizations was uncomfortable:
A successfully authenticated user could no longer be assumed trustworthy.

Ransomware Lost Center Stage

Ransomware-as-a-Service was a major concern in 2024, and it absolutely continued to cause damage. But by 2026, it was no longer the dominant threat model.

Attackers increasingly favored quieter approaches:

• Data theft without encryption

• Identity persistence instead of disruption

• Monetization through fraud, resale of access, or secondary abuse

Ransomware was noisy and expensive. Silent compromise was easier to sustain and harder to detect.

In hindsight, I overweighted ransomware relative to the broader shift toward identity-driven attacks and long-lived access.

IoT Security Mattered — Just Not Everywhere

In 2024, I highlighted IoT as a growing risk due to weak security controls and rapid adoption. That risk didn’t disappear, but it didn’t materialize evenly across industries.

IoT proved most critical in:

• Healthcare

• Manufacturing

• Critical infrastructure

• Nation-state activity

For most enterprises, however, IoT wasn’t the primary breach vector. Identity systems, SaaS platforms, and cloud control planes were far more attractive targets.

The risk was real—but narrower than I anticipated.

Regulations Increased Accountability, Not Safety

I expected evolving cybersecurity regulations to materially improve organizational security posture. What actually improved was visibility, not resilience.

Disclosure requirements, audits, and compliance frameworks forced organizations to acknowledge incidents more transparently. They did not, on their own, prevent breaches or meaningfully reduce impact.

By 2026, it became clear that compliance answers “Did you follow the rules?”
It does not answer “Can you withstand failure?”

That distinction matters.

Insider Threats Were Mostly About Access, Not People

In 2024, I pointed to insider threats as a growing concern. What changed was my understanding of the root cause.

Most “insider” incidents weren’t driven by malicious employees. They were driven by:

• Excessive access

• Weak authorization boundaries

• Stolen sessions operating under legitimate identities

Attackers didn’t need insiders. They simply became them.

What Defines Cybersecurity in 2026

The biggest change between 2024 and 2026 wasn’t a new technology or a breakthrough attack technique. It was a shift in how security failures actually happen.

Most incidents didn’t occur because defenses were missing. They happened because trust was granted too easily and held for too long.

That reality forces a different starting point for modern security programs:

• Compromise is not an edge case—it’s something to plan for

• Authentication buys you a moment, not lasting confidence

• Trust has to be reevaluated continuously, not assumed

• Limiting blast radius matters as much as trying to prevent intrusion

The organizations that adapted weren’t the ones that bought the most tools. They were the ones willing to challenge long-held assumptions about users, access, and control.

Closing Thoughts

My predictions in 2024 weren’t wrong, but they missed the center of gravity. I spent too much time focused on emerging threats and not enough on how existing trust models would be exploited at scale.

By 2026, cybersecurity is less about keeping attackers out and more about controlling the damage once they’re in.

That shift has fundamentally changed how I think about identity, access, and what “secure” really means.

For years, organizations have treated multi-factor authentication as the finish line of identity security. Deploy MFA, check the box, and assume phishing is solved.

It isn’t.

Over the past year, a class of attack that was once niche has gone fully mainstream: Adversary-in-the-Middle (AiTM). It’s fast, scalable, and brutally effective—and it breaks one of the most deeply held assumptions in modern security programs:

“If a user completes MFA, the session can be trusted.”

It can’t.

AiTM doesn’t defeat MFA. It steps around it by stealing the authenticated session after MFA succeeds. Once an attacker has the session token, they are the user. No password. No second factor. No additional friction.

This is why AiTM matters, how it actually works, and what organizations can do about it.

Why AiTM Exists: Attackers Adapted

Attackers didn’t suddenly get smarter. They adjusted their business model.

As MFA became widespread, password theft stopped being enough. To keep compromising accounts at scale, attackers needed a way to impersonate users after authentication.

The answer was reverse-proxy phishing.

What began as a technique used by sophisticated actors is now sold as turnkey SaaS. These kits come with dashboards, analytics, and automation that would look familiar to anyone in marketing ops.

AiTM isn’t an advanced trick anymore. It’s a commodity.

How Adversary-in-the-Middle Works

When a user clicks an AiTM phishing link, here’s what happens:

1. The link routes through a reverse proxy
   The attacker places themselves between the victim and the real login service.

2. The real login page is displayed
   This isn’t a fake page—it’s the legitimate one, transparently proxied.

3. Credentials are captured in real time
   Everything the user types passes through the attacker.

4. MFA succeeds
   The user completes their second factor normally.

5. The session token is intercepted
   This is the critical moment. The authenticated session cookie is copied.

6. The attacker replays the session
   They now have a fully authenticated, high-trust session—no MFA required.

7. Persistence and abuse begin
   Common follow-on actions include:
   

   • Registering a new MFA method

   • Granting OAuth access

   • Creating mailbox rules for BEC

   • Exfiltrating data from SaaS platforms

   • Spinning up cloud resources

From the platform’s perspective, this all looks like legitimate user activity. Many detections fail precisely because nothing “breaks.”

Why This Keeps Working: Identity Is the Perimeter

AiTM thrives because modern environments are built this way:

• SSO is everywhere

• Sessions last hours—or weeks

• Conditional access trusts authenticated sessions by default

• OTPs, push prompts, and SMS aren’t cryptographically bound

• Browser sessions aren’t tied to a specific device

Attackers don’t need malware.

They don’t need a zero-day.

They just need a user to log in—through them.

Phishing-Resistant MFA Is the Real Fix

The only authentication methods that reliably stop AiTM are those that are cryptographically bound and origin-verified:

• FIDO2 security keys

• Passkeys

• WebAuthn device-bound credentials

These methods don’t transmit secrets that can be intercepted. Authentication is bound to:

• the device

• the browser

• the legitimate domain

A proxy can’t fake that.

If your organization relies on MFA but hasn’t standardized on phishing-resistant MFA, AiTM isn’t hypothetical. It’s an active risk.

Detecting AiTM in Practice

AiTM leaves fingerprints if you’re looking in the right places:

• MFA completion from one device followed by session use from another location minutes later

• Browser or JA4 fingerprint mismatches between authentication and session activity

• Unexpected OAuth consent grants

• New MFA methods added by the “user”

• Silent mailbox forwarding rules

• Near-instant impossible travel events

• Token refreshes from unusual IP ranges (residential proxies, cloud VMs)

When identity telemetry is collected and correlated properly, AiTM stands out clearly. The challenge isn’t visibility—it’s knowing what matters.

More on that later. This part gets long.

Red teaming started as a military practice and eventually became a staple in cybersecurity, but it’s now being reshaped by the rise of artificial intelligence. As generative AI systems weave deeper into everyday life and major industries, both companies and policymakers have leaned on red teaming to spot weaknesses and improve safety. But a recent paper from Stanford and Georgetown researchers argues that today’s AI red-teaming efforts are too limited — and that we may be focusing on small issues while overlooking much bigger risks.

The Problem: A Narrow Focus on Models

Most AI red-teaming work today zeroes in on finding weaknesses in a single model — what the authors call “micro-level” red teaming. These tests usually revolve around prompt injection, jailbreaks, and other adversarial tactics to coax out bad behavior. Useful as that is, it doesn’t get at the broader, more complicated risks that emerge once AI systems are deployed in real-world settings.

The authors argue that this narrow focus misses the bigger picture. Most red-teaming efforts ignore the sociotechnical side of things — how AI systems behave once real people, institutions, and environments get involved. Many of the most serious risks don’t come from a model misfiring on its own, but from how it’s used in the world: misinformation that snowballs, feedback loops that amplify bias, or users finding and exploiting gaps in the overall system rather than the model itself.

A Broader Vision: Two Levels of AI Red Teaming

To fill this gap, the authors propose a comprehensive framework that distinguishes between two complementary levels of AI red teaming:

• Micro-level red teaming targets the model itself, evaluating prompt behavior, guardrails, and output reliability. This is the current norm in most AI labs.

• Macro-level red teaming, by contrast, considers the entire AI system lifecycle, from data sourcing and training pipeline to deployment environment and downstream impacts. It involves assessing risks that emerge from complex interactions between models, users, interfaces, and institutions.

By operationalizing red teaming at both levels, organizations can shift from reactive patchwork to proactive risk management.

Recommendations for the Future

Drawing on decades of experience from cybersecurity and systems engineering, the authors offer a set of actionable recommendations to improve AI red teaming practices:

1. Treat Red Teaming as an Ongoing Process: Red teaming should not be a one-time product launch checklist but an iterative, continuous evaluation across the AI lifecycle.

2. Build Multifunctional Red Teams: Effective red teams should include experts in machine learning, human-computer interaction, social science, cybersecurity, and domain-specific knowledge. This diversity is essential to uncovering emergent, cross-cutting vulnerabilities.

3. Evaluate Sociotechnical Contexts: Red teams must consider how AI systems are used in practice—including how users might manipulate them, how they interact with other tools, and how organizational incentives shape deployment.

4. Incorporate Systems Thinking: Rather than viewing AI safety as a matter of model robustness alone, organizations must assess systemic resilience—identifying feedback loops, cascades, and coordination failures that lead to large-scale harm.

5. Embrace Transparency and External Testing: Independent red teaming and disclosure mechanisms can surface risks that internal teams may miss due to institutional blind spots or incentive structures.

The field of AI safety is at an inflection point. As AI systems become more powerful and socially embedded, the traditional model-focused approach to red teaming must evolve. This paper offers a timely and much-needed reframing of red teaming—not as a narrow adversarial testing tool, but as a holistic practice rooted in systems thinking, interdisciplinary collaboration, and continuous scrutiny.

Organizations that adopt this broader vision will be better positioned to identify and mitigate not just model-level bugs, but the real-world harms that arise when AI meets messy, unpredictable human systems.

Citation:
Sharkey, L., Pasquinelli, M., Cheng, B., Dobbe, R., et al. Operationalizing Red Teaming for AI Systems. arXiv preprint arXiv:2507.05538. July 2025.
https://arxiv.org/abs/2507.05538

It’s been a while since I’ve posted, so I want to start stretching this muscle again over the coming months. As it is timely and pertinent, I wanted to cover information operations in the context of influencing others through the use of AI.

AI-Driven Social Engineering

The advent of AI has ushered in a new era of social engineering, where attackers leverage machine learning algorithms to craft phishing emails or messages that are not just personalized but dynamically adaptive. These attackers leverage machine learning (ML) algorithms to craft phishing emails or messages that are not just personalized but dynamically adaptive. Here's how it works:

• Personalization: AI analyzes vast datasets, including social media profiles, purchase histories, and browsing habits, to tailor messages that resonate with the target's interests, fears, or needs. This personalization goes beyond just using the recipient's name; it might reference recent activities or events in their life, making the communication seem genuinely relevant.

• Dynamic Adaptation: Unlike traditional phishing where messages are static, AI-driven systems can adjust in real-time based on user interaction. If a recipient engages with the message, the AI might alter its approach, perhaps offering more incentives or changing the tone to sound more urgent or authoritative.

• Natural Language Generation (NLG): AI uses NLG to create text that's indistinguishable from human writing, often incorporating emotional triggers or persuasive language that traditional phishing emails might lack.

• Multi-Channel Engagement: These attacks don't just stop at email. AI can orchestrate campaigns across multiple platforms, including SMS, social media, or even voice calls, creating a multi-faceted assault that's harder to dismiss as spam.

The implications of AI-driven social engineering are profound:

• Misinformation Propagation: By impersonating trusted sources like banks, government officials, or even friends and family, these attacks can spread misinformation with alarming efficiency. For instance, an email purportedly from a health organization might spread false information about a health crisis, influencing public behavior or policy.

• Corporate Espionage: In corporate settings, these techniques can be used to extract sensitive information or manipulate decisions. An AI-crafted email from a supposed executive could direct employees to transfer funds or disclose confidential data, all under the guise of legitimate business operations.

• Public Opinion Manipulation: On a larger scale, AI-driven social engineering can sway public opinion by flooding social media with bots that push specific narratives or by creating fake news stories that resonate with targeted demographics, potentially influencing elections or market trends.

• Increased Effectiveness: The dynamic nature of these attacks means they can bypass traditional security measures like spam filters or user training, which often look for static signs of phishing. The adaptability of AI makes each interaction potentially unique, reducing the effectiveness of static defense strategies.

• Long-term Impact: Beyond immediate actions like clicking a link or downloading a file, these attacks can erode trust in digital communications, leading to broader societal impacts where skepticism towards any form of digital communication becomes the norm.

This evolution in social engineering represents a significant leap in sophistication, requiring not just technical countermeasures but also a shift in how individuals and organizations approach digital trust and verification. The challenge lies in developing defenses that are equally adaptive, leveraging AI to detect and counteract these nuanced attacks while fostering a culture of skepticism without paralyzing digital communication.

If you’re interested in more nuanced and specific attack vectors, please feel free to connect with me on LinkedIn and X.

Just as the internet evolved from HTTP to HTTPS to protect data-in-transit, the field of cryptography is undergoing a transformation to safeguard against the looming threat of quantum computing. This blog post explores the analogy between the shift from HTTP to HTTPS and the current transition from classical cryptography to post-quantum cryptography (PQC), highlighting why this evolution is essential for our digital future.

Understanding the Evolution: HTTP to HTTPS

HTTP (Hypertext Transfer Protocol) was the original protocol for transferring web data. It served its purpose well in the early days of the internet, enabling the exchange of information between web servers and clients. However, as the internet grew, so did the capabilities of attackers. HTTP's lack of robust security features became a significant vulnerability, allowing for data to be intercepted and manipulated.

To address these security concerns, HTTPS (Hypertext Transfer Protocol Secure) was developed. By adding layers of encryption through SSL/TLS protocols, HTTPS ensures that data transmitted over the internet is secure and cannot be easily intercepted or tampered with. This transition from HTTP to HTTPS marked a significant improvement in web security, making it the standard for protecting online communications.

Classical Cryptography vs. Post-Quantum Cryptography

Classical cryptography has been the cornerstone of secure communications for decades. Protocols like RSA and ECC rely on mathematical problems that are challenging for classical computers to solve, providing a robust defense against unauthorized access. However, the advent of quantum computing threatens to undermine these cryptographic methods. Quantum computers, with their immense computational power, have the potential to break classical cryptographic algorithms, rendering them obsolete.

Enter Post-Quantum Cryptography (PQC). Similar to how HTTPS was created to address the security shortcomings of HTTP, PQC is being developed to counter the future threat posed by quantum computers. PQC algorithms are designed to be secure against both classical and quantum attacks, ensuring that our data remains protected in the quantum era.

The Analogy: HTTP/HTTPS and Classical/PQC

The analogy between HTTP/HTTPS and classical/PQC helps illustrate the need for this cryptographic evolution:

• HTTP (Classical Cryptography): Just as HTTP was sufficient for the early internet but became vulnerable as attack methods advanced, classical cryptographic methods are currently effective but will become vulnerable with the advent of quantum computers.

• HTTPS (PQC): HTTPS was developed to address the security shortcomings of HTTP, adding encryption to secure web communications. Similarly, PQC is being developed to address the security shortcomings of classical cryptography in the face of quantum computing threats.

Key Points of the Analogy:

1. Coexistence:

   • HTTP still exists but is largely replaced by HTTPS for secure communications. Similarly, classical cryptography will still exist but is likely to be supplemented or replaced by PQC algorithms for enhanced security.

2. Enhanced Security:

   • HTTPS provides a more secure alternative to HTTP. Likewise, PQC offers more secure alternatives to classical cryptographic methods to protect against advanced threats.

3. Transition Period:

   • The shift from HTTP to HTTPS involved a transition period with the development of standards, tools, and widespread adoption. The shift to PQC will also involve a transition period with ongoing research, standardization, and gradual adoption.

Preparing for the Quantum Future

The move from classical cryptography to post-quantum cryptography is not just an upgrade but a necessary evolution to secure our digital future. As quantum computing continues to advance, the urgency to develop and implement PQC solutions grows. Just as HTTPS became the standard for secure web communications, PQC will become essential to protect our data in a post-quantum world.

By understanding this analogy and recognizing the importance of PQC, we can better prepare for the challenges and opportunities that lie ahead. The future of cybersecurity depends on our ability to adapt and evolve, ensuring that our digital infrastructure remains robust and secure against the threats of tomorrow.

Quantum computing is stepping out of science fiction and into reality, challenging our current cryptographic standards. With quantum algorithms such as Shor’s potentially unraveling the complexities of RSA and ECC, and Grover's accelerating the brute force attacks, the cryptographic community is steering towards Post-Quantum Cryptography (PQC). PQC aims to devise systems that can withstand the onslaught of both quantum and traditional computational threats, thereby securing our digital communications and data.

This proactive approach, spearheaded by the National Institute of Standards and Technology (NIST), involves evaluating quantum-resistant algorithms through a meticulous public process. As we edge closer to the quantum era, understanding and integrating PQC is not just advisable; it's imperative for safeguarding sensitive information across various sectors.

For a deeper dive into the specifics of these quantum challenges and the ongoing efforts in cryptographic innovations, check out NIST's dedicated project page on post-quantum cryptography: NIST Post-Quantum Cryptography.

As a small or medium business owner, you may think you're not a prime target for cyber attacks. Unfortunately, that misconception can leave you vulnerable. In 2022, 61% of data breaches involved small businesses.

Cyber criminals go after SMBs because they typically have fewer security resources than larger enterprises. But implementing some basic cybersecurity practices can go a long way in protecting your business data and assets.

Use Strong Passwords and Multi-Factor Authentication

Weak passwords are one of the most common entry points for hackers. Require all employees to use long, complex passwords that are unique for each account and application. Better yet, use a password manager to store and encrypt passwords.

You should also enable multi-factor authentication (MFA) whenever possible. MFA adds an extra layer of security by requiring a second form of verification, like a code sent to your phone, in addition to a password.

Keep Software Up-To-Date

Hackers are constantly finding new software vulnerabilities to exploit. That's why it's critical to install security updates for operating systems, applications, and firmware as soon as they are released.

Set up automatic updates whenever possible. For systems that don't allow automatic patching, assign an employee to regularly check for and install updates.

Back Up Data Regularly

Ransomware and other malware can make your business data unusable or permanently delete it unless you pay a ransom. The best way to recover from such attacks is to maintain frequent backups of critical data.

Use the 3-2-1 approach: Keep at least 3 backup copies on 2 different storage types, with 1 copy offsite or in the cloud. Test restoring from backups periodically.

Provide Cybersecurity Training

Your employees are the first line of defense against phishing, social engineering, and other cyber threats that exploit human vulnerability. Train them on cybersecurity best practices and how to spot potential attacks.

Conduct routine security awareness activities to keep cybersecurity top of mind. Consider requiring all employees to complete annual cybersecurity training.

Encrypt Sensitive Data

If sensitive customer, employee or financial data falls into the wrong hands, it could devastate your business. Encryption encodes information so it appears like gibberish to anyone not authorized to access it.

Encrypt data both at rest (stored data) and in transit (data being transmitted) using industry-standard encryption protocols.

No business is too small to ignore cybersecurity. Invest time and resources into protecting your digital assets. It's one of the smartest business decisions you can make.

If you’re interested in learning more about any of these topics or more, please do not hesitate to reach out.

The digital age is witnessing a significant transformation in software security, primarily driven by advancements in Artificial Intelligence (AI). As this technology continues to evolve, it brings both opportunities and challenges for cybersecurity. Understanding the nuanced role of AI in enhancing and complicating software security is crucial for industry professionals and organizations aiming to navigate this evolving era.

The Essence of AI in Enhancing Software Security

AI embodies the creation of intelligent systems that can simulate human thought processes, decision-making, and problem-solving abilities. In the realm of software security, AI's capability to mimic and exceed human cognitive functions is a game-changer. It offers unprecedented advantages in detecting, analyzing, and responding to cyber threats with speed and efficiency far beyond human capabilities.

AI systems in cybersecurity can process vast datasets to identify patterns and anomalies indicative of malicious activity. They can predict potential vulnerabilities and threats by understanding the nuances of cyberattack strategies. Furthermore, AI-driven security systems can automate response protocols, shutting down attacks or isolating affected systems before significant damage is done.

Distinguishing Between AI and ML in Cybersecurity

While discussing AI's role in software security, it's essential to clarify the distinction between AI and its subset, Machine Learning (ML). AI is the broader discipline that encompasses creating machines capable of performing tasks that typically require human intelligence. This includes reasoning, learning, problem-solving, and understanding natural language.

ML, a subset of AI, focuses specifically on the ability of machines to learn from data, improve from experience, and make decisions based on that learning. In cybersecurity, ML algorithms analyze patterns in data to detect anomalies that could indicate a security breach, learning and adapting to new threats over time.

However, AI's application in cybersecurity is not limited to ML. It also includes other technologies like natural language processing (NLP) and rule-based systems, which can interpret and respond to human language or follow complex sets of instructions to identify threats. This broad arsenal of AI technologies enhances the ability of security systems to protect against a wide range of cyberattacks with greater precision and efficiency.

The Challenges Posed by AI

The integration of AI into software security does not come without its challenges. The sophistication of AI systems also provides cybercriminals with powerful tools to develop more complex and adaptive forms of malware and phishing attacks. AI can be used to automate the generation of malicious software that can learn and adapt to evade detection or to craft personalized phishing emails at scale, which are more likely to deceive users.

Moreover, the reliance on AI for cybersecurity raises significant ethical and privacy concerns. The extensive data analysis capabilities of AI systems pose risks to personal privacy and data protection, necessitating stringent safeguards and ethical guidelines for AI use in cybersecurity.

Looking Ahead

As we look to the future, the role of AI in software security is set to become even more pivotal. The dynamic nature of cyber threats requires security systems that are not just reactive but predictive and adaptive. AI, with its comprehensive capabilities, stands at the forefront of this shift, offering solutions that can evolve in tandem with emerging threats.

However, harnessing the full potential of AI in cybersecurity will require ongoing efforts to address the ethical and privacy concerns associated with its use. It will also necessitate a synergistic approach where human expertise and AI capabilities complement each other, ensuring a robust and resilient cybersecurity posture.

AI is reshaping the landscape of software security, offering sophisticated tools to protect digital assets while also presenting new challenges. Understanding the distinction between AI and its subsets, such as ML, is crucial for leveraging these technologies effectively. As we continue to explore the possibilities, it's clear that its impact will be profound, driving innovations that will define the future of digital security.

The following references offer a deeper insight into the topics discussed, such as the role of AI in cybersecurity, the distinction between AI and ML, and the ethical considerations surrounding the use of AI:

• Artificial Intelligence and Cybersecurity: The New Age of Protection by Cade Metz. This book provides an overview of how artificial intelligence is revolutionizing the field of cybersecurity, offering both opportunities and challenges for protection against cyber threats.

• Machine Learning and Security: Protecting Systems with Data and Algorithms by Clarence Chio and David Freeman. This book dives into the specific role of machine learning within the broader context of AI in cybersecurity, detailing how algorithms can detect and defend against cyber attacks.

• Cybersecurity Ethics: An Introduction by Mary Manjikian. As ethical considerations become increasingly important in the deployment of AI technologies, this book explores the ethical dilemmas and responsibilities facing professionals in cybersecurity.

• The Future of Cybersecurity: AI and Autonomous Attacks in the Harvard Business Review. This article discusses the future implications of AI in cybersecurity, including the potential for AI to both enhance defense mechanisms and be used in autonomous cyber attacks.

• Ethics of Artificial Intelligence and Robotics in the Stanford Encyclopedia of Philosophy. This comprehensive entry examines the ethical considerations surrounding the development and use of artificial intelligence, including privacy concerns and the moral implications of AI decisions.

Before delving into the intricacies of auditing and accountability in logging, it's essential to lay down a foundational understanding that caters to a broad audience. While this content may be familiar to some, it serves as a valuable primer for those seeking to learn and explore this crucial aspect of information technology.

Ensuring accountability and maintaining accurate logs is paramount for businesses across all industries. Whether managing physical infrastructure or operating within cloud environments like SaaS, PaaS, or IaaS, robust auditing practices are essential for security, compliance, and operational transparency.

Auditing Physical Systems:

In traditional IT setups with physical servers and networks, logging plays a critical role in tracking system activities, user actions, and potential security breaches. Auditing these systems involves capturing events such as login attempts, file access, configuration changes, and network traffic. By maintaining comprehensive logs, organizations can trace the root cause of issues, detect anomalies, and demonstrate compliance with regulatory requirements.

Key Aspects of Auditing Physical Systems:

1. Log Integrity: Ensuring that logs are tamper-proof and securely stored to prevent unauthorized alterations or deletions.

2. Access Controls: Implementing strict access controls to limit who can view, modify, or delete log files, thus preventing unauthorized tampering.

3. Regular Reviews: Conducting periodic reviews of logs to identify suspicious activities, security gaps, or compliance violations.

Auditing Cloud Services:

With the widespread adoption of cloud computing, auditing becomes more complex yet equally crucial. Cloud service models such as SaaS, PaaS, and IaaS offer scalability and flexibility but require tailored logging and auditing approaches.

Challenges in Auditing Cloud Environments:

1. Shared Responsibility Model: Understanding the division of responsibilities between the cloud service provider and the customer regarding security and logging.

2. Multi-Tenancy Concerns: Ensuring that logs are isolated and protected in multi-tenant cloud environments to prevent data leakage or unauthorized access.

3. Dynamic Infrastructure: Adapting auditing practices to the dynamic nature of cloud infrastructure where resources are provisioned and de-provisioned on-demand.

Best Practices for Auditing in the Cloud:

1. Centralized Logging: Implementing centralized logging solutions that aggregate logs from various cloud services and instances for unified monitoring and analysis.

2. Encryption and Access Controls: Encrypting log data both in transit and at rest, and enforcing granular access controls to protect sensitive information.

3. Automated Auditing: Leveraging automation tools to streamline auditing processes, identify deviations from compliance standards, and trigger alerts for remediation.

Auditing and accountability are integral components of logging, whether managing physical systems or operating in cloud environments. By adopting best practices tailored to the specific infrastructure, organizations can enhance security, ensure compliance, and maintain trust in their systems and services.

Here's a list of resources (URLs) that readers can explore to delve deeper into the topic of auditing, logging, and accountability:

1. NIST Special Publication 800-92: Guide to Computer Security Log Management: https://csrc.nist.gov/publications/detail/sp/800-92/final

2. OWASP Logging Cheat Sheet: https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html

3. Cloud Security Alliance (CSA) Security Guidance v4: https://cloudsecurityalliance.org/research/security-guidance/v4/

4. PCI DSS Logging Standard: https://www.pcisecuritystandards.org/documents/Logging_Standard.pdf

5. ISO/IEC 27002:2013 Information technology -- Security techniques -- Code of practice for information security controls: https://www.iso.org/standard/54534.html

6. Logging and Auditing in AWS: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html

7. Azure Monitor Logs overview: https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-platform-logs

8. Google Cloud's Operations suite (formerly Stackdriver): https://cloud.google.com/stackdriver

9. ELK Stack (Elasticsearch, Logstash, Kibana): https://www.elastic.co/what-is/elk-stack

10. Splunk: https://www.splunk.com/

These resources cover a wide range of topics related to auditing, logging, and accountability in both physical systems and cloud environments, providing readers with valuable insights, best practices, and technical guidance.

Security advisors and consultants have the critical responsibility of steering investors and firms through the complexities of new technology capital investment and acquisitions. A key aspect of this guidance involves evaluating how thoroughly the technology has been assessed and vetted, particularly in terms of market context and potential risks. This process is not just about scrutinizing the technology itself, but also about establishing a level of assurance regarding its viability and security. Below are the three pivotal due diligence activities that technology consultants should focus on to ensure informed and secure technology investments.

1. Comprehensive Market Analysis and Vendor Evaluation

Understanding the Technology's Market Relevance

A deep dive into the market landscape is essential to gauge the relevance and potential of the technology. This includes analyzing market trends, consumer demands, and the competitive landscape. Consultants should assess if the technology addresses a genuine market need or presents a unique solution that sets it apart from existing offerings.

Assessing Vendor’s Market Standing and Track Record

Evaluating the vendor's position in the market and their history is crucial. This step involves examining the vendor's financial health, customer feedback, and their record in managing technology deployments. Understanding the vendor's stability and reputation in the market is key to assessing the longevity and support for the technology.

2. Verification of Prior Assessment and Threat Modeling

Reviewing Existing Assessments and Security Measures

An important part of due diligence is determining if the product or service has undergone thorough assessment and security analysis. If the vendor has already completed a qualitative threat model or similar assessments, advisors should review these findings in detail. This review helps in understanding the rigor of the vendor's testing and risk management processes.

Recommending Assessment Steps if Needed

In cases where comprehensive assessments haven't been performed, advisors must be able to recommend necessary steps. This includes suggesting security audits, threat modeling, and other risk assessment strategies. The goal is to ensure that every potential risk is identified and addressed, building a strong foundation of assurance for the client.

3. Pilot Testing and Performance Evaluation

Conducting Real-World Testing

Implementing pilot tests in real-life scenarios is critical for evaluating the technology’s practical performance and integration capabilities. These tests help in identifying any operational issues and assessing the overall user experience.

Detailed Analysis of Test Outcomes

Analyzing the results from pilot testing provides valuable insights into the technology's efficiency, scalability, and reliability. This step is crucial for confirming that the technology not only meets current requirements but is also capable of adapting to future demands and challenges.

In summary…

For technology consultants and advisors, ensuring a high level of assurance in technology investments requires a strategic approach that encompasses thorough market analysis, rigorous vendor and product assessment, and comprehensive testing. By focusing on these key areas, advisors can provide their clients with the confidence that their technology choices are not only innovative but also secure, market-relevant, and future-proof.

The security of Operational Technology (OT) has emerged as a critical concern in the public domain. OT, which encompasses hardware and software that monitors or controls equipment, assets, and processes, has traditionally been isolated from Information Technology (IT) systems. However, with increasing integration and connectivity, the once distinct line between IT and OT is blurring, bringing unique security challenges to the forefront.

The Evolution of OT and Emerging Security Risks

OT systems, historically standalone and disconnected from networks, are increasingly interconnected with IT systems for efficiency and data analysis benefits. This integration, while beneficial, exposes OT systems to cyber threats that were previously limited to IT networks. The stakes are high in OT environments—compromises can lead to operational disruptions, safety hazards, and significant financial losses.

Unique Challenges in OT Security

1. Legacy Systems: Many OT systems were designed without cybersecurity in mind and updating them can be complex, costly, and could disrupt critical operations.

2. Different Priorities: Unlike IT systems where data integrity and confidentiality are key, OT security emphasizes system availability and physical safety. This difference requires a distinct approach to security.

3. Limited Patching and Downtime: Regularly updating and patching, a common practice in IT security, is challenging in OT due to the need for continuous operation and minimal downtime.

Call to Action

As the landscape of OT security continues to evolve, staying informed and prepared is crucial. If you're interested in learning more about how to safeguard your OT environment, or if you have specific questions about OT security, don't hesitate to reach out. Our team of experts is here to provide insights, strategies, and solutions tailored to your unique operational needs. Contact us to explore how you can strengthen your OT security posture.

With the increasing reliance on digital technologies, the sophistication of cyber threats is also reaching new heights. In this article, we'll explore the key trends shaping cybersecurity in 2024, offering insights into how businesses can stay protected in this ever-changing environment.

The Rise of AI-Driven Cyber Attacks

AI is a double-edged sword in cybersecurity. While it's used for defensive purposes, such as threat detection and response, cybercriminals are also leveraging AI to orchestrate more complex attacks. These AI-driven attacks can adapt to security measures in real-time, making them particularly challenging to detect and counter.

Proliferation of AI-driven Attacks Anticipated in 2024

Trend Micro Incorporated (TYO: 4704; TSE: 4704), a global cybersecurity leader, today warned of the transformative role of generative AI (GenAI) in the cyber threat landscape and a coming tsunami...

newsroom.trendmicro.com/2023-12-05-Proliferation-of-AI-driven-Attacks-Anticipated-in-2024

The Expanding Cybersecurity Perimeter

The traditional security perimeter is dissolving. With the rise of remote work and cloud computing, the line between internal and external networks is blurring. Organizations need to adopt a zero-trust security model, ensuring robust verification processes for anyone trying to access their systems, irrespective of their location.

Top 5 Cyber Predictions for 2024: A CISO Perspective

Read the top 5 cyberthreat predictions for 2024, covering generative AI-driven attacks, Ransomware-as-a-Service, the response to SEC regulations, and more.

www.zscaler.com/blogs/security-research/top-5-cyber-predictions-2024-ciso-perspective

The Growing Importance of IoT Security

The Internet of Things (IoT) continues to expand rapidly. However, this growth also presents numerous security challenges. Many IoT devices lack robust security features, making them vulnerable to attacks. Strengthening IoT security protocols is imperative to prevent these devices from becoming entry points for cybercriminals.

IoT Security Predictions for 2024 and Beyond | Asimily

Organizations planning their security programs would do well to consider these IoT security predictions in weeks and months ahead. 

asimily.com/blog/iot-security-predictions-for-2024-and-beyond

The Surge in Ransomware-as-a-Service (RaaS)

RaaS is transforming the landscape of cybercrime. It allows individuals without extensive technical knowledge to launch ransomware attacks. This democratization of cybercrime means that businesses of all sizes are at risk and must enhance their defenses against ransomware.

2024 Cybersecurity Forecast: Ransomware's New Tactics and Targets

In the past, cybercriminals often operated with the motive to "do it for lulz," engaging in malicious activities purely for the sake of amusement or creating chaos.

www.bitdefender.com/blog/businessinsights/2024-cybersecurity-forecast-ransomwares-new-tactics-and-targets

The Evolution of Cybersecurity Regulations

Regulatory frameworks are evolving to keep up with cybersecurity challenges. Businesses must stay informed about these changes to ensure compliance. Non-compliance not only poses legal risks but also leaves companies vulnerable to cyber threats.

The Impact of Cybersecurity Regulation in 2024 - Spiceworks

Greg Bulmash explores the future of cybersecurity regulations in 2024. Prepare your strategy with insights from GitGuardian.

www.spiceworks.com/it-security/cyber-risk-management/guest-article/future-of-cybersecurity-regulation

Enhanced Focus on Insider Threats

Insider threats, both intentional and accidental, are on the rise. Organizations must implement comprehensive strategies to detect and mitigate risks from within. This includes regular security training, robust access controls, and employing behavior analysis to detect anomalies.

Navigating the threat landscape in 2024

Cybersecurity is a critical aspect of any organisation and implementing a robust security strategy is essential for preventing cyber threats

cybermagazine.com/articles/navigating-the-threat-landscape-in-2024

Conclusion

The cybersecurity landscape in 2024 is complex and requires a proactive approach. By understanding these emerging trends, businesses can better prepare and protect themselves from new threats. Investing in advanced security solutions, continuous employee training, and staying abreast of regulatory changes are key steps towards a more secure future.