Cloud_Enum - Multi-cloud OSINT Tool. Enumerate Public Resources In AWS, Azure, And Google Cloud

Multi-cloud OSINT tool. Enumerate public resources in AWS, Azure, and Google Cloud.

Currently enumerates the following:

Amazon Web Services: - Open / Protected S3 Buckets - awsapps (WorkMail, WorkDocs, Connect, etc.)

Microsoft Azure: - Storage Accounts - Open Blob Storage Containers - Hosted Databases - Virtual Machines - Web Apps

Google Cloud Platform - Open / Protected GCP Buckets - Open / Protected Firebase Realtime Databases - Google App Engine sites - Cloud Functions (enumerates project/regions with existing functions, then brute forces actual function names) - Open Firebase Apps

See it in action in Codingo's video demo here.

Usage

Setup

Several non-standard libaries are required to support threaded HTTP requests and dns lookups. You'll need to install the requirements as follows:

pip3 install -r ./requirements.txt

Running

The only required argument is at least one keyword. You can use the built-in fuzzing strings, but you will get better results if you supply your own with -m and/or -b.

You can provide multiple keywords by specifying the -k argument multiple times.

Keywords are mutated automatically using strings from enum_tools/fuzz.txt or a file you provide with the -m flag. Services that require a second-level of brute forcing (Azure Containers and GCP Functions) will also use fuzz.txt by default or a file you provide with the -b flag.

Let's say you were researching "somecompany" whose website is "somecompany.io" that makes a product called "blockchaindoohickey". You could run the tool like this:

./cloud_enum.py -k somecompany -k somecompany.io -k blockchaindoohickey

HTTP scraping and DNS lookups use 5 threads each by default. You can try increasing this, but eventually the cloud providers will rate limit you. Here is an example to increase to 10.

./cloud_enum.py -k keyword -t 10

IMPORTANT: Some resources (Azure Containers, GCP Functions) are discovered per-region. To save time scanning, there is a "REGIONS" variable defined in cloudenum/azure_regions.py and cloudenum/gcp_regions.py that is set by default to use only 1 region. You may want to look at these files and edit them to be relevant to your own work.

Complete Usage Details

usage: cloud_enum.py [-h] -k KEYWORD [-m MUTATIONS] [-b BRUTE]

Multi-cloud enumeration utility. All hail OSINT!

optional arguments:
  -h, --help            show this help message and exit
  -k KEYWORD, --keyword KEYWORD
                        Keyword. Can use argument multiple times.
  -kf KEYFILE, --keyfile KEYFILE
                        Input file with a single keyword per line.
  -m MUTATIONS, --mutations MUTATIONS
                        Mutations. Default: enum_tools/fuzz.txt
  -b BRUTE, --brute BRUTE
                        List to brute-force Azure container names. Default: enum_tools/fuzz.txt
  -t THREADS, --threads THREADS
                        Threads for HTTP brute-force. Default = 5
  -ns NAMESERVER, --nameserver NAMESERVER
                        DNS server to use in brute-force.
  -l LOGFILE, --logfile LOGFILE
                        Will APPEND found items to specified file.
  -f FORMAT, --format FORMAT
                        Format for log file (text,json,csv - defaults to text)
  --disable-aws         Disable Amazon checks.
  --disable-azure       Disable Azure checks.
  --disable-gcp         Disable Google checks.
  -qs, --quickscan      Disable all mutations and second-level scans

Thanks

So far, I have borrowed from: - Some of the permutations from GCPBucketBrute

GATOR - GCP Attack Toolkit For Offensive Research, A Tool Designed To Aid In Research And Exploiting Google Cloud Environments

GATOR - GCP Attack Toolkit for Offensive Research, a tool designed to aid in research and exploiting Google Cloud Environments. It offers a comprehensive range of modules tailored to support users in various attack stages, spanning from Reconnaissance to Impact.

Modules

Installation

Python 3.11 or newer should be installed. You can verify your Python version with the following command:

python --version

Manual Installation via setup.py

git clone https://github.com/anrbn/GATOR.git
cd GATOR
python setup.py install

Automated Installation via pip

pip install gator-red

Documentation

Have a look at the GATOR Documentation for an explained guide on using GATOR and it's module!

Issues

Reporting an Issue

If you encounter any problems with this tool, I encourage you to let me know. Here are the steps to report an issue:

1. Check Existing Issues: Before reporting a new issue, please check the existing issues in this repository. Your issue might have already been reported and possibly even resolved.

2. Create a New Issue: If your problem hasn't been reported, please create a new issue in the GitHub repository. Click the Issues tab and then click New Issue.

3. Describe the Issue: When creating a new issue, please provide as much information as possible. Include a clear and descriptive title, explain the problem in detail, and provide steps to reproduce the issue if possible. Including the version of the tool you're using and your operating system can also be helpful.

4. Submit the Issue: After you've filled out all the necessary information, click Submit new issue.

Your feedback is important, and will help improve the tool. I appreciate your contribution!

Resolving an Issue

I'll be reviewing reported issues on a regular basis and try to reproduce the issue based on your description and will communicate with you for further information if necessary. Once I understand the issue, I'll work on a fix.

Please note that resolving an issue may take some time depending on its complexity. I appreciate your patience and understanding.

Contributing

I warmly welcome and appreciate contributions from the community! If you're interested in contributing on any existing or new modules, feel free to submit a pull request (PR) with any new/existing modules or features you'd like to add.

Once you've submitted a PR, I'll review it as soon as I can. I might request some changes or improvements before merging your PR. Your contributions play a crucial role in making the tool better, and I'm excited to see what you'll bring to the project!

Thank you for considering contributing to the project.

Questions and Issues

If you have any questions regarding the tool or any of its modules, please check out the documentation first. I've tried to provide clear, comprehensive information related to all of its modules. If however your query is not yet solved or you have a different question altogether please don't hesitate to reach out to me via Twitter or LinkedIn. I'm always happy to help and provide support. :)

TerraformGoat - "Vulnerable By Design" Multi Cloud Deployment Tool

TerraformGoat is selefra research lab's "Vulnerable by Design" multi cloud deployment tool.

Currently supported cloud vendors include Alibaba Cloud, Tencent Cloud, Huawei Cloud, Amazon Web Services, Google Cloud Platform, Microsoft Azure.

Scenarios

Install

TerraformGoat is deployed using Docker images and therefore requires Docker Engine environment support, Docker Engine installation can be found in https://docs.docker.com/engine/install/

Depending on the cloud service provider you are using, choose the corresponding installation command.

Alibaba Cloud

docker pull registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_aliyun:0.0.4
docker run -itd --name terraformgoat_aliyun_0.0.4 registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_aliyun:0.0.4
docker exec -it terraformgoat_aliyun_0.0.4 /bin/bash

Tencent Cloud

docker pull registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_tencentcloud:0.0.4
docker run -itd --name terraformgoat_tencentcloud_0.0.4 registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_tencentcloud:0.0.4
docker exec -it terraformgoat_tencentcloud_0.0.4 /bin/bash

Huawei Cloud

docker pull registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_huaweicloud:0.0.4
docker run -itd --name terraformgoat_huaweicloud_0.0.4 registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_huaweicloud:0.0.4
docker exec -it terraformgoat_huaweicloud_0.0.4 /bin/bash

Amazon Web Services

docker pull registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_aws:0.0.4
docker run -itd --name terraformgoat_aws_0.0.4 registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_aws:0.0.4
docker exec -it terraformgoat_aws_0.0.4 /bin/bash

Google Cloud Platform

docker pull registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_gcp:0.0.4
docker run -itd --name terraformgoat_gcp_0.0.4 registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_gcp:0.0.4
docker exec -it terraformgoat_gcp_0.0.4 /bin/bash

Microsoft Azure

docker pull registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_azure:0.0.4
docker run -itd --name terraformgoat_azure_0.0.4 registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_azure:0.0.4
docker exec -it terraformgoat_azure_0.0.4 /bin/bash

Demo

After entering the container, cd to the corresponding scenario directory and you can start deploying the scenario.

Here is a demonstration of the Alibaba Cloud Bucket Object Traversal scenario build.

docker pull registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_aliyun:0.0.4
docker run -itd --name terraformgoat_aliyun_0.0.4 registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat_aliyun:0.0.4
docker exec -it terraformgoat_aliyun_0.0.4 /bin/bash

cd /TerraformGoat/aliyun/oss/bucket_object_traversal/
aliyun configure
terraform init
terraform apply

The program prompts Enter a value:, type yes and enter, use curl to access the bucket, you can see the object traversed.

To avoid the cloud service from continuing to incur charges, remember to destroy the scenario in time after using it.

terraform destroy



Uninstall

If you are in a container, first execute the exit command to exit the container, and then execute the following command under the host.

docker stop $(docker ps -a -q -f "name=terraformgoat*")
docker rm $(docker ps -a -q -f "name=terraformgoat*")
docker rmi $(docker images -a -q -f "reference=registry.cn-beijing.aliyuncs.com/huoxian_pub/terraformgoat*")

Notice

1. The README of each vulnerable environment is executed within the TerraformGoat container environment, so the TerraformGoat container environment needs to be deployed first.
2. Due to the horizontal risk of intranet horizontal on the cloud in some scenarios, it is strongly recommended that users use their own test accounts to configure the scenarios, avoid using the cloud account of the production environment, and install TerraformGoat using Dockerfile to isolate the user's local cloud vendor token and the test account token.
3. TerraformGoat is used for educational purposes only, It is not allowed to use it for illegal and criminal purposes, any consequences arising from TerraformGoat are the responsibility of the person using it, and not the selefra organization.

Contributing

Contributions are welcomed and greatly appreciated. Further reading — CONTRIBUTING.md for details on contribution workflow.

License

TerraformGoat is under the Apache 2.0 license. See the LICENSE file for details.

Wrongsecrets - Examples With How To Not Use Secrets

Welcome to the OWASP WrongSecrets p0wnable app. With this app, we have packed various ways of how to not store your secrets. These can help you to realize whether your secret management is ok. The challenge is to find all the different secrets by means of various tools and techniques.

Can you solve all the 16 challenges? 

Support

Need support? Contact us via OWASP Slack for which you sign up here, file a PR, file an issue , or use discussions. Please note that this is an OWASP volunteer based project, so it might take a little while before we respond.

Basic docker exercises

Can be used for challenges 1-4, 8, 12-15

For the basic docker exercises you currently require:

• Docker Install from here
• Some Browser that can render HTML

You can install it by doing:

docker run -p 8080:8080 jeroenwillemsen/wrongsecrets:1.4.0-no-vault

Now you can try to find the secrets by means of solving the challenge offered at:

• localhost:8080/challenge/1
• localhost:8080/challenge/2
• localhost:8080/challenge/3
• localhost:8080/challenge/4
• localhost:8080/challenge/8
• localhost:8080/challenge/12
• localhost:8080/challenge/13
• localhost:8080/challenge/14
• localhost:8080/challenge/15
• localhost:8080/challenge/16

Note that these challenges are still very basic, and so are their explanations. Feel free to file a PR to make them look better ;-).

Running these on Heroku

You can test them out at https://wrongsecrets.herokuapp.com/ as well! But please understand that we have NO guarantees that this works. Given we run in Heroku free-tier, please do not fuzz and/or try to bring it down: you would be spoiling it for others that want to testdrive it.

Deploying the app under your own heroku account

1. Sign up to Heroku and log in to your account
2. Click the button below and follow the instructions

Basic K8s exercise

Can be used for challenges 1-6, 8, 12-16

Minikube based

Make sure you have the following installed:

• Docker Install from here
• Minikube Install from here

The K8S setup currently is based on using Minikube for local fun:

    minikube start
    kubectl apply -f k8s/secrets-config.yml
    kubectl apply -f k8s/secrets-secret.yml
    kubectl apply -f k8s/secret-challenge-deployment.yml
    while [[ $(kubectl get pods -l app=secret-challenge -o 'jsonpath={..status.conditions[?(@.type=="Ready")].status}') != "True" ]]; do echo "waiting for secret-challenge" && sleep 2; done
    kubectl expose deployment secret-challenge --type=LoadBalancer --port=8080
    minikube service secret-challenge

now you can use the provided IP address and port to further play with the K8s variant (instead of localhost).

• localhost:8080/challenge/5
• localhost:8080/challenge/6

k8s based

Want to run vanilla on your own k8s? Use the commands below:

    kubectl apply -f k8s/secrets-config.yml
    kubectl apply -f k8s/secrets-secret.yml
    kubectl apply -f k8s/secret-challenge-deployment.yml
    while [[ $(kubectl get pods -l app=secret-challenge -o 'jsonpath={..status.conditions[?(@.type=="Ready")].status}') != "True" ]]; do echo "waiting for secret-challenge" && sleep 2; done
    kubectl port-forward \
        $(kubectl get pod -l app=secret-challenge -o jsonpath="{.items[0].metadata.name}") \
        8080:8080

now you can use the provided IP address and port to further play with the K8s variant (instead of localhost).

• localhost:8080/challenge/5
• localhost:8080/challenge/6

Vault exercises with minikube

Can be used for challenges 1-8, 12-16 Make sure you have the following installed:

• minikube with docker (or comment out line 8 and work at your own k8s setup),
• docker,
• helm Install from here,
• kubectl Install from here,
• jq Install from here,
• vault Install from here,
• grep, Cat, and Sed

Run ./k8s-vault-minkube-start.sh, when the script is done, then the challenges will wait for you at http://localhost:8080 . This will allow you to run challenges 1-8, 12-15.

When you stopped the k8s-vault-minikube-start.sh script and want to resume the port forward run: k8s-vault-minikube-resume.sh. This is because if you run the start script again it will replace the secret in the vault and not update the secret-challenge application with the new secret.

Cloud Challenges

Can be used for challenges 1-16

READ THIS: Given that the exercises below contain IAM privilege escalation exercises, never run this on an account which is related to your production environment or can influence your account-over-arching resources.

Running WrongSecrets in AWS

Follow the steps in the README in the AWS subfolder.

Running WrongSecrets in GCP

Follow the steps in the README in the GCP subfolder.

Running WrongSecrets in Azure

Follow the steps in the README in the Azure subfolder.

Running Challenge15 in your own cloud only

When you want to include your own Canarytokens for your cloud-deployment, do the following:

1. Fork the project.
2. Make sure you use the GCP ingress or AWS ingress scripts to generate an ingress for your project.
3. Go to canarytokens.org and select AWS Keys, in the webHook URL field add <your-domain-created-at-step1>/canaries/tokencallback.
4. Encrypt the received credentials so that Challenge15 can decrypt them again.
5. Commit the unencrypted and encrypted materials to Git and then commit again without the decrypted materials.
6. Adapt the hints of Challenge 15 in your fork to point to your fork.
7. Create a container and push it to your registry
8. Override the K8s definition files for either AWS or GCP.

Do you want to play without guidance?

Each challenge has a Show hints button and a What's wrong? button. These buttons help to simplify the challenges and give explanation to the reader. Though, the explanations can spoil the fun if you want to do this as a hacking exercise. Therefore, you can manipulate them by overriding the following settings in your env:

• hints_enabled=false will turn off the Show hints button.
• reason_enabled=false will turn of the What's wrong? explanation button.

Special thanks & Contributors

Leaders:

• Ben de Haan @bendehaan
• Jeroen willemsen @commjoen

Top contributors:

• Nanne Baars @nbaars
• Marcin Nowak @MarcinNowak-codes
• Tibor Hercz @tiborhercz
• Filip Chyla @fchyla
• Dmitry Litosh @Dlitosh
• Josh Grossman @tghosth
• Spyros @northdpole
• Mike Woudenberg @mikewoudenberg
• Ruben Kruiver @RubenAtBinx
• Finn @f3rn0s
• Joss Sparkes @remakingeden

Testers:

• Dave van Stein @davevs
• Marcin Nowak @MarcinNowak-codes
• Marc Chang Sing Pang @mchangsp

Special mentions for helping out:

• Madhu Akula @madhuakula
• Björn Kimminich @bkimminich
• Avinash Pancham @avinashpancham

Help Wanted

You can help us by the following methods:

• Star us
• Share this app with others
• Of course, we can always use your help to get more flavors of "wrongly" configured secrets in to spread awareness! We would love to get some help with other cloudproiders, like Alibabaor Tencent cloud for instance. Do you miss something else than a cloud provider as an example? File an issue or create a PR! See our guide on contributing for more details. Contributors will be listed in releases, in the "Special thanks & Contributors"-section, and the web-app.

Use OWASP WrongSecrets as a secret detection benchmark

As tons of secret detection tools are coming up for both Docker and Git, we are creating a Benchmark testbed for it. Want to know if your tool detects everything? We will keep track of the embedded secrets in this issue and have a branch in which we put additional secrets for your tool to detect. The branch will contain a Docker container generation script using which you can eventually test your container secret scanning.

Notes on development

For development on local machine use the local profile ./mvnw spring-boot:run -Dspring-boot.run.profiles=local

If you want to test against vault without K8s: start vault locally with

 export VAULT_ADDR='http://127.0.0.1:8200'
 export VAULT_API_ADDR='http://127.0.0.1:8200'
 vault server -dev

and in your next terminal, do (with the token from the previous commands):

export VAULT_ADDR='http://127.0.0.1:8200'
export VAULT_TOKEN='<TOKENHERE>'
vault token create -id="00000000-0000-0000-0000-000000000000" -policy="root"
vault kv put secret/secret-challenge vaultpassword.password="$(openssl rand -base64 16)"

Now use the local-vault profile to do your development.

./mvnw spring-boot:run -Dspring-boot.run.profiles=local,local-vault

If you want to dev without a Vault instance, use additionally the without-vault profile to do your development:

./mvnw spring-boot:run -Dspring-boot.run.profiles=local,without-vault

Want to push a container? See .github/scripts/docker-create-and-push.sh for a script that generates and pushes all containers. Do not forget to rebuild the app before composing the container

Dependency management

We have CycloneDX and OWASP Dependency-check integrated to check dependencies for vulnerabilities. You can use the OWASP Dependency-checker by calling mvn dependency-check:aggregate and mvn cyclonedx:makeBom to use CycloneDX to create an SBOM.

Automatic reload during development

To make changes made load faster we added spring-dev-tools to the Maven project. To enable this in IntelliJ automatically, make sure:

• Under Compiler -> Automatically build project is enabled, and
• Under Advanced settings -> Allow auto-make to start even if developed application is currently running.

You can also manually invoke: Build -> Recompile the file you just changed, this will also force reloading of the application.

How to add a Challenge

Follow the steps below on adding a challenge:

1. First make sure that you have an Issue reported for which a challenge is really wanted.
2. Add the new challenge in the org.owasp.wrongsecrets.challenges folder. Make sure you add an explanation in src/main/resources/explanations and refer to it from your new Challenge class.
3. Add a unit and integration test to show that your challenge is working.
4. Don't forget to add @Order annotation to your challenge ;-).

If you want to move existing cloud challenges to another cloud: extend Challenge classes in the org.owasp.wrongsecrets.challenges.cloud package and make sure you add the required Terraform in a folder with the separate cloud identified. Make sure that the environment is added to org.owasp.wrongsecrets.RuntimeEnvironment. Collaborate with the others at the project to get your container running so you can test at the cloud account.