Login
Web Path Finder is a Python program that provides information about a website. It retrieves various details such as page title, last updated date, DNS information, subdomains, firewall names, technologies used, certificate information, and more.Β
Clone the repository:
FreshRSS git clone https://github.com/HalilDeniz/PathFinder.git
Install the required packages:
pip install -r requirements.txt
This will install all the required modules and their respective versions.
Run the program using the following command:
Γ’βΕΓ’ββ¬Γ’ββ¬(rootΓ°ΕΈββ¬denizhalil)-[~/MyProjects/]
Γ’ββΓ’ββ¬# python3 web-info-explorer.py --help
usage: wpathFinder.py [-h] url
Web Information Program
positional arguments:
url Enter the site URL
options:
-h, --help show this help message and exitReplace <url> with the URL of the website you want to explore.
Here is an example output of running the program:
Γ’βΕΓ’ββ¬Γ’ββ¬(rootΓ°ΕΈββ¬denizhalil)-[~/MyProjects/]
Γ’ββΓ’ββ¬# python3 pathFinder.py https://www.facebook.com/
Site Information:
Title: Facebook - Login or Register
Last Updated Date: None
First Creation Date: 1997-03-29 05:00:00
Dns Information: []
Sub Branches: ['157']
Firewall Names: []
Technologies Used: javascript, php, css, html, react
Certificate Information:
Certificate Issuer: US
Certificate Start Date: 2023-02-07 00:00:00
Certificate Expiration Date: 2023-05-08 23:59:59
Certificate Validity Period (Days): 90
Bypassed JavaScript content:
Contributions are welcome! To contribute to PathFinder, follow these steps:
This project is licensed under the MIT License - see the LICENSE file for details.
For any inquiries or further information, you can reach me through the following channels:
TLDHunt is a command-line tool designed to help users find available domain names for their online projects or businesses. By providing a keyword and a list of TLD (top-level domain) extensions, TLDHunt checks the availability of domain names that match the given criteria. This tool is particularly useful for those who want to quickly find a domain name that is not already taken, without having to perform a manual search on a domain registrar website.
For red teaming or phishing purposes, this tool can help you to find similar domains with different extensions from the original domain.
This tool is written in Bash and the only dependency required is whois. Therefore, make sure that you have installed whois on your system. In Debian, you can install whois using the following command:
sudo apt install whois -y
To detect whether a domain is registered or not, we search for the words "Name Server" in the output of the WHOIS command, as this is a signature of a registered domain. If you have a better signature or detection method, please feel free to submit a pull request.
You can use your custom tlds.txt list, but make sure that it is formatted like this:
.aero
.asia
.biz
.cat
.com
.coop
.info
.int
.jobs
.mobi
Γ’ΕΎΕ TLDHunt ./tldhunt.sh
_____ _ ___ _ _ _
|_ _| | | \| || |_ _ _ _| |_
| | | |__| |) | __ | || | ' \ _|
|_| |____|___/|_||_|\_,_|_||_\__|
Domain Availability Checker
Keyword is required.
Usage: ./tldhunt.sh -k <keyword> [-e <tld> | -E <exts>] [-x]
Example: ./tldhunt.sh -k linuxsec -E tlds.txt
Example of TLDHunt usage:
./tldhunt.sh -k linuxsec -E tlds.txt
You can add -x flag to print only Not Registered domain. Example:
./tldhunt.sh -k linuxsec -E tlds.txt -x
This script scans every file from a given folder recursively, extracts every IPv4 and IPv6 address, filters out the public IP addresses and then queries these IPs against TOR relays, WhoIs service and Location service. It outputs 4 files: a CSV file and a JSON file with all details of the IP addresses, a CSV file containing an index of every found IP address and a KML file with all the locations gathered.
Python 3.9.x
To install dependencies, run:
pip install -r requirements.txt
pip install pyinstaller
pyinstaller --onefile osips.py
python osips.py [-h] [-F INPUTFOLDER] [-f INPUTFILE] [-p INPUTFILESPATTERN] [-o OUTPUTFILESNAME] [-t {y,n,Y,N}] [-w {y,n,Y,N}] [-l {y,n,Y,N}] [-locationEndpoint LOCATIONENDPOINT]
python osips.py
python osips.py --checkTor Y --checkWhoIs Y --checkLocations Y --inputFolder testFolder/ --inputFilesPattern * --outputFilesName test
python osips.py -h
-h, --help Show the help message and exit-F INPUTFOLDER, --inputFolder INPUTFOLDER A folder containing files with IPs (log files, email files, text files etc). The IPs can be IPv4 and/or IPv6 and can be placed anywhere in the files. The script will parse every file in the folder and will scan for every IP-f INPUTFILE, --inputFile INPUTFILE Use this option to scan only one file instead of a folder with files. The file can be any text file that contains IP addresses (log files, email files, text files etc.)-p INPUTFILESPATTERN, --inputFilesPattern INPUTFILESPATTERN Enter the pattern to use for the files in the input folder. Example: * for all files, *.txt for text files.-o OUTPUTFILESNAME, --outputFilesName OUTPUTFILESNAME The name for the output files, WITHOUT EXTENSION. 4 different files will be created with this name: a CSV file storing the IP details, an index file storing the IP indexes, a JSON file storing the details and a KML file storing the locations.-t {y,n,Y,N}, --checkTor {y,n,Y,N} Specify if you want to check every public IP address against the TOR relays IP addresses-w {y,n,Y,N}, --checkWhoIs {y,n,Y,N} Specify if you want to extract WhoIs information for every public IP address-l {y,n,Y,N}, --checkLocations {y,n,Y,N} Specify if you want to geocode every public IP address and extract location info using https://reallyfreegeoip.org
-locationEndpoint LOCATIONENDPOINT Default REST endpoint for location is "https://reallyfreegeoip.org/json/". You can specify a diffrent end point, includding placeholder. The result must be in JSON and must include the following fields: country_name, region_code, region_name, city, zip_code, time_zone, latitude, longitude, metro_codeThis article originally appeared in The Domain Name Industry Brief (Volume 18, Issue 3)
Earlier this year, the Internet Engineering Task Forceβs (IETFβs) Internet Engineering Steering Group (IESG) announced that several Proposed Standards related to the Registration Data Access Protocol (RDAP), including three that I co-authored, were being promoted to the prestigious designation of Internet Standard. Initially accepted as proposed standards six years ago, RFC 7480, RFC 7481, RFC 9082 and RFC 9083 now comprise the new Standard 95. RDAP allows users to access domain registration data and could one day replace its predecessor the WHOIS protocol. RDAP is designed to address some widely recognized deficiencies in the WHOIS protocol and can help improve the registration data chain of custody.
In the discussion that follows, Iβll look back at the registry data model, given the evolution from WHOIS to the RDAP protocol, and examine how the RDAP protocol can help improve upon the more traditional, WHOIS-based registry models.
In 1998, Network Solutions was responsible for providing both consumer-facing registrar and back-end registry functions for the legacy .com, .net and .org generic top-level domains (gTLDs). Network Solutions collected information from domain name registrants, used that information to process domain name registration requests, and published both collected data and data derived from processing registration requests (such as expiration dates and status values) in a public-facing directory service known as WHOIS.
From Network Solutionβs perspective as the registry, the chain of custody for domain name registration data involved only two parties: the registrant (or their agent) and Network Solutions. With the introduction of a Shared Registration System (SRS) in 1999, multiple registrars began to compete for domain name registration business by using the registry services operated by Network Solutions. The introduction of additional registrars and the separation of registry and registrar functions added parties to the chain of custody of domain name registration data. Information flowed from the registrant, to the registrar, and then to the registry, typically crossing multiple networks and jurisdictions, as depicted in Figure 1.
Over time, new gTLDs and new registries came into existence, new WHOIS services (with different output formats) were launched, and countries adopted new laws and regulations focused on protecting the personal information associated with domain name registration data. As time progressed, it became clear that WHOIS lacked several needed features, such as:
The IETF made multiple attempts to add features to WHOIS to address some of these issues, but none of them were widely adopted. A possible replacement protocol known as the Internet Registry Information Service (IRIS) was standardized in 2005, but it was not widely adopted. Something else was needed, and the IETF went back to work to produce what became known as RDAP.
RDAP was specified in a series of five IETF Proposed Standard RFC documents, including the following, all of which were published in March 2015:
Only when RDAP was standardized did we start to see broad deployment of a possible WHOIS successor by domain name registries, domain name registrars and address registries.
The broad deployment of RDAP led to RFCs 7480 and 7481 becoming Internet Standard RFCs (part of Internet Standard 95) without modification in March 2021. As operators of registration data directory services implemented and deployed RDAP, they found places in the other specifications where minor corrections and clarifications were needed without changing the protocol itself. RFC 7482 was updated to become Internet Standard RFC 9082, which was published in June 2021. RFC 7483 was updated to become Internet Standard RFC 9083, which was also published in June 2021. All were added to Standard 95. As of the writing of this article, RFC 7484 is in the process of being reviewed and updated for elevation to Internet Standard status.
Operators of registration data directory services who implemented RDAP can take advantage of key features not available in the WHOIS protocol. Iβve highlighted some of these important features in the table below.
| RDAP Feature | Benefit |
| Standard, well-understood, and widely available HTTP transport | Relatively easy to implement, deploy and operate using common web service tools, infrastructure and applications. |
| Securable via HTTPS | Helps provide confidentiality for RDAP queries and responses, reducing the amount of information that is disclosed to monitors. |
| Structured output in JavaScript Object Notation (JSON) | JSON is well-understood and tool friendly, which makes it easier for clients to parse and format responses from all servers without the need for software thatβs customized for different service providers. |
| Easily extensible | Designed to support the addition of new features without breaking existing implementations. This makes it easier to address future function needs with less risk of implementation incompatibility. |
| Internationalized output, with full support for Unicode character sets | Allows implementations to provide human-readable inputs and outputs that are represented in a language appropriate to the local operating environment. |
| Referral capability, leveraging HTTP constructs | Provides information to software clients that allow the client to retrieve additional information from other RDAP servers. This can be used to hide complexity from human users. |
| Support of standardized authentication | RDAP can take full advantage of all of the client identification, authentication and authorization methods that are available to web services. This means that RDAP can be used to provide the basic framework for differentiated access to registration data based on attributes associated with the user and the userβs query. |
Verisignβs RDAP service, which was originally launched as an experimental implementation several years before gaining widespread adoption, allows users to look up records in the registry database for all registered .com, .net, .name, .cc and .tv domain names. It also supports Internationalized Domain Names (IDNs).
We at Verisign were pleased not only to see the IETF recognize the importance of RDAP by elevating it to an Internet Standard, but also that the protocol became a requirement for ICANN-accredited registrars and registries as of August 2019. Widespread implementation of the RDAP protocol makes registration data more secure, stable and resilient, and we are hopeful that the community will evolve the prescribed implementation of RDAP such that the full power of this rich protocol will be deployed.
You can learn more in the RDAP Help section of the Verisign website, and access helpful documents such as the RDAP technical implementation guide and the RDAP response profile.
The post Industry Insights: RDAP Becomes Internet Standard appeared first on Verisign Blog.
