v1.0 public

README.md

@@ -0,0 +1,84 @@
+# Introduction
+Yaset (Yet Another Subdomain Enumeration Tool) is a subdomain enumeration tool which main function is the passive enumeration. The APIs used as sources in the passive enumeration are added creating templates in YAML, making Yaset a tool that can grow easily.
+
+# Installation
+To install the tool and the templates, you have to compile the file `yaset.go`, move it to some directory in your path and then, clone the templates repository in your home directory. You can execute the following line:
+`git clone https://github.com/whoissecure/yaset && cd yaset && go build yaset.go && mkdir ~/.config/yaset && mv config.ini ~/.config/yaset/. && sudo mv yaset /usr/bin/. && cd && git clone https://github.com/whoissecure/yaset-templates`
+
+After that, you can delete the folder called "yaset" created where you executed the commands, and you can execute the tool from anywhere because it was moved to `/usr/bin/`, which should be in your path.
+
+# Modes
+To execute Yaset, you must choose one mode or both. The passive mode interacts with third party APIs to obtain subdomains, and the bruteforce mode uses a wordlist to check if the formed domain resolves to an IP.
+
+## Passive mode
+This is the main mode of Yaset and to use it, you need to specify the `-p` flag. There are templates that need variables to be defined and for that, a ini file is used. An example for this file is in the repository and it is placed in `~/.config/yaset/config.ini`, the default path that will use the tool automatically. In this file, the main things defined will be the API keys, but it is possible to add variables if needed (this is better explained in the templates section). It is possible to check how many templates are loaded correctly and how many need the variables to be added to the ini file because there are variables missing executing Yaset with the `-c` argument.
+
+## Bruteforce
+To bruteforce subdomains, the argument `-w` with a wordlist. This mode combines each line of the text file with the domain to be bruteforced and tries to resolve it.
+
+# Usage
+```
+Usage of yaset:
+  -c	Check templates status and errors.
+  -config string
+    	Path to config file (default "~/.config/yaset/config.ini")
+  -d string
+    	The domain to be enumerated.
+  -o string
+    	File to write the results.
+  -p	Passive mode to enumerate using APIs.
+  -r	Resolve all the domains.
+  -w string
+    	Wordlist to bruteforce domains.
+```
+
+# Templates
+The templates are stored in other repository (https://github.com/whoissecure/yaset-templates). To add APIs, templates with YAML are created. The format of the templates to define the HTTP requests is the following:
+
+```yaml
+name: yasetDB # Optional. Now, it is only a reference, the tool does not use it.
+use: true # Optional. If it is not present or if it is in false, the template is not used.
+url: http://yaset.local/subdomains/:target?limit=1000 # Obligatory. Example of URL. It can use GET parameters.
+verb: POST # Obligatory. HTTP method.
+headers: # Optional. You can add headers to the request.
+  X-API-Key: :apikey
+data: # Optional. Request body.
+  domain: :target
+  test: value
+vars: # Variables to replace in URL, headers and request body. Target or equivalent is mandatory as engine.target to be replaced by the domain introduced as target.
+  target: engine.target
+  apikey: engine.config.service.key
+# Choose between parse or regexUse to get the domains from the response body.
+parse: "@this.#.host" # Parse the JSON from the response body (if it is in json format) with a GJSON (https://gjson.dev) expression.
+regexUse: true # Use a regex to match all the domains in the response body.
+```
+
+First of all, when parsing a template, the tool replaces "engine.target" by the domain introduced to be enumerated and then, it replaces the variables with the format "engine.config.X.Y". 
+
+The variables of type "engine.config" are used to get a value from the config.ini file. The default path for the config file is `~/.config/yaset/config.ini` and, for example, the variable defined as "engine.config.X.Z" in the template, should be defined in the ini file as:
+
+```ini
+[X]
+Z = "API_KEY_OR_VAR_HERE"
+```
+
+The vars field make replacements in URL, headers and request body, and then Yaset forges the HTTP request to be done.
+
+The parser used in the requests for the APIs with json format is GJSON (https://gjson.dev/). It is possible to parse other types of responses indicating in the template the use of the regex, instead of the gjson expression.
+
+To check if the created template is valid for its syntax, you can place it in the `~/yaset-templates` directory and execute Yaset with the `-c` flag.
+
+# Contributing
+If you have any feature that you want to be added, please, open an issue or make a pull request.
+
+# References
+- Post written in spanish in my personal blog: https://www.whoissecure.xyz/2023/05/yaset.html
+
+# To do
+- [ ] Change the structure of the project to be used as a library or like cli tool
+- [ ] Support multiple requests in one template (Used for example to obtain a token and use then before or to get a list of links and then visit and parse them)
+- [ ] Support yaset-scripts too?
+- [ ] Enumeration with other methods than APIs and brute force or maintain just passive enumeration?
+- [ ] Support `engine.iterate` with variables like `start`, `end` and `max`, to make multiple requests changing a numeric parameter.
+- [X] Some domains are taken as url encoded "%252FX.Z.Y" due to double URL encoding in the results of the APIs, fix it some way, maybe upgrading the regex (?
+- [ ] Change parse in templates to engine.regex/gjson.expression/cookies.CookieName to use when supporting various reqs.

config.ini

@@ -0,0 +1,43 @@
+# Config file for yaset (Yet Another Subdomain Enumeration Tool)
+# The info of this tool is in https://github.com/whoissecure/yaset
+# You can open issues with the tool in the main repo or if they are template related, in https://github.com/whoissecure/yaset-templates
+
+# https://otx.alienvault.com/ [Free]
+[alienvault]
+key = ""
+
+# https://www.binaryedge.io/ [Free]
+[binaryedge]
+key = ""
+
+# https://rapidapi.com/projectxio/api/bufferover-run-tls [Free]
+[bufferover]
+key = ""
+
+# https://chaos.projectdiscovery.io/#/ [Free]
+[chaos]
+key = ""
+
+# https://fullhunt.io/ [Free]
+[fullhunt]
+key = ""
+
+# https://hunter.io/ [Free]
+[hunterio]
+key = ""
+
+# https://netlas.io/ [Free]
+[netlas]
+key = ""
+
+# https://securitytrails.com/ [Free]
+[securitytrails]
+key = ""
+
+# https://www.shodan.io/ [Free]
+[shodan]
+key = ""
+
+# https://www.virustotal.com
+[virustotal]
+key = ""

go.mod

@@ -0,0 +1,15 @@
+module yaset
+
+go 1.20
+
+require (
+	github.com/tidwall/gjson v1.14.4
+	gopkg.in/ini.v1 v1.67.0
+	gopkg.in/yaml.v2 v2.4.0
+)
+
+require (
+	github.com/stretchr/testify v1.8.2 // indirect
+	github.com/tidwall/match v1.1.1 // indirect
+	github.com/tidwall/pretty v1.2.0 // indirect
+)

go.sum

@@ -0,0 +1,27 @@
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
+github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
+github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
+github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
+github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
+github.com/tidwall/gjson v1.14.4 h1:uo0p8EbA09J7RQaflQ1aBRffTR7xedD2bcIVSYxLnkM=
+github.com/tidwall/gjson v1.14.4/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
+github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
+github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
+github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
+github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/ini.v1 v1.67.0 h1:Dgnx+6+nfE+IfzjUEISNeydPJh9AXNNsWbGP9KzCsOA=
+gopkg.in/ini.v1 v1.67.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
+gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
+gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=