Packages and Binaries:

uro

Declutter URLs for crawling/pentesting
Uro declutters a list of URLs by removing

• Incremental URLs,
• Blog posts and other similar human-written content,
• URLs with the same path but parameter value differences,
• Images, JS, CSS, and other “useless” files.

Installed size: 38 KB
How to install: sudo apt install uro

uro

root@kali:~# uro -h
usage: uro [-h] [-i INPUT_FILE] [-o OUTPUT_FILE]
           [-w WHITELIST [WHITELIST ...]] [-b BLACKLIST [BLACKLIST ...]]
           [-f FILTERS [FILTERS ...]]

options:
  -h, --help            show this help message and exit
  -i INPUT_FILE         file containing urls
  -o OUTPUT_FILE        output file
  -w, --whitelist WHITELIST [WHITELIST ...]
                        only keep these extensions and extensionless urls
  -b, --blacklist BLACKLIST [BLACKLIST ...]
                        remove these extensions
  -f, --filters FILTERS [FILTERS ...]
                        additional filters, read docs