
Hurls 作成者: Device1
Hurls short for historic urls is a Firefox Extension that was created so that it'd be easier to interact with the wayback CDX API, and with the purpose of using it in security research.
この拡張機能を使用するには Firefox が必要です
拡張機能メタデータ
この拡張機能について
The user can create a wayback CDX API request with the following:
- Add the domain, by placing
- Add a limit, by placing an integer like "50000", which is the default input, into the limit input, which basically adds "&limit=50000" into the final url. NOTE: The higher this is the more resources are required from your PC!
- Check the subdomain option, that will add "*." before the domain, which will make the CDX API pull results with subdomains aswell.
- Add a date from which point in time the results should be pulled from, by placing a year like "2020" and "2021" into the date inputs, which will basically just add "&from=2020&to=2021" into the final url. According to the CDX API documentation it's also possible to use regular timestamps aswell, however I have not tried that out myself.
- Add various filters available for the API such as, "mimetype" and "statuscode", which allow you to pull more specific type of urls from the CDX API, by pressing the "+ Filter" button and adding the filters such as "mimetype:application/json", which would basically add "&filter=mimetype:application/json" into the final url, allowing you to pull urls that use the corresponding mimetype form the CDX API. There are other filters available that can be read from the CDX API github page: https://github.com/internetarchive/wayback/tree/master/wayback-cdx-server
The results of the request will then be placed into a new tab, which allows you to check the results in various forms:
- You can get all the urls from the response, with no filters or anything placed, by pressing the "All Urls" button.
- You can get all the endpoints from the response, with some regex filters placed to remove trashy endpoints from the results, by pressing the "All Endpoints" button.
- You can get all the parameters from the response, with some regex filters placed to remove trashy parameters from the results, by pressing the "All Parameters" button. This differs from the endpoints in a way as you get all the parameters as single values like "query" rather than "?query=x&anotherparam=y" type of format.
NOTE: The CDX API is very slow and if your target is massive it can take a good while for the request to complete. To somewhat mitigate the wait time you can use a smaller limit or place specific dates to get data from like placing 2023, 2025 into the date input.
- Add the domain, by placing
domain.com
into the domain input, which basically adds "?url=domain.com/*" into the final url.- Add a limit, by placing an integer like "50000", which is the default input, into the limit input, which basically adds "&limit=50000" into the final url. NOTE: The higher this is the more resources are required from your PC!
- Check the subdomain option, that will add "*." before the domain, which will make the CDX API pull results with subdomains aswell.
- Add a date from which point in time the results should be pulled from, by placing a year like "2020" and "2021" into the date inputs, which will basically just add "&from=2020&to=2021" into the final url. According to the CDX API documentation it's also possible to use regular timestamps aswell, however I have not tried that out myself.
- Add various filters available for the API such as, "mimetype" and "statuscode", which allow you to pull more specific type of urls from the CDX API, by pressing the "+ Filter" button and adding the filters such as "mimetype:application/json", which would basically add "&filter=mimetype:application/json" into the final url, allowing you to pull urls that use the corresponding mimetype form the CDX API. There are other filters available that can be read from the CDX API github page: https://github.com/internetarchive/wayback/tree/master/wayback-cdx-server
The results of the request will then be placed into a new tab, which allows you to check the results in various forms:
- You can get all the urls from the response, with no filters or anything placed, by pressing the "All Urls" button.
- You can get all the endpoints from the response, with some regex filters placed to remove trashy endpoints from the results, by pressing the "All Endpoints" button.
- You can get all the parameters from the response, with some regex filters placed to remove trashy parameters from the results, by pressing the "All Parameters" button. This differs from the endpoints in a way as you get all the parameters as single values like "query" rather than "?query=x&anotherparam=y" type of format.
NOTE: The CDX API is very slow and if your target is massive it can take a good while for the request to complete. To somewhat mitigate the wait time you can use a smaller limit or place specific dates to get data from like placing 2023, 2025 into the date input.
あなたの体験を評価
Permissions and data詳細情報
必要な権限:
- ブラウザーのタブへのアクセス
- すべてのウェブサイトの保存されたデータへのアクセス
詳しい情報
- バージョン
- 1.0.1
- サイズ
- 11.84 KB
- 最終更新日
- 3ヶ月前 (2025年5月6日)
- 関連カテゴリー
- ライセンス
- MIT License
- バージョン履歴
コレクションへ追加
1.0.1 のリリースノート
- Added default dark mode
- Tiny UI changes
Device1 が公開している他の拡張機能
- まだ評価されていません
- まだ評価されていません
- まだ評価されていません
- まだ評価されていません
- まだ評価されていません
- まだ評価されていません