matejaputic/Site Archiver
Archive web pages from a sitemap as Markdown files using Defuddle
View on GitHubTrust Signals
- Scorecard Score
- not yet scored
- Maintenance Recency
- Stale
- License
- None
Inputs
| name | description | required | default |
|---|---|---|---|
| sitemap_url | URL to fetch sitemap.xml from (e.g., https://example.com/sitemap.xml) | yes | — |
| xpath | XPath expression to filter URLs to include (e.g., //url/loc[contains(text(), '/blog/')]) | no | //url/loc |
| dry_run | When true, only lists URLs without archiving (useful for testing XPath expressions) | no | false |
| output_dir | Output directory for archived pages | no | pages |
| delay_ms | Delay between requests in milliseconds (rate limiting) | no | 5000 |
| max_retries | Maximum number of retry attempts per page | no | 3 |
| retry_delay_ms | Delay before retrying a failed request in milliseconds | no | 5000 |
| commit | Whether to commit and push changes | no | true |
| commit_message | Commit message (use {date} for current date) | no | Archive website content {date} |
| git_user_name | Git user name for commits | no | Site Archiver Bot |
| git_user_email | Git user email for commits | no | archiver@localhost |
Outputs
| name | description |
|---|---|
| success_count | Number of pages successfully archived |
| failed_count | Number of pages that failed |