{"id":1094,"date":"2024-11-14T14:27:24","date_gmt":"2024-11-14T22:27:24","guid":{"rendered":"https:\/\/www.gauntletwizard.net\/wordpress\/?p=1094"},"modified":"2024-11-14T14:27:24","modified_gmt":"2024-11-14T22:27:24","slug":"real-time-constant-backups-with-zfszrepl","status":"publish","type":"post","link":"https:\/\/www.gauntletwizard.net\/wordpress\/?p=1094","title":{"rendered":"Real time constant backups with ZFS+Zrepl"},"content":{"rendered":"\n<p>This is a guide to making your home network backups seamless, secure, and awesome. It&#8217;s comparable in many ways to Apple Time Machine.<br \/><br \/>Prerequisites:<br \/>A machine in the cloud with lots of disk space<br \/>Mine comes from zfs.rent, where I sent some physical hard drives and rent a small machine with them attached.<br \/>CA Infrastructure.<br \/>Creating your own private CA is a whole topic of it&#8217;s own, one I hope to make simpler. For the short term, creating a private CA just for Zrepl is the best option. Create the key with <code>cfssl genkey -initca zrepl-ca.json | cfssljson -bare zrepl-ca<\/code><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/zrepl\/zrepl-ca.json - On your backup host\n\n{\n  \"CN\": \"Personal Zrepl Root CA\",\n  \"key\": {\n    \"algo\": \"rsa\",\n    \"size\": 4096\n  },\n  \"names\": &#91;\n    {\n      \"C\": \"US\",\n      \"L\": \"Seattle\",\n      \"O\": \"Thomas Hahn\",\n      \"OU\": \"Zrepl\",\n      \"ST\": \"Washington\"\n    }\n  ]\n}<\/code><\/pre>\n\n\n\n<p><br \/>Setup<br \/>Create certificates for your hosts. They should use the hostname as the CN, but also have appropriate Subject Alternative Names. I recommend CFSSL to do so.<\/p>\n\n\n\n<p>Example contents of  your cfssl csr json:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># \/etc\/zrepl\/zrepl.json\n{\n  \"CN\": \"timemachine\",\n  \"hosts\": &#91;\n    \"timemachine.internal\",\n    \"timemachine.zfs.rent\",\n    \"timemachine.gauntletwizard.net\"\n  ],\n  \"key\": {\n    \"algo\": \"rsa\",\n    \"size\": 4096\n  },\n  \"names\": &#91;\n    {\n      \"C\": \"US\",\n      \"L\": \"Seattle\",\n      \"O\": \"Thomas Hahn\",\n      \"OU\": \"Zrepl\",\n      \"ST\": \"Washington\"\n    }\n  ]\n}<\/code><\/pre>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir \/etc\/zrepl\ncfssl genkey zrepl.json | cfssljson -bare zrepl-$HOSTNAME<\/code><\/pre>\n\n\n\n<p>You should now have a <code>zrepl-key.pem<\/code> and a <code>zrepl.csr<\/code> in your \/etc\/zrepl folder, on each of your machines. Copy all of the .csr files to your CA for signing. Don&#8217;t touch the -key.pem files! These are your private keys, and need to be secret. Once you have the csrs on your CA&#8217;s machine, sign them. Sign them with: <br \/><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>host=HOSTNAME # Replace HOSTNAME with the name of each host\ncfssl sign -ca ca.pem -ca-key ca-key.pem \"zrepl-${host}.csr\" | cfssljson -bare ${host}<\/code><\/pre>\n\n\n\n<p>Signing them with the above will leave you with a set of .pem files, named host.pem. You should verify that they signed correctly: <code>openssl verify -CAfile ca.pem host.pem<\/code>. It should print &#8216;host.pem: ok&#8217;. Copy these files back to their respective hosts, and rename them on that host to zrepl.crt. Also copy the ca.pem file to each host as \/etc\/zrepl\/ca.pem<\/p>\n\n\n\n<p>Next, set up the backup as a sink:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>global:\n  logging:\n    # use syslog instead of stdout because it makes journald happy\n    - type: syslog\n      format: human\n      level: warn\n  monitoring:\n    - type: prometheus\n      listen: ':9811'\n\n\njobs:\n  - name: backups\n    type: sink\n    root_fs: tank\/backups\n    recv:\n      placeholder:\n        encryption: inherit\n    serve:\n      type: tls\n      listen: \":8826\"\n      ca: \"\/etc\/zrepl\/ca.pem\"\n      cert: \"\/etc\/zrepl\/zrepl.crt\"\n      key: \"\/etc\/zrepl\/zrepl-key.pem\"\n      client_cns:\n        - \"desktop\"\n        - \"laptop\"\n        - \"timemachine\"<\/code><\/pre>\n\n\n\n<p>zrepl uses the CN field for disambiguation. Add each host you signed above to the  client_cns section in zrepl.yaml.  zrepl will create a new zfs filesystem under your root_fs for each of these client cns upon their first backup, i..e. <code>tank\/backups\/desktop<\/code> and so on. <\/p>\n\n\n\n<p>Next, configure your machines as sources:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>global:\n  logging:\n    # use syslog instead of stdout because it makes journald happy\n    - type: syslog\n      format: human\n      level: warn\n  monitoring:\n    - type: prometheus\n      listen: ':9811'\n\n\njobs:\n  - name: desktop\n    type: push\n    filesystems:\n      \"desktop\/home&lt;\": true\n    send:\n      encrypted: false\n    connect:\n      type: tls\n      address: \"timemachine:8826\"\n      ca: \/etc\/zrepl\/ca.pem\n      cert: \/etc\/zrepl\/zrepl.crt\n      key:  \/etc\/zrepl\/zrepl-key.pem\n      server_cn: \"timemachine\"\n\n    snapshotting:\n      type: periodic\n      prefix: zrepl_\n      interval: 5m\n    pruning:\n      keep_sender:\n        - type: not_replicated\n        - type: regex\n          regex: \".*\"\n      keep_receiver:\n      - type: grid\n        grid: 1x1h(keep=all) | 24x1h | 30x1d | 6x30d\n        regex: \"^zrepl_\"<\/code><\/pre>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is a guide to making your home network backups seamless, secure, and awesome. It&#8217;s comparable in many ways to Apple Time Machine. Prerequisites:A machine in the cloud with lots of disk spaceMine comes from zfs.rent, where I sent some physical hard drives and rent a small machine with them attached.CA Infrastructure.Creating your own private &hellip; <a href=\"https:\/\/www.gauntletwizard.net\/wordpress\/?p=1094\" class=\"more-link\">Continue reading <span class=\"screen-reader-text\">Real time constant backups with ZFS+Zrepl<\/span> <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1094"}],"collection":[{"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1094"}],"version-history":[{"count":3,"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1094\/revisions"}],"predecessor-version":[{"id":1099,"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=\/wp\/v2\/posts\/1094\/revisions\/1099"}],"wp:attachment":[{"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1094"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1094"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gauntletwizard.net\/wordpress\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1094"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}