01 Jan 2025
+27 Oct 2024
arcK is a command-line, cross-platform archival tool to preserve data.
Thank you for stopping by and taking an interest in this project. This is an exciting project to archive important data to ensure successful recovery. To achieve this, the directory is encoded into 5 parts with 2 extra parts for redundancy, which means you only need 3 parts to restore.
This may take longer to do, but file integrity is of the utmost importance and this project ensures that even with 2 missing parts, data can be restored; an excellent use case is to store across multiple cloud storage providers or regional buckets with the same cloud provider. I have stored my personal photos across two S3 regions. Each bucket does not need to have all files, therefore I have stored 3 in one and 3 in another; with one part stored in both (bucket_US: 0,1,2; bucket_EU: 2,3,4).
Before:
After:
Original data remains intact. A new archive is created. You can specify a different output directory - the above is example only.
This is not a backup tool and cannot update the archive with incremental changes.
Let's get started: visit the download page.
"},{"location":"bugs/","title":"Bugs","text":"It's empty in here. As we discovery bugs they will be reported here.
"},{"location":"coding/","title":"Erasure code","text":"Erasure code is a method to preserve data integrity by recovery from damaged parts.
Ark uses this code to split archives into 5 parts. The archive can be restored with only 3 of the original parts.
Wikipedia: Erasure Code
"},{"location":"contact/","title":"Contact","text":"For more information or queries, please email:
arck-email24@karlhunter.co.uk
"},{"location":"create/","title":"Create","text":""},{"location":"create/#wizard","title":"Wizard","text":"Run the app via command line:
arck
Follow on-screen prompts:
Provide a source directory, destination directory and name of archive.
The name of the archive denotes the name of the part files. For example, if you choose 'pic24' the files will be named:
arck -s /path/to/source -d /path/to/destination -n name_archive
Name of archive cannot have equals sign.
"},{"location":"create/#backup","title":"Backup","text":"Please ensure you keep additional copies of the metadata file, ending with the .m. You can never have too many copies.
"},{"location":"create/#create-technical","title":"Create Technical","text":"Following process is followed during creation:
If you find this software useful, and choose to make a donation, I would be grateful.
click here to donate.
"},{"location":"download/","title":"Downloads","text":""},{"location":"download/#ubuntudebian-based-system-including-linux-mint","title":"Ubuntu/Debian based system (including Linux Mint)","text":"Recommended
Install guide
"},{"location":"download/#windows","title":"Windows","text":"Not GUI, needs to be run through command prompt or powershell.
./arck.exe
Choose the right architecture and operating system.
Architecture Linux Windows .deb FreeBSD OpenBSD NetBSD darwin amd64 Download Download Download Download Download Download Download i386 (32-bit) Download Arm64 Download Download Download Download Download Arm6 Download Download"},{"location":"download/#time-to-install","title":"Time to install","text":"Install guide
"},{"location":"errors/","title":"Exit Error Codes","text":"Code Description 0 Completed with no errors 1 General input error; read message given during exit 2 Encode error 3 Archive error (encoding) 5 Hash fail find file 6 Metadata error 7 Reconstruction error 8 Metadata load error 9 Unable to reconstruct due to checksum errors 10 Extraction of file"},{"location":"install/","title":"Install","text":""},{"location":"install/#deb-package","title":"Deb package","text":"Recommended methods for debian/ubuntu distros.
Download the compiled binary package.
From a graphical interface, right-click and choose Install
From command line, type:
dpkg --install arck.deb
Before you can use the binary, you need to make it executable. Use command:
chmod +x arck
Then, copy to bin directory:
cp -a arck /usr/local/bin
Test this worked:
arck
You should be greeted with the wizard.
"},{"location":"license/","title":"MIT License","text":"Copyright (c) 2024 Karl Hunter
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"},{"location":"metadata/","title":"Metadata File","text":""},{"location":"metadata/#about","title":"About","text":"The metadata file is important and must be backed up and also stored alongside the parts too. In a nutshell, keep many copies of this file -- it makes restoring easier.
The file contains the checksum values of each part. This is important because if you feed the erasure coding module with a damaged part, it could result in failure to reconstruct the archive. If a part fails, arcK will automatically delete the damaged part - afterall, it's useless now. But remember, you can restore the archive with just 3 parts.
"},{"location":"metadata/#fixing-damanged-or-lost-metadata-file","title":"Fixing damanged or lost metadata file","text":"If you need to recreate the metadata file, generate sha256 checksum of each part and store in the metadata file on one line. This can be in any order. Nb. End the line with a semi-colon.
archive_name.0=sha256hash;archive_name.1=sha256hash; ...
Name the file with the same name of the parts. For example, if the archive parts are called 'pictures.X' then name the metadata file 'pictures.m'
If you are not confident of the integrity of the parts, then use 'skip'. For example:
archive_name.0=936a185caaa266bb9cbe981e9e05cb78cd732b0b3280eb944412bb6f8f8f07af;archive_name.1=skip; ...
This will ensure the program continues without giving you an error, then the erasure coding module will attempt to check the archive parts integrity. This may fail, as giving this module bad data may result in failure to restore; hence why the metadata file is of utmost importance In this case, try renaming each part and then see if the restoration works until you find the bad part. There may be more than one bad part, which makes this process complex.
"},{"location":"release/","title":"Release Notes","text":""},{"location":"release/#112","title":"1.1.2","text":"27 Oct 2024
19 Oct 2024
13 Oct 2024
12 Oct 2024
Welcome to the first release of ark. We hope you find it useful.
"},{"location":"restore/","title":"Restore","text":""},{"location":"restore/#wizard","title":"Wizard","text":"Run the app via command line:
arck
Choose option 3
3. Restore an archive
Provide path to the metadata file (the file ending in .m)
Ensure the metadata file is in the same directory with the part files
If you do not give an output directory, the default will be used: current working directory.
The directory/file will be restored to original state.
"},{"location":"restore/#command-line","title":"Command Line","text":"arck -x /path/to/file.m -d /path/to/restore/
Supported flags:
--skipFollowing process is followed during extraction:
Directory structure restored.
"},{"location":"start/","title":"Getting Started","text":""},{"location":"start/#quick-start","title":"Quick Start","text":""},{"location":"start/#install","title":"Install","text":"It's easy to get started. Frist, download and run or install the pre-built binary.
Test installation has worked. Run:
arck
This will start the wizard. If you have not installed into your bin directory, CD to directory and run:
.\\arck
You will be prompted to choose operating mode, which are as follows:
Either use the wizard (called by arck), or via single command as below.
Simply follow the on-screen prompts:
To create an archive, simply run:
arck -s /path/to/source/ -d /output/dir/ -n archive_name
And to restore (extract) the archive:
arck -x /output/dir/archive_name.m -d /path/to/source
more
"},{"location":"tech/","title":"Technical Manual","text":"arck {-s -d -n | -v | -x -d}
Flag Description -s Source directory used for creating the archive -d Target directory used to output archive. If not specified, working directory will be used instead -n Specify the name of the archive. Must be given when creating archive -v Verifies integrity of part files. Specify metadata file to test archives -x Restore archive. Specify path to metadata file"},{"location":"usage/#create-archive","title":"Create archive","text":"You can specify a directory (recommended) or file. You do not need to tar archive a directory yourself.
arck -s /path/to/source/ -d /path/to/outputdir/ -n archive_name
You cannot create archive names with an equals (=) sign. If you want to use a space remember to enclose in perensese: -n \"name of archive\"
Nb: the output directory must have double the available space to match the source directory. After processing, the space requirement will the same as the source directory.
How this works.
"},{"location":"usage/#verify-archive","title":"Verify archive","text":"A useful feature is to verify the integrity of the archive parts. Testing backups are an important aspect of data preservation. Verifying is a faster method to test parts integrity without restoring the archive.
arck -v /path/to/archive.m
You will need to specify the archive metadata file. As a reminder, you should keep this backed up and ideally keep multiple copies. More on the (metadata file)[metadata.md].
You will be given an output like:
project.0 PASS project.1 PASS project.2 PASS project.3 PASS project.4 PASS
If there are any failed parts, the output will read:
project.3 FAIL
In this case, consider restoring broken part from backup and re-testing. There are no methods to fix broken parts, but if there are enough \"working\" parts, you can restore. To restore, there needs to be 3 of the 5 parts undamaged.
"},{"location":"usage/#restore","title":"Restore","text":"Before you restore, ensure metadate file is located with parts. Example, metadata file named [archive_name].m
File structure:
archive.m archive.0 archive.1 archive.2 archive.3 archive.4
You can still restore if parts are missing. Let the program handle this.
arck -x /path/to/metadata_file.m -d /path/to/outputdir/
How this works.
"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Welcome to arcK","text":"arcK is a command-line, cross-platform archival tool to preserve data.
Thank you for stopping by and taking an interest in this project. This is an exciting project to archive important data to ensure successful recovery. To achieve this, the directory is encoded into 5 parts with 2 extra parts for redundancy, which means you only need 3 parts to restore.
This may take longer to do, but file integrity is of the utmost importance and this project ensures that even with 2 missing parts, data can be restored; an excellent use case is to store across multiple cloud storage providers or regional buckets with the same cloud provider. I have stored my personal photos across two S3 regions. Each bucket does not need to have all files, therefore I have stored 3 in one and 3 in another; with one part stored in both (bucket_US: 0,1,2; bucket_EU: 2,3,4).
Before:
After:
Original data remains intact. A new archive is created. You can specify a different output directory - the above is example only.
This is not a backup tool and cannot update the archive with incremental changes.
Let's get started: visit the download page.
"},{"location":"bugs/","title":"Bugs","text":"It's empty in here. As we discovery bugs they will be reported here.
"},{"location":"coding/","title":"Erasure code","text":"Erasure code is a method to preserve data integrity by recovery from damaged parts.
Ark uses this code to split archives into 5 parts. The archive can be restored with only 3 of the original parts.
Wikipedia: Erasure Code
"},{"location":"contact/","title":"Contact","text":"For more information or queries, please email:
arck-email24@karlhunter.co.uk
"},{"location":"create/","title":"Create","text":""},{"location":"create/#wizard","title":"Wizard","text":"Run the app via command line:
arck
Follow on-screen prompts:
Provide a source directory, destination directory and name of archive.
The name of the archive denotes the name of the part files. For example, if you choose 'pic24' the files will be named:
arck -s /path/to/source -d /path/to/destination -n name_archive
Name of archive cannot have equals sign.
"},{"location":"create/#backup","title":"Backup","text":"Please ensure you keep additional copies of the metadata file, ending with the .m. You can never have too many copies.
"},{"location":"create/#create-technical","title":"Create Technical","text":"Following process is followed during creation:
If you find this software useful, and choose to make a donation, I would be grateful.
click here to donate.
"},{"location":"download/","title":"Downloads","text":""},{"location":"download/#ubuntudebian-based-system-including-linux-mint","title":"Ubuntu/Debian based system (including Linux Mint)","text":"Recommended
Install guide
"},{"location":"download/#windows","title":"Windows","text":"Not GUI, needs to be run through command prompt or powershell.
./arck.exe
Choose the right architecture and operating system.
Architecture Linux Windows .deb FreeBSD OpenBSD NetBSD darwin amd64 Download Download Download Download Download Download Download i386 (32-bit) Download Arm64 Download Download Download Download Download Arm6 Download Download"},{"location":"download/#time-to-install","title":"Time to install","text":"Install guide
"},{"location":"errors/","title":"Exit Error Codes","text":"Code Description 0 Completed with no errors 1 General input error; read message given during exit 2 Encode error 3 Archive error (encoding) 5 Hash fail find file 6 Metadata error 7 Reconstruction error 8 Metadata load error 9 Unable to reconstruct due to checksum errors 10 Extraction of file"},{"location":"install/","title":"Install","text":""},{"location":"install/#deb-package","title":"Deb package","text":"Recommended methods for debian/ubuntu distros.
Download the compiled binary package.
From a graphical interface, right-click and choose Install
From command line, type:
dpkg --install arck.deb
Before you can use the binary, you need to make it executable. Use command:
chmod +x arck
Then, copy to bin directory:
cp -a arck /usr/local/bin
Test this worked:
arck
You should be greeted with the wizard.
"},{"location":"license/","title":"MIT License","text":"Copyright (c) 2024 Karl Hunter
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"},{"location":"metadata/","title":"Metadata File","text":""},{"location":"metadata/#about","title":"About","text":"The metadata file is important and must be backed up and also stored alongside the parts too. In a nutshell, keep many copies of this file -- it makes restoring easier.
The file contains the checksum values of each part. This is important because if you feed the erasure coding module with a damaged part, it could result in failure to reconstruct the archive. If a part fails, arcK will automatically delete the damaged part - afterall, it's useless now. But remember, you can restore the archive with just 3 parts.
"},{"location":"metadata/#fixing-damanged-or-lost-metadata-file","title":"Fixing damanged or lost metadata file","text":"If you need to recreate the metadata file, generate sha256 checksum of each part and store in the metadata file on one line. This can be in any order. Nb. End the line with a semi-colon.
archive_name.0=sha256hash;archive_name.1=sha256hash; ...
Name the file with the same name of the parts. For example, if the archive parts are called 'pictures.X' then name the metadata file 'pictures.m'
If you are not confident of the integrity of the parts, then use 'skip'. For example:
archive_name.0=936a185caaa266bb9cbe981e9e05cb78cd732b0b3280eb944412bb6f8f8f07af;archive_name.1=skip; ...
This will ensure the program continues without giving you an error, then the erasure coding module will attempt to check the archive parts integrity. This may fail, as giving this module bad data may result in failure to restore; hence why the metadata file is of utmost importance In this case, try renaming each part and then see if the restoration works until you find the bad part. There may be more than one bad part, which makes this process complex.
"},{"location":"release/","title":"Release Notes","text":""},{"location":"release/#113","title":"1.1.3","text":"01 Jan 2025
27 Oct 2024
19 Oct 2024
13 Oct 2024
12 Oct 2024
Welcome to the first release of ark. We hope you find it useful.
"},{"location":"restore/","title":"Restore","text":""},{"location":"restore/#wizard","title":"Wizard","text":"Run the app via command line:
arck
Choose option 3
3. Restore an archive
Provide path to the metadata file (the file ending in .m)
Ensure the metadata file is in the same directory with the part files
If you do not give an output directory, the default will be used: current working directory.
The directory/file will be restored to original state.
"},{"location":"restore/#command-line","title":"Command Line","text":"arck -x /path/to/file.m -d /path/to/restore/
Supported flags:
--skipFollowing process is followed during extraction:
Directory structure restored.
"},{"location":"start/","title":"Getting Started","text":""},{"location":"start/#quick-start","title":"Quick Start","text":""},{"location":"start/#install","title":"Install","text":"It's easy to get started. Frist, download and run or install the pre-built binary.
Test installation has worked. Run:
arck
This will start the wizard. If you have not installed into your bin directory, CD to directory and run:
.\\arck
You will be prompted to choose operating mode, which are as follows:
Either use the wizard (called by arck), or via single command as below.
Simply follow the on-screen prompts:
To create an archive, simply run:
arck -s /path/to/source/ -d /output/dir/ -n archive_name
And to restore (extract) the archive:
arck -x /output/dir/archive_name.m -d /path/to/source
more
"},{"location":"tech/","title":"Technical Manual","text":"arck {-s -d -n | -v | -x -d}
Flag Description -s Source directory used for creating the archive -d Target directory used to output archive. If not specified, working directory will be used instead -n Specify the name of the archive. Must be given when creating archive -v Verifies integrity of part files. Specify metadata file to test archives -x Restore archive. Specify path to metadata file"},{"location":"usage/#create-archive","title":"Create archive","text":"You can specify a directory (recommended) or file. You do not need to tar archive a directory yourself.
arck -s /path/to/source/ -d /path/to/outputdir/ -n archive_name
You cannot create archive names with an equals (=) sign. If you want to use a space remember to enclose in perensese: -n \"name of archive\"
Nb: the output directory must have double the available space to match the source directory. After processing, the space requirement will the same as the source directory.
How this works.
"},{"location":"usage/#verify-archive","title":"Verify archive","text":"A useful feature is to verify the integrity of the archive parts. Testing backups are an important aspect of data preservation. Verifying is a faster method to test parts integrity without restoring the archive.
arck -v /path/to/archive.m
You will need to specify the archive metadata file. As a reminder, you should keep this backed up and ideally keep multiple copies. More on the (metadata file)[metadata.md].
You will be given an output like:
project.0 PASS project.1 PASS project.2 PASS project.3 PASS project.4 PASS
If there are any failed parts, the output will read:
project.3 FAIL
In this case, consider restoring broken part from backup and re-testing. There are no methods to fix broken parts, but if there are enough \"working\" parts, you can restore. To restore, there needs to be 3 of the 5 parts undamaged.
"},{"location":"usage/#restore","title":"Restore","text":"Before you restore, ensure metadate file is located with parts. Example, metadata file named [archive_name].m
File structure:
archive.m archive.0 archive.1 archive.2 archive.3 archive.4
You can still restore if parts are missing. Let the program handle this.
arck -x /path/to/metadata_file.m -d /path/to/outputdir/
How this works.
"}]} \ No newline at end of file diff --git a/arck/sitemap.xml.gz b/arck/sitemap.xml.gz index 549c605..646d307 100644 Binary files a/arck/sitemap.xml.gz and b/arck/sitemap.xml.gz differ