Archive

Articles taggués ‘rsync’

Ignore existing files or update only newer files with rsync

10/03/2024 Aucun commentaire

Rsync is a useful command line utility for synchronising files and directories across two different file systems. I recently needed to use rsync to only copy over files that did not already exist at the other end, so this post documents how to do this.

Copying from local to remote

Note that all the examples shown in the post are for copying files from the local computer to a remote server/computer.

Default behavior

The following command will recursively copy all files from the local filesystem from /var/www to the remote system at 10.1.1.1. Note the following:

  1. Any files that do not exist on the remote system are copied over
  2. Any that have been updated will be copied over, although note that rsync is extremely efficient in that only the changed parts of files are copied and if the file is exactly the same if it is not copied over at all
  3. Any that have been deleted on the local system are deleted on the remote
rsync -raz --progress /var/www 10.1.1.1:/var

Ignore existing files

Use the –ignore-existing flag to prevent files from being copied over that already exist on the remote server. By adding this, we eliminate behaviors 2 and 3 in the list above and all that is done is this:

  1. Any files that do not exist on the remote system are copied over
--ignore-existing -raz --progress /var/www 10.1.1.1:/var 

Lire la suite…

Categories: Système Tags:

Using Rsync and SSH

23/01/2024 Comments off

Source: troy.jdmz.net

Keys, Validating, and Automation

This document covers using cron, ssh, and rsync to backup files over a local network or the Internet. Part of my goal is to ensure no user intervention is required when the computer is restarted (for passwords, keys, or key managers).
I like to backup some logging, mail, and configuration information sometimes on hosts across the network and Internet, and here is a way I have found to do it. You’ll need these packages installed:
  • rsync
  • openssh
  • cron (or vixie-cron)

Please note these instructions may be specific to Red Hat Linux versions 7.3, 9, and Fedora Core 3, but I hope they won’t be too hard to adapt to almost any *NIX type OS. The man pages for ‘ssh’ and ‘rsync’ should be helpful to you if you need to change some things (use the “man ssh” and “man rsync” commands). Lire la suite…

Categories: Système Tags: , , ,

Sauvegarder ses données avec Rsync

21/01/2024 Comments off

Source: journaldunadminlinux.fr

En parcourant pas mal de forums, j’ai pu remarquer que pas mal de jeunes admins ne connaissaient pas Rync.

Rsync est un petit outil de sauvegarde extrêmement puissant. Grâce à cette outil j’ai pu sauvegarder des centaines de  giga-octet de données sans problèmes.

Rsync se charge en plus de garder tous les droits et attributs de vos fichiers et gère en natif SSH pour faire une sauvegarde d’une machine vers une autre. Lire la suite…

Synchronizing folders with rsync

10/01/2024 Comments off

Source: Juan Valencia’s website

In this post I cover the basics of rsync, in preparation for a subsequent post that will cover backups and it’s use in conjunction with cronjobs to automatize the backup process. From the copying and synchronization of local files and folders, to it’s use for transfer information among computers. Itsuse as a daemon when SSH is unavailable was moved to it’s own section.

Topics
The basics of rsync
Copying local files and folders
Dealing with whitespace and rare characters
Update the contents of a folder
Synchronizing two folders with rsync
Compressing the files while transferring them
Transferring files between two remote systems
Excluding files and directories
Running rsync as a daemon (moved to it’s own section)
Some additional rsync parameters
Footnotes

The basics of rsync

rsync is a very versatile copying and backup tool that is included by default in almost every Linux distribution. It can be used as an advanced copying tool, allowing us to copy files both locally and remotely. It can also be used as a backup tool. It supports the creation of incremental backups.

rsync counts with a famous delta-transfer algorithm that allows us to transfer new files as well as recent changes to existent files, while ignoring unchanged files. Additionally to this, the behavior ofrsync can be throughly customized, helping us to automatize backups, it can also be run as a daemon to turn the computer into a host and allow rsync clients connect to it.

Besides the copying of local files and folders, rsync allow us to copy over SSH (Secure Shell), RSH (Remote Shell) and it can be run as a daemon in a computer and allow other computers to connect to it, when rsync is run as a daemon it listens to the port TCP 873.

When we use rsync as a daemon or when we use RSH, the data that is send between computers travels unencrypted, so, if you are transferring files between two computers in the same local network, this is useful, but this shouldn’t be used to transfer files over insecure networks, such as the Internet. For this purpose SSH is the way to go.

This is the main reason why I favor the use of SSH for my transfers, besides, since SSH is secure, many servers have the SSH daemon available. But the use of rsync as a daemon for transfers over fast connections, as is usually the case in a local network, is useful. I don’t have the RSH daemon running in my computers so you may find me a bit biased about SSH in the examples. The examples covering the transfer of files between two computers use SSH as the medium of transport, but in a separate post I cover the use of rsync as a daemon.

Lire la suite…

Categories: Réseau, Système Tags: , ,

NAS Synology : résoudre l’erreur rsync “permission denied” lors de la connexion au NAS après mise à jour du DSM

25/12/2023 Comments off

Mon NAS Synology vient de mettre à jour son firmware DSM et je constate en lançant ma sauvegarde rsync que la connexion rsync vers le NAS ne se fait plus : après saisie du mot de passe, on obtient une erreur “permission denied”.

Voici comment remédier à ce petit désagrément en deux minutes montre en main.

Problème : connexion SSH refusée

Lors de la connexion initiale, démarrée par :

rsync --ignore-existing --progress -vr --rsh='ssh -p22222' /home/backup/* root@example.com:/volume1/video

on obtient le message d’erreur suivant, après saisie du mot de passe:

Permission denied, please try again.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.1]

Après vérification que les identifiants (user/password) sont bien corrects, il s’avère que la solution réside dans l’utilisation de l’argument --rsync-path afin d’expliciter le chemin de l’exécutable rsync présent sur le NAS.

Lire la suite…

Categories: Logiciel, Système Tags: ,