Creating a hard disk image using the dd utility. Creating a bootable USB flash drive for installing Windows using Rufus Cloning a hard drive

Quite often, system administrators need to copy various binary data. For example, sometimes you may need to make a backup hard drive, create an empty file filled with zeros to organize swap space or other virtual file system.

To solve all these problems, the dd linux utility is used, which simply copies data from one place to another at the binary level. It can copy a CD/DVD disc, a section on a disc, or even an entire HDD. In this article we will look at what the linux dd command is, its main options and parameters, and how to use it.

First you need to understand how the dd command works and what it does. In fact, this is an analogue of the utility only for block data. The utility simply transfers one block of data of the specified size from one place to another. Since everything, including devices, is considered a file in Linux, you can transfer devices to files and vice versa.

Using various utility options, you can influence the block size, and this, in turn, already affects the speed of the program. Next we will look at the main options of the utility and its capabilities.

dd command

The syntax of the utility is quite unusual, but at the same time very simple, once you remember and get used to it:

$dd if= copy_source of= destination options

Using the if parameter, you need to specify the source from which the blocks will be copied, this can be a device, for example, /dev/sda or a file - disk.img. Next, using the of parameter, you need to specify the destination device or file. Other parameters have the same syntax as if and of.

Now let's look at the additional options:

bs- indicates how many bytes to read and write at a time;
CBS- how many bytes need to be written at a time;
count- copy the specified number of blocks, the size of one block is indicated in the bs parameter;
conv- apply filters to the data stream;
ibs- read the specified number of bytes at a time;
obs- write the specified number of bytes at a time;
seek- skip the specified number of bytes at the beginning of the reading device;
skip- skip the specified number of bytes at the beginning of the output device;
status- indicates how detailed the conclusion should be;
iflag, oflag- allows you to set additional operation flags for the input and output device, the main ones: nocache, nofollow.

These were all the basic options you might need. Now let's move closer to practice and look at several examples of how to use the dd linux utility.

How to use dd?

Regular users use the dd command most often to create images DVDs or CD. For example, to save a disk image to a file, you can use the following command:

sudo dd if=/dev/sr0 of=~/CD.iso bs=2048 conv=noerror

The noerror filter allows you to disable response to errors. Next, you can create an image of the hard drive or partition on it and save this image to disk. Just be careful not to save to the same hard drive or partition, so as not to cause recursion:

dd if=/dev/sda of=~/disk.img

A file named disk1.img will be created in your home folder, which in the future can be deployed and restored to the damaged system. To write an image to a hard drive or partition, just swap the device addresses:

dd if=~/disk.img of=/dev/sda

A very important and useful option is bs. It allows you to greatly influence the speed of the utility. This parameter allows you to set the size of one block when transferring data. Here you need to specify a digital value with one of these format modifiers:

With- one character;
b- 512 bytes;
kB- 1000 bytes;
K- 1024 bytes;
M.B.- 1000 kilobytes;
M- 1024 kilobytes;
G.B.- 1000 megabytes;
G- 1024 megabytes.

The dd linux command uses just such a system, it is complex, but there is no escape from it. It will have to be understood and remembered. For example, 2b is 1 kilobyte, and 1k is also 1 kilobyte, 1M is 1 megabyte. By default, the utility uses a block size of 512 bytes. For example, to speed up disk copying, you can take blocks of 5 megabytes in size. To do this, use the following command:

dd if=/dev/sda of=~/disk.img bs=5M

The next parameter is count. Using it you can specify how many blocks need to be copied. For example, we can create a 512 megabyte file by filling it with zeros from /dev/zero or random numbers from /dev/random:

sudo dd if=/dev/zero of=file.img bs=1M count=512

Please note that this parameter does not indicate the size in megabytes, but only the number of blocks. Therefore, if you specify a block size of 1b, then you only need to take two blocks to create a 1KB file. This option can also be used to backup the MBR partition table. To do this, copy the first 512 bytes of the hard drive to a file:

sudo dd if=/dev/sda of=mbr.img bs=1b count=1

To restore, use the usual command to deploy the image to disk.

If the disk image is too large, you can redirect all output to the non-standard output stream of the gzip utility:

dd if =/dev/sda2 | bzip2 disk.img.bz2

You can also use the dd linux utility to copy files, although this is not its intended purpose:

dd if=/home/sergiy/test.txt of=/home/sergiy/test1.txt

As you know, the linux dd command writes data to disk directly to binary, this means that zeros and ones are written. They override what was previously placed on the recording device. Therefore, to erase a disk, you can simply fill it with zeros from /dev/zero.

sudo dd if=/dev/zero of=/dev/sdb

Using dd this way results in the entire disk being completely erased.

conclusions

In this article we looked at how to use dd linux, what this utility can be used for and how useful it can be. It's an almost indispensable tool. system administrator, because it can be used to make backup copies of an entire system. And now you know how. If you have any questions, ask in the comments!

dd is a simple utility that is included with most Unix-like software. operating systems- Linux, FreeBSD, Solaris, etc.
Its purpose is to read data from one device or file and write to another.

dd can be effectively used to create an image of a hard drive, without using commercial utilities like Acronis True Image or Norton Ghost.

Let's assume we have two disks:

/dev/sda - the disk whose image needs to be created;
/dev/sdb - the disk on which the image will be written.

If necessary, you need to substitute your own values.

The first thing you need to do is boot from any available Live-CD disc, which has the dd utility, and enter command line as a superuser. Create a mount point for backup.

mkdir /mnt/backup

We mount the hard drive on which you want to save the image.

Creating a hard drive image

dd if=/dev/sda of=/mnt/backup/sda.img bs=8M conv=sync,noerror

if=/dev/sda - copy the entire hard drive sda;
of=/mnt/backup/sda.img - copy to /mnt/backup/sda.img;
bs=8M — set the size of the hard drive cache to speed up the copying procedure (otherwise the data will be reset in small portions of 512 bytes);
conv=sync,noerror - we indicate to dd the need for bit-for-bit copying and ignoring read errors.

To reduce the size of a hard disk image, you can compress it with any archiver.

dd if=/dev/sda bs=8M conv=sync,noerror | gzip -c > /mnt/backup/sda.img

Recovering a hard drive image

To restore a hard disk image, you must follow the reverse procedure to the procedure for creating this image.

dd if=/mnt/backup/sda.img of=/dev/sda bs=8M conv=sync,noerror

When using compression, you must unzip the image in parallel.

gunzip -c /mnt/backup/sda.img | dd of=/dev/sda conv=sync,noerror bs=8M

Migrating the system to another hard drive

To migrate the entire system to another hard drive, you must set the location of the new drive as the destination.

dd if=/dev/sda of=/dev/sdb bs=8M conv=sync,noerror

Then, if necessary, install the boot from this tough disk. Provided that new hard the disk is larger than the old one, there will be an unallocated area on it. It should be marked up and formatted according to existing requirements.

Copy statistics in dd

The main disadvantage of dd is the lack of a visual representation of the statistics of the copying procedure. However, this disadvantage can be easily circumvented. All you need to do is connect to another terminal.

Determine the process number under which dd is running.

Send periodically this process command kill -USR1 process_number_dd .

watch -n 5 kill -USR1 process_number_dd

watch -n 5 - execute the command every 5 seconds;
kill -USR1 process_number_dd — show copy statistics.

Cons of using dd to create disk images

Everything has pros and cons. dd is a free and very flexible tool, but it can only do full copy volumes Special programs They can only copy data that is stored on disk.

Thus, the volume of a disk image created using dd will be equal to the volume of the disk itself - regardless of how much data is on the disk.

Team dd is designed to use the utility of the same name, designed for low-level copying and conversion of data. Its name stands for “data duplicator” or “data duplicator”. This utility is used mainly for writing images of installation disks of Linux distributions to flash drives and creating images of optical media, however, its range of functions is not limited to the listed operations. For example, dd can be used to simply copy files or change the case of text strings. In general, the utility in question is to some extent unique, because it involves the use of its own format for passing parameters.

The standard command syntax is as follows:

$dd if=<имя исходного файла>of=<имя целевого файла>[options]

It is easy to notice that the recording format is used to pass parameters to the utility <имя параметра>=<значение параметра> . The utility can read source data from standard input and output the resulting data using standard output if parameters are not used if And of, but in the vast majority of cases these parameters are necessary to specify file names with the corresponding data. The utility reads and writes data in blocks, and the block size can be changed using the parameter bs(Blocks of 512 KB are used by default). There are separate parameters for setting the sizes of readable and writable blocks, namely, ibs And obs. The number of blocks read can be limited using the parameter count. The parameter can be used to skip a specified number of blocks in the source file. skip, target file - parameter seek. The parameter can be used to specify read and write flags separated by commas iflag

append- activation of the mode of appending data to the target file.
direct- data processing mode bypassing the file system cache (increases speed).
dsync- data recording mode with synchronization (increases reliability).
sync- data and metadata recording mode with synchronization (increases reliability).
fullblock- reading only complete blocks.
nonblock- activation of non-blocking I/O mode (increases speed).
noatime- disabling the mechanism for updating file system element timestamps (increases speed).
nofollow- refusal to follow symbolic links.

Finally, the parameter can be used to specify comma-separated conversion flags conv. The most commonly used flags are:

lcase- converting string characters in ASCII encoding to lower case.
ucase- converting string characters in ASCII encoding to upper case.
nocreat- display an error message if the target file is missing.
excl- display an error message if the target file exists.
notrunc- refusal to trim the target file.
swab- changing the locations of every two bytes from the source file.
noerror- continuation of work even if errors occur.
fdatasync- activation of the mode of writing data to the target file before completing the utility.
fsync- activation of the mode of writing data and metadata to the target file before completing the utility.

Examples of using

Backing up disk drive data

Let's assume we are using a hard drive represented by a device file /dev/sda, and we need to create a sector-by-sector backup of all the data located on it, saving it in a file in the partition of the removable USB drive represented by the device file /dev/sdb1 and mounted in the directory /mnt/sdb1. These backup files are usually called dumps or disk images. Our disk image file will be named backup.img. This is the command with which you can create it:

# dd if=/dev/sda of=/mnt/sdb1/backup.img

In this command, using the parameter if the path to the source file is specified, and using the parameter of- to the target.

Restoring data from a backup

To recover data from the created backup copy you should boot the system from installation disk distribution and execute the reverse command.

# dd if=/mnt/sdb1/backup1.img of=/dev/sda

Warning: executing this command will overwrite the entire contents of the specified hard drive, so you should treat such commands with special attention.

Hard drive cloning

Before cloning a hard drive, you must ensure that you have a hard drive that is the same size as the original one. The same operation can be performed in the case of flash drives with a USB interface of similar sizes. Let's assume that the source flash drive is represented by a device file /dev/sdb, and the target - the device file /dev/sdc. In this case, you can clone the source drive using the following command:

# dd if=/dev/sdb of=/dev/sdc

Even if the target drive has a larger capacity, you will only have access to the size of the source flash drive stored at the file system level.

Transferring a disk image file to another computer

To transfer a disk image file over a network to another computer named target the following command can be used:

# dd if=/dev/sdb | ssh root@target "(cat >backup.img)"

Compressing a Disk Image File

To make the backup disk partition take up less space, you can compress it using a compressor such as bzip2 :

# dd if=/dev/sdb | bzip2 backup.img.bz2

Creating an ISO optical disc image

To create an image of an optical disk CD, DVD or BD, simply read its contents block by block and save this contents in a file:

# dd if=/dev/sr0 of=image.iso bs=2048

Saving a file from damaged media or creating an image of such media

If your favorite movie or music track is no longer readable due to media corruption, you can try to copy it using the utility dd, ignoring bad blocks:

# dd if=movie.avi of=/home/alex/movie.avi conv=noerror,sync

You can also create an image file of the damaged media and try to extract files from it:

# dd if=/dev/sdb of=/home/alex/movie.iso bs=2048 conv=noerror,sync

Burning an installation disk image to a USB flash drive

For installation Linux distribution from a USB flash drive, you must write an ISO installation disk image to this flash drive. A similar command can be used for this purpose:

# dd if=/home/alex/Fedora-Workstation-Live-x86_64-26_Alpha-1.7.iso of=/dev/sdc

It is important to remember that even if there are partitions on the flash drive, you should not specify the path to the device file of one of the partitions, but the path to the device file of the drive itself, in our case this is /dev/sdc.

Hard drive content analysis

Utility dd is an excellent tool for exploring file systems. To analyze the contents of a hard drive with data output from individual blocks, in our case, a block 1001 on the partition represented by the device file /dev/sdc1, just use the following command:

# dd if=/dev/sdc1 count=1 skip=1000

In order to see the first 40 bytes of your hard drive in hexadecimal notation, use the command:

# dd if=/dev/sda bs=1 count=40 | hexdump -C

IN in this case using the parameter bs sets the disk block size.

Testing disk drive performance

To test the performance of a disk drive represented, for example, by a device file /dev/sda When reading blocks of different sizes, a similar command can be used:

1000000+0 records in
1000000+0 records out

# dd if=/dev/sda of=/dev/null bs=4096 count=1000000
1000000+0 records in
1000000+0 records out
4096000000 bytes (4.1 GB) copied, 29.8747 s, 137 MB/s

Thanks to the file system caching mechanism, you may encounter a mystical speedup of read operations that should not be surprising:

# dd if=/dev/sda of=/dev/null bs=512 count=1000000
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 4.25186 s, 120 MB/s

# dd if=/dev/sda of=/dev/null bs=512 count=1000000
1000000+0 records in
1000000+0 records out
512000000 bytes (512 MB) copied, 0.417317 s, 1.2 GB/s

To obtain correct test results, it is recommended to bypass the file system caching mechanism altogether by using I/O mode without caching:

# dd if=/dev/sda of=/dev/null bs=512 count=100000 iflag=direct
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 5.01053 s, 10.2 MB/s

Copying files

Yes, utility dd can even be used for regular file copying. Of course, for this purpose it is better to use a utility specially designed for this purpose, namely, cp. In any case, you can copy the file using a similar command:

$ dd if=/home/alex/test.txt /home/alex/test_copy.txt

Rufus- free software with open source code for formatting removable USB storage media and creating bootable disks with various operating systems. The program is easy to use, high speed work and supports a multilingual interface.

You can download the program on the developer's website. The page contains links to download the standard version Rufus, and portable Rufus portable, which do not differ in anything except the name of the executable file and the location where the settings are stored. The program settings include the language used and settings for checking for updates. The standard version stores these settings in the registry, while the portable version stores these settings in a file rufus.ini program directory. Rufus does not require installation on the system - just download executable file and run it. The program interface is very simple:

Overall, the program Rufus is not something unique in the field of tools for creating bootable media and its main advantage is ease of use. In order to create a bootable flash drive with its help, it is enough to have the original image of the bootable system and be able to click on the “Start” button. All selected parameters and settings, by default, are already designed for using the program to work on a standard computer.

Easiest and most convenient to use Rufus to create a bootable flash drive (bootable USB drive) from installation ISO images Windows drives or Linux, as well as system emergency recovery disks and diagnostic tools.

When creating a bootable Windows flash drive, just select the device to which the recording and file will be performed bootable iso image. The program will set other parameters itself.

If you do not have an ISO image file, you can create one from a physical CD (or from a set of distribution files) using CD/DVD burning programs such as the well-known Nero, Alcohol, or the free CDBurnerXP or ImgBurn.

The procedure for creating a bootable USB flash drive with Windows is as follows:

select the flash drive on which the image will be written. Unlike many similar programs, Rufus displays the volume label, drive letter and size, so if you have several removable drives on the system, it is easy to select the one to write to.

select the partition scheme and system interface type. Rufus allows you to create flash drives for booting in a regular BIOS interface and for booting in a UEFI environment, create boot records for MBR volumes and GPT volumes. The default mode is "MBR for computers with BIOS or UEFI" - the most common mode for bootable flash drives today.

select the file system that will be used on the bootable flash drive to be created. By default, bootable Windows flash drives use the file system FAT32, but if necessary, you can choose NTFS, if you need to use files larger than 4 GB.

set the cluster size. The cluster size is selected by the program based on the image data and file system type, but if necessary, it can be changed.

specify the volume label that will be specified for the created flash drive.

set formatting options. It is best to leave these parameters as default and just select the file ISO image. For images created by the program dd on Linux, you need to select the option DD image.

After pressing the button Start the program will format the flash drive, set the sign of the active partition, write down the main boot entry and the partition boot record, as well as bootable media data from the ISO image. After completion of work Rufus You can boot using the resulting bootable flash drive.

Using virtualization technology to test bootable flash drives. Links to download free and convenient programs to simplify the process of creating, debugging and testing created bootable media.

To create an image of a hard drive, it is not necessary to use utilities like Acronis True Image or Norton Ghost; a simple dd utility, which is included in most Unix-like operating systems (Linux, FreeBSD, Solaris, etc.), is sufficient. The article discusses a simple method create a backup copy of the hard disk image using dd. The first step is to prepare for backup. In this article we introduce the following notation:

/dev/sda - the disk whose image needs to be created;
/dev/sdb - the disk on which the image will be written.