Azure AZcopy — Moving files across Buckets or Containers
→ This is a helpful tool when we copy data between different Azure subscriptions or cross Clouds like Amazon S3 bucket or Google Cloud Bucket
Pre-requisites:
Azure Storage account with some files.
a. For creating an Azure account
→ This is important because using AZcopy we will copy files from one Azure storage account to the other.
→ Before starting copying Azure files. Let us go to our storage account.
STEP 1: Search for a storage account on the Azure home page
STEP 2: Inside the storage account, click on the container whose file we want to copy.
eg. storageamir
→ Click on the containers.
→ Click on the storage container where files reside.
→ We can now see files inside our storage account’s container:
→ Till now we have located the place in the storage account from where we will copy files.
→ Now the first way, of course, is the basic way of copying
Solution 1: Manual:
→ By the manual solution we mean downloading the file locally and then uploading it to the storage account where want to copy it.
→ Click on the files you want to copy.
→ Scroll to the right, and click the three dots.
→ Click on Download. Our files are downloaded locally to our system and can be read.
→ We see files can be read by clicking the downloaded file ( i.e. image for this case)
→ For small files, this approach is okay.
→ But if we download large data (say in TBs), it will take time to download.
→ Secondly, Azure also charges if we have TBs or multiple TBs of data.
→ For this there is another solution that is fast, free and can easily be done i.e. AZcopy
Solution 2: AZcopy:
→ AZcopy is the efficient way to copy data because instead of downloading locally or uploading in some other storage account.
→ AZ Copy is the Azure terminal to copy blobs or files to or from a storage account. We can connect to our storage account, and then transfer data.
→ Before using AZ copy, two things are required:
a. Get source URL:
→ Let us go to the file we need to copy.
→ Click on the image, scroll right
→ Click Generate SAH
→ Scroll down and click the Generate SAS button:
→ We need to copy the file path — Click the copy button
→ Let us paste this source URL in a Notepad or Editor:
→ If we hit this URL in the browser, the file will open:
b. Get destination URL:
→ We need to copy files from one storage account to another.
→ For this we should have another storage account where we need to copy.
→ Let us create a storage account.
→ Search for Storage Account in Search
→ Click on Create + button:
→ Enter the name of the destination container eg. secondcontainer
→ Click the Review button:
→ Click on Create button:
→ Within some seconds, a new storage account is created.
→ Click the Go to resource button
→ We are going to create containers inside the storage account where files will be copied.
→ Click on Containers
→ Click on + Container button
→ Enter the Storage account name. eg. second-storage-container
→ Click Create button:
→ Within some seconds new container will be created.
→ The next step is to get the destination URL of this container.
→ Click on three dots of the newly created container. Click Generate SAS.
→ This step is important. By default only Read permission is selected
→ We have to give the Write permission for copying files into it. For now, giving most access permissions
→ Click on Generate SAS Token and URL button
→ We have to copy BLOB SAS URL and paste into a notepad.
AZcopy Terminal:
→ Now we have source (file) and destination (container) URLs
There are two ways we can use azcopy terminal:
a. Download locally and use terminal:
· Windows 64-bit (zip)
· Windows 32-bit (zip)
· Linux x86–64 (tar)
· Linux ARM64 Preview (tar)
· macOS (zip)
b. Using Cloud Shell:
→ We can directly open the terminal from the Azure portal and run commands.
→ Click on the Cloud terminal icon
→ Click Powershell
→ Click Create Storage
→ Finally our terminal is set up and ready to be used:
→ Let us clear the screen using the clear command
→ For using azcopy, we have to check in the terminal whether the copy is installed. This is similar to checking git — help
azcopy --help
→ We observe terminal responds to us with commands to use. This means azcopy is ready to be used.
→ In the newly created container, we observe that it is currently empty.
Command to Copy:
→ The command to copy is:
azcopy copy '<source_url>' '<destination_url>'→ We already have copied files to our notepad — both source (file URL)and destination (container URL).
→ Let us run the command
→ This copy method is very fast. No double efforts of downloading and re-upload files.
→ We observe getting 100% result and it took 0.0334 seconds to copy.
→ Let us now go to the destination container and click the Refresh button.
→ Our file is successfully copied. 😀
→ If we are copying TBs of data. This solution will save your day. It is ultra fast in copying files ⚡⚡
Closing Thoughts:
In this article, we have understood why AZcopy is important and how it can save us. Let us summarize it:
a. AZcopy is a tool that we download and use, or from Azure cloud shell.
b. Basically it is used to move files across cloud accounts, and keep files in sync at ultra-speed.
c. We can even move files across Amazon S3 cloud buckets or Google Cloud buckets.
Thank you for reading till the end 🙌 . If you enjoyed this article or learned something new, support me by clicking the share button below to reach more people and/or give me a follow on Twitter and subscribe Happy Learnings !! to see some other tips, articles, and things I learn about and share there.
