top

Search

GIT Tutorial

1. Git LFSWhat is LFSStoring Large content files eventually consumes a huge amount of space on the Git Server.Moreover, since Git is a Distributed Version Control System, every clone and every checkout or pull of this repository will have to download each version of this huge file present on the Server; unlike Centralized Systems wherein just the latest version is downloaded.Git LFS server comes to our rescue in terms of storing huge files.Let’s see how?Git LFS is a system for managing and versioning large files of upto 2GM in association with a Git repository.  Instead of storing the large files within the Git repository as blobs, Git LFS stores special "pointer files" in the repository, while storing the actual file contents on a Git LFS server.  The contents of the large file are downloaded automatically when needed, for example when a Git branch containingthe large file is checked out.Referenced: Git LFS manual page.Summary:Git LFS handles large files by storing references to the file in the repository, but not the actual file itself. Diagram: Distribution of normal files and LFS files over the remote and LFS serverDiagram: git checkout of LFS files.Diagram: after a ‘git checkout’How to install git LFSStep 1: Setup git LFS on your system. Or Just for a repository.git lfs installhugeProj [master] $git lfs install Updated git hooks. Git LFS initialized.Step 2: Choose the type of files you want to track as LFS files.git lfs track “*.iso” hugeProj [master] $git lfs track "*.iso" "*.tar" Tracking "*.iso" Tracking "*.tar" --List the tracked files: hugeProj [master] $git lfs track Listing tracked patterns    *.iso (.gitattributes)    *.tar (.gitattributes) Listing excluded patternsStep 3: Tracking information is stored in .gitattributes file, add this file and rest of the files to the repositorygit add .gitattributesUntracked files:  (use "git add <file>..." to include in what will be committed) .gitattributes CentOS-7-x86_64-Minimal-1804.iso code/Step 4: Commit, push the files normally to the Git server.hugeProj [master] $git add . hugeProj [master] $git commit -m 'Add LFS objects' hugeProj [master] $git push origin master Uploading LFS objects: 100% (1/1), 950 MB | 0 B/s, done                                                         Enumerating objects: 7, done. Counting objects: 100% (7/7), done. Delta compression using up to 4 threads Compressing objects: 100% (5/5), done. Writing objects: 100% (6/6), 875 bytes | 875.00 KiB/s, done. Total 6 (delta 0), reused 0 (delta 0) To github.com:divyabhushan/hugeProject.git    2beb247..3f8184b  master -> masterStep 5: Verify the files on the Git server and local repository.FLS files stored on the LFS Server and not on the Git Server.Snapshot from GitHub Server shows an LFS file is available only in the RAW form; as it is stored on a LFS Server.hugeProj [master] $ls -lt total 1855496 -rw-r--r--  1 Divya1 staff         13 Jan 28 12:23 README drwxr-xr-x  3 Divya1 staff        102 Jan 28 12:22 code -rw-r--r--@ 1 Divya1  staff 950009856 Jan 28 12:20 CentOS-7-x86_64-Minimal-1804.isoNOTE: LFS files are denoted with an ‘@’ attribute on the client machine-@      Display extended attribute keys and sizes in long (-l) output.Step 6: Clone this repository as a different local repository.Regular files are pulled/downloaded from Git Server and LFS files from the LFS Server.git clone http://github.com/divyabhushan/hugeProject.git largeProj Uploading LFS objects: 100% (1/1), 950 MB | 0 B/s, donegit lfs commands:git lfs install:Install Git LFS configuration.git lfs version:Report the version number.git lfs env:Display the Git LFS environment.git lfs ls-files:Show information about Git LFS files in the index and working tree.hugeProj [master] $git lfs ls-files 714acc0aef * CentOS-7-x86_64-Minimal-1804.isogit lfs track “*.ext”:View or add Git LFS paths to Git attributes.git lfs untrack “*.ext”Remove Git LFS paths from Git Attributes.git lfs statusShow the status of Git LFS files in the working tree.git lfs push:Push queued large files to the Git LFS endpoint.git lfs update:Update Git hooks for the current Git repository.--Status after ‘git add’ myProj [initialBranch] $git lfs status On branch initialBranch Git LFS objects to be pushed to origin/initialBranch: Git LFS objects to be committed: CentOS-7-x86_64-Minimal-1804.iso Git LFS objects not staged for commit:CONCLUSION:Data is increasing tremendously over the internet; with IoT(“Internet Of Things”) technology booming.There will always be a need of a good Version system like “Git” and the filesystem like “LFS” integrated with Git to store, process and analyze the ever-growing data.We learnt:What is LFS how do we store are large files in the LFS cloud and checkout them to our local repository.2. Git patch operationGit format-patch - Prepare patches for e-mail submissionPrepare each commit with its patch in one file per commit, formatted to resemble UNIX mailbox format. The output of this command is convenient for e-mail submissionCommand:git format-patch1) Create a patch from the commit id (HEAD)myProj [master] $git format-patch -1 0001-Add-the-Linux-iso-image.patch2) Create a patch from an older selected commit id:myProj [master] $git format-patch -1 367b4ed 0001-Removing-temp-and-useless-files.patch3) Read the patch in a diff format:[master] $cat 0001-Add-the-Linux-iso-image.patch  From 310ddc5504a0533ab5612d7387a97b08450851d9 Mon Sep 17 00:00:00 2001 From: divya bhushan <divya_bhushan@hotmail.com> Date: Mon, 28 Jan 2019 12:09:12 +0100 Subject: [PATCH] Add the Linux iso image. ---  .gitattributes                   | 2 ++  CentOS-7-x86_64-Minimal-1804.iso | 3 +++  2 files changed, 5 insertions(+)  create mode 100644 .gitattributes  create mode 100644 CentOS-7-x86_64-Minimal-1804.iso diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..539acb8 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +*.iso filter=lfs diff=lfs merge=lfs -text +*.tar filter=lfs diff=lfs merge=lfs -text diff --git a/CentOS-7-x86_64-Minimal-1804.iso b/CentOS-7-x86_64-Minimal-1804.iso new file mode 100644 index 0000000..9598fdb --- /dev/null +++ b/CentOS-7-x86_64-Minimal-1804.iso @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:714acc0aefb32b7d51b515e25546835e55a90da9fb00417fbee2d03a62801efd +size 950009856 --  2.19.04) To list the Statistics of the patch but do not apply it, use the command: git apply --stat <patch_name>myProj [master] $git apply --stat 0001-Add-the-Linux-iso-image.patch .gitattributes                   | 2 ++ CentOS-7-x86_64-Minimal-1804.iso |    3 +++ 2 files changed, 5 insertions(+)5) Check for errors before applying the patch; with the command:git apply --check <patch_name>Let’s say you received the patch via email; and you would want to apply it in your present projectcd hugeProject --copy the patch here. hugeProj [master] $cp ../myProj/0001-Add-the-Linux-iso-image.patch hugeProj [master] $git apply --check 0001-Add-the-Linux-iso-image.patch error: .gitattributes: already exists in working directory error: CentOS-7-x86_64-Minimal-1804.iso: already exists in working directoryThe patch won’t apply here as the files are already present in the projectLet’s test another patch: 0001-Removing-temp-and-useless-files.patch--Read the patch content and the stats hugeProj [master] $git apply --stat 0001-Removing-temp-and-useless-files.patch mytest.mine |    2 -- runtest.t   | 0 2 files changed, 2 deletions(-) --copy the 2 files from the 367b4ed~1 commit id of ‘myProj’ repository here myProj [master] $checkout 367b4ed~1 myProj [(HEAD detached at d1cd657)] $cp runtest.t ../hugeProj/ myProj [(HEAD detached at d1cd657)] $cp mytest.mine ../hugeProj/ --Test if the patch would apply successfully! --No output means a success cd ../hugeProj hugeProj [master] $git apply --check 0001-Removing-temp-and-useless-files.patchhugeProj [master] $6) Apply the patch now.Command:git apply <patch_name> git apply --verbose <patch_name>--Files mytest.mine and runtest.t will be deleted. hugeProj [master] $git apply --verbose 0001-Removing-temp-and-useless-files.patch Checking patch mytest.mine... Checking patch runtest.t... Applied patch mytest.mine cleanly. Applied patch runtest.t cleanly.Use the --verbose flag to see the progress report of the patch being applied.To get a summary of the patch run: ‘git apply --summary <patch_id>myProj [master] $git apply --summary 0001-Add-the-Linux-iso-image.patch create mode 100644 .gitattributes create mode 100644 CentOS-7-x86_64-Minimal-1804.iso Divya1@Divya:myProj [master] $git apply --summary 0001-Removing-temp-and-useless-files.patch delete mode 100644 mytest.mine delete mode 100644 runtest.tSUMMARY:We learnt to create, test, and apply the patch differences of a committed snapshot of your repository work. These patches are email friendly and are shared among repository users.
logo

GIT Tutorial

Git Filesystem

1. Git LFS

What is LFS

Storing Large content files eventually consumes a huge amount of space on the Git Server.

Moreover, since Git is a Distributed Version Control System, every clone and every checkout or pull of this repository will have to download each version of this huge file present on the Server; unlike Centralized Systems wherein just the latest version is downloaded.

Git LFS server comes to our rescue in terms of storing huge files.

Let’s see how?

Git LFS is a system for managing and versioning large files of upto 2GM in association with a Git repository.  Instead of storing the large files within the Git repository as blobs, Git LFS stores special "pointer files" in the repository, while storing the actual file contents on a Git LFS server.  The contents of the large file are downloaded automatically when needed, for example when a Git branch containing

the large file is checked out.

Referenced: Git LFS manual page.

Summary:

Git LFS handles large files by storing references to the file in the repository, but not the actual file itself. 

Distribution of normal files and LFS files over the remote and LFS server

Diagram: Distribution of normal files and LFS files over the remote and LFS server

git checkout of LFS files.

Diagram: git checkout of LFS files.after a ‘git checkout’

Diagram: after a ‘git checkout’

How to install git LFS

Step 1: Setup git LFS on your system. Or Just for a repository.

git lfs install

hugeProj [master] $git lfs install
Updated git hooks.
Git LFS initialized.

Step 2: Choose the type of files you want to track as LFS files.

git lfs track “*.iso”

 hugeProj [master] $git lfs track "*.iso" "*.tar"
Tracking "*.iso"
Tracking "*.tar"

--List the tracked files:
hugeProj [master] $git lfs track
Listing tracked patterns
   *.iso (.gitattributes)
   *.tar (.gitattributes)
Listing excluded patterns

Step 3: Tracking information is stored in .gitattributes file, add this file and rest of the files to the repository

git add .gitattributes

Untracked files:
 (use "git add <file>..." to include in what will be committed)

.gitattributes
CentOS-7-x86_64-Minimal-1804.iso
code/

Step 4: Commit, push the files normally to the Git server.

hugeProj [master] $git add .
hugeProj [master] $git commit -m 'Add LFS objects'
hugeProj [master] $git push origin master
Uploading LFS objects: 100% (1/1), 950 MB | 0 B/s, done                                                        
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 4 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 875 bytes | 875.00 KiB/s, done.
Total 6 (delta 0), reused 0 (delta 0)
To github.com:divyabhushan/hugeProject.git
   2beb247..3f8184b  master -> master

Step 5: Verify the files on the Git server and local repository.

FLS files stored on the LFS Server and not on the Git Server.

Snapshot from GitHub Server shows an LFS file is available only in the RAW form; as it is stored on a LFS Server.

Verify the files on the Git server and local repository.

Git LFS file

hugeProj [master] $ls -lt
total 1855496
-rw-r--r--  1 Divya1 staff         13 Jan 28 12:23 README
drwxr-xr-x  3 Divya1 staff        102 Jan 28 12:22 code
-rw-r--r--@ 1 Divya1  staff 950009856 Jan 28 12:20 CentOS-7-x86_64-Minimal-1804.iso

NOTE: LFS files are denoted with an ‘@’ attribute on the client machine

-@      Display extended attribute keys and sizes in long (-l) output.

Step 6: Clone this repository as a different local repository.

Regular files are pulled/downloaded from Git Server and LFS files from the LFS Server.

git clone http://github.com/divyabhushan/hugeProject.git largeProj 

Uploading LFS objects: 100% (1/1), 950 MB | 0 B/s, done

git lfs commands:

git lfs install:

Install Git LFS configuration.

git lfs version:

Report the version number.

git lfs env:

Display the Git LFS environment.

git lfs ls-files:

Show information about Git LFS files in the index and working tree.

hugeProj [master] $git lfs ls-files
714acc0aef * CentOS-7-x86_64-Minimal-1804.iso

git lfs track “*.ext”:
View or add Git LFS paths to Git attributes.

git lfs untrack “*.ext”

Remove Git LFS paths from Git Attributes.

git lfs status

Show the status of Git LFS files in the working tree.

git lfs push:
Push queued large files to the Git LFS endpoint.

git lfs update:

Update Git hooks for the current Git repository.

--Status after ‘git add’
myProj [initialBranch] $git lfs status
On branch initialBranch
Git LFS objects to be pushed to origin/initialBranch:

Git LFS objects to be committed:

          CentOS-7-x86_64-Minimal-1804.iso

Git LFS objects not staged for commit:

CONCLUSION:

Data is increasing tremendously over the internet; with IoT(“Internet Of Things”) technology booming.

There will always be a need of a good Version system like “Git” and the filesystem like “LFS” integrated with Git to store, process and analyze the ever-growing data.

We learnt:

What is LFS how do we store are large files in the LFS cloud and checkout them to our local repository.

2. Git patch operation

Git format-patch - Prepare patches for e-mail submission

Prepare each commit with its patch in one file per commit, formatted to resemble UNIX mailbox format. The output of this command is convenient for e-mail submission

Command:

git format-patch

1) Create a patch from the commit id (HEAD)

myProj [master] $git format-patch -1
0001-Add-the-Linux-iso-image.patch

2) Create a patch from an older selected commit id:

myProj [master] $git format-patch -1 367b4ed
0001-Removing-temp-and-useless-files.patch

3) Read the patch in a diff format:

[master] $cat 0001-Add-the-Linux-iso-image.patch 
From 310ddc5504a0533ab5612d7387a97b08450851d9 Mon Sep 17 00:00:00 2001
From: divya bhushan <divya_bhushan@hotmail.com>
Date: Mon, 28 Jan 2019 12:09:12 +0100
Subject: [PATCH] Add the Linux iso image.
---
 .gitattributes                   | 2 ++
 CentOS-7-x86_64-Minimal-1804.iso | 3 +++
 2 files changed, 5 insertions(+)
 create mode 100644 .gitattributes
 create mode 100644 CentOS-7-x86_64-Minimal-1804.iso

diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 0000000..539acb8
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,2 @@
+*.iso filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
diff --git a/CentOS-7-x86_64-Minimal-1804.iso b/CentOS-7-x86_64-Minimal-1804.iso
new file mode 100644
index 0000000..9598fdb
--- /dev/null
+++ b/CentOS-7-x86_64-Minimal-1804.iso
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:714acc0aefb32b7d51b515e25546835e55a90da9fb00417fbee2d03a62801efd
+size 950009856
-- 
2.19.0

4) To list the Statistics of the patch but do not apply it, use the command: 

git apply --stat <patch_name>

myProj [master] $git apply --stat 0001-Add-the-Linux-iso-image.patch
.gitattributes                   | 2 ++
CentOS-7-x86_64-Minimal-1804.iso |    3 +++
2 files changed, 5 insertions(+)

5) Check for errors before applying the patch; with the command:

git apply --check <patch_name>

Let’s say you received the patch via email; and you would want to apply it in your present project

cd hugeProject
--copy the patch here.
hugeProj [master] $cp ../myProj/0001-Add-the-Linux-iso-image.patch

hugeProj [master] $git apply --check 0001-Add-the-Linux-iso-image.patch
error: .gitattributes: already exists in working directory
error: CentOS-7-x86_64-Minimal-1804.iso: already exists in working directory

The patch won’t apply here as the files are already present in the project

Let’s test another patch: 0001-Removing-temp-and-useless-files.patch

--Read the patch content and the stats
hugeProj [master] $git apply --stat 0001-Removing-temp-and-useless-files.patch
mytest.mine |    2 --
runtest.t   | 0
2 files changed, 2 deletions(-)
--copy the 2 files from the 367b4ed~1 commit id of ‘myProj’ repository here
myProj [master] $checkout 367b4ed~1
myProj [(HEAD detached at d1cd657)] $cp runtest.t ../hugeProj/
myProj [(HEAD detached at d1cd657)] $cp mytest.mine ../hugeProj/
--Test if the patch would apply successfully!
--No output means a success
cd ../hugeProj
hugeProj [master] $git apply --check 0001-Removing-temp-and-useless-files.patch

hugeProj [master] $

6) Apply the patch now.

Command:

git apply <patch_name>
git apply --verbose <patch_name>

--Files mytest.mine and runtest.t will be deleted.
hugeProj [master] $git apply --verbose 0001-Removing-temp-and-useless-files.patch
Checking patch mytest.mine...
Checking patch runtest.t...
Applied patch mytest.mine cleanly.
Applied patch runtest.t cleanly.

Use the --verbose flag to see the progress report of the patch being applied.

To get a summary of the patch run: ‘git apply --summary <patch_id>

myProj [master] $git apply --summary 0001-Add-the-Linux-iso-image.patch
create mode 100644 .gitattributes
create mode 100644 CentOS-7-x86_64-Minimal-1804.iso

Divya1@Divya:myProj [master] $git apply --summary 0001-Removing-temp-and-useless-files.patch
delete mode 100644 mytest.mine
delete mode 100644 runtest.t

SUMMARY:

We learnt to create, test, and apply the patch differences of a committed snapshot of your repository work. These patches are email friendly and are shared among repository users.

Leave a Reply

Your email address will not be published. Required fields are marked *

Comments

stephen

Firstly thanks for providing this tutorial from this article I have gathered lots of information. I can say that whatever the information provided by knowledgeHut is more useful to know more about the git.

Saurabh

Written nicely