This is a mirror of the wiki at pascal.chat. Editing, history and sign-in are there.

Git Conversion Lazarus

This page has the detailed steps used to convert the Lazarus SVN repro to git.

SubGIT

The initial conversion will be done using TMate Subgit.

Install

Unpack subgit to

text

   ~/subgit/subgit-3.3.11

Required

An author.txt file

text

 svn-name - git-name <email>

Run / main repro

text

   cd ~/subgit/subgit-3.3.11
   ./bin/subgit configure https://svn.freepascal.org/svn/lazarus laz.git
   
   gedit laz.git/subgit/config

And add the following (the first line, replaces an existing setting)

text

   [svn]
     trunk = trunk:refs/heads/main
     gitCommitMessage = %message\\n\\ngit-svn-id: %branch@%revision -\\n
     
   [translate]  
     eols = false
     otherProperties = false
     ignores = false

The translate options prevent .gitignore and .gitattributes. Those can be translated, but will later be replaced...

text

   cp authors.txt laz.git/subgit/authors.txt

text

   ./bin/subgit install laz.git

Without skipping the gitignore/gitattributes, there will be an error at revision 47330. The process will hang. nterrupt it and edit

text

   gedit lazarus.git/svn/.metadata

and set branches-maxRev and tags-maxRev to 47332 (it should be at 47330) https://stackoverflow.com/questions/18918215/skip-revisions-when-installing-subgit-repository Continue with

text

   ./bin/subgit install laz-bin.git

Stop the daemon

text

   ./bin/subgit shutdown laz-bin.git/

Run / binaries repro

text

   ./bin/subgit configure https://svn.freepascal.org/svn/lazarus  laz-bin.git

text

   gedit laz-bin.git/subgit/config 
     [svn]
     gitCommitMessage = %message\\n\\ngit-svn-id: %branch@%revision -\\n
     trunk = binaries:refs/heads/main

text

     cp authors.txt laz-bin.git/subgit/authors.txt

follow the steps above

Run / website repro

text

   ./bin/subgit configure https://svn.freepascal.org/svn/lazaruswebsite laz-web.git

text

   gedit laz-web.git/subgit/config 
     [svn]
     gitCommitMessage = %message\\n\\ngit-svn-id: %branch@%revision -\\n
     trunk = trunk:refs/heads/main

text

     cp authors.txt laz-web.git/subgit/authors.txt

follow the steps above

Part 2

Updating gitignore
Prefixing all issue references with #
translating svn revisions to git hashes

Set a git user

text

   git config --local user.email "lazarus@lazarus-ide.org"
   git config --local user.name "lazarus"

Clean out left overs from subgit

text

   git for-each-ref --format="%(refname)" refs/svn/ | xargs -n 1 git update-ref -d

Clean left overs from binaries

There are a few tags, that lead to binaries (gdb.exe) ending up in the data

text

  git tag -d binaries_0_9_26
  git tag -d binaries_0_9_26_2
  git tag -d binaries_0_9_28
  git tag -d binaries_0_9_30
  git tag -d binaries_0_9_30_2
  git tag -d binaries_0_9_30_2RC1
  git tag -d binaries_1_0

Updating gitignore/gitattributes / Prefix issues with #

The below combines the 2 steps. It can be run in 2 separate calls.

It will run for about one hour.

text

   FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch --index-filter '  cp ~/subgit/subgit-3.3.11/.gitignore  ~/subgit/subgit-3.3.11/.gitattributes . ; git add .gitignore .gitattributes '  --msg-filter ' perl -ne "while (s/((?:iss?ues?|[ (](?:bugs?|mantis|fix|fixes|patch)\b)(?: +id)?[ :]+(?:#?0*[1-9][0-9]{2,4}(?: *(?:[ ,]| and ) *))*)(0*[1-9][0-9]{2,4})\b/\1#\2/i) {}; print" ' -f  -- --all

text

   FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch  --msg-filter ' perl -ne "while (s/((?:iss?ues?|[ (](?:bugs?|mantis|fix|fixes|patch)\b)(?: +id)?[ :]+(?:#?0*[1-9][0-9]{2,4}(?: *(?:[ ,]| and ) *))*)(0*[1-9][0-9]{2,4})\b/\1#\2/i) {}; print" ' -f  -- --all

Cleanup

text

   git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
   git reflog expire --expire=now --all ;git gc --prune=now

Translating references to svn rev (r12345) to #hash

This must be the very last filter-branch. As each filter-branch creates new hashes for the commits, and that would invalidate the hash refs created here.

The first batch will update only commits it branch main.

This will also only be references to other commits in main.

Merges from branches to main can not be updated.

The 2nd batch will update only commits in branches. This means the hashes for the main branch will be left intact.

This will update merges from main to fixes

To verify the 2nd batch does not update the main branch compare the hash for git log -n1 main. It should not be changed by the 2nd batch.

text

   rm -rf /tmp/rmap
   mkdir /tmp/rmap
   git rev-list  main | xargs -L1 /bin/sh -c 'a=$( git log -n1 --pretty='%B' $0 | grep "git-svn-id: .*@[0-9][0-9]* "  | sed "s/.*@\([0-9]*\) .*/\1/" ) ; echo $0 > /tmp/rmap/$a'
   git rev-list  main | xargs -L1 /bin/sh -c 'echo 1 > /tmp/rmap/m_$0'

text

   FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch --msg-filter 'perl -wne "use strict; 
   unless ( -e qq#/tmp/rmap/m_\$ENV{GIT_COMMIT}# ) { print;  next; }
   my %f;  foreach(/(?:(?:commit|rev|revisions?|revision\(s\)|revert|reverted) +| r)([1-9][0-9]{3,4}\b(?:[\/,-] *[1-9][0-9]{3,4}\b)*)\b/gi) {
     my \$rvl=\$1;    
     foreach my \$a (split / *[\/,] */, \$rvl) {
     unless (\$f{\$a}){
    \$f{\$a}=q/#/;
       my(\$a1, \$a2) = split / *- */, \$a;
    my \$c; my \$rh; if( open \$rh, q/</, qq#/tmp/rmap/\$a1#) { \$c=<\$rh>;   chomp \$c; }
    my \$h; if(\$c && (open \$h, q/</, qq#../map/\$c#)) {     
       \$c=<\$h>;   chomp \$c;  \$c = q/#/.substr \$c, 0,10; 
         if (\$a2) {
        my \$c2; my \$rh2; if( open \$rh2, q/</, qq#/tmp/rmap/\$a1#) { \$c2=<\$rh2>;   chomp \$c2; }
        my \$h; if(\$c2 && (open \$h, q/</, qq#../map/\$c2#)) {       \$c2=<\$h>;   chomp \$c2;        
           \$c = \$c.q/-#/.substr( \$c2, 0,10); 
        }
        else { \$c = \$c.q/..# / unless \$c eq q/#/;  warn q/MAP2 FAIL /.\$c2 }
      }
      \$f{\$a}=\$c;
    }
    else {warn q/MAP1 fail/.\$a;}
     }} 
   }  
   foreach my \$k (keys %f) {   my \$v = \$f{\$k};
     unless (\$v eq q/#/) {     s/([ r,\/]\${k}\b)/\$1 \$v/g;   }
   }   
   warn \$_ if keys %f;
   print; "
   ' -f -- --all

text

   git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d

text

   rm -rf /tmp/rmap
   mkdir /tmp/rmap
   git rev-list  --all | xargs -L1 /bin/sh -c 'a=$( git log -n1 --pretty='%B' $0 | grep "git-svn-id: .*@[0-9][0-9]* "  | sed "s/.*@\([0-9]*\) .*/\1/" ) ; echo $0 > /tmp/rmap/$a'
   git rev-list  main | xargs -L1 /bin/sh -c 'echo 1 > /tmp/rmap/m_$0'

text

   FILTER_BRANCH_SQUELCH_WARNING=1 git filter-branch --msg-filter 'perl -wne "use strict; 
   if ( -e qq#/tmp/rmap/m_\$ENV{GIT_COMMIT}# ) { print;  next; }
   my %f;  foreach(/(?:(?:commit|rev|revisions?|revision\(s\)|revert|reverted) +| r)([1-9][0-9]{3,4}\b(?:[\/,-] *[1-9][0-9]{3,4}\b)*)\b/gi) {
     my \$rvl=\$1;    
     foreach my \$a (split / *[\/,] */, \$rvl) {
     unless (\$f{\$a}){
    \$f{\$a}=q/#/;
       my(\$a1, \$a2) = split / *- */, \$a;
    my \$c; my \$rh; if( open \$rh, q/</, qq#/tmp/rmap/\$a1#) { \$c=<\$rh>;   chomp \$c; }
       if (\$c) {   unless ( -e qq#/tmp/rmap/m_\$c# ) { 
          my \$h; if(open \$h, q/</, qq#../map/\$c#) {   \$c=<\$h>;   chomp \$c; }
    }}
    if(\$c ) {
       \$c = q/#/.substr \$c, 0,10; 
         if (\$a2) {
        my \$c2; my \$rh2; if( open \$rh2, q/</, qq#/tmp/rmap/\$a1#) { \$c2=<\$rh2>;   chomp \$c2; }
           if (\$c2) {  unless ( -e qq#/tmp/rmap/m_\$c2# ) { 
              my \$h; if(open \$h, q/</, qq#../map/\$c2#) {   \$c2=<\$h>;   chomp \$c2; }
        }}
        if(\$c2) {   \$c = \$c.q/-#/.substr( \$c2, 0,10);     }
        else { \$c = \$c.q/..# / unless \$c eq q/#/;  warn q/MAP2 FAIL /.\$c2 }
      }
      \$f{\$a}=\$c;
    }
    else {warn q/MAP1 fail/.\$a;}
     }} 
   }  
   foreach my \$k (keys %f) {   my \$v = \$f{\$k};
     unless (\$v eq q/#/) {     s/([ r,\/]\${k}\b)/\$1 \$v/g;   }
   }   
   warn \$_ if keys %f;
   print; "
   ' -f -- --all

text

   git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
   git reflog expire --expire=now --all ;git gc --prune=now
   rm -rf /tmp/rmap

Updating tags

This must be done after all filter-branches

Annotate all tags.
Create some more tags

This needs the git user being set

text

   git for-each-ref --format="%(refname:short)" refs/tags/ | xargs -L1 /bin/sh -c 'git tag -f -a $0 -m $0 $0 '

text

   git rev-list --in-commit-order  --reverse   fixes_2_2..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-2_3" -m "main to version 2.3" $0'
   git rev-list --in-commit-order  --reverse   fixes_2_0..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-2_1" -m "main to version 2.1" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_8..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-1_9" -m "main to version 1.9" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_6..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-1_7" -m "main to version 1.7" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_4..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-1_5" -m "main to version 1.5" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_2..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-1_3" -m "main to version 1.3" $0'
   git rev-list --in-commit-order  --reverse   fixes_1_0..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-1_1" -m "main to version 1.1" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_30..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-0_9_31" -m "main to version 0.9.31" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_28..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-0_9_29" -m "main to version 0.9.29" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_26..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-0_9_27" -m "main to version 0.9.27" $0'
   git rev-list --in-commit-order  --reverse   fixes_0_9_24..main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-0_9_25" -m "main to version 0.9.25" $0'
   git rev-list --in-commit-order  --reverse   main | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "main-0_0" -m "main init" $0'
   
   git rev-list --in-commit-order  --reverse   main..fixes_2_2 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-2_2" -m "fixes 2.2 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_2_0 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-2_0" -m "fixes 2.0 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_1_8 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-1_8" -m "fixes 1.8 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_1_6 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-1_6" -m "fixes 1.6 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_1_4 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-1_4" -m "fixes 1.4 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_1_2 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-1_2" -m "fixes 1.2 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_1_0 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-1_0" -m "fixes 1.0 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_0_9_30 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-0_9_30" -m "fixes 0.9.30 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_0_9_28 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-0_9_28" -m "fixes 0.9.28 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_0_9_26 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-0_9_26" -m "fixes 0.9.26 branch" $0'
   git rev-list --in-commit-order  --reverse   main..fixes_0_9_24 | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-fixes-0_9_24" -m "fixes 0.9.24 branch" $0'
   
   git rev-list --in-commit-order  --reverse   main..free-sparta | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-free-sparta" -m "free-sparta branch" $0'
   git rev-list --in-commit-order  --reverse   main..gtk-splitup | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-gtk-splitup" -m "gtk-splitup branch" $0'
   git rev-list --in-commit-order  --reverse   main..lcl-smartlink | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-lcl-smartlink" -m "lcl-smartlink branch" $0'
   git rev-list --in-commit-order  --reverse   main..listviewsortindicator | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-listviewsortindicator" -m "listviewsortindicator branch" $0'
   git rev-list --in-commit-order  --reverse   main..macosfullkeyboardaccess | head -n 1 | xargs -L1 /bin/sh -c 'git tag -a "t-macosfullkeyboardaccess" -m "macosfullkeyboardaccess branch" $0'

Finish

edit HEAD to point to main

text

   gedit HEAD

text

   git remote add origin https://gitlab.com/freepascal.org/lazarus-ide/lazarus_test_conversion_X.git

text

   git push --all origin
   git push --tags origin

On gitlab

Settings / Repository
- set default gitlab branch
  - no autoclose issues on default branch // decide later
- set "reject unverified users"
- protect branches * // no force push
- protect tags: fixes* lazarus* release* main* // maintainers only
Settings / Integrations
- External wiki
Settings / General
- Disable Wiki, Snippets, Papges, Requirements, Operations, Service Desk, Container, Packages
  For now also CI, until we use it
- Merge request approvals: Enable "Prevent users from modifying MR approval rules"
- set the avatar
- Add "description"
- Make public
  - Disable "Users can request access"

Comparing data before/after filter-branch

With subject line / Will show changes in subject

text

   git log --pretty='format:%ae %an %cd  %s %d' --graph --all > ../log.txt

Without subject line / Should have no diff

This compares the graph and committers => Thus it will show if all commits are present, and if all branches are correct

text

   git log --pretty='format:%ae %an %cd %d' --graph --all > ../log.txt

compare the changes to files (gitignore / gitattributes will diff for the top commit(s)) / --graph can be added too

(slow)

text

   git log --all --pretty='%ae %an %cd %d' --stat > ../statlog.txt

Compare the overall amount of lines in the log.

text

   git log --all | wc -l

Make sure you do that after "clean up" in each step. Otherwise the internal subgit or filter-branch revs will go into the compare.

Also "removing the binaries" introduces a diff.

Data

Please mail me for any changes. Do not make changes here. Otherwise the changes will not make it into the conversion.

.gitignore

text

   **/*.o
   **/*.ppu
   **/*.exe
   **/*.bak
   **/*.diff
   **/*.patch
   **/units/
   **/lib/
   **/*;[1-9]
   **/*.~*
   **/*.*~
   **/*.fpcunit.ini
   **/*.lps
   **/*.compiled
   /docs/chm/**/*.chm
   /docs/chm/**/*.xct
   /docs/chm/**/*.kwd
   /docs/chm/**/*.txt
   /libeay32.dll
   /ssleay32.dll
   /lazarus.cfg
   ide/revision.inc
   
   !**/patches/*.patch
   !**/patches/*.diff

.gitattributes

text

   * text=auto !eol
   **/*.pp text
   **/*.pas text
   **/*.inc text
   **/*.lfm text
   **/*.lrs text
   **/*.lps text
   **/*.lpr text
   **/*.lpi text
   **/*.lpk text
   **/*.lpl text
   **/*.lrj text
   **/*.dpr text
   **/*.dfm text
   **/*.pot text
   **/*.po text
   **/*.rc text
   **/*.txt text
   **/*.txt.sample text
   **/*.ini text
   **/*.cfg text
   **/*.iss text
   **/*.isl text
   **/*.xml text
   **/*.xsl text
   **/*.html text
   **/*.css text
   **/*.php text
   **/*.js text
   **/.htaccess text
   **/*.patch text
   **/*.md text
   **/*.bat text
   **/*.sh text
   **/*.compiled text
   **/*.bdsproj text
   **/*.plist text
   **/*.h text
   **/*.c text
   **/*.cpp text
   **/*.in text
   **/*.xpm text
   **/*.svg text
   **/Makefile text
   **/Makefile.compiled text
   **/Makefile.fpc text
   **/*.exe -text
   **/*.dll -text
   **/*.so -text
   **/*.gif -text
   **/*.jpg -text
   **/*.ico -text
   **/*.bmp -text
   **/*.res -text
   **/*.ttf -text
   **/*.cur -text
   **/*.db3 -text
   **/*.dbf -text
   **/*.mbf -text
   **/*.mdx -text
   **/*.fbk -text
   **/*.odt -text
   **/*.ods -text
   **/*.odg -text
   **/*.odp -text
   **/*.xls -text
   **/*.pdf -text
   **/*.zip -text
   **/*.rar -text
   **/*.tar -text
   **/*.tgz -text
   **/*.gz -text
   **/*.icns -text
   **/*.dcr -text
   components/aggpas/**/*.ppm -text
   **/*.app/Contents/MacOS/* -text

From the Free Pascal wiki, page "Git Conversion Lazarus", as it stood on 2021-07-28 after a change by Martin, out of the wiki's dump of 2024-05-26. Copyright stays with the author who wrote it, according to the old wiki's Licensing page. If you have concerns about this material, message the admin.

Revision 3, last changed by WikiArchive 11-6-2026.