Using pbuilder and sbuild and debuild to build debian package (by quqi99)

我不是女神ヾ 2022-09-05 14:58 32阅读 0赞

**作者：张华 发表于：2021-08-18  
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明**

## 问题 ##

在ussuri (16.3.0)里有一个bug, 但在steain(15.3.4)里没有，需要二分，先通过’git tag’找到其中的16.0.0, 16.1.0, 16.3.0三个tag

15.3.4
    16.0.0 https://launchpad.net/ubuntu/+source/neutron/2:16.0.0-0ubuntu2
    16.0.0.0b1
    16.0.0.0rc1
    16.0.0.0rc2
    16.1.0 https://launchpad.net/ubuntu/+source/neutron/2:16.1.0-0ubuntu2
    16.2.0 https://launchpad.net/ubuntu/+source/neutron/2:16.2.0-0ubuntu2
    16.3.0

前一篇关于debian的文章是：　https://blog.csdn.net/quqi99/article/details/50511173

一些感受:

*  在bionic中运行git-buildpackage的pristine-tar checkout这步会报一个XD3\_INVALID\_INPUT错误，在focal上无此问题
 *  在focal上运行git-buildpackage + git-pbuiler(cowpbuilder)时会报一个dbus相关的错误。运行git-buildpackage + sbuild或git-buildpackage + debuild都无此问题。直接运行pdebuild也无此问题(git-pbuilder用的是cowpbuilder是cow格式) .git-pbuilder创建出来的镜像里没有dbus包和环境变量（DBUS\_SESSION\_BUS\_ADDRESS=unix:path=/run/user/1000/bus），暂不清楚是否和这相关. 后来，使用git-pubilder时又提示输入什么密码。看来git-pbuilder确实有问题不能用.
 *  git-buildpackage从git源码编译时都要用到tarball，但tarball有通过prinstine-tar命令人工准备和它从neutron git源码中自动tarball两种方式
 *  使用git-buildpackage + sbuild即可
 *  源码包看debian/changelog就能确实是适合在bionic上还是bionic-steain上还是focal上编译，这样能避免包依赖问题

## 准备测试机器 ##

juju deploy ubuntu debbuild --series=bionic --config hostname=debbuild --constraints "mem=8G cores=2 root-disk=80G"
    #  juju add-machine --series bionic
    sudo cp /etc/apt/sources.list /etc/apt/sources.list_bak
    cat <<EOF | sudo tee /etc/apt/sources.list
    deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
    deb http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse
    deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
    #deb http://archive.ubuntu.com/ubuntu bionic-proposed main restricted universe multiverse
    #deb http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse
    deb-src http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
    deb-src http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse
    deb-src http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
    #deb-src http://archive.ubuntu.com/ubuntu bionic-proposed main restricted universe multiverse
    #deb-src http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse
    EOF
    sudo apt clean && sudo apt update

## 使用pbuilder ##

会报一些包依赖的错（openstack-pkg-tools (>= 85ubuntu3~) ），见最后，后来证明和pbuilder无关，是该dsc文件就不是针对bionic的所以一堆问题。

sudo apt install pbuilder debootstrap devscripts -y
    sudo apt install gnupg ubuntu-dev-tools bzr-builddeb apt-file debhelper dh-systemd openstack-pkg-tools -y
    sudo apt install build-essential quilt dh-autoreconf fakeroot dpkg-dev python-sphinx dh-make -y
    #sudo apt install apt-cacher-ng -y
    #echo 'Acquire::http::Proxy "http://127.0.0.1:3142";' | sudo tee /etc/apt/apt.conf.d/01acng
    # sometimes base.tgz can't be generated when using pbuilderrc
    sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
    sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/bionic-train-base.tgz
    # need to use '--basetgz' when using no pbuilderrc
    sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
    #chroot to image to ajust what you want
    sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
      echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
      apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
      add-apt-repository cloud-archive:train
      #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
      #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
      sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
      apt update
    # then exit chroot, press ctrl-d to save
    
    # but pdebuild needs pbuilderrc
    cat <<EOF | sudo tee /etc/pbuilderrc
    MIRRORSITE=http://archive.ubuntu.com/ubuntu/
    DEBFULLNAME='Zhang Hua'
    DEBEMAIL='xxx@xxx.com'
    DISTRIBUTION='bionic'
    EXTRAPACKAGES=''
    #EXTRAPACKAGES+=' vim sudo bash-completion wget git build-essential apt-transport-https ca-certificates ubuntu-cloud-keyring'
    # Cloud Archive
    #OS_RELEASE='train'
    if [ -n "\$OS_RELEASE" ]; then
        BASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-\$OS_RELEASE-base.tgz"
    else
        BASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-base.tgz"
    fi
    EOF
    mkdir -p /bak/work/debian/neutron/16.1.0 && cd /bak/work/debian/neutron/16.1.0
    dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.1.0-0ubuntu2/neutron_16.1.0-0ubuntu2.dsc
    dpkg-source -x neutron_16.1.0-0ubuntu2.dsc
    cd /bak/work/debian/neutron/16.1.0/neutron-16.1.0
    # install dependency, use mk-build-deps instead of 'apt build-dep neutron'
    add-apt-repository cloud-archive:train  #also add uca
    sudo apt install devscripts equivs -y
    sudo mk-build-deps --install debian/control
    # dpkg-source: error: aborting due to unexpected upstream changes
    rm -rf debian/source/fomat
    sudo pdebuild --debug
    #sudo pdebuild --debbuildopts -sa #when hitting 'Unable to find xxx.orig.tar.gz in upload or distribution'
    #DEB_BUILD_OPTIONS=nocheck debuild -us -uc

报这个错：

ubuntu@debbuild:/bak/work/debian/neutron/16.1.0/neutron-16.1.0$ sudo pdebuild 
    W: /home/ubuntu/.pbuilderrc does not exist
    dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 12) openstack-pkg-tools (>= 85ubuntu3~) python3-pbr (>= 4.0.0)
    W: Unmet build-dependency in source
    dpkg-source: info: using options from neutron-16.1.0/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/
    pyversions: missing X(S)-Python-Version in control file, fall back to debian/pyversions
    pyversions: missing debian/pyversions file, fall back to supported versions
    py3versions: no X-Python3-Version in control file, using supported versions
    debian/rules:42: warning: overriding recipe for target 'override_dh_installinit'
    /usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit'
    dh clean  --with python3 --buildsystem=pybuild
    dh: Compatibility levels before 5 are no longer supported (level 1 requested)
    debian/rules:17: recipe for target 'clean' failed
    make: *** [clean] Error 25

原因是运行pbuilder的机器上也得添加train uca并安装(pbuilder镜像里也记得添加train uca哦)，也别记了将pbuilderrc中的RELEASE放开来构建正确的BASETGZ变量：

apt install debhelper-compat openstack-pkg-tools python3-pbr -y

但是接着会报下列错，这是debian包的问题，因为我们编译的neutron-16.1.0是位于train和ussuri之间的版本，并不是严格的train (train中的debain/control中的确实是python3-neutron-lib (>= 1.29.1),)

pbuilder-satisfydepends-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installed
                                     Depends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installed
                                     Depends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed
     python3-cryptography : Depends: libssl1.1 (>= 1.1.1) but 1.1.0g-2ubuntu4 is installed

后来又这样：

sudo mv /etc/pbuilderrc /tmp/
    sudo pbuilder --create --distribution focal --architecture amd64 --debootstrapopts --variant=buildd
    sudo mv /tmp/pbuilderrc /etc/
    sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/focal-base.tgz
    sudo pbuilder --login --basetgz /var/cache/pbuilder/focal-base.tgz --save-after-login
    #/etc/pbuilderrc will point out 'BASETGZ='
    gbp buildpackage --git-pbuilder --git-dist=focal --git-arch=amd64 --git-ignore-branch --git-ignore-new -uc -us -j8

结果又提示输出密码：

[sudo] password for pbuilder:

看来gbp与pbuilder结合这一块确实有问题

## 使用sbuild ##

最后遇到错误”Not removing build depends: cloned chroot in use"

#using sbuild - https://wiki.ubuntu.com/SecurityTeam/BuildEnvironment
    sudo apt install sbuild debhelper ubuntu-dev-tools piuparts moreutils schroot gnupg-agent -y
    sudo adduser $USER sbuild
    mkdir -p $HOME/ubuntu/scratch
    cat <<EOF | sudo tee -a  /etc/schroot/sbuild/fstab
    /home/$USER/ubuntu/scratch  /scratch          none  rw,bind  0  0
    EOF
    cat <<EOF | tee ~/.sbuildrc
    # Name to use as override in .changes files for the Maintainer: field
    # (mandatory, no default!).
    \$maintainer_name='Zhang Hua <xxx.zhang@xxx.com>';
    # Default distribution to build.
    \$distribution = "bionic";
    # Build arch-all by default.
    \$build_arch_all = 0;
    
    # When to purge the build directory afterwards; possible values are "never",
    # "successful", and "always".  "always" is the default. It can be helpful
    # to preserve failing builds for debugging purposes.  Switch these comments
    # if you want to preserve even successful builds, and then use
    # "schroot -e --all-sessions" to clean them up manually.
    \$purge_build_directory = 'successful';
    \$purge_session = 'successful';
    \$purge_build_deps = 'successful';
    # $purge_build_directory = 'never';
    # $purge_session = 'never';
    # $purge_build_deps = 'never';
    
    # Directory for writing build logs to
    \$log_dir=\$ENV{HOME}."/ubuntu/logs";
    
    # don't remove this, Perl needs it:
    1;
    EOF
    mkdir -p $HOME/ubuntu/{build,logs}
    cat <<EOF | tee ~/.mk-sbuild.rc
    SCHROOT_CONF_SUFFIX="source-root-users=root,sbuild,admin
    source-root-groups=root,sbuild,admin
    preserve-environment=true"
    SKIP_UPDATES="1"
    SKIP_PROPOSED="1"
    EOF
    #mk-sbuild bionic --arch=amd64 --skip-updates --debootstrap-mirror=http://<mirror>/ubuntu --distro=ubuntu
    # sg sbuild  #do not need this line if you reenter ssh
    mk-sbuild bionic --arch=amd64 --skip-updates
    
    # install uca
    schroot -l
    schroot -c source:bionic-amd64 -u root
      echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
      apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
      add-apt-repository cloud-archive:train
      #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
      #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
      apt update
      chmod 1777 /tmp && apt update
    exit
    
     To CHANGE the golden image: sudo schroot -c source:bionic-amd64 -u root
     To ENTER an image snapshot: schroot -c bionic-amd64
     To BUILD within a snapshot: sbuild -A -d bionic-amd64 PACKAGE*.dsc
     To BUILD for : sbuild -A -d bionic-amd64 --host  PACKAGE*.dsc
    
    mkdir -p /bak/work/debian/neutron/16.1.0 && cd /bak/work/debian/neutron/16.1.0
    dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.1.0-0ubuntu2/neutron_16.1.0-0ubuntu2.dsc
    dpkg-source -x neutron_16.1.0-0ubuntu2.dsc
    sbuild-update -udc bionic-amd64 #it equals: apt update && apt upgrade
    cd /bak/work/debian/neutron/16.1.0/neutron-16.1.0
    sbuild --verbose -d bionic-amd64 ../neutron_*.dsc  #use .. to point out the parent directory
    #dsc: amd64 not in arch list or does not match any arch wildcards: all -- skipping
    # ../neutron_*.dsc is using arch=all so need to use --arch-all for sbuild
    sbuild --verbose -d bionic-amd64 --arch-all ../neutron_*.dsc
    # Not removing build depends: cloned chroot in use
    # E: Unable to correct problems, you have held broken packages.

镜像里添加了UCA后，报的错也和pbuilder是一样的，当然，这种错误和publider和sbuild本身都无关,所以sbuild也运行正确。

sbuild-build-depends-neutron-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installed
                                          Depends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installed
                                          Depends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed

注意，要改为使用focal的话，或者修改默认值（sed -i -e “s/bionic/focal/g” ~/.sbuildrc），或者使用（’–git-builder=sbuild -As -d focal-amd64’）

gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d focal-amd64' --git-dist=focal --git-arch=amd64 -j8

## 使用debuild ##

先使用sbuild快速弄一个bionic环境出来：

sudo cp /etc/schroot/chroot.d/sbuild-bionic-amd64 /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
    sudo cp -r /var/lib/schroot/chroots/bionic-amd64 /var/lib/schroot/chroots/bionic-amd64-tmp
    sudo sed -i -e "s/bionic-amd64/bionic-amd64-tmp/g" /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
    #sudo rm /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
    #sudo rm -rf /var/lib/schroot/chroots/bionic-amd64-tmp
    schroot -l
    schroot -c source:bionic-amd64-tmp -u root

再试debuild

apt install devscripts curl ca-certificates bash-completion -y
    dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.0.0-0ubuntu2/neutron_16.0.0-0ubuntu2.dsc
    dpkg-source -x neutron_16.0.0-0ubuntu2.dsc
    cd neutron-16.0.0/
    # install dependency, use mk-build-deps instead of 'apt build-dep neutron'
    apt install devscripts equivs -y
    mk-build-deps --install debian/control
    apt install debhelper-compat openstack-pkg-tools python3-pbr -y
    # install some dependency according to the output of 'debuild -i -us -uc -S'
    apt install debhelper-compat dh-python openstack-pkg-tools python3-all python3-pbr -y
    #build unsigned source package
    debuild -i -us -uc -S

debuild的分解动作如下：

*  清理源代码树(debian/rules clean)
 *  构建源代码包(dpkg-source -b)
 *  构建程序(debian/rules build)
 *  构建二进制包(fakeroot debian/rules binary)
 *  使用 gpg 签署 .dsc 文件 使用
 *  dpkg-genchanges 和 gpg 创建并签署上传用的.changes 文件

报这个错：

dpkg-buildpackage: info: binary and diff upload (original source NOT included)

那就加-sa参数。

debuild -i -us -uc -S -sa

上面测16.0.1因为源码有问题不典型，让我们改测train(15.3.4)来只证明debuild可以即可(注意：下列参数-S也改成了-b):

apt-get source neutron
    mk-build-deps --install debian/control
    debuild -i -us -uc -b

## 通过自己从头构建熟悉git-buildpackage原理 ##

https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html  
git-buildpackage的原理是：

*  准备git工程. 注意：这里初始添加了README.md之后会报：will cause dpkg-source: error: aborting due to unexpected upstream changes，但初始不加，在git branch时看不到master与upstream分支，还是先添加，呆会再删除

mkdir neutron-git-buildpackage && cd neutron-git-buildpackage
    echo "# neutron-git-buildpackage" >> README.md
    git init
    git add README.md
    git commit -m "first commit"
    #git remote add origin git@github.com:zhhuabj/neutron-git-buildpackage.git
    #git push origin master

*  会将neutron source code (not debian/ directory)提交到upstream branch，故先创建它：

git checkout -b upstream

*  回到master分支(此时好像不能创建新分支，是都测试好之后再创建分支）并准备neutron source code, 'gbp import-dsc’会自动将neutron source code下载下来，并提交到upstream branch

git checkout master
    # gbp import-orig /tmp/tarball/new-upstream.tar.gz -u 15.3.4
    gbp import-dsc https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc

*  也在master分支准备debianized repository, 这时master中同时有upstream中的neutron源码和debian目录（Imported the debian/ directory into master as well as upstream’s files）

git checkout master
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    git add *
    git commit -m 'init master'

*  此时状态是：

ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git branch
    * master
      upstream
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ ls
    AUTHORS           LICENSE      api-ref     debian    neutron           releasenotes      setup.py
    CONTRIBUTING.rst  PKG-INFO     babel.cfg   devstack  neutron.egg-info  requirements.txt  test-requirements.txt
    ChangeLog         README.rst   bin         doc       playbooks         roles             tools
    HACKING.rst       TESTING.rst  bindep.txt  etc       rally-jobs        setup.cfg         tox.ini
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git checkout upstream
    Switched to branch 'upstream'
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ ls
    AUTHORS           HACKING.rst  README.rst   babel.cfg   devstack  neutron           rally-jobs        roles      test-requirements.txt
    CONTRIBUTING.rst  LICENSE      TESTING.rst  bin         doc       neutron.egg-info  releasenotes      setup.cfg  tools
    ChangeLog         PKG-INFO     api-ref      bindep.txt  etc       playbooks         requirements.txt  setup.py   tox.ini
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git tag
    debian/2%15.3.4-0ubuntu1_cloud1
    upstream/15.3.4

*  pristine-tar分支中准备tarball

git checkout -b pristine-tar
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4.orig.tar.gz
    chmod +x neutron_15.3.4.orig.tar.gz
    git add neutron_15.3.4.orig.tar.gz
    git commit -m 'neutron_15.3.4.orig.tar.gz'
    # pristine-tar: committed neutron_15.3.4.orig.tar.gz.delta to branch pristine-tar
    pristine-tar commit ./neutron_15.3.4.orig.tar.gz 15.3.4
    pristine-tar list |grep 15
    #Remember to tell git to push both the pristine-tar branch, and your tag:
    #git push --all --tags
    #once your local tarball has lost - http://manpages.ubuntu.com/manpages/trusty/man1/pristine-tar.1.html
    #pristine-tar checkout ../hello-1.0.tar.gz

或者直接从上游git中获取

# generate tarball from upstream
    cd /bak/openstack/neutron
    git checkout master
    git checkout -b 15.3.4 15.3.4
    git archive --output=../neutron_15.3.4.tar.gz --format=tar HEAD
    
    # commit tarball into this git repo
    pristine-tar commit /bak/openstack/neutron_15.3.4.orig.tar.gz 15.3.4

*  之前就可以回master分支，安装依赖并运行gbp buildpackage了，它会调用’dpkg-buildpackage -rfakeroot -us -uc -ui -i -I’。都测试完之后，就可以将master分支打tag了

git checkout master
    # also need to enable UCA
    sudo mk-build-deps --install debian/control   #install dependency
    gbp buildpackage --git-ignore-branch --git-ignore-new

总结，这个重点，即要安装UCA，也要安装依赖，git有改还得添加–git-ignore-new来忽略改动。  
但是它会报这个错：\#neutron\_15.3.4.orig.tar.gz does not match stored hash，用下列workaround就OK啦：

#neutron_15.3.4.orig.tar.gz does not match stored hash
    pristine-tar checkout ../neutron_15.3.4.orig.tar.gz
    gbp buildpackage --git-tarball-dir=/bak/work --git-ignore-branch --git-ignore-new

## 解决运行gbp import-dsc过程中遇到的问题 ##

https://wiki.debian.org/PackagingWithGit  
得将neutron source code 导入到debianized repository中去，  
1, 先看通过"gbp import-dsc"从dsc中导入neutron source code  
可以直接从neutron的上游下载source code:

https://releases.openstack.org/train/#train-neutron
    https://tarballs.openstack.org/neutron/neutron-15.3.4.tar.gz

也可以用’gbp import-dsc’从dsc中导入neutron source code

sudo apt install git-buildpackage -y
    mkdir -p /tmp/deb
    https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
    sudo chown -R $USER ~/.gnupg/
    # bakup gpg keys
    gpg --output mygpgkey_pub.gpg --armor --export $GPGKEY
    gpg --output mygpgkey_sec.gpg --armor --export-secret-key $GPGKEY
    # import gpg keys from other machine
    sudo apt-get install rng-tools gnupg-agent -y && sudo rngd -r /dev/urandom && killall -q gpg-agent && eval $(gpg-agent --daemon)
    gpg --import mygpgkey_pub.gpg
    gpg --allow-secret-key-import --import mygpgkey_sec.gpg
    gpg --list-public
    gpg --list-sig
    export GPGKEY=C7100A4CD0D8F3AE44212746E6A84FFFCF31A67F
    gpg --send-keys --keyserver keyserver.ubuntu.com $GPGKEY 
    
    # import dsc. the dsc provider signs dsc, and dsc user verifies dsc
    gpg --verify ./neutron_15.3.4-0ubuntu1~cloud1.dsc
    gpg --search-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
    gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
    # you can ignore 'There is no indication that the signature belongs to the owner'
    # as it simply means you have not established a web of trust with other GPG users
    gpg --verify ./neutron_15.3.4-0ubuntu1~cloud1.dsc
    rm -rf ./neutron_15.3.4-0ubuntu1~cloud1.dsc
    git config --global user.email "xxx.zhang@xxx.com"
    git config --global user.name "Zhang Hua"
    ＃　Then export the key to your local trustedkeys to make it trusted:
    gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -
    # but it still says: gpg: Can't check signature: No public key
    gbp import-dsc https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc

尽管已经导入key, 但上面使用’gbp import-dsc’仍然报“Can’t check signature: No public key"。是因为还要将dsc provider的公钥 export the key to your local trustedkeys to make it trusted

＃　Then export the key to your local trustedkeys to make it trusted:
    gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -

如果还是不行，可以使用下列dget命令作为workaround，dget即使没有证书也会下载文件：

sudo apt-get install debian-keyring　-y
    dget --download-only -q https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
    dpkg-source -x neutron_15.3.4-0ubuntu1~cloud1.dsc
    gpg --keyserver keyserver.ubuntu.com --recv-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
    ＃　Then export the key to your local trustedkeys to make it trusted:
    gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -

## 使用ubuntu-openstack-dev debian源码构建 ##

上面熟悉了从零构建git-buildpackage工程，实际上ubuntu-openstack-dev中已经有了这个工程了。

*  master分支应该同时有debian目录和neutron源码
 *  upstream分支只有neutron源码，没有debain目录
 *  pristine-tar分支只有tarball，并且使用pristine-tar list命令能看到，也要能运行pristine-tar checkout …/xxx.tar.gz
 *  编译时是在master分支运行
 *  暂不清楚如果编译15.3.4分支，因为15.3.4下没有debain目录，有debian目录的是这个：

ubuntu@debbuild:/bak/work/debsource/neutron$ git tag |grep debian |grep 15.3.4
    debian/2%15.3.4-0ubuntu1_cloud0
    debian/2%15.3.4-0ubuntu1_cloud1

git-buildpackage命令运行在neutron代码分支之后，会先使用’pristine-tar checkout …/xxx.orig.tar.gz’先tarball弄到neutron代码分支（tarball存在pristine-tar分支中），然后再build.

# https://wiki.ubuntu.com/OpenStack/CorePackages
    sudo apt install git-buildpackage -y
    sudo mkdir -p /bak/work/debsource && sudo chown -R $USER /bak && cd /bak/work/debsource
    git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
    cd neutron
    git checkout pristine-tar
    git checkout upstream
    git checkout master
    
    # 仅编译15.3.4, 这个tag (debian/2%15.3.4-0ubuntu1_cloud0)有debian目录，应该用它
    git checkout -b debian/2%15.3.4-0ubuntu1_cloud0 debian/2%15.3.4-0ubuntu1_cloud0
    git checkout debian/2%15.3.4-0ubuntu1_cloud0
    gbp buildpackage  --git-ignore-branch --git-ignore-new -us -uc

但是上面命令报下面这个错：

xdelta3: target window checksum mismatch: XD3_INVALID_INPUT

所以本想用下列workaround (–git-tarball-dir=/bak/openstack/tarball)来避开上面问题，但是又报不匹配的错

# generate tarball from upstream
    cd /bak/openstack/neutron
    git checkout master
    git checkout -b 15.3.4 15.3.4
    git archive --output=/bak/openstack/tarball/neutron_15.3.4.orig.tar.gz --format=tar HEAD
    
    gbp buildpackage --git-tarball-dir=/bak/openstack/tarball --git-ignore-branch --git-ignore-new -us -uc

看来还是得想办法解决XD3\_INVALID\_INPUT的问题。得想办法运行下列命令成功：

pristine-tar checkout ../neutron_15.3.4.orig.tar.gz

但这个命令在focal上是能运行成功的，可能是bionic的pristine-tar版本有问题吧，可以在focal导入tarball后再拷到bionic中，再使用’–git-tarball-dir='来workaroud.  
或者我们可以直接在focal中使用pbuilder + git-buildpackage 来规避这个问题。

## 使用pbuilder + ubuntu-openstack-dev debian源码构建 ##

第一步，创建pbuilder环境。这里的pbuilder实际上是cowpbuilder, 镜像格式是cow格式。

sudo apt-get install git-buildpackage cowbuilder debian-archive-keyring -y
    #it will delete /var/cache/pbuilder/aptcache
    #sudo pbuilder clean
    
    #sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
    rm -rf sudo rm -rf /var/cache/pbuilder/base-bionic-amd64.cow
    sudo mv /etc/pbuilderrc /tmp/
    DIST=bionic ARCH=amd64 git-pbuilder create
    sudo mv /tmp/pbuilderrc /etc/
    
    #sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
    DIST=bionic ARCH=amd64 git-pbuilder update
    #sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
    DIST=bionic ARCH=amd64 git-pbuilder login --save-after-login
      echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
      apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
      add-apt-repository cloud-archive:train
      #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
      #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
      sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
      apt update
      exit

第二步，使用pubilder (在gbp buuildpackage中添加–git-pbuilder --git-dist=bionic --git-arch=amd64）编译

sudo apt install devscripts equivs -y
    sudo mkdir -p /bak/work/debsource && sudo chown -R $USER /bak && cd /bak/work/debsource
    git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
    cd neutron
    git checkout pristine-tar
    git checkout upstream
    git checkout master
    git checkout -b debian/2%15.3.4-0ubuntu1_cloud0 debian/2%15.3.4-0ubuntu1_cloud0
    git checkout debian/2%15.3.4-0ubuntu1_cloud0
    
    #sudo mk-build-deps --install debian/control
    #gbp buildpackage  --git-ignore-branch --git-ignore-new -us -uc
    gbp buildpackage --git-pbuilder --git-dist=bionic --git-arch=amd64 --git-ignore-branch --git-ignore-new -uc -us -j16
    #debuild clean

## 在neutron源码中自动导入tarball构建 ##

下面直接从neutron源码中使用gbp来编译:  
https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html  
1, 首先，neutron source code必须从orgin更名为upstream避免和debian repository混淆

git clone --no-checkout -o upstream https://github.com/openstack/neutron

2, 将15.3.4的源码切换到debian/15.3.4分支，并且将debian目录加入到debian分支

cd neutron
    git checkout -b debian/15.3.4 15.3.4
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    git add debian/
    git commit -m 'add debian'
    #gbp:error: Couldn't commit to 'pristine-tar' with upstream '15.3.4': pristine-tar: more than one ref matches "15.3.4":

3, 因为debian/15.3.4分支有debian目录，应该是在这个分支下运行编译命令，**这们这里没有指定tarball包因为’–git-pristine-tar --git-pristine-tar-commit '会自动从源码中准备tarball**

gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 -us -uc

若更新了源文件的话, ‘gbp dch’会在changelog中添加一记录（15.3.4-0ubuntu1cloud1ubuntu11.gbp5d92a7），’–git-ignore-new’用于告诉　'gbp buildpackage’工作目录不干净。

git fetch upstream
    git merge 15.3.4
    # gbp:info: Changelog 2:15.3.4-0ubuntu1~cloud1ubuntu1~1.gbp5d92a7 (snapshot #1) prepared up to 5d92a7b
    gbp dch --debian-branch=debian/15.3.4 --snapshot --auto debian/
    gbp buildpackage --git-ignore-new --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' -us -uc

测试没问题之后，修改changelog和build一个新版本

gbp dch --release --auto --debian-branch=debian/15.3.4
    git commit -m"Release 1.1-1" debian/changelog
    gbp buildpackage --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 -us -uc

## 再总结 - 通过自己从头构建熟悉git-buildpackage原理 ##

这个是对”通过自己从头构建熟悉git-buildpackage原理“这一节的总结，**这里还是要自己准备tarball**麻烦，最好参考”通过neutron git来使用gbp“这一节由git来自动处理tarball. 还最好考虑git-buildpackage怎么与pbuilder结合(git-pbuilder)。  
1, 准备git工程, 注意：这里初始添加了README.md之后会报：will cause dpkg-source: error: aborting due to unexpected upstream changes，但初始不加，在git branch时看不到master与upstream分支，还是先添加，呆会再删除 (注意：最后一次试就又遇到这个问题，不需要删除）.

mkdir neutron-git-buildpackage2 && cd neutron-git-buildpackage2
    echo "# neutron-git-buildpackage" >> README.md
    git init
    git add README.md
    git commit -m "first commit"
    #git remote add origin git@github.com:zhhuabj/neutron-git-buildpackage.git
    #git push origin master

2, neutron源码工程下准备 tarball

mkdir -p /bak/openstack/tarball/ && cd /bak/openstack
    git clone https://github.com/openstack/neutron
    cd neutron
    git checkout master
    git checkout -b 15.3.4 15.3.4
    mkdir -p /tmp/tarball
    git archive --output=/tmp/tarball/neutron_15.3.4.orig.tar.gz --format=tar.gz HEAD

3, 创建upstream分支

git checkout -b upstream

4, 回到master分支（这时不能使用15.3.4，但master测试没问题后再加tag为15.3.4)导入tarball

git checkout master
    gbp import-orig /tmp/tarball/neutron_15.3.4.orig.tar.gz -u 15.3.4

只有使用’gbp import-orig’导入tarball包才会提交到upstream分支，并merge到master分支（也就是upstream与master分支都有）, 此时状态为：

ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git branch
    * master
      upstream
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
    CONTRIBUTING.rst  LICENSE     TESTING.rst  babel.cfg  bindep.txt  doc  neutron    rally-jobs    requirements.txt  setup.cfg  test-requirements.txt  tox.ini
    HACKING.rst       README.rst  api-ref      bin        devstack    etc  playbooks  releasenotes  roles             setup.py   tools
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git checkout upstream
    Switched to branch 'upstream'
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
    CONTRIBUTING.rst  LICENSE     TESTING.rst  babel.cfg  bindep.txt  doc  neutron    rally-jobs    requirements.txt  setup.cfg  test-requirements.txt  tox.ini
    HACKING.rst       README.rst  api-ref      bin        devstack    etc  playbooks  releasenotes  roles             setup.py   tools
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git tag
    upstream/15.3.4

5, master分支中添加debian目录(此时只能用master，不能创建tag, tag可在master中测试好了再打)

6, pristine-tar commit tarball, 总之，通过’pristine-tar list’ 得能看到neutron\_15.3.4.orig.tar.gz 15.3.4, 这样编译系统在编译的时候才能通过’pristine-tar checkout …/neutron\_15.3.4.orig.tar.gz’获取它。

pristine-tar commit /tmp/tarball/neutron_15.3.4.orig.tar.gz 15.3.4
    pristine-tar list

pristine-tar commit会自动添加pristine-tar分支：

ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git checkout pristine-tar
    Switched to branch 'pristine-tar'
    ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
    neutron_15.3.4.orig.tar.gz.delta  neutron_15.3.4.orig.tar.gz.id

７，回到master分支编译. 它是使用debuild来编译的，应该也可以和pbuilder等集成。

git checkout master
    gbp buildpackage --git-ignore-branch --git-ignore-new

上面会遇到这个错误”pristine-tar: /bak/work/build-area/neutron\_15.3.4.orig.tar.gz does not match stored hash“，下面使用pristine-tar checkout时也会报这个错，直接用–git-tarall-dir来指定之前使用git archive创建的tarball来workaround就好了（-us -uc意为不签名)：

#pristine-tar checkout /tmp/tarball/neutron_15.3.4.orig.tar.gz
    #gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -S -us -uc -sa
    gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -us -uc

8，编译后的包在/bak/work/build-area目录中  
9, 都没问题之后，打tag归档，今后可以在该tag下运行编译命令了：

git checkout master
    git tag '15.3.4'
    git checkout -b 15.3.4 15.3.4
    
    git checkout 15.3.4
    gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -us -uc

## 再总结 - 在neutron源码中自动导入tarball构建 ##

这是针对"在neutron源码中**自动导入tarball构建**”一节更快速的总结.

sudo apt install git-buildpackage pristine-tar -y
    # git clone --no-checkout -o upstream https://github.com/openstack/neutron
    git clone https://github.com/openstack/neutron
    cd neutron
    # create upstream branch
    git checkout master && git checkout -b upstream
    # create pristine-tar branch, but we don't need to prepare tarball because '--git-pristine-tar --git-pristine-tar-commit' will help us create it automatically
    git checkout master && git checkout -b pristine-tar
    # create debain/15.3.4 branch with upstream neutron=15.3.4
    git checkout master && git checkout -b debian/15.3.4 15.3.4
    # add debian package into debian/15.3.4
    git checkout debian/15.3.4
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    git add debian
    git commit -m 'add debian for 15.3.4'
    # run 'gbp buildpackage' command under the directory which has debian
    # --git-pristine-tar and --git-pristine-tar-commit will create tarball automatically
    git checkout debian/15.3.4
    gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc

报下列错，那是因为要检出15.3.4分支(git checkout master && git checkout -b 15.3.4 15.3.4)：

ubuntu@debbuild:/bak/openstack/neutron$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
    gbp:error: Couldn't commit to 'pristine-tar' with upstream '15.3.4': pristine-tar: more than one ref matches "15.3.4":
    480f4b7479bc0b19752edde46fe90660a94cc654 refs/heads/debian/15.3.4
    729f6aefe103a0ba10b71888f84b6247d694bc04 refs/tags/15.3.4

## 附录 - 如何检查包依赖问题 ##

上面种种方法都有这么一个问题（openstack-pkg-tools (>= 85ubuntu3~)），所以我们有必要检查这个源码包究竟有没有问题。  
原来，错误原因是必须添加：sudo add-apt-repository cloud-archive:stein

sudo add-apt-repository cloud-archive:stein
    sudo sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
    sudo apt update
    sudo apt-cache policy neutron-l3-agent
    sudo apt-get source neutron=2:15.3.4-0ubuntu1~cloud1
    cd neutron-15.3.4/
    $ grep -r 'openstack-pkg-tools' debian/control 
     openstack-pkg-tools (>= 85ubuntu3~),
    $ sudo apt-cache policy openstack-pkg-tools |grep Candidate
      Candidate: 99ubuntu1~cloud0

## 附录 - 其他 ##

1, gbp buildpackage之后无问题，可以加tag，会自动将neutron source code和debian source code加tag: debian/2%15.3.4-0ubuntu1\_cloud1

gbp buildpackage --git-tag-only --git-sign-tags --git-ignore-new
    #git checkout -b debian/2%15.3.4-0ubuntu1_cloud1 debian/2%15.3.4-0ubuntu1_cloud1

2, 然后可以推入远程repo

git push origin --all --follow-tags

## 最佳实践 ##

*  使用pbuilder避免在物理机上安装依赖
 *  直接使用neutron源码，并在neutron源码自动产生tarball的方式

第一步，创建pbuilder环境

#sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
    rm -rf sudo rm -rf /var/cache/pbuilder/base-bionic-amd64.cow
    sudo mv /etc/pbuilderrc /tmp/
    DIST=bionic ARCH=amd64 git-pbuilder create
    sudo mv /tmp/pbuilderrc /etc/
    
    #sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
    DIST=bionic ARCH=amd64 git-pbuilder update
    #sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
    DIST=bionic ARCH=amd64 git-pbuilder login --save-after-login
      echo 'precedence ::ffff:0:0/96 100' >> /etc/gai.conf
      apt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y
      # Failed to connect to socket /run/user/1000/bus: No such file or directory
      apt install dbus -y && /etc/init.d/dbus start
      add-apt-repository cloud-archive:train
      #deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main
      #apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EA
      sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
      apt update
      exit

第二步，准备neutron源码环境，还要创建upstream与prinstine-tar两个分支，以及要编译的15.3.4分支，以及存放15.3.4源码及debian目录的debian/15.3.4分支

sudo apt install git-buildpackage pristine-tar -y
    git clone https://github.com/openstack/neutron
    cd neutron
    git checkout master && git checkout -b upstream
    git checkout master && git checkout -b pristine-tar
    git checkout master && git checkout -b debian/15.3.4 15.3.4
    git checkout master && git checkout -b 15.3.4 15.3.4
    
    git checkout debian/15.3.4
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    git add debian
    git commit -m 'add debian for 15.3.4'

第三步，使用了git-builder来编译。并且通过“–git-pristine-tar --git-pristine-tar-commit ”来配置自动导出tarball

git checkout debian/15.3.4
    # dont's use git-builder
    #gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
    
    # use git-builder
    gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc --git-pbuilder --git-dist=bionic --git-arch=amd64 -j8

但是会报下列dbus的错：

dbus.exceptions.DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /run/user/1000/bus: No such file or directory
    debian/rules:16: recipe for target 'override_dh_install' failed

试了下面的还是不行。

DIST=bionic ARCH=amd64 git-pbuilder login --save-after-login
    apt install dbus
    /etc/init.d/dbus start
    exit

git-pbuilder创建出来的镜像里没有dbus包和环境变量（DBUS\_SESSION\_BUS\_ADDRESS=unix:path=/run/user/1000/bus），暂不清楚是否和这相关。  
如果不使用git-builder，直接运行’gpb buildpackage’ + debuild模式，it works. 注意：下列这个测试没用pbuilder只能在bionic上进行。

# NOTE: must run on bionic
    sudo mk-build-deps --install debian/control
    gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc -j8

将pbuilder换成sbuild也成功了（换sbuild时要去掉-us -uc, 见：https://www.pseudorandom.co.uk/2008/sbuild-dm/ , 另外，使用的是–git-builder=‘sbuild -A’， -A代表all-arch，构建源码包时用: --git-builder=‘sbuild -As’

gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new --git-builder='sbuild -A' --git-dist=bionic --git-arch=amd64 -j8

在neutron debian source code(https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron)中运行pbuilder也出现了dbus的错误，现在确认是这个dbus错误只是在pbuilder中才出现。

## 示例：ubuntu debian源码上编译16.1.0 ##

neutron debian source 上运行：

git checkout -b debian/2%16.1.0-0ubuntu2 debian/2%16.1.0-0ubuntu2
    gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As' --git-dist=bionic --git-arch=amd64 -j8

但会报错：

sbuild-build-depends-main-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installed
                                       Depends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installed
                                       Depends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed

那是因为镜像中使用的是train UCA(python3-neutron-lib=1.29.1)，train对应的是15.3.4，并不是16.1.0，所以有问题。vimdiff一下：

git checkout debian/2%16.1.0-0ubuntu2
    mkdir -p /tmp/16.1.0 && cp -r debian /tmp/16.1.0/
    git checkout debian/2%15.3.4-0ubuntu1_cloud0
    mkdir -p /tmp/15.3.4 && cp -r debian /tmp/15.3.4/
    vimdiff /tmp/15.3.4/debian/control /tmp/16.1.0/debian/control
    
    15.3.4 -> python3-neutron-lib (>= 1.29.1), 
    16.1.0 -> python3-neutron-lib (>= 2.2.0),

python3-neutron-lib 2.2.0来自focal前的一个版本也就是19.10

所以上面的16.1.0的源码显然是针对focal来做的（这里也有显示：https://launchpad.net/ubuntu/+source/neutron/2%3A16.1.0-0ubuntu2/+publishinghistory），所以：

*  目前bionic-stein的版本是15.3.4, focal-ussuri的版本是16.4.0, 现在想要测的是16.1.0
 *  如果在bionic里测16.1.0的话，得修改debian/control与requirment.txt中的发生问题的pip版本号，但这也不一定行，因为neutron代码也变了既然requirement.txt说依赖某个包的高版本可能就真是需要高版本
 *  那样就只能在focal里测16.1.0了。将focal现有的16.4.0改成16.1.0来测，sbuild的镜像什么的都得由bionic改成16.1.0

# create focal sbuid image
    mk-sbuild focal --arch=amd64
    schroot -l
    schroot -c source:focal-amd64 -u root
    
    # clone neutron debian source code
    cd /bak/work/debsource
    git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
    cd neutron
    git checkout pristine-tar
    git checkout upstream
    git checkout master
    
    # build with focal image
    git checkout -b debian/2%16.1.0-0ubuntu2 debian/2%16.1.0-0ubuntu2
    git checkout debian/2%16.1.0-0ubuntu2
    
    # 注意：下面使用了--git-dist=focal --git-arch=amd64来指定用focal但实际用的还是bionic，
    # 原来git-dist只是针对pbuilder，这样使用了~/.sbuildrc的默认项：sed -i -e "s/bionic/focal/g" ~/.sbuildrc
    #gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As' --git-dist=focal --git-arch=amd64 -j8
    
    #所以或者使用（sed -i -e "s/bionic/focal/g" ~/.sbuildrc），或者使用（'--git-builder=sbuild -As -d focal-amd64'）
    gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d focal-amd64' --git-dist=focal --git-arch=amd64 -j8

## 示例：neutron upsteam源码上编译15.3.4 ##

# prepare bionic-train sbuild image
    sudo cp /etc/schroot/chroot.d/sbuild-bionic-amd64 /etc/schroot/chroot.d/sbuild-bionic-train-amd64
    sudo cp -r /var/lib/schroot/chroots/bionic-amd64 /var/lib/schroot/chroots/bionic-train-amd64
    sudo sed -i -e "s/bionic-amd64/bionic-train-amd64/g" /etc/schroot/chroot.d/sbuild-bionic-train-amd64
    schroot -l
    schroot -c source:bionic-train-amd64 -u root
      apt install devscripts curl ca-certificates bash-completion software-properties-common vim -y
      # Couldn't create temporary file /tmp/apt.conf.5gKxoB for passing config to apt-key
      chmod 1777 /tmp
      # /usr/bin/apt-key: cannot create /dev/null: Permission denied
      # https://askubuntu.com/questions/764719/fix-permissions-after-upgrade-to-16-04
      chmod go+rw /dev/null
      add-apt-repository cloud-archive:train
    
    git clone https://github.com/openstack/neutron
    cd neutron
    #upstream与pristine-tar不是必须的吧
    #git checkout master && git checkout -b upstream
    #git checkout master && git checkout -b pristine-tar
    git checkout master && git checkout -b debian/15.3.4 15.3.4
    git checkout master && git checkout -b 15.3.4 15.3.4
    
    git checkout debian/15.3.4
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    git add debian
    git commit -m 'add debian for 15.3.4'
    
    # 已经在15.3.4分支下，--git-debian-branch=debian/15.3.4也不是必须的吧
    git checkout debian/15.3.4
    #gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
    gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d bionic-train-amd64' -j8

## 示例 - 在15.3.4与16.0.0之间二分准备15.3.5的测试包 ##

gbp buildpackage要想不自己准备tarball包的话，只能根据tag来做，不能根据commit来做，所以需自己准备如15.3.5与debian/15.3.5 (不自己准备tag的话似乎tarball包不更新使用15.3.4).  
当然，想自己准备tarball的话，就是另外一种方式。这里讲的是不自己准备tarball包的模式。

e562ecec2669c39b3628f7744011467c7394088e (在15.3.4基础上提交debian目录的commit id)
    15.3.4 - 0f18d406a4134cf769fdf6f2a004c98cbe409215
    16.0.0 - c06cb95b6f46574e74244fd738873290fd1773cb
    
    # 下载代码
    # first fork https://github.com/openstack/neutron
    git clone https://github.com/zhhuabj/neutron.git
    git config user.name
    git config user.email
    git remote add upstream https://github.com/openstack/neutron
    git remote set-url --push upstream no_push
    # update
    git fetch upstream
    git checkout master
    git merge upstream/master
    git push origin master
    #git rebase upstream/master
    #git push origin master -f
    
    # http://www.360doc.com/content/18/0208/09/19862658_728539054.shtml
    # git二分，先找到15.3.4与16.0.0之间的中间版本1c2e10f859
    cd neutron
    git checkout master
    git bisect start
    git bisect good 15.3.4
    $ git bisect bad 16.0.0
    Bisecting: a merge base must be tested
    [1c2e10f8595d2286bd9bec513bc5a346a84a6f7c] Merge "Remove get_external_network_id for router"
    $ git status
    HEAD detached at 1c2e10f859
    
    # 在上面1c2e10f859的基础上创建几个分支
    git checkout -b 15.3.5
    git checkout -b debian/15.3.5
    # 往debian/15.3.5中添加15.3.4的debian目录
    cp -r ../debian .
    git add debian/ && git commit -m 'add debian'
    
    # 修改debian/changelog
    dch -i
    neutron (2:15.3.5-0ubuntu1~cloud0) bionic-train; urgency=medium
    git commit -m "releasing package 2:15.3.5-0ubuntu1~cloud0" debian/changelog
    
    # 视情况清空debian/patches/避免一些错误, 并记得用debcommit提交
    cp -r debian/patches/ /tmp/
    rm -rf debian/patches/* && touch debian/patches/series
    git add -u && debcommit -m 'delete series'
    # 然后git log会看到：commit f1ed620b2abd4a04013ed9549fcafff5a290ff5b (HEAD -> debian/15.3.5)
    
    # 编译
    git checkout 15.3.5 && git tag --no-sign 15.3.5 -m "15.3.5"
    git checkout debian/15.3.5 && git tag --no-sign debian/15.3.5 -m "debian/15.3.5"
    git checkout debian/15.3.5
    #git push --all
    #git push --tags
    
    # 测试之后可以使用'git bisect bad'或'git bisect good'来标记, 使用’git bisect replay‘可以看到曾经测试过的内容
    git bisect log > bisectlog
    git bisect bad | git bisect good
    
    # 下一个二分可以先回到15.3.4基准，然后再git merge下一个二分点
    git reset --hard e562ecec2669c39b3628f7744011467c7394088e
    git merge <下一个二分点>
    
    # 二分完成之后
    git bisect reset

一个错误：

# 错误 - https://paste.ubuntu.com/p/DtfMXg3Yq2/
    # The 'weakrefmethod>=1.0.2' distribution was not found and is required by the application
    1, 15.3.4与1c2e10f859源码中有对weakrefmethod的定义
    # 15.3.4
    $ grep -r 'weakrefmethod' requirements.txt
    weakrefmethod>=1.0.2;python_version=='2.7' # PSF
    # git checkout -b 1c2e10f859 1c2e10f859
    $ grep -r 'weakrefmethod' requirements.txt 
    weakrefmethod>=1.0.2;python_version=='2.7' # PSF
    
    2, 在tarball中也有啊
    hua@node1:/bak/work/tmp/build-area/neutron-15.3.5$ grep -r 'weakrefmethod' requirements.txt 
    weakrefmethod>=1.0.2;python_version=='2.7' # PSF
    
    3, 难道是要在/bak/work/tmp/neutron/debian/control中加吗？
    
    git add -u && debcommit -m 'add python-weakrefmethod'
    
    hua@node1:/bak/work/tmp/neutron$ git diff
    diff --git a/debian/control b/debian/control
    index f25fd6f9ea..3f65330fd2 100644
    --- a/debian/control
    +++ b/debian/control
    @@ -13,6 +13,7 @@ Build-Depends:
      python3-all,
      python3-pbr (>= 4.0.0),
      python3-setuptools,
    + python-weakrefmethod,
     Build-Depends-Indep:
      crudini,
      python3-alembic (>= 0.8.10),

通过debuild调试上速问题的测试环境快速搭建：

sudo cp /etc/schroot/chroot.d/sbuild-bionic-train-amd64 /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
    sudo cp -r /var/lib/schroot/chroots/bionic-train-amd64 /var/lib/schroot/chroots/bionic-train-amd64-tmp
    sudo sed -i -e "s/bionic-train-amd64/bionic-train-amd64-tmp/g" /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
    #sudo rm /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
    #sudo rm -rf /var/lib/schroot/chroots/bionic-train-amd64-tmp
    schroot -l
    schroot -c source:bionic-train-amd64-tmp -u root
    chmod 1777 /tmp
    apt install git vim wget devscripts equivs -y
    git clone https://github.com/zhhuabj/neutron.git && cd neutron
    git bisect start
    git bisect good 15.3.4
    git bisect bad 16.0.0
    git checkout -b 15.3.5
    git checkout -b debian/15.3.5
    wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
    # mkdir /home/hua
    #git config --global user.email "you@example.com"
    #git config --global user.name "Your Name"
    git add debian/ && git commit -m 'add debian'
    dch -i
    #neutron (2:15.3.5-0ubuntu1~cloud0) bionic-train; urgency=medium
    git commit -m "releasing package 2:15.3.5-0ubuntu1~cloud0" debian/changelog
    
    cp -r debian/patches/ /tmp/
    rm -rf debian/patches/* && touch debian/patches/series
    git add -u && debcommit -m 'delete series'
    
    git checkout 15.3.5 && git tag --no-sign 15.3.5 -m "15.3.5"
    git checkout debian/15.3.5 && git tag --no-sign debian/15.3.5 -m "debian/15.3.5"
    
    git checkout 15.3.5
    git archive --format=tar HEAD |gzip > ../neutron_15.3.5.orig.tar.gz
    git checkout debian/15.3.5
    mk-build-deps --install debian/control
    rm -rf ../neutron_15.3.5-0ubuntu1~cloud0* && rm -rf .stestr
    debuild -i -us -uc

有很多种方式有可以很轻易重现上面问题：  
一种是直接运行’stestr run --subunit’，不清楚要不要运行’tox -e cover’（运行这句也hang在那）

stestr取代了testrepository - https://storyboard.openstack.org/#!/story/1666232
    # https://github.com/mtreinish/stestr/
    apt install python-tox python3-stestr python3-mock -y
    #tox -e cover
    stestr run --subunit

二是直接运行debian/rules build：

debian/rules clean && debian/rules build

三是运行’pkgos-dh\_auto\_test --no-py2’：

ls build/lib.linux-x86_64-2.7/neutron/
    cat /usr/share/openstack-pkg-tools/pkgos.make
    pkgos-dh_auto_test --no-py2
    
    vim debian/rules
    ifeq (,$(findstring nocheck, $(DEB_BUILD_OPTIONS)))
    override_dh_auto_test:
            pkgos-dh_auto_test --no-py2
    endif

所以最后通过‘DEB\_BUILD\_OPTIONS=nocheck’（不再运行unit test)来bypass了上述错误

echo 'DEBOOTSTRAP_PROXY=http://t440p:3142/' >> ~/.mk-sbuild.rc
    gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.5 --git-ignore-branch --git-ignore-new --git-builder='DEB_BUILD_OPTIONS=nocheck sbuild -As -d bionic-train-amd64' -j8

## Reference ##

\[1\] https://www.debian.org/doc/manuals/maint-guide/build.zh-cn.html  
\[2\] https://wiki.softwareheritage.org/wiki/Debian\_packaging