[pve-devel] [RFC kernel 2/2] debian/scripts: add import-upstream-tag

[Fabian Grünbichler]

On Wed, Apr 04, 2018 at 11:31:44AM +0200, Thomas Lamprecht wrote:
> Am 04/03/2018 um 01:30 PM schrieb Fabian Grünbichler:
> > $ import-upstream-tag path/to/kernel/submodule path/to/kernel/patches tag [rebase]
> > 
> > fetches 'tag' from default remote, optionally imports, rebases and exports
> > patchqueue, checks out 'tag' and commits the resulting changes.
> > 
> > Signed-off-by: Fabian Grünbichler <f.gruenbichler at proxmox.com>
> > ---
> >   debian/scripts/import-upstream-tag | 115 +++++++++++++++++++++++++++++++++++++
> >   1 file changed, 115 insertions(+)
> >   create mode 100755 debian/scripts/import-upstream-tag
> > 
> > diff --git a/debian/scripts/import-upstream-tag b/debian/scripts/import-upstream-tag
> > new file mode 100755
> > index 0000000..59daa5b
> > --- /dev/null
> > +++ b/debian/scripts/import-upstream-tag
> > @@ -0,0 +1,115 @@
> > +#!/bin/bash
> > +
> > +set -e
> > +
> > +top=$(pwd)
> > +
> > +# parameters
> > +kernel_submodule=
> > +kernel_patchdir=
> > +new_tag=
> > +rebase=
> > +
> > +# generated based on new_tag
> > +pq_branch=
> > +# previously checked out in submodule
> > +old_ref=
> > +
> > +function cleanup_pq_branch {
> > +    if [[ -n $pq_branch ]]; then
> > +	echo "cleaning up PQ branch '$pq_branch'"
> > +	cd "${top}/${kernel_submodule}"
> > +	git checkout --quiet $old_ref
> > +	git reset --hard
> > +	git branch -D "$pq_branch"
> > +    fi
> > +}
> > +
> > +function error_exit {
> > +    echo "$1"
> > +    set +e
> > +
> > +    cleanup_pq_branch
> > +
> > +    cd "${top}"
> > +
> > +    exit 1
> > +}
> > +
> > +if [ "$#" -lt 3 ]; then
> > +    error_exit "at least three parameters required."
> > +fi
> > +
> > +kernel_submodule=$1
> > +if [ ! -d "${kernel_submodule}" ]; then
> > +    error_exit "'${kernel_submodule}' must be a directory!"
> > +fi
> > +
> > +kernel_patchdir=$2
> > +if [ ! -d "${kernel_patchdir}" ]; then
> > +    error_exit "'${kernel_patchdir}' must be a directory!"
> > +fi
> > +
> > +new_tag=$3
> > +rebase=$4
> > +
> > +if [[ -n $(git status --untracked-files=no --porcelain) ]]; then
> > +    error_exit "working directory unclean, aborting"
> > +fi
> > +
> > +
> > +cd "${kernel_submodule}"
> > +## check for tag and fetch if needed
> > +echo "checking for tag '${new_tag}'"
> > +if [[ -z $(git tag -l "${new_tag}") ]]; then
> > +    echo "tag not found, fetching and retrying"
> > +    git fetch --tags
> > +fi
> > +if [[ -z $(git tag -l "${new_tag}") ]]; then
> > +    error_exit "tag not found, aborting"
> > +fi
> > +echo "tag found"
> > +cd "${top}"
> > +
> > +if [[ -n "$rebase" ]]; then
> > +    echo ""
> > +    echo "automatic patchqueue rebase enabled"
> > +    cd "${kernel_submodule}"
> > +    ## preparing patch queue branch
> > +    old_ref=$(git rev-parse HEAD)
> > +    pq_branch="auto_pq/${new_tag}"
> > +    cd "${top}"
> > +
> > +    echo "previous HEAD: ${old_ref}"
> > +
> > +    echo ""
> > +    "${top}/debian/scripts/import-patchqueue" "${kernel_submodule}" "${kernel_patchdir}" "${pq_branch}" || error_exit "failed to import patchqueue"
> > +
> > +    cd "${kernel_submodule}"
> > +    ## rebase patches
> > +    echo ""
> > +    echo "rebasing patchqueue on top of '${new_tag}'"
> > +    git rebase "${new_tag}"
> > +    cd "${top}"
> > +
> > +    ## regenerate exported patch queue
> > +    echo ""
> > +    "${top}/debian/scripts/export-patchqueue" "${kernel_submodule}" "${kernel_patchdir}" "${new_tag}" || error_exit "failed to export patchqueue"
> > +
> > +    cleanup_pq_branch
> > +    cd "${top}"
> > +    pq_branch=
> > +fi
> > +
> > +cd "${kernel_submodule}"
> > +echo ""
> > +echo "checking out '${new_tag}' in submodule"
> > +git checkout --quiet "${new_tag}"
> > +cd "${top}"
> > +
> > +echo ""
> > +echo "committing results"
> > +git commit --verbose -s -m "update sources to ${new_tag}" -m "(generated with debian/scripts/import-upstream-tag)" "${kernel_submodule}"
> > +if [[ -n "$rebase" ]]; then
> > +    git commit --verbose -s -m "rebase patches on top of ${new_tag}" -m "(generated with debian/scripts/import-upstream-tag)" "${kernel_patchdir}"
> > +fi
> 
> This is a bit weird, while there may have been changes nothing got added to
> the staging area,
> so no commit gets produced. e.g., see following run:

I think we need to do "git rm" and "git add" explicitly, because
"git commit [...] files" only commits tracked files.. will do some more
testing and send a v2 with this and the other feedback.

thanks for catching it!

> 
> root at dev5 sources/pve/pve-kernel±# debian/scripts/import-upstream-tag
> submodules/ubuntu-artful patches/kernel Ubuntu-4.13.0-38.43 yes
> checking for tag 'Ubuntu-4.13.0-38.43'
> tag found
> 
> automatic patchqueue rebase enabled
> previous HEAD: 23155b1c4c2fb5eececde1750aa9ba44a5da614b
> 
> creating patchqeueue branch 'auto_pq/Ubuntu-4.13.0-38.43'
> Switched to a new branch 'auto_pq/Ubuntu-4.13.0-38.43'
> importing patches from 'patches/kernel'
> Applying: Make mkcompile_h accept an alternate timestamp string
> Applying: bridge: keep MAC of first assigned port
> Applying: pci: Enable overrides for missing ACS capabilities (4.12+)
> Applying: kvm: disable default dynamic halt polling growth
> Applying: cgroup: Add mount flag to enable cpuset to use v2 behavior in v1
> cgroup
> Applying: cpuset: Allow v2 behavior in v1 cgroup
> Applying: IB/core: Avoid crash on pkey enforcement failed in received MADs
> Applying: IB/core: Don't enforce PKey security on SMI MADs
> Applying: KVM: SVM: obey guest PAT
> Applying: net: sched: em_nbyte: don't add the data offset twice
> Applying: net: sched: fix TCF_LAYER_LINK case in tcf_get_base_ptr
> Applying: KVM: x86: Add memory barrier on vmcs field lookup
> Applying: EDAC, sb_edac: Fix missing break in switch
> Applying: sched/wait: Fix add_wait_queue() behavioral change
> Applying: module/retpoline: Warn about missing retpoline in module
> Applying: net: tcp: close sock if net namespace is exiting
> Applying: sctp: fix dst refcnt leak in sctp_v4_get_dst
> Applying: sctp: fix dst refcnt leak in sctp_v6_get_dst()
> Applying: lockd: lost rollback of set_grace_period() in lockd_down_net()
> Applying: ocfs2: make metadata estimation accurate and clear
> Applying: ocfs2: try to reuse extent block in dealloc without meta_alloc
> Applying: mm/shmem: do not wait for lock_page() in
> shmem_unused_huge_shrink()
> Applying: mm/thp: Do not wait for lock_page() in deferred_split_scan()
> 
> rebasing patchqueue on top of 'Ubuntu-4.13.0-38.43'
> Current branch auto_pq/Ubuntu-4.13.0-38.43 is up to date.
> 
> clearing old exported patchqueue
> exporting patchqueue using 'git format-patch [...] Ubuntu-4.13.0-38.43..
> cleaning up PQ branch 'auto_pq/Ubuntu-4.13.0-38.43'
> HEAD is now at 23155b1c4c2f UBUNTU: Ubuntu-4.13.0-38.43
> Deleted branch auto_pq/Ubuntu-4.13.0-38.43 (was 3369573a9da2).
> 
> checking out 'Ubuntu-4.13.0-38.43' in submodule
> 
> committing results
> On branch rev_fabian_git-scripts
> Changes not staged for commit:
>         modified: patches/kernel/0015-module-retpoline-Warn-about-missing-retpoline-in-mod.patch
>         modified: patches/kernel/0021-ocfs2-try-to-reuse-extent-block-in-dealloc-without-m.patch
>         deleted: patches/kernel/0022-mm-thp-Do-not-wait-for-lock_page-in-deferred_split_s.patch
> 
> Untracked files:
>         buildlog
>         buildlog-4.15
>         config-4.13.13.org
>         linux-tools-4.13-dbgsym_4.13.16-43_amd64.deb
>         linux-tools-4.13_4.13.16-43_amd64.deb
>         linux-tools-4.15-dbgsym_4.15.10-2_amd64.deb
>         linux-tools-4.15_4.15.10-2_amd64.deb
> patches/kernel/0023-mm-thp-Do-not-wait-for-lock_page-in-deferred_split_s.patch
>         pve-headers-4.13.16-1-pve-dbgsym_4.13.16-43_amd64.deb
>         pve-headers-4.13.16-1-pve_4.13.16-43_amd64.deb
>         pve-headers-4.15.10-1-pve-dbgsym_4.15.10-2_amd64.deb
>         pve-headers-4.15.10-1-pve_4.15.10-2_amd64.deb
>         pve-headers/
>         pve-kernel-4.13.16-1-pve_4.13.16-43_amd64.deb
>         pve-kernel-4.15.10-1-pve_4.15.10-2_amd64.deb
>         pve-kernel_4.13.16-43_amd64.buildinfo
>         pve-kernel_4.13.16-43_amd64.changes
>         pve-kernel_4.15.10-2_amd64.buildinfo
>         pve-kernel_4.15.10-2_amd64.changes
>         submodules/spl-module/
>         submodules/ubuntu-bionic/
>         submodules/zfs-module/
> 
> no changes added to commit
> 
> root at dev5 sources/pve/pve-kernel±# git st
> On branch rev_fabian_git-scripts
> Changes not staged for commit:
>   (use "git add/rm <file>..." to update what will be committed)
>   (use "git checkout -- <file>..." to discard changes in working directory)
> 
>         modified: patches/kernel/0015-module-retpoline-Warn-about-missing-retpoline-in-mod.patch
>         modified: patches/kernel/0021-ocfs2-try-to-reuse-extent-block-in-dealloc-without-m.patch
>         deleted: patches/kernel/0022-mm-thp-Do-not-wait-for-lock_page-in-deferred_split_s.patch
> 
> no changes added to commit (use "git add" and/or "git commit -a")
>