git repositories for cygwin packaging - please test

[Jon TURNEY]

While a number of maintainers keep their cygwin packaging under some
sort of version control, there is currently no central collection of
these repositories.

To remedy this lack, using the same ssh key you use for sftp package
upload, package maintainers can now also push to git repositories, like so:

git push [hidden email]:/git/cygwin-packages/<packagename>

where <packagename> is a package name you are listed as a maintainer for
in http://cygwin.com/cygwin-pkg-maint.

These repositories are lazily created on the first push.

Since it's intended that these repositories will only contain cygport
scripts, patches, and other packaging files, and to prevent the
accidental committing of upstream archives, pushes containing large
binary files will be rejected.

These repositories are viewable via gitweb at
https://cygwin.com/git-cygwin-packages/ (URL may be subject to change),
and should be cloneable via anonymous git/http with the URLs shown there.

Please give this a test, if possible, and report any problems here.

[Ken Brown-6]

On 8/4/2019 4:08 PM, Jon Turney wrote:

>
> While a number of maintainers keep their cygwin packaging under some sort of
> version control, there is currently no central collection of these repositories.
>
> To remedy this lack, using the same ssh key you use for sftp package upload,
> package maintainers can now also push to git repositories, like so:
>
> git push [hidden email]:/git/cygwin-packages/<packagename>
>
> where <packagename> is a package name you are listed as a maintainer for in
> http://cygwin.com/cygwin-pkg-maint.
>
> These repositories are lazily created on the first push.
>
> Since it's intended that these repositories will only contain cygport scripts,
> patches, and other packaging files, and to prevent the accidental committing of
> upstream archives, pushes containing large binary files will be rejected.
>
> These repositories are viewable via gitweb at
> https://cygwin.com/git-cygwin-packages/ (URL may be subject to change), and
> should be cloneable via anonymous git/http with the URLs shown there.
>
> Please give this a test, if possible, and report any problems here.

Great idea!  Thanks for doing this.  I've pushed a few of my repos, and I've
cloned one of yours.  So far everything works as expected.

By the way, people who want to switch to using these new repos as their upstream
can issue the following command:

   git remote set-url origin [hidden email]:/git/cygwin-packages/<packagename>

Ken

[cygwin-apps mailing list]

> While a number of maintainers keep their cygwin packaging under some
> sort of version control, there is currently no central collection of
> these repositories.
>
> To remedy this lack, using the same ssh key you use for sftp package
> upload, package maintainers can now also push to git repositories, like so:
>
> git push [hidden email]:/git/cygwin-packages/<packagename>
>
> where <packagename> is a package name you are listed as a maintainer for
> in http://cygwin.com/cygwin-pkg-maint.

Nice work, thanks. And, a big step towards an automated package build
server! Gold star! https://cygwin.com/goldstars/#Jty

[cygwin-apps mailing list]

>
> While a number of maintainers keep their cygwin packaging under some
> sort of version control, there is currently no central collection of
> these repositories.
>
> To remedy this lack, using the same ssh key you use for sftp package
> upload, package maintainers can now also push to git repositories, like so:
>
> git push [hidden email]:/git/cygwin-packages/<packagename>
>
> where <packagename> is a package name you are listed as a maintainer for
> in http://cygwin.com/cygwin-pkg-maint.
>
> These repositories are lazily created on the first push.

In my testing, git push hangs on the first push, after "Initialized empty
git repositories". After I interrupt, it finishes normally. Maybe a
misconfiguration on my end.

~/d/c/fish> git push --set-upstream origin master
Initialized empty Git repository in
/sourceware1/projects/cygwin-home/cygwin-packages/fish.git/
# Insert Ctrl-C here
Counting objects: 92, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (75/75), done.
Writing objects: 100% (92/92), 14.87 KiB | 249.00 KiB/s, done.
Total 92 (delta 33), reused 0 (delta 0)
To cygwin.com:/git/cygwin-packages/fish
 * [new branch]      master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.

[Ken Brown-6]

On 8/8/2019 10:04 AM, Andrew Schulman via cygwin-apps wrote:

>>
>> While a number of maintainers keep their cygwin packaging under some
>> sort of version control, there is currently no central collection of
>> these repositories.
>>
>> To remedy this lack, using the same ssh key you use for sftp package
>> upload, package maintainers can now also push to git repositories, like so:
>>
>> git push [hidden email]:/git/cygwin-packages/<packagename>
>>
>> where <packagename> is a package name you are listed as a maintainer for
>> in http://cygwin.com/cygwin-pkg-maint.
>>
>> These repositories are lazily created on the first push.
>
> In my testing, git push hangs on the first push, after "Initialized empty
> git repositories". After I interrupt, it finishes normally. Maybe a
> misconfiguration on my end.

I'm seeing something like this too now, though I didn't when I first tested.  In
my case it doesn't finish normally after the interrupt; I have to do a second
git push, which then works.

Ken

[Jon TURNEY]

On 04/08/2019 21:08, Jon Turney wrote:
> Since it's intended that these repositories will only contain cygport
> scripts, patches, and other packaging files, and to prevent the
> accidental committing of upstream archives, pushes containing large
> binary files will be rejected.

Actually, it seems that I'd configured things so that pushes containing
binary files larger than 1kB were being rejected.

This was catching some repos which contain icon(s) (for a menu item), so
I've adjusted rule to permit those (based on file extension).

[Jon TURNEY]

On 08/08/2019 18:09, Ken Brown wrote:

> On 8/8/2019 10:04 AM, Andrew Schulman via cygwin-apps wrote:
>>>
>>> These repositories are lazily created on the first push.
>>
>> In my testing, git push hangs on the first push, after "Initialized empty
>> git repositories". After I interrupt, it finishes normally. Maybe a
>> misconfiguration on my end.
>
> I'm seeing something like this too now, though I didn't when I first tested.  In
> my case it doesn't finish normally after the interrupt; I have to do a second
> git push, which then works.
>

Thanks for reporting this problem.

I need to look into this some more, but this looks like push is getting
stuck due to creating the repo taking an unusually long time (perhaps
due to lock contention with someone else creating a repo...).

[Ken Brown-6]

On 8/9/2019 12:12 PM, Jon Turney wrote:

> On 08/08/2019 18:09, Ken Brown wrote:
>> On 8/8/2019 10:04 AM, Andrew Schulman via cygwin-apps wrote:
>>>>
>>>> These repositories are lazily created on the first push.
>>>
>>> In my testing, git push hangs on the first push, after "Initialized empty
>>> git repositories". After I interrupt, it finishes normally. Maybe a
>>> misconfiguration on my end.
>>
>> I'm seeing something like this too now, though I didn't when I first tested.  In
>> my case it doesn't finish normally after the interrupt; I have to do a second
>> git push, which then works.
>>
>
> Thanks for reporting this problem.
>
> I need to look into this some more, but this looks like push is getting stuck
> due to creating the repo taking an unusually long time (perhaps due to lock
> contention with someone else creating a repo...).

Whatever it was, it seems to be fixed now.

Ken

[Brian Inglis]

On 2019-08-04 14:08, Jon Turney wrote:

>
> While a number of maintainers keep their cygwin packaging under some sort of
> version control, there is currently no central collection of these repositories.
>
> To remedy this lack, using the same ssh key you use for sftp package upload,
> package maintainers can now also push to git repositories, like so:
>
> git push [hidden email]:/git/cygwin-packages/<packagename>
>
> where <packagename> is a package name you are listed as a maintainer for in
> http://cygwin.com/cygwin-pkg-maint.
>
> These repositories are lazily created on the first push.
>
> Since it's intended that these repositories will only contain cygport scripts,
> patches, and other packaging files, and to prevent the accidental committing of
> upstream archives, pushes containing large binary files will be rejected.
>
> These repositories are viewable via gitweb at
> https://cygwin.com/git-cygwin-packages/ (URL may be subject to change), and
> should be cloneable via anonymous git/http with the URLs shown there.
>
> Please give this a test, if possible, and report any problems here.

WJFFM - nice resource - got me to standardize some other stuff in my main
package dirs.

--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.

[Jon TURNEY]

On 09/08/2019 20:12, Ken Brown wrote:

> On 8/9/2019 12:12 PM, Jon Turney wrote:
>> On 08/08/2019 18:09, Ken Brown wrote:
>>> On 8/8/2019 10:04 AM, Andrew Schulman via cygwin-apps wrote:
>>>>>
>>>>> These repositories are lazily created on the first push.
>>>>
>>>> In my testing, git push hangs on the first push, after "Initialized empty
>>>> git repositories". After I interrupt, it finishes normally. Maybe a
>>>> misconfiguration on my end.
>>>
>>> I'm seeing something like this too now, though I didn't when I first tested.  In
>>> my case it doesn't finish normally after the interrupt; I have to do a second
>>> git push, which then works.
>>>
>>
>> Thanks for reporting this problem.
>>
>> I need to look into this some more, but this looks like push is getting stuck
>> due to creating the repo taking an unusually long time (perhaps due to lock
>> contention with someone else creating a repo...).
>
> Whatever it was, it seems to be fixed now.

I've updated gitolite with a fix so creating a repository isn't
grovelling over all existing repositories (which was taking a
ridiculously long time after lots of repositories had been created)

[Jon TURNEY]

On 04/08/2019 21:08, Jon Turney wrote:
>
> These repositories are viewable via gitweb at
> https://cygwin.com/git-cygwin-packages/ (URL may be subject to change),
> and should be cloneable via anonymous git/http with the URLs shown there.
>
> Please give this a test, if possible, and report any problems here.

Thanks for the feedback.

I've made the following adjustments:

Updated the gitolite version to get a performance fix for creating a
repository (so it's ~O(1) rather than ~O(number of repositories))

https://cygwin.com/git-cygwin-packages/ is now paginated to improve
pageload.  This works moderately well when using the default sorting (by
project), but unfortunately, it still needs to run thousands of git
commands to get the information when sorting by last change or owner.
The popular solutions to that problem seem to be "don't use gitweb" :).

Patched gitolite to permit repository names starting with '_', since we
have a couple of those.

[Achim Gratz]

Jon Turney writes:
> To remedy this lack, using the same ssh key you use for sftp package
> upload, package maintainers can now also push to git repositories,
> like so:
>
> git push [hidden email]:/git/cygwin-packages/<packagename>
>
> where <packagename> is a package name you are listed as a maintainer
> for in http://cygwin.com/cygwin-pkg-maint.

I spent the better part of last weekend to split up my private
repositories to fit this structure and then to rebase my local repos on
top of the history of co-maintained packages (from Yaakov) that had
already been uploaded.

The package listing page was a bit overwhelming to start with
and even more so after my uploads, perhaps ther could be a way to
pre-filter on first letter or prefix to more easily search?


Regards,
Achim.
--
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Samples for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra

[Jon TURNEY]

On 04/08/2019 21:08, Jon Turney wrote:
> To remedy this lack, using the same ssh key you use for sftp package
> upload, package maintainers can now also push to git repositories, like so:

Package maintainers may have noticed that the output from pushing to
these git repositories now includes a line like:

"remote: scallywag: build nnn queued"

This is a *prototype* of a system to automatically build the packages,
where the results appear (some time later) at [1] (URL subject to change)

[1] https://cygwin.com/cgi-bin2/jobs.cgi

Currently, many packages will fail to build correctly due to:

(i) missing or insufficient 'BUILD_REQUIRES',
(ii) missing prerequisites implied by an 'inherit' (a bug in this system),
(iii) resource limits imposed by AppVeyor's free service which is used
to perform the actual builds, or
(iv) other bugs in this system.

At this stage, this is only probably useful for verifying that
BUILD_REQUIRES is correct.

(Note that a successful build doesn't always mean that you have
reproduced your build: You'll need to check the configuration step
output and/or dependencies of the produced packages are the same.
Consider explicitly enabling the functionality you are expecting in the
options to the configure script , if it supports that (e.g.
CYGCONF_ARGS, etc.), to avoid it potentially silently turning off in
future, if it's requirements change)

To allow experimentation without messing up the version history
unnecessarily:

- All package repositories allow the maintainer(s) to create, push,
rewind and delete a branch named 'playground'.

- An additional package repository called 'playground' exists, that all
maintainers can do anything to.

[szgyg-2]

On Wed, May 27, 2020 at 11:27:49PM +0100, Jon Turney wrote:
> Currently, many packages will fail to build correctly due to:
>
> (iii) resource limits imposed by AppVeyor's free service which is used to
> perform the actual builds, or

Azure Devops may worth a try.

s

[Alexey Sokolov]

04.08.2019 21:08, Jon Turney пишет:

>
> While a number of maintainers keep their cygwin packaging under some
> sort of version control, there is currently no central collection of
> these repositories.
>
> To remedy this lack, using the same ssh key you use for sftp package
> upload, package maintainers can now also push to git repositories, like so:
>
> git push [hidden email]:/git/cygwin-packages/<packagename>
>
> where <packagename> is a package name you are listed as a maintainer for
> in http://cygwin.com/cygwin-pkg-maint.
>
> These repositories are lazily created on the first push.
>
> Since it's intended that these repositories will only contain cygport
> scripts, patches, and other packaging files, and to prevent the
> accidental committing of upstream archives, pushes containing large
> binary files will be rejected.
>
> These repositories are viewable via gitweb at
> https://cygwin.com/git-cygwin-packages/ (URL may be subject to change),
> and should be cloneable via anonymous git/http with the URLs shown there.
>
> Please give this a test, if possible, and report any problems here.

I've tried to push the playground branch to
cygwin.com:/git/cygwin-packages/znc and the output was full of this:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "",
        LC_ALL = (unset),
        LANG = "ru_RU.utf8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = "",
        LC_ALL = (unset),
        LANG = "ru_RU.utf8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

[cygwin-apps mailing list]

On 5/27/2020 6:27 PM, Jon Turney wrote:

> On 04/08/2019 21:08, Jon Turney wrote:
>> To remedy this lack, using the same ssh key you use for sftp package upload,
>> package maintainers can now also push to git repositories, like so:
>
> Package maintainers may have noticed that the output from pushing to these git
> repositories now includes a line like:
>
> "remote: scallywag: build nnn queued"
>
> This is a *prototype* of a system to automatically build the packages, where the
> results appear (some time later) at [1] (URL subject to change)
>
> [1] https://cygwin.com/cgi-bin2/jobs.cgi

This is great!  Thanks for doing this.  One strange thing I've noticed is that
sometimes a package will build fine on x86_64 but will fail on x86 with an error
like this:

distutils.errors.CompileError: command 'gcc' terminated by signal 11
make[4]: *** [/usr/share/gobject-introspection-1.0/Makefile.introspection:160:
HarfBuzz-0.0.gir] Error 1

Every time I've seen this, the build works fine on my local x86 install, so it's
not a big deal.  But I'm curious what might cause this.

Ken

[Brian Inglis]

On 2020-06-04 10:01, Ken Brown via Cygwin-apps wrote:

> On 5/27/2020 6:27 PM, Jon Turney wrote:
>> On 04/08/2019 21:08, Jon Turney wrote:
>>> To remedy this lack, using the same ssh key you use for sftp package upload,
>>> package maintainers can now also push to git repositories, like so:
>> Package maintainers may have noticed that the output from pushing to these git
>> repositories now includes a line like:
>> "remote: scallywag: build nnn queued"
>> This is a *prototype* of a system to automatically build the packages, where
>> the results appear (some time later) at [1] (URL subject to change)
>> [1] https://cygwin.com/cgi-bin2/jobs.cgi

> This is great!  Thanks for doing this.  One strange thing I've noticed is that
> sometimes a package will build fine on x86_64 but will fail on x86 with an error
> like this:
> distutils.errors.CompileError: command 'gcc' terminated by signal 11
> make[4]: *** [/usr/share/gobject-introspection-1.0/Makefile.introspection:160:
> HarfBuzz-0.0.gir] Error 1
> Every time I've seen this, the build works fine on my local x86 install, so it's
> not a big deal.  But I'm curious what might cause this.

$ kill -l 11
SEGV

distutils bug?

--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in IEC units and prefixes, physical quantities in SI.]

[cygwin-apps mailing list]

> 04.08.2019 21:08, Jon Turney wrote:

>> While a number of maintainers keep their cygwin packaging under some
>> sort of version control, there is currently no central collection of
>> these repositories.
>>
>> To remedy this lack, using the same ssh key you use for sftp package
>> upload, package maintainers can now also push to git repositories, like so:
>>
>> git push [hidden email]:/git/cygwin-packages/<packagename>
>>
>> where <packagename> is a package name you are listed as a maintainer for
>> in http://cygwin.com/cygwin-pkg-maint.
>>
>> These repositories are lazily created on the first push.
>>
>> Since it's intended that these repositories will only contain cygport
>> scripts, patches, and other packaging files, and to prevent the
>> accidental committing of upstream archives, pushes containing large
>> binary files will be rejected.
>>
>> These repositories are viewable via gitweb at
>> https://cygwin.com/git-cygwin-packages/ (URL may be subject to change),
>> and should be cloneable via anonymous git/http with the URLs shown there.
>>
>> Please give this a test, if possible, and report any problems here.

Works well for me, thank you. I was just wondering how I was going to
update the cygport and patches, so very useful.

Hamish McIntyre-Bhatty

[Jon TURNEY]

On 04/06/2020 21:33, Brian Inglis wrote:

> On 2020-06-04 10:01, Ken Brown via Cygwin-apps wrote:
>> On 5/27/2020 6:27 PM, Jon Turney wrote:
>>> On 04/08/2019 21:08, Jon Turney wrote:
>>>> To remedy this lack, using the same ssh key you use for sftp package upload,
>>>> package maintainers can now also push to git repositories, like so:
>>> Package maintainers may have noticed that the output from pushing to these git
>>> repositories now includes a line like:
>>> "remote: scallywag: build nnn queued"
>>> This is a *prototype* of a system to automatically build the packages, where
>>> the results appear (some time later) at [1] (URL subject to change)
>>> [1] https://cygwin.com/cgi-bin2/jobs.cgi
>
>> This is great!  Thanks for doing this.  One strange thing I've noticed is that
>> sometimes a package will build fine on x86_64 but will fail on x86 with an error
>> like this:
>> distutils.errors.CompileError: command 'gcc' terminated by signal 11
>> make[4]: *** [/usr/share/gobject-introspection-1.0/Makefile.introspection:160:
>> HarfBuzz-0.0.gir] Error 1
>> Every time I've seen this, the build works fine on my local x86 install, so it's
>> not a big deal.  But I'm curious what might cause this.
>
> $ kill -l 11
> SEGV
>
> distutils bug?

There should be no input that distutils can give gcc that makes it SEGV,
so it's at least a gcc bug as well.

I've adjusted things so any .stackdump files created by the build should
be preserved, which might give a start at investigating this.

[Brian Inglis]

On 2020-06-09 07:26, Jon Turney wrote:

> On 04/06/2020 21:33, Brian Inglis wrote:
>> On 2020-06-04 10:01, Ken Brown via Cygwin-apps wrote:
>>> On 5/27/2020 6:27 PM, Jon Turney wrote:
>>>> On 04/08/2019 21:08, Jon Turney wrote:
>>>>> To remedy this lack, using the same ssh key you use for sftp package upload,
>>>>> package maintainers can now also push to git repositories, like so:
>>>> Package maintainers may have noticed that the output from pushing to these git
>>>> repositories now includes a line like:
>>>> "remote: scallywag: build nnn queued"
>>>> This is a *prototype* of a system to automatically build the packages, where
>>>> the results appear (some time later) at [1] (URL subject to change)
>>>> [1] https://cygwin.com/cgi-bin2/jobs.cgi
>>
>>> This is great!  Thanks for doing this.  One strange thing I've noticed is that
>>> sometimes a package will build fine on x86_64 but will fail on x86 with an error
>>> like this:
>>> distutils.errors.CompileError: command 'gcc' terminated by signal 11
>>> make[4]: *** [/usr/share/gobject-introspection-1.0/Makefile.introspection:160:
>>> HarfBuzz-0.0.gir] Error 1
>>> Every time I've seen this, the build works fine on my local x86 install, so it's
>>> not a big deal.  But I'm curious what might cause this.
>>
>> $ kill -l 11
>> SEGV
>>
>> distutils bug?
>
> There should be no input that distutils can give gcc that makes it SEGV, so it's
> at least a gcc bug as well.
>
> I've adjusted things so any .stackdump files created by the build should be
> preserved, which might give a start at investigating this.

Any chance it may have a similar or related cause to:

        https://bugzilla.redhat.com/show_bug.cgi?id=1737186#c11

"I went ahead and enabled the introspection build. The issue that was causing
the introspection scanner to not find libharfbuzz-gobject.so.0 was that the spec
file was removing the rpaths that the introspection scanner relies on."

--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in IEC units and prefixes, physical quantities in SI.]