Questions regarding improving the Apache Commons release process.

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Questions regarding improving the Apache Commons release process.

Rob Tompkins-2
Hello all,

Pardon, maybe this should have gone to your @user list, but why not ping the dev crew. I’ve been playing around the ideas surrounding our fairly manual release process for the components in Commons, and I was hoping for some insights.

Scripting the version changes isn’t really that big of a deal for us, and that I can manage. But, when it comes to publishing our artifacts to the apache nexus repository, and then separately publishing our -src and -bin assemblies to the dev dist subversion repository (and consequently deleting those artifacts from nexus as they’re “attached” for the purpose of gpg signing), I feel it a tad cumbersome.

I’ve fiddled around a little with the idea of detaching the -src and -bin assemblies after gpg signing with some success, but then I have to delve into the mechanics of publishing those up to the subversion repository, and clearly that problem has already been solved. So I find myself in the space of trying to shoehorn our process into its the main maven-release-plugin, which I’ve found a tad difficult, versus writing our own release plugin, which feels like I would be duplicating tons of code (which I don’t want to do).

I’m curious if you guys have any thoughts on the matter as I’ve been playing around in the space for a little while now.

Cheers and happy holidays from UTC-5,
-Rob
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Stephen Connolly
On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]> wrote:

> Hello all,
>
> Pardon, maybe this should have gone to your @user list, but why not ping
> the dev crew. I’ve been playing around the ideas surrounding our fairly
> manual release process for the components in Commons, and I was hoping for
> some insights.
>
> Scripting the version changes isn’t really that big of a deal for us, and
> that I can manage. But, when it comes to publishing our artifacts to the
> apache nexus repository, and then separately publishing our -src and -bin
> assemblies to the dev dist subversion repository (and consequently deleting
> those artifacts from nexus as they’re “attached” for the purpose of gpg
> signing), I feel it a tad cumbersome.
>
> I’ve fiddled around a little with the idea of detaching the -src and -bin
> assemblies after gpg signing with some success, but then I have to delve
> into the mechanics of publishing those up to the subversion repository, and
> clearly that problem has already been solved.


Is your problem you don’t want those going to Nexus staging but only to
dist? Or is it that you want them *also* going to dist.

Personally... I see no reason to remove them from going to Nexus staging
(in fact I have a background plan to add secondary signing support to
staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
though. That would mean that the PMC would be able to *add* their GPG
signature to the staged artifacts as part of the voting... in which case
you’d want to hold off uploading to dist until *after* the vote so you get
the full set of signatures)

If you want to upload a subset of attached artifacts to dist as part of the
release, that seems much more tenable... just a derivative of the
scm-publish plugin from what I can see.

So I find myself in the space of trying to shoehorn our process into its
> the main maven-release-plugin, which I’ve found a tad difficult, versus
> writing our own release plugin, which feels like I would be duplicating
> tons of code (which I don’t want to do).


So the release plugin is really two parts:

1. A toolkit for writing release plugins

2. An example that does the job for the requirements of the Maven TLP and
has seemed “sufficient” for a lot of other people.

As such, if you have different needs, do not feel bad about having to
encode differently... hopefully the toolkit half of the codebase is
sufficient for you.

>
>
> I’m curious if you guys have any thoughts on the matter as I’ve been
> playing around in the space for a little while now.
>
> Cheers and happy holidays from UTC-5,
> -Rob
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
> --
Sent from my phone
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Matt Sicker
On 26 December 2017 at 04:27, Stephen Connolly <
[hidden email]> wrote:

> Personally... I see no reason to remove them from going to Nexus staging
> (in fact I have a background plan to add secondary signing support to
> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
> though. That would mean that the PMC would be able to *add* their GPG
> signature to the staged artifacts as part of the voting... in which case
> you’d want to hold off uploading to dist until *after* the vote so you get
> the full set of signatures)
>

Wow, this sounds really cool! So the PMC votes can add a vote by adding
their GPG signature? That's a really nifty idea.

--
Matt Sicker <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

sebb-2-2
On 26 December 2017 at 18:48, Matt Sicker <[hidden email]> wrote:

> On 26 December 2017 at 04:27, Stephen Connolly <
> [hidden email]> wrote:
>
>> Personally... I see no reason to remove them from going to Nexus staging
>> (in fact I have a background plan to add secondary signing support to
>> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
>> though. That would mean that the PMC would be able to *add* their GPG
>> signature to the staged artifacts as part of the voting... in which case
>> you’d want to hold off uploading to dist until *after* the vote so you get
>> the full set of signatures)
>>

It would be better to upload the original files (including RM-only
sig) to dist/dev.
The updated sigs can always be replaced on dist/dev just before publication.

> Wow, this sounds really cool! So the PMC votes can add a vote by adding
> their GPG signature? That's a really nifty idea.
>
> --
> Matt Sicker <[hidden email]>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Rob Tompkins
In reply to this post by Stephen Connolly
Stephen,

I can’t thank you enough for your reply. I’ll take your suggestions and continue to sandbox around using the maven-release-plugin as a guideline.

All the best and happy holidays,
-Rob

> On Dec 26, 2017, at 5:27 AM, Stephen Connolly <[hidden email]> wrote:
>
> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email] <mailto:[hidden email]>> wrote:
>
>> Hello all,
>>
>> Pardon, maybe this should have gone to your @user list, but why not ping
>> the dev crew. I’ve been playing around the ideas surrounding our fairly
>> manual release process for the components in Commons, and I was hoping for
>> some insights.
>>
>> Scripting the version changes isn’t really that big of a deal for us, and
>> that I can manage. But, when it comes to publishing our artifacts to the
>> apache nexus repository, and then separately publishing our -src and -bin
>> assemblies to the dev dist subversion repository (and consequently deleting
>> those artifacts from nexus as they’re “attached” for the purpose of gpg
>> signing), I feel it a tad cumbersome.
>>
>> I’ve fiddled around a little with the idea of detaching the -src and -bin
>> assemblies after gpg signing with some success, but then I have to delve
>> into the mechanics of publishing those up to the subversion repository, and
>> clearly that problem has already been solved.
>
>
> Is your problem you don’t want those going to Nexus staging but only to
> dist? Or is it that you want them *also* going to dist.
>
> Personally... I see no reason to remove them from going to Nexus staging
> (in fact I have a background plan to add secondary signing support to
> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
> though. That would mean that the PMC would be able to *add* their GPG
> signature to the staged artifacts as part of the voting... in which case
> you’d want to hold off uploading to dist until *after* the vote so you get
> the full set of signatures)
>
> If you want to upload a subset of attached artifacts to dist as part of the
> release, that seems much more tenable... just a derivative of the
> scm-publish plugin from what I can see.
>
> So I find myself in the space of trying to shoehorn our process into its
>> the main maven-release-plugin, which I’ve found a tad difficult, versus
>> writing our own release plugin, which feels like I would be duplicating
>> tons of code (which I don’t want to do).
>
>
> So the release plugin is really two parts:
>
> 1. A toolkit for writing release plugins
>
> 2. An example that does the job for the requirements of the Maven TLP and
> has seemed “sufficient” for a lot of other people.
>
> As such, if you have different needs, do not feel bad about having to
> encode differently... hopefully the toolkit half of the codebase is
> sufficient for you.
>
>>
>>
>> I’m curious if you guys have any thoughts on the matter as I’ve been
>> playing around in the space for a little while now.
>>
>> Cheers and happy holidays from UTC-5,
>> -Rob
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
>>
>> --
> Sent from my phone

Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Hervé BOUTEMY
Hi Rob,

one additional point: currently, for Maven itself, instead of adding new
Maven-specific ReleasePhase(s) to the default configuration, or configure them in
our parent pom (I'm not sure documentation on how to do that is available: I
could not find it), we chose first to create a separate "dist-tool" to check
consistency of what is currently published in misc places and provide some
commands to fix when an inconsistency is found.

This happens through daily reports done by a Jenkins job [1]:
- distribution area vs Maven Central [2]
- version from Maven site vs Maven Central [3]

We did not produce any release nor made it really configurable for conventions
different from Maven ones (like Common's -src & -bin), but at least there is a
configuration file to define artifacts to check [4]

HTH

Hervé


[1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/

[2] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
dist-tool-check-source-release.html

[3] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
dist-tool-check-index-page.html

[4] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
dist-tool.conf.html

Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :

> Stephen,
>
> I can’t thank you enough for your reply. I’ll take your suggestions and
> continue to sandbox around using the maven-release-plugin as a guideline.
>
> All the best and happy holidays,
> -Rob
>
> > On Dec 26, 2017, at 5:27 AM, Stephen Connolly
> > <[hidden email]> wrote:>
> > On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
<mailto:[hidden email]>> wrote:

> >> Hello all,
> >>
> >> Pardon, maybe this should have gone to your @user list, but why not ping
> >> the dev crew. I’ve been playing around the ideas surrounding our fairly
> >> manual release process for the components in Commons, and I was hoping
> >> for
> >> some insights.
> >>
> >> Scripting the version changes isn’t really that big of a deal for us, and
> >> that I can manage. But, when it comes to publishing our artifacts to the
> >> apache nexus repository, and then separately publishing our -src and -bin
> >> assemblies to the dev dist subversion repository (and consequently
> >> deleting
> >> those artifacts from nexus as they’re “attached” for the purpose of gpg
> >> signing), I feel it a tad cumbersome.
> >>
> >> I’ve fiddled around a little with the idea of detaching the -src and -bin
> >> assemblies after gpg signing with some success, but then I have to delve
> >> into the mechanics of publishing those up to the subversion repository,
> >> and
> >> clearly that problem has already been solved.
> >
> > Is your problem you don’t want those going to Nexus staging but only to
> > dist? Or is it that you want them *also* going to dist.
> >
> > Personally... I see no reason to remove them from going to Nexus staging
> > (in fact I have a background plan to add secondary signing support to
> > staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
> > though. That would mean that the PMC would be able to *add* their GPG
> > signature to the staged artifacts as part of the voting... in which case
> > you’d want to hold off uploading to dist until *after* the vote so you get
> > the full set of signatures)
> >
> > If you want to upload a subset of attached artifacts to dist as part of
> > the
> > release, that seems much more tenable... just a derivative of the
> > scm-publish plugin from what I can see.
> >
> > So I find myself in the space of trying to shoehorn our process into its
> >
> >> the main maven-release-plugin, which I’ve found a tad difficult, versus
> >> writing our own release plugin, which feels like I would be duplicating
> >> tons of code (which I don’t want to do).
> >
> > So the release plugin is really two parts:
> >
> > 1. A toolkit for writing release plugins
> >
> > 2. An example that does the job for the requirements of the Maven TLP and
> > has seemed “sufficient” for a lot of other people.
> >
> > As such, if you have different needs, do not feel bad about having to
> > encode differently... hopefully the toolkit half of the codebase is
> > sufficient for you.
> >
> >> I’m curious if you guys have any thoughts on the matter as I’ve been
> >> playing around in the space for a little while now.
> >>
> >> Cheers and happy holidays from UTC-5,
> >> -Rob
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> <mailto:[hidden email]> For additional commands,
> >> e-mail: [hidden email] <mailto:[hidden email]>
> >>
> >> --
> >
> > Sent from my phone



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Rob Tompkins


> On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email]> wrote:
>
> Hi Rob,
>
> one additional point: currently, for Maven itself, instead of adding new
> Maven-specific ReleasePhase(s) to the default configuration, or configure them in
> our parent pom (I'm not sure documentation on how to do that is available: I
> could not find it), we chose first to create a separate "dist-tool" to check
> consistency of what is currently published in misc places and provide some
> commands to fix when an inconsistency is found.
>
> This happens through daily reports done by a Jenkins job [1]:
> - distribution area vs Maven Central [2]
> - version from Maven site vs Maven Central [3]
>
> We did not produce any release nor made it really configurable for conventions
> different from Maven ones (like Common's -src & -bin), but at least there is a
> configuration file to define artifacts to check [4]

Interesting. Thanks,.

-Rob

>
> HTH
>
> Hervé
>
>
> [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/
>
> [2] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool-check-source-release.html
>
> [3] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool-check-index-page.html
>
> [4] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool.conf.html
>
> Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
>> Stephen,
>>
>> I can’t thank you enough for your reply. I’ll take your suggestions and
>> continue to sandbox around using the maven-release-plugin as a guideline.
>>
>> All the best and happy holidays,
>> -Rob
>>
>>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
>>> <[hidden email]> wrote:>
>>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
> <mailto:[hidden email]>> wrote:
>>>> Hello all,
>>>>
>>>> Pardon, maybe this should have gone to your @user list, but why not ping
>>>> the dev crew. I’ve been playing around the ideas surrounding our fairly
>>>> manual release process for the components in Commons, and I was hoping
>>>> for
>>>> some insights.
>>>>
>>>> Scripting the version changes isn’t really that big of a deal for us, and
>>>> that I can manage. But, when it comes to publishing our artifacts to the
>>>> apache nexus repository, and then separately publishing our -src and -bin
>>>> assemblies to the dev dist subversion repository (and consequently
>>>> deleting
>>>> those artifacts from nexus as they’re “attached” for the purpose of gpg
>>>> signing), I feel it a tad cumbersome.
>>>>
>>>> I’ve fiddled around a little with the idea of detaching the -src and -bin
>>>> assemblies after gpg signing with some success, but then I have to delve
>>>> into the mechanics of publishing those up to the subversion repository,
>>>> and
>>>> clearly that problem has already been solved.
>>>
>>> Is your problem you don’t want those going to Nexus staging but only to
>>> dist? Or is it that you want them *also* going to dist.
>>>
>>> Personally... I see no reason to remove them from going to Nexus staging
>>> (in fact I have a background plan to add secondary signing support to
>>> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
>>> though. That would mean that the PMC would be able to *add* their GPG
>>> signature to the staged artifacts as part of the voting... in which case
>>> you’d want to hold off uploading to dist until *after* the vote so you get
>>> the full set of signatures)
>>>
>>> If you want to upload a subset of attached artifacts to dist as part of
>>> the
>>> release, that seems much more tenable... just a derivative of the
>>> scm-publish plugin from what I can see.
>>>
>>> So I find myself in the space of trying to shoehorn our process into its
>>>
>>>> the main maven-release-plugin, which I’ve found a tad difficult, versus
>>>> writing our own release plugin, which feels like I would be duplicating
>>>> tons of code (which I don’t want to do).
>>>
>>> So the release plugin is really two parts:
>>>
>>> 1. A toolkit for writing release plugins
>>>
>>> 2. An example that does the job for the requirements of the Maven TLP and
>>> has seemed “sufficient” for a lot of other people.
>>>
>>> As such, if you have different needs, do not feel bad about having to
>>> encode differently... hopefully the toolkit half of the codebase is
>>> sufficient for you.
>>>
>>>> I’m curious if you guys have any thoughts on the matter as I’ve been
>>>> playing around in the space for a little while now.
>>>>
>>>> Cheers and happy holidays from UTC-5,
>>>> -Rob
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> <mailto:[hidden email]> For additional commands,
>>>> e-mail: [hidden email] <mailto:[hidden email]>
>>>>
>>>> --
>>>
>>> Sent from my phone
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[commons-release-plugin] Progress report. (Was: Re: Questions regarding improving the Apache Commons release process.)

Rob Tompkins
In reply to this post by Hervé BOUTEMY
Hello all,

I just wanted to let everyone know where I’ve been running lately. I’m writing a new commons component specifically “commons-release-plugin” for the sake of making a maven plugin that adheres to our release process. I’m sandboxing it in my git work area:

https://github.com/chtompki/commons-release-plugin <https://github.com/chtompki/commons-release-plugin>

My goal is to get it functional an then bring it up for vote on creating it as a full fledged component (in the same vein as the commons-build-plugin).

Currently, I have it declared locally in [text] with:

        <groupId>org.apache.commons</groupId>
        <artifactId>commons-release-plugin</artifactId>
        <version>0.1-SNAPSHOT</version>
        <configuration>
          <pubScmStagingUrl>scm:svn:https://dist.apache.org/repos/dist/dev/commons/text</pubScmStagingUrl>
        </configuration>
        <executions>
          <execution>
            <id>detatch-assemblies</id>
            <phase>verify</phase>
            <goals>
              <goal>detatch-assemblies</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

immediately following this line: https://github.com/apache/commons-text/blob/master/pom.xml#L164 <https://github.com/apache/commons-text/blob/master/pom.xml#L164> in the pom. As it stands now, it detaches the distributions from the deployment to nexus (after gpg signing, and then copies them and the sha1’s and md5’s into a working directory in /target), My plan here is to use the maven-release-plugin as a guideline for how to publish these up to the dist svn repository. I think I’m further going to set up a mojo to do site deployment to somewhere (open to thoughts on where the site should go, maybe the dist svn area in a zip file like what Gary did with the latest [dbcp] release??).

Given this is a progress report. I’m open to anyone telling me that I’m wasting my time and pointing me in a new direction.

Cheers,
-Rob

P.S. I haven’t figured out how to write tests for maven plugins, so I have no testing in the project. I’m open to suggestions/help in that department if anyone wants to chip in and help.


> On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email]> wrote:
>
> Hi Rob,
>
> one additional point: currently, for Maven itself, instead of adding new
> Maven-specific ReleasePhase(s) to the default configuration, or configure them in
> our parent pom (I'm not sure documentation on how to do that is available: I
> could not find it), we chose first to create a separate "dist-tool" to check
> consistency of what is currently published in misc places and provide some
> commands to fix when an inconsistency is found.
>
> This happens through daily reports done by a Jenkins job [1]:
> - distribution area vs Maven Central [2]
> - version from Maven site vs Maven Central [3]
>
> We did not produce any release nor made it really configurable for conventions
> different from Maven ones (like Common's -src & -bin), but at least there is a
> configuration file to define artifacts to check [4]
>
> HTH
>
> Hervé
>
>
> [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/
>
> [2] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool-check-source-release.html
>
> [3] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool-check-index-page.html
>
> [4] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool.conf.html
>
> Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
>> Stephen,
>>
>> I can’t thank you enough for your reply. I’ll take your suggestions and
>> continue to sandbox around using the maven-release-plugin as a guideline.
>>
>> All the best and happy holidays,
>> -Rob
>>
>>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
>>> <[hidden email]> wrote:>
>>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
> <mailto:[hidden email]>> wrote:
>>>> Hello all,
>>>>
>>>> Pardon, maybe this should have gone to your @user list, but why not ping
>>>> the dev crew. I’ve been playing around the ideas surrounding our fairly
>>>> manual release process for the components in Commons, and I was hoping
>>>> for
>>>> some insights.
>>>>
>>>> Scripting the version changes isn’t really that big of a deal for us, and
>>>> that I can manage. But, when it comes to publishing our artifacts to the
>>>> apache nexus repository, and then separately publishing our -src and -bin
>>>> assemblies to the dev dist subversion repository (and consequently
>>>> deleting
>>>> those artifacts from nexus as they’re “attached” for the purpose of gpg
>>>> signing), I feel it a tad cumbersome.
>>>>
>>>> I’ve fiddled around a little with the idea of detaching the -src and -bin
>>>> assemblies after gpg signing with some success, but then I have to delve
>>>> into the mechanics of publishing those up to the subversion repository,
>>>> and
>>>> clearly that problem has already been solved.
>>>
>>> Is your problem you don’t want those going to Nexus staging but only to
>>> dist? Or is it that you want them *also* going to dist.
>>>
>>> Personally... I see no reason to remove them from going to Nexus staging
>>> (in fact I have a background plan to add secondary signing support to
>>> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
>>> though. That would mean that the PMC would be able to *add* their GPG
>>> signature to the staged artifacts as part of the voting... in which case
>>> you’d want to hold off uploading to dist until *after* the vote so you get
>>> the full set of signatures)
>>>
>>> If you want to upload a subset of attached artifacts to dist as part of
>>> the
>>> release, that seems much more tenable... just a derivative of the
>>> scm-publish plugin from what I can see.
>>>
>>> So I find myself in the space of trying to shoehorn our process into its
>>>
>>>> the main maven-release-plugin, which I’ve found a tad difficult, versus
>>>> writing our own release plugin, which feels like I would be duplicating
>>>> tons of code (which I don’t want to do).
>>>
>>> So the release plugin is really two parts:
>>>
>>> 1. A toolkit for writing release plugins
>>>
>>> 2. An example that does the job for the requirements of the Maven TLP and
>>> has seemed “sufficient” for a lot of other people.
>>>
>>> As such, if you have different needs, do not feel bad about having to
>>> encode differently... hopefully the toolkit half of the codebase is
>>> sufficient for you.
>>>
>>>> I’m curious if you guys have any thoughts on the matter as I’ve been
>>>> playing around in the space for a little while now.
>>>>
>>>> Cheers and happy holidays from UTC-5,
>>>> -Rob
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> <mailto:[hidden email]> For additional commands,
>>>> e-mail: [hidden email] <mailto:[hidden email]>
>>>>
>>>> --
>>>
>>> Sent from my phone
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [commons-release-plugin] Progress report. (Was: Re: Questions regarding improving the Apache Commons release process.)

garydgregory
Thank you for spearheading making our release process better.

One tricky thing to watch out for are components like Lang and DBCP which
have folder names on dist that are not the artifact ID. IOW lang vs lang3,
dbcp vs dbcp2.

Gary

On Dec 30, 2017 08:29, "Rob Tompkins" <[hidden email]> wrote:

> Hello all,
>
> I just wanted to let everyone know where I’ve been running lately. I’m
> writing a new commons component specifically “commons-release-plugin” for
> the sake of making a maven plugin that adheres to our release process. I’m
> sandboxing it in my git work area:
>
> https://github.com/chtompki/commons-release-plugin <
> https://github.com/chtompki/commons-release-plugin>
>
> My goal is to get it functional an then bring it up for vote on creating
> it as a full fledged component (in the same vein as the
> commons-build-plugin).
>
> Currently, I have it declared locally in [text] with:
>
>         <groupId>org.apache.commons</groupId>
>         <artifactId>commons-release-plugin</artifactId>
>         <version>0.1-SNAPSHOT</version>
>         <configuration>
>           <pubScmStagingUrl>scm:svn:https://dist.apache.org/repos/
> dist/dev/commons/text</pubScmStagingUrl>
>         </configuration>
>         <executions>
>           <execution>
>             <id>detatch-assemblies</id>
>             <phase>verify</phase>
>             <goals>
>               <goal>detatch-assemblies</goal>
>             </goals>
>           </execution>
>         </executions>
>       </plugin>
>
> immediately following this line: https://github.com/apache/
> commons-text/blob/master/pom.xml#L164 <https://github.com/apache/
> commons-text/blob/master/pom.xml#L164> in the pom. As it stands now, it
> detaches the distributions from the deployment to nexus (after gpg signing,
> and then copies them and the sha1’s and md5’s into a working directory in
> /target), My plan here is to use the maven-release-plugin as a guideline
> for how to publish these up to the dist svn repository. I think I’m further
> going to set up a mojo to do site deployment to somewhere (open to thoughts
> on where the site should go, maybe the dist svn area in a zip file like
> what Gary did with the latest [dbcp] release??).
>
> Given this is a progress report. I’m open to anyone telling me that I’m
> wasting my time and pointing me in a new direction.
>
> Cheers,
> -Rob
>
> P.S. I haven’t figured out how to write tests for maven plugins, so I have
> no testing in the project. I’m open to suggestions/help in that department
> if anyone wants to chip in and help.
>
>
> > On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email]>
> wrote:
> >
> > Hi Rob,
> >
> > one additional point: currently, for Maven itself, instead of adding new
> > Maven-specific ReleasePhase(s) to the default configuration, or
> configure them in
> > our parent pom (I'm not sure documentation on how to do that is
> available: I
> > could not find it), we chose first to create a separate "dist-tool" to
> check
> > consistency of what is currently published in misc places and provide
> some
> > commands to fix when an inconsistency is found.
> >
> > This happens through daily reports done by a Jenkins job [1]:
> > - distribution area vs Maven Central [2]
> > - version from Maven site vs Maven Central [3]
> >
> > We did not produce any release nor made it really configurable for
> conventions
> > different from Maven ones (like Common's -src & -bin), but at least
> there is a
> > configuration file to define artifacts to check [4]
> >
> > HTH
> >
> > Hervé
> >
> >
> > [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/
> >
> > [2] https://builds.apache.org/view/M-R/view/Maven/job/dist-
> tool-plugin/site/
> > dist-tool-check-source-release.html
> >
> > [3] https://builds.apache.org/view/M-R/view/Maven/job/dist-
> tool-plugin/site/
> > dist-tool-check-index-page.html
> >
> > [4] https://builds.apache.org/view/M-R/view/Maven/job/dist-
> tool-plugin/site/
> > dist-tool.conf.html
> >
> > Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
> >> Stephen,
> >>
> >> I can’t thank you enough for your reply. I’ll take your suggestions and
> >> continue to sandbox around using the maven-release-plugin as a
> guideline.
> >>
> >> All the best and happy holidays,
> >> -Rob
> >>
> >>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
> >>> <[hidden email]> wrote:>
> >>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
> > <mailto:[hidden email]>> wrote:
> >>>> Hello all,
> >>>>
> >>>> Pardon, maybe this should have gone to your @user list, but why not
> ping
> >>>> the dev crew. I’ve been playing around the ideas surrounding our
> fairly
> >>>> manual release process for the components in Commons, and I was hoping
> >>>> for
> >>>> some insights.
> >>>>
> >>>> Scripting the version changes isn’t really that big of a deal for us,
> and
> >>>> that I can manage. But, when it comes to publishing our artifacts to
> the
> >>>> apache nexus repository, and then separately publishing our -src and
> -bin
> >>>> assemblies to the dev dist subversion repository (and consequently
> >>>> deleting
> >>>> those artifacts from nexus as they’re “attached” for the purpose of
> gpg
> >>>> signing), I feel it a tad cumbersome.
> >>>>
> >>>> I’ve fiddled around a little with the idea of detaching the -src and
> -bin
> >>>> assemblies after gpg signing with some success, but then I have to
> delve
> >>>> into the mechanics of publishing those up to the subversion
> repository,
> >>>> and
> >>>> clearly that problem has already been solved.
> >>>
> >>> Is your problem you don’t want those going to Nexus staging but only to
> >>> dist? Or is it that you want them *also* going to dist.
> >>>
> >>> Personally... I see no reason to remove them from going to Nexus
> staging
> >>> (in fact I have a background plan to add secondary signing support to
> >>> staging... i’m Waiting to see the Nexus 3 staging APIs before
> attempting
> >>> though. That would mean that the PMC would be able to *add* their GPG
> >>> signature to the staged artifacts as part of the voting... in which
> case
> >>> you’d want to hold off uploading to dist until *after* the vote so you
> get
> >>> the full set of signatures)
> >>>
> >>> If you want to upload a subset of attached artifacts to dist as part of
> >>> the
> >>> release, that seems much more tenable... just a derivative of the
> >>> scm-publish plugin from what I can see.
> >>>
> >>> So I find myself in the space of trying to shoehorn our process into
> its
> >>>
> >>>> the main maven-release-plugin, which I’ve found a tad difficult,
> versus
> >>>> writing our own release plugin, which feels like I would be
> duplicating
> >>>> tons of code (which I don’t want to do).
> >>>
> >>> So the release plugin is really two parts:
> >>>
> >>> 1. A toolkit for writing release plugins
> >>>
> >>> 2. An example that does the job for the requirements of the Maven TLP
> and
> >>> has seemed “sufficient” for a lot of other people.
> >>>
> >>> As such, if you have different needs, do not feel bad about having to
> >>> encode differently... hopefully the toolkit half of the codebase is
> >>> sufficient for you.
> >>>
> >>>> I’m curious if you guys have any thoughts on the matter as I’ve been
> >>>> playing around in the space for a little while now.
> >>>>
> >>>> Cheers and happy holidays from UTC-5,
> >>>> -Rob
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: [hidden email]
> >>>> <mailto:[hidden email]> For additional commands,
> >>>> e-mail: [hidden email] <mailto:[hidden email]>
> >>>>
> >>>> --
> >>>
> >>> Sent from my phone
> >
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Rob Tompkins
In reply to this post by Rob Tompkins
Hello again Maven Team,

I’ve made substantive progress towards getting the commons team a release plugin (https://github.com/chtompki/commons-release-plugin <https://github.com/chtompki/commons-release-plugin>). However, in my attempt at writing plugin unit tests for the first time, I’ve found myself running into difficulties in dealing with which dependencies need to be in the classpath to effectively run the maven-plugin-testing-harness.

I was hoping to get another set of eyes on my work, namely the pom and the unit test that is in flight (see the above repository), such that I can get around these classpath issues and start writing proper tests for the plugin. It is quite easy to see the issues that I’m having by cloning the project and running “mvn test:"

Running org.apache.commons.release.plugin.mojos.CommonsSiteCompressionMojoTest
Jan 06, 2018 2:00:13 PM org.eclipse.sisu.inject.Logs$JULSink warn
WARNING: Error injecting: org.apache.maven.artifact.resolver.DefaultArtifactResolver
java.lang.NoClassDefFoundError: Lorg/apache/maven/artifact/transform/ArtifactTransformationManager;
        at java.lang.Class.getDeclaredFields0(Native Method)
        at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
        at java.lang.Class.getDeclaredFields(Class.java:1916)

Any guidance here would be much welcomed, and moreover I can’t thank you guys enough for the previous insights because they gave me the ability to make solid progress on our release automation.

Many thanks and all the best,
-Rob


> On Dec 28, 2017, at 4:53 PM, Rob Tompkins <[hidden email]> wrote:
>
>
>
>> On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email]> wrote:
>>
>> Hi Rob,
>>
>> one additional point: currently, for Maven itself, instead of adding new
>> Maven-specific ReleasePhase(s) to the default configuration, or configure them in
>> our parent pom (I'm not sure documentation on how to do that is available: I
>> could not find it), we chose first to create a separate "dist-tool" to check
>> consistency of what is currently published in misc places and provide some
>> commands to fix when an inconsistency is found.
>>
>> This happens through daily reports done by a Jenkins job [1]:
>> - distribution area vs Maven Central [2]
>> - version from Maven site vs Maven Central [3]
>>
>> We did not produce any release nor made it really configurable for conventions
>> different from Maven ones (like Common's -src & -bin), but at least there is a
>> configuration file to define artifacts to check [4]
>
> Interesting. Thanks,.
>
> -Rob
>
>>
>> HTH
>>
>> Hervé
>>
>>
>> [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/
>>
>> [2] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>> dist-tool-check-source-release.html
>>
>> [3] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>> dist-tool-check-index-page.html
>>
>> [4] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>> dist-tool.conf.html
>>
>> Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
>>> Stephen,
>>>
>>> I can’t thank you enough for your reply. I’ll take your suggestions and
>>> continue to sandbox around using the maven-release-plugin as a guideline.
>>>
>>> All the best and happy holidays,
>>> -Rob
>>>
>>>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
>>>> <[hidden email]> wrote:>
>>>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>>>> Hello all,
>>>>>
>>>>> Pardon, maybe this should have gone to your @user list, but why not ping
>>>>> the dev crew. I’ve been playing around the ideas surrounding our fairly
>>>>> manual release process for the components in Commons, and I was hoping
>>>>> for
>>>>> some insights.
>>>>>
>>>>> Scripting the version changes isn’t really that big of a deal for us, and
>>>>> that I can manage. But, when it comes to publishing our artifacts to the
>>>>> apache nexus repository, and then separately publishing our -src and -bin
>>>>> assemblies to the dev dist subversion repository (and consequently
>>>>> deleting
>>>>> those artifacts from nexus as they’re “attached” for the purpose of gpg
>>>>> signing), I feel it a tad cumbersome.
>>>>>
>>>>> I’ve fiddled around a little with the idea of detaching the -src and -bin
>>>>> assemblies after gpg signing with some success, but then I have to delve
>>>>> into the mechanics of publishing those up to the subversion repository,
>>>>> and
>>>>> clearly that problem has already been solved.
>>>>
>>>> Is your problem you don’t want those going to Nexus staging but only to
>>>> dist? Or is it that you want them *also* going to dist.
>>>>
>>>> Personally... I see no reason to remove them from going to Nexus staging
>>>> (in fact I have a background plan to add secondary signing support to
>>>> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
>>>> though. That would mean that the PMC would be able to *add* their GPG
>>>> signature to the staged artifacts as part of the voting... in which case
>>>> you’d want to hold off uploading to dist until *after* the vote so you get
>>>> the full set of signatures)
>>>>
>>>> If you want to upload a subset of attached artifacts to dist as part of
>>>> the
>>>> release, that seems much more tenable... just a derivative of the
>>>> scm-publish plugin from what I can see.
>>>>
>>>> So I find myself in the space of trying to shoehorn our process into its
>>>>
>>>>> the main maven-release-plugin, which I’ve found a tad difficult, versus
>>>>> writing our own release plugin, which feels like I would be duplicating
>>>>> tons of code (which I don’t want to do).
>>>>
>>>> So the release plugin is really two parts:
>>>>
>>>> 1. A toolkit for writing release plugins
>>>>
>>>> 2. An example that does the job for the requirements of the Maven TLP and
>>>> has seemed “sufficient” for a lot of other people.
>>>>
>>>> As such, if you have different needs, do not feel bad about having to
>>>> encode differently... hopefully the toolkit half of the codebase is
>>>> sufficient for you.
>>>>
>>>>> I’m curious if you guys have any thoughts on the matter as I’ve been
>>>>> playing around in the space for a little while now.
>>>>>
>>>>> Cheers and happy holidays from UTC-5,
>>>>> -Rob
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> <mailto:[hidden email]> For additional commands,
>>>>> e-mail: [hidden email] <mailto:[hidden email]>
>>>>>
>>>>> --
>>>>
>>>> Sent from my phone

Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Rob Tompkins
I’ve found some success now using org.apache.maven.plugin-testing:maven-plugin-testing-harness:3.3.0. You may disregard the following questions.

Cheers,
-Rob

> On Jan 6, 2018, at 2:03 PM, Rob Tompkins <[hidden email]> wrote:
>
> Hello again Maven Team,
>
> I’ve made substantive progress towards getting the commons team a release plugin (https://github.com/chtompki/commons-release-plugin <https://github.com/chtompki/commons-release-plugin>). However, in my attempt at writing plugin unit tests for the first time, I’ve found myself running into difficulties in dealing with which dependencies need to be in the classpath to effectively run the maven-plugin-testing-harness.
>
> I was hoping to get another set of eyes on my work, namely the pom and the unit test that is in flight (see the above repository), such that I can get around these classpath issues and start writing proper tests for the plugin. It is quite easy to see the issues that I’m having by cloning the project and running “mvn test:"
>
> Running org.apache.commons.release.plugin.mojos.CommonsSiteCompressionMojoTest
> Jan 06, 2018 2:00:13 PM org.eclipse.sisu.inject.Logs$JULSink warn
> WARNING: Error injecting: org.apache.maven.artifact.resolver.DefaultArtifactResolver
> java.lang.NoClassDefFoundError: Lorg/apache/maven/artifact/transform/ArtifactTransformationManager;
>         at java.lang.Class.getDeclaredFields0(Native Method)
>         at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
>         at java.lang.Class.getDeclaredFields(Class.java:1916)
>
> Any guidance here would be much welcomed, and moreover I can’t thank you guys enough for the previous insights because they gave me the ability to make solid progress on our release automation.
>
> Many thanks and all the best,
> -Rob
>
>
>> On Dec 28, 2017, at 4:53 PM, Rob Tompkins <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>
>>
>>> On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>> Hi Rob,
>>>
>>> one additional point: currently, for Maven itself, instead of adding new
>>> Maven-specific ReleasePhase(s) to the default configuration, or configure them in
>>> our parent pom (I'm not sure documentation on how to do that is available: I
>>> could not find it), we chose first to create a separate "dist-tool" to check
>>> consistency of what is currently published in misc places and provide some
>>> commands to fix when an inconsistency is found.
>>>
>>> This happens through daily reports done by a Jenkins job [1]:
>>> - distribution area vs Maven Central [2]
>>> - version from Maven site vs Maven Central [3]
>>>
>>> We did not produce any release nor made it really configurable for conventions
>>> different from Maven ones (like Common's -src & -bin), but at least there is a
>>> configuration file to define artifacts to check [4]
>>
>> Interesting. Thanks,.
>>
>> -Rob
>>
>>>
>>> HTH
>>>
>>> Hervé
>>>
>>>
>>> [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/ <https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/>
>>>
>>> [2] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/ <https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/>
>>> dist-tool-check-source-release.html
>>>
>>> [3] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/ <https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/>
>>> dist-tool-check-index-page.html
>>>
>>> [4] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/ <https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/>
>>> dist-tool.conf.html
>>>
>>> Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
>>>> Stephen,
>>>>
>>>> I can’t thank you enough for your reply. I’ll take your suggestions and
>>>> continue to sandbox around using the maven-release-plugin as a guideline.
>>>>
>>>> All the best and happy holidays,
>>>> -Rob
>>>>
>>>>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
>>>>> <[hidden email] <mailto:[hidden email]>> wrote:>
>>>>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email] <mailto:[hidden email]>
>>> <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>>>>>> Hello all,
>>>>>>
>>>>>> Pardon, maybe this should have gone to your @user list, but why not ping
>>>>>> the dev crew. I’ve been playing around the ideas surrounding our fairly
>>>>>> manual release process for the components in Commons, and I was hoping
>>>>>> for
>>>>>> some insights.
>>>>>>
>>>>>> Scripting the version changes isn’t really that big of a deal for us, and
>>>>>> that I can manage. But, when it comes to publishing our artifacts to the
>>>>>> apache nexus repository, and then separately publishing our -src and -bin
>>>>>> assemblies to the dev dist subversion repository (and consequently
>>>>>> deleting
>>>>>> those artifacts from nexus as they’re “attached” for the purpose of gpg
>>>>>> signing), I feel it a tad cumbersome.
>>>>>>
>>>>>> I’ve fiddled around a little with the idea of detaching the -src and -bin
>>>>>> assemblies after gpg signing with some success, but then I have to delve
>>>>>> into the mechanics of publishing those up to the subversion repository,
>>>>>> and
>>>>>> clearly that problem has already been solved.
>>>>>
>>>>> Is your problem you don’t want those going to Nexus staging but only to
>>>>> dist? Or is it that you want them *also* going to dist.
>>>>>
>>>>> Personally... I see no reason to remove them from going to Nexus staging
>>>>> (in fact I have a background plan to add secondary signing support to
>>>>> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
>>>>> though. That would mean that the PMC would be able to *add* their GPG
>>>>> signature to the staged artifacts as part of the voting... in which case
>>>>> you’d want to hold off uploading to dist until *after* the vote so you get
>>>>> the full set of signatures)
>>>>>
>>>>> If you want to upload a subset of attached artifacts to dist as part of
>>>>> the
>>>>> release, that seems much more tenable... just a derivative of the
>>>>> scm-publish plugin from what I can see.
>>>>>
>>>>> So I find myself in the space of trying to shoehorn our process into its
>>>>>
>>>>>> the main maven-release-plugin, which I’ve found a tad difficult, versus
>>>>>> writing our own release plugin, which feels like I would be duplicating
>>>>>> tons of code (which I don’t want to do).
>>>>>
>>>>> So the release plugin is really two parts:
>>>>>
>>>>> 1. A toolkit for writing release plugins
>>>>>
>>>>> 2. An example that does the job for the requirements of the Maven TLP and
>>>>> has seemed “sufficient” for a lot of other people.
>>>>>
>>>>> As such, if you have different needs, do not feel bad about having to
>>>>> encode differently... hopefully the toolkit half of the codebase is
>>>>> sufficient for you.
>>>>>
>>>>>> I’m curious if you guys have any thoughts on the matter as I’ve been
>>>>>> playing around in the space for a little while now.
>>>>>>
>>>>>> Cheers and happy holidays from UTC-5,
>>>>>> -Rob
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
>>>>>> <mailto:[hidden email] <mailto:[hidden email]>> For additional commands,
>>>>>> e-mail: [hidden email] <mailto:[hidden email]> <mailto:[hidden email] <mailto:[hidden email]>>
>>>>>>
>>>>>> --
>>>>>
>>>>> Sent from my phone
>