[crypto] Interest in adding support for cryptographic hash function?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[crypto] Interest in adding support for cryptographic hash function?

Alex Remily
I'm working on a project that makes heavy use of hashing, and I'd like to
use the OpenSSL implementation.  Thoughts on adding support for the SHA-2
and/or Blake2 family into commons crypto?  I'm happy to do the work if
there's someone out there willing to review and advise.

Alex
Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Gilles Sadowski-2
Hi.

Le sam. 27 févr. 2021 à 15:51, Alex Remily <[hidden email]> a écrit :
>
> I'm working on a project that makes heavy use of hashing, and I'd like to
> use the OpenSSL implementation.  Thoughts on adding support for the SHA-2

http://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/digest/Sha2Crypt.html

> and/or Blake2 family into commons crypto?

[Codec] seems to be the appropriate component.

> I'm happy to do the work

Thanks!

> if
> there's someone out there willing to review and advise.
>
> Alex

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

garydgregory
See also
https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/digest/DigestUtils.html

Gary


On Sat, Feb 27, 2021, 09:58 Gilles Sadowski <[hidden email]> wrote:

> Hi.
>
> Le sam. 27 févr. 2021 à 15:51, Alex Remily <[hidden email]> a
> écrit :
> >
> > I'm working on a project that makes heavy use of hashing, and I'd like to
> > use the OpenSSL implementation.  Thoughts on adding support for the SHA-2
>
>
> http://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/digest/Sha2Crypt.html
>
> > and/or Blake2 family into commons crypto?
>
> [Codec] seems to be the appropriate component.
>
> > I'm happy to do the work
>
> Thanks!
>
> > if
> > there's someone out there willing to review and advise.
> >
> > Alex
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Bernd Eckenfels
Hello,

I don’t think it’s a Good idea to introduce native dependencies to formerly pure Java projects. So i think native optimized hash implementations would fit better in commons-crypto. So I would say go for it, keep in mind license clearance and portability.

Gruß
Bernd

--
https://Bernd.eckenfels.net
________________________________
From: Gary Gregory <[hidden email]>
Sent: Saturday, February 27, 2021 6:45:12 PM
To: Commons Developers List <[hidden email]>
Subject: Re: [crypto] Interest in adding support for cryptographic hash function?

See also
https://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/digest/DigestUtils.html

Gary


On Sat, Feb 27, 2021, 09:58 Gilles Sadowski <[hidden email]> wrote:

> Hi.
>
> Le sam. 27 févr. 2021 à 15:51, Alex Remily <[hidden email]> a
> écrit :
> >
> > I'm working on a project that makes heavy use of hashing, and I'd like to
> > use the OpenSSL implementation.  Thoughts on adding support for the SHA-2
>
>
> http://commons.apache.org/proper/commons-codec/apidocs/org/apache/commons/codec/digest/Sha2Crypt.html
>
> > and/or Blake2 family into commons crypto?
>
> [Codec] seems to be the appropriate component.
>
> > I'm happy to do the work
>
> Thanks!
>
> > if
> > there's someone out there willing to review and advise.
> >
> > Alex
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Gilles Sadowski-2
Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
<[hidden email]> a écrit :
>
> Hello,
>
> I don’t think it’s a Good idea to introduce native dependencies to formerly pure Java projects.

+1
[I thought that the idea was a (pure) Java implementation.]

> So i think native optimized hash implementations would fit better in commons-crypto. So I would say go for it, keep in mind license clearance and portability.
>
> Gruß
> Bernd
>
> > [...]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Alex Remily
I'd be exposing additional elements of the OpenSSL API via additions to the
commons crypto API.  Since I wouldn't be adding any additional
dependencies, I would expect that licensing and portability would remain
unchanged.  Would it not?

Thanks,

Alex

On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <[hidden email]>
wrote:

> Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> <[hidden email]> a écrit :
> >
> > Hello,
> >
> > I don’t think it’s a Good idea to introduce native dependencies to
> formerly pure Java projects.
>
> +1
> [I thought that the idea was a (pure) Java implementation.]
>
> > So i think native optimized hash implementations would fit better in
> commons-crypto. So I would say go for it, keep in mind license clearance
> and portability.
> >
> > Gruß
> > Bernd
> >
> > > [...]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

garydgregory
That sounds reasonable to me. I think once we see a PR, we'll get a better
idea. Start small IMO.

Gary


On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]> wrote:

> I'd be exposing additional elements of the OpenSSL API via additions to the
> commons crypto API.  Since I wouldn't be adding any additional
> dependencies, I would expect that licensing and portability would remain
> unchanged.  Would it not?
>
> Thanks,
>
> Alex
>
> On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <[hidden email]>
> wrote:
>
> > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > <[hidden email]> a écrit :
> > >
> > > Hello,
> > >
> > > I don’t think it’s a Good idea to introduce native dependencies to
> > formerly pure Java projects.
> >
> > +1
> > [I thought that the idea was a (pure) Java implementation.]
> >
> > > So i think native optimized hash implementations would fit better in
> > commons-crypto. So I would say go for it, keep in mind license clearance
> > and portability.
> > >
> > > Gruß
> > > Bernd
> > >
> > > > [...]
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Bernd Eckenfels
... and also do benchmarking early, the native interface overhead might be a problem so that pure java (intrinsic) code is much faster


--
http://bernd.eckenfels.net
________________________________
Von: Gary Gregory <[hidden email]>
Gesendet: Sunday, February 28, 2021 2:45:26 PM
An: Commons Developers List <[hidden email]>
Betreff: Re: [crypto] Interest in adding support for cryptographic hash function?

That sounds reasonable to me. I think once we see a PR, we'll get a better
idea. Start small IMO.

Gary


On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]> wrote:

> I'd be exposing additional elements of the OpenSSL API via additions to the
> commons crypto API.  Since I wouldn't be adding any additional
> dependencies, I would expect that licensing and portability would remain
> unchanged.  Would it not?
>
> Thanks,
>
> Alex
>
> On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <[hidden email]>
> wrote:
>
> > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > <[hidden email]> a écrit :
> > >
> > > Hello,
> > >
> > > I don’t think it’s a Good idea to introduce native dependencies to
> > formerly pure Java projects.
> >
> > +1
> > [I thought that the idea was a (pure) Java implementation.]
> >
> > > So i think native optimized hash implementations would fit better in
> > commons-crypto. So I would say go for it, keep in mind license clearance
> > and portability.
> > >
> > > Gruß
> > > Bernd
> > >
> > > > [...]
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Matt Sicker
I'd also be interested in benchmarking comparisons as I've been
working on a proof of concept using Blake3 to do similarly (I have a
pure Java implementation and a JNI version that ultimately invokes the
reference C implementation, though I've also wondered about linking
the reference Rust implementation). The potential advantage to linking
in the native implementations here would be in taking advantage of
CPU-specific extensions like SSE, AVX, NEON, etc., which aren't
accessible from the JVM without JNI or other patches to the JDK
itself. If the overhead turns out to be non-negligible, then we should
probably keep the native code bindings to commons-crypto while porting
pure Java implementations to commons-codec.

On Sun, 28 Feb 2021 at 09:25, Bernd Eckenfels <[hidden email]> wrote:

>
> ... and also do benchmarking early, the native interface overhead might be a problem so that pure java (intrinsic) code is much faster
>
>
> --
> http://bernd.eckenfels.net
> ________________________________
> Von: Gary Gregory <[hidden email]>
> Gesendet: Sunday, February 28, 2021 2:45:26 PM
> An: Commons Developers List <[hidden email]>
> Betreff: Re: [crypto] Interest in adding support for cryptographic hash function?
>
> That sounds reasonable to me. I think once we see a PR, we'll get a better
> idea. Start small IMO.
>
> Gary
>
>
> On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]> wrote:
>
> > I'd be exposing additional elements of the OpenSSL API via additions to the
> > commons crypto API.  Since I wouldn't be adding any additional
> > dependencies, I would expect that licensing and portability would remain
> > unchanged.  Would it not?
> >
> > Thanks,
> >
> > Alex
> >
> > On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <[hidden email]>
> > wrote:
> >
> > > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > > <[hidden email]> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > I don’t think it’s a Good idea to introduce native dependencies to
> > > formerly pure Java projects.
> > >
> > > +1
> > > [I thought that the idea was a (pure) Java implementation.]
> > >
> > > > So i think native optimized hash implementations would fit better in
> > > commons-crypto. So I would say go for it, keep in mind license clearance
> > > and portability.
> > > >
> > > > Gruß
> > > > Bernd
> > > >
> > > > > [...]
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >
> > >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Matt Sicker
For those who are unfamiliar with the Blake family of hash functions,
Blake3 is essentially an updated version of Blake2s that unifies the
other Blake2* variants into a single API to make things easier for
developers and users. Blake2 has several variants, 2b and 2s being the
most commonly used (blake2b is the default hash used in libsodium, a
popular C crypto library). Some links:

https://github.com/BLAKE3-team/BLAKE3
https://www.blake2.net/

Specs:
https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blake3.pdf

On Sun, 28 Feb 2021 at 14:14, Matt Sicker <[hidden email]> wrote:

>
> I'd also be interested in benchmarking comparisons as I've been
> working on a proof of concept using Blake3 to do similarly (I have a
> pure Java implementation and a JNI version that ultimately invokes the
> reference C implementation, though I've also wondered about linking
> the reference Rust implementation). The potential advantage to linking
> in the native implementations here would be in taking advantage of
> CPU-specific extensions like SSE, AVX, NEON, etc., which aren't
> accessible from the JVM without JNI or other patches to the JDK
> itself. If the overhead turns out to be non-negligible, then we should
> probably keep the native code bindings to commons-crypto while porting
> pure Java implementations to commons-codec.
>
> On Sun, 28 Feb 2021 at 09:25, Bernd Eckenfels <[hidden email]> wrote:
> >
> > ... and also do benchmarking early, the native interface overhead might be a problem so that pure java (intrinsic) code is much faster
> >
> >
> > --
> > http://bernd.eckenfels.net
> > ________________________________
> > Von: Gary Gregory <[hidden email]>
> > Gesendet: Sunday, February 28, 2021 2:45:26 PM
> > An: Commons Developers List <[hidden email]>
> > Betreff: Re: [crypto] Interest in adding support for cryptographic hash function?
> >
> > That sounds reasonable to me. I think once we see a PR, we'll get a better
> > idea. Start small IMO.
> >
> > Gary
> >
> >
> > On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]> wrote:
> >
> > > I'd be exposing additional elements of the OpenSSL API via additions to the
> > > commons crypto API.  Since I wouldn't be adding any additional
> > > dependencies, I would expect that licensing and portability would remain
> > > unchanged.  Would it not?
> > >
> > > Thanks,
> > >
> > > Alex
> > >
> > > On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <[hidden email]>
> > > wrote:
> > >
> > > > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > > > <[hidden email]> a écrit :
> > > > >
> > > > > Hello,
> > > > >
> > > > > I don’t think it’s a Good idea to introduce native dependencies to
> > > > formerly pure Java projects.
> > > >
> > > > +1
> > > > [I thought that the idea was a (pure) Java implementation.]
> > > >
> > > > > So i think native optimized hash implementations would fit better in
> > > > commons-crypto. So I would say go for it, keep in mind license clearance
> > > > and portability.
> > > > >
> > > > > Gruß
> > > > > Bernd
> > > > >
> > > > > > [...]
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > > >
> > > >
> > >

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

sebb-2-2
In reply to this post by Matt Sicker
On Sun, 28 Feb 2021 at 20:14, Matt Sicker <[hidden email]> wrote:

>
> I'd also be interested in benchmarking comparisons as I've been
> working on a proof of concept using Blake3 to do similarly (I have a
> pure Java implementation and a JNI version that ultimately invokes the
> reference C implementation, though I've also wondered about linking
> the reference Rust implementation). The potential advantage to linking
> in the native implementations here would be in taking advantage of
> CPU-specific extensions like SSE, AVX, NEON, etc., which aren't
> accessible from the JVM without JNI or other patches to the JDK
> itself. If the overhead turns out to be non-negligible, then we should
> probably keep the native code bindings to commons-crypto while porting
> pure Java implementations to commons-codec.

Of course the disadvantage is that the code is not portable.
Also each OS version will need separate testing and makes releasing
binaries harder.
This is an issue we have already faced with Crypto.
And there will be versions of Java for which it will not work.

So unless the is a great performance improvement, I don't think it's
worth the extra overheads.

> On Sun, 28 Feb 2021 at 09:25, Bernd Eckenfels <[hidden email]> wrote:
> >
> > ... and also do benchmarking early, the native interface overhead might be a problem so that pure java (intrinsic) code is much faster
> >
> >
> > --
> > http://bernd.eckenfels.net
> > ________________________________
> > Von: Gary Gregory <[hidden email]>
> > Gesendet: Sunday, February 28, 2021 2:45:26 PM
> > An: Commons Developers List <[hidden email]>
> > Betreff: Re: [crypto] Interest in adding support for cryptographic hash function?
> >
> > That sounds reasonable to me. I think once we see a PR, we'll get a better
> > idea. Start small IMO.
> >
> > Gary
> >
> >
> > On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]> wrote:
> >
> > > I'd be exposing additional elements of the OpenSSL API via additions to the
> > > commons crypto API.  Since I wouldn't be adding any additional
> > > dependencies, I would expect that licensing and portability would remain
> > > unchanged.  Would it not?
> > >
> > > Thanks,
> > >
> > > Alex
> > >
> > > On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <[hidden email]>
> > > wrote:
> > >
> > > > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > > > <[hidden email]> a écrit :
> > > > >
> > > > > Hello,
> > > > >
> > > > > I don’t think it’s a Good idea to introduce native dependencies to
> > > > formerly pure Java projects.
> > > >
> > > > +1
> > > > [I thought that the idea was a (pure) Java implementation.]
> > > >
> > > > > So i think native optimized hash implementations would fit better in
> > > > commons-crypto. So I would say go for it, keep in mind license clearance
> > > > and portability.
> > > > >
> > > > > Gruß
> > > > > Bernd
> > > > >
> > > > > > [...]
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > > >
> > > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Matt Sicker
That's why I'm interested in proper benchmarks before supporting a
release of something with platform-specific code. The CPU extensions
are ostensibly for optimizing the code where possible (the Blake3 code
has a dynamic dispatch mechanism for checking CPU features at runtime
to select the assembler-optimized variants), but if JNI overhead
negates the gains there, then I'd agree that sticking to pure Java
code here would be optimal.

On Sun, 28 Feb 2021 at 17:18, sebb <[hidden email]> wrote:

>
> On Sun, 28 Feb 2021 at 20:14, Matt Sicker <[hidden email]> wrote:
> >
> > I'd also be interested in benchmarking comparisons as I've been
> > working on a proof of concept using Blake3 to do similarly (I have a
> > pure Java implementation and a JNI version that ultimately invokes the
> > reference C implementation, though I've also wondered about linking
> > the reference Rust implementation). The potential advantage to linking
> > in the native implementations here would be in taking advantage of
> > CPU-specific extensions like SSE, AVX, NEON, etc., which aren't
> > accessible from the JVM without JNI or other patches to the JDK
> > itself. If the overhead turns out to be non-negligible, then we should
> > probably keep the native code bindings to commons-crypto while porting
> > pure Java implementations to commons-codec.
>
> Of course the disadvantage is that the code is not portable.
> Also each OS version will need separate testing and makes releasing
> binaries harder.
> This is an issue we have already faced with Crypto.
> And there will be versions of Java for which it will not work.
>
> So unless the is a great performance improvement, I don't think it's
> worth the extra overheads.
>
> > On Sun, 28 Feb 2021 at 09:25, Bernd Eckenfels <[hidden email]> wrote:
> > >
> > > ... and also do benchmarking early, the native interface overhead might be a problem so that pure java (intrinsic) code is much faster
> > >
> > >
> > > --
> > > http://bernd.eckenfels.net
> > > ________________________________
> > > Von: Gary Gregory <[hidden email]>
> > > Gesendet: Sunday, February 28, 2021 2:45:26 PM
> > > An: Commons Developers List <[hidden email]>
> > > Betreff: Re: [crypto] Interest in adding support for cryptographic hash function?
> > >
> > > That sounds reasonable to me. I think once we see a PR, we'll get a better
> > > idea. Start small IMO.
> > >
> > > Gary
> > >
> > >
> > > On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]> wrote:
> > >
> > > > I'd be exposing additional elements of the OpenSSL API via additions to the
> > > > commons crypto API.  Since I wouldn't be adding any additional
> > > > dependencies, I would expect that licensing and portability would remain
> > > > unchanged.  Would it not?
> > > >
> > > > Thanks,
> > > >
> > > > Alex
> > > >
> > > > On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <[hidden email]>
> > > > wrote:
> > > >
> > > > > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > > > > <[hidden email]> a écrit :
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I don’t think it’s a Good idea to introduce native dependencies to
> > > > > formerly pure Java projects.
> > > > >
> > > > > +1
> > > > > [I thought that the idea was a (pure) Java implementation.]
> > > > >
> > > > > > So i think native optimized hash implementations would fit better in
> > > > > commons-crypto. So I would say go for it, keep in mind license clearance
> > > > > and portability.
> > > > > >
> > > > > > Gruß
> > > > > > Bernd
> > > > > >
> > > > > > > [...]
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: [hidden email]
> > > > > For additional commands, e-mail: [hidden email]
> > > > >
> > > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Alex Remily
I echo all sentiments regarding benchmarking.  The deck at the link is a
bit dated, but it contains some benchmarking of the existing commons
crypto implementation against the JCE on JDK 8 and 9 that provides reason
for optimism.  Even if more recent versions of the JCE significantly narrow
the performance gap, which remains to be seen, I think commons crypto will
endure for the foreseeable future for users who need the performance and
can't or won't upgrade.

https://events.static.linuxfound.org/sites/events/files/slides/apache-commons-crypto-another-wheel.pdf

Regarding portability, the JNI libraries for each supported OS distribution
are bundled with the JAR and transparent to the caller of the commons
crypto api.  Yes, that makes it a hassle to build a release, but users need
only have libcrypto installed locally.



On Sun, Feb 28, 2021 at 6:21 PM Matt Sicker <[hidden email]> wrote:

> That's why I'm interested in proper benchmarks before supporting a
> release of something with platform-specific code. The CPU extensions
> are ostensibly for optimizing the code where possible (the Blake3 code
> has a dynamic dispatch mechanism for checking CPU features at runtime
> to select the assembler-optimized variants), but if JNI overhead
> negates the gains there, then I'd agree that sticking to pure Java
> code here would be optimal.
>
> On Sun, 28 Feb 2021 at 17:18, sebb <[hidden email]> wrote:
> >
> > On Sun, 28 Feb 2021 at 20:14, Matt Sicker <[hidden email]> wrote:
> > >
> > > I'd also be interested in benchmarking comparisons as I've been
> > > working on a proof of concept using Blake3 to do similarly (I have a
> > > pure Java implementation and a JNI version that ultimately invokes the
> > > reference C implementation, though I've also wondered about linking
> > > the reference Rust implementation). The potential advantage to linking
> > > in the native implementations here would be in taking advantage of
> > > CPU-specific extensions like SSE, AVX, NEON, etc., which aren't
> > > accessible from the JVM without JNI or other patches to the JDK
> > > itself. If the overhead turns out to be non-negligible, then we should
> > > probably keep the native code bindings to commons-crypto while porting
> > > pure Java implementations to commons-codec.
> >
> > Of course the disadvantage is that the code is not portable.
> > Also each OS version will need separate testing and makes releasing
> > binaries harder.
> > This is an issue we have already faced with Crypto.
> > And there will be versions of Java for which it will not work.
> >
> > So unless the is a great performance improvement, I don't think it's
> > worth the extra overheads.
> >
> > > On Sun, 28 Feb 2021 at 09:25, Bernd Eckenfels <[hidden email]>
> wrote:
> > > >
> > > > ... and also do benchmarking early, the native interface overhead
> might be a problem so that pure java (intrinsic) code is much faster
> > > >
> > > >
> > > > --
> > > > http://bernd.eckenfels.net
> > > > ________________________________
> > > > Von: Gary Gregory <[hidden email]>
> > > > Gesendet: Sunday, February 28, 2021 2:45:26 PM
> > > > An: Commons Developers List <[hidden email]>
> > > > Betreff: Re: [crypto] Interest in adding support for cryptographic
> hash function?
> > > >
> > > > That sounds reasonable to me. I think once we see a PR, we'll get a
> better
> > > > idea. Start small IMO.
> > > >
> > > > Gary
> > > >
> > > >
> > > > On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]>
> wrote:
> > > >
> > > > > I'd be exposing additional elements of the OpenSSL API via
> additions to the
> > > > > commons crypto API.  Since I wouldn't be adding any additional
> > > > > dependencies, I would expect that licensing and portability would
> remain
> > > > > unchanged.  Would it not?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Alex
> > > > >
> > > > > On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <
> [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > > > > > <[hidden email]> a écrit :
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I don’t think it’s a Good idea to introduce native
> dependencies to
> > > > > > formerly pure Java projects.
> > > > > >
> > > > > > +1
> > > > > > [I thought that the idea was a (pure) Java implementation.]
> > > > > >
> > > > > > > So i think native optimized hash implementations would fit
> better in
> > > > > > commons-crypto. So I would say go for it, keep in mind license
> clearance
> > > > > > and portability.
> > > > > > >
> > > > > > > Gruß
> > > > > > > Bernd
> > > > > > >
> > > > > > > > [...]
> > > > > >
> > > > > >
> ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: [hidden email]
> > > > > > For additional commands, e-mail: [hidden email]
> > > > > >
> > > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [crypto] Interest in adding support for cryptographic hash function?

Matt Sicker
Coming back to this thread since I started porting the Blake3 code.
Seeing the AES differences makes a lot of sense since most of the
software optimizations possible are insecure, and the only secure
software designs for AES have noticeable performance penalties
compared to AES intrinsics.

I'd still be interested to see what kind of improvements we can get
for other algorithms that can rely on CPU extensions similarly. I
recall seeing several on Intel's list [1] plus rdseed, etc..

[1]: https://software.intel.com/sites/landingpage/IntrinsicsGuide/#cats=Cryptography

On Sun, 28 Feb 2021 at 18:26, Alex Remily <[hidden email]> wrote:

>
> I echo all sentiments regarding benchmarking.  The deck at the link is a
> bit dated, but it contains some benchmarking of the existing commons
> crypto implementation against the JCE on JDK 8 and 9 that provides reason
> for optimism.  Even if more recent versions of the JCE significantly narrow
> the performance gap, which remains to be seen, I think commons crypto will
> endure for the foreseeable future for users who need the performance and
> can't or won't upgrade.
>
> https://events.static.linuxfound.org/sites/events/files/slides/apache-commons-crypto-another-wheel.pdf
>
> Regarding portability, the JNI libraries for each supported OS distribution
> are bundled with the JAR and transparent to the caller of the commons
> crypto api.  Yes, that makes it a hassle to build a release, but users need
> only have libcrypto installed locally.
>
>
>
> On Sun, Feb 28, 2021 at 6:21 PM Matt Sicker <[hidden email]> wrote:
>
> > That's why I'm interested in proper benchmarks before supporting a
> > release of something with platform-specific code. The CPU extensions
> > are ostensibly for optimizing the code where possible (the Blake3 code
> > has a dynamic dispatch mechanism for checking CPU features at runtime
> > to select the assembler-optimized variants), but if JNI overhead
> > negates the gains there, then I'd agree that sticking to pure Java
> > code here would be optimal.
> >
> > On Sun, 28 Feb 2021 at 17:18, sebb <[hidden email]> wrote:
> > >
> > > On Sun, 28 Feb 2021 at 20:14, Matt Sicker <[hidden email]> wrote:
> > > >
> > > > I'd also be interested in benchmarking comparisons as I've been
> > > > working on a proof of concept using Blake3 to do similarly (I have a
> > > > pure Java implementation and a JNI version that ultimately invokes the
> > > > reference C implementation, though I've also wondered about linking
> > > > the reference Rust implementation). The potential advantage to linking
> > > > in the native implementations here would be in taking advantage of
> > > > CPU-specific extensions like SSE, AVX, NEON, etc., which aren't
> > > > accessible from the JVM without JNI or other patches to the JDK
> > > > itself. If the overhead turns out to be non-negligible, then we should
> > > > probably keep the native code bindings to commons-crypto while porting
> > > > pure Java implementations to commons-codec.
> > >
> > > Of course the disadvantage is that the code is not portable.
> > > Also each OS version will need separate testing and makes releasing
> > > binaries harder.
> > > This is an issue we have already faced with Crypto.
> > > And there will be versions of Java for which it will not work.
> > >
> > > So unless the is a great performance improvement, I don't think it's
> > > worth the extra overheads.
> > >
> > > > On Sun, 28 Feb 2021 at 09:25, Bernd Eckenfels <[hidden email]>
> > wrote:
> > > > >
> > > > > ... and also do benchmarking early, the native interface overhead
> > might be a problem so that pure java (intrinsic) code is much faster
> > > > >
> > > > >
> > > > > --
> > > > > http://bernd.eckenfels.net
> > > > > ________________________________
> > > > > Von: Gary Gregory <[hidden email]>
> > > > > Gesendet: Sunday, February 28, 2021 2:45:26 PM
> > > > > An: Commons Developers List <[hidden email]>
> > > > > Betreff: Re: [crypto] Interest in adding support for cryptographic
> > hash function?
> > > > >
> > > > > That sounds reasonable to me. I think once we see a PR, we'll get a
> > better
> > > > > idea. Start small IMO.
> > > > >
> > > > > Gary
> > > > >
> > > > >
> > > > > On Sat, Feb 27, 2021, 13:51 Alex Remily <[hidden email]>
> > wrote:
> > > > >
> > > > > > I'd be exposing additional elements of the OpenSSL API via
> > additions to the
> > > > > > commons crypto API.  Since I wouldn't be adding any additional
> > > > > > dependencies, I would expect that licensing and portability would
> > remain
> > > > > > unchanged.  Would it not?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Alex
> > > > > >
> > > > > > On Sat, Feb 27, 2021 at 1:08 PM Gilles Sadowski <
> > [hidden email]>
> > > > > > wrote:
> > > > > >
> > > > > > > Le sam. 27 févr. 2021 à 19:00, Bernd Eckenfels
> > > > > > > <[hidden email]> a écrit :
> > > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > I don’t think it’s a Good idea to introduce native
> > dependencies to
> > > > > > > formerly pure Java projects.
> > > > > > >
> > > > > > > +1
> > > > > > > [I thought that the idea was a (pure) Java implementation.]
> > > > > > >
> > > > > > > > So i think native optimized hash implementations would fit
> > better in
> > > > > > > commons-crypto. So I would say go for it, keep in mind license
> > clearance
> > > > > > > and portability.
> > > > > > > >
> > > > > > > > Gruß
> > > > > > > > Bernd
> > > > > > > >
> > > > > > > > > [...]
> > > > > > >
> > > > > > >
> > ---------------------------------------------------------------------
> > > > > > > To unsubscribe, e-mail: [hidden email]
> > > > > > > For additional commands, e-mail: [hidden email]
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]