[VFS] developer ping - future direction

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[VFS] developer ping - future direction

Bernd Eckenfels
Dear [VFS] Developer and Contributors,

Please excuse the spam (bcc to all emails mentioned as developers (8) or  
contributors (6) in the project pom).

The project is currently a bit in sleeping state. I raised a few concerns  
and questions on the commons-dev mailinglist and wanted to direct your  
attention to the list - it would also help me to provide more patches when  
I know how you would prefer to solve things I raised.

Could you maybe get ack to the list and let us know if you have any  
current plans/needs with/for VFS2 and if you had looked at the recent VFS  
discussions on the commons developer list.

Currently I am mostly concerend with concurrency and atomic transactions,  
but also some unclear API meanings, dirty (commented out) code and the  
unit test "suite" system is somewhat confusing to use if you want to write  
providers outside of the main archive (with a -tests.jar dependency only).

I have two new providers, one which allows to project VFS on Blobs in a  
JDBC table and one which simulates a virtual filesystem on top of  
git-style trees. (https://github.com/ecki/seeburger-vfs2). For the former  
I need to implement some atomicity and concurrency (for the content). And  
for the later some questions around injecting the datasource and having  
additional fs operations come to mind.

Besides that it might be time to think about  
java.nio.file.spi.FileSystemProvider as well.

If you feel fluent in one of the points mentioned let me know. (if too  
much discussion is coming out of that we can conser moving to an google  
group for those, but for now it is just a dream that that could happen :)

What would be the quickest way for me to get acccess to the VFS wiki?

Greetings
Bernd
--
http://www.zusammenkunft.net

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

Mark Fortner-3
Hi Bernd,
I'm a user rather than a developer of VFS.  I have a weekend project to
create a file browser in JavaFX. Prior to that I wrote an NFS provider for
a company I was working for.

Concurrency/thread safety should definitely be a priority.  Part of my
weekend project uses a threadpool to handle a variety of different types of
file operations.

If there was an adapter that could adapt VFS2 calls to NIO2 calls that
would be nice.  But I think that rewriting all of the current providers to
make them NIO2 providers would be a bit of a stretch.

In your previous posting about VFS-490 you mentioned not wanting to browse
JAR files as though they were directories.  It would be nice if that was
configurable -- I actually use that functionality and find it pretty useful.

Metadata support would also be useful:
http://apache-commons.680414.n4.nabble.com/vfs-File-Metadata-td739249.html

Plugin download support would also be useful.  One of the current problems
that most users face is that the available file systems aren't downloadable
from a single point.  So if you encounter a URL that requires a different
provider, you have to hunt around for the provider and manually install it.
 It would be useful VFS could simply check a registry of providers, and
download the appropriate provider.  It would keep the core download fairly
small and make it easy for 3rd party providers to be used.

The biggest challenge for most 3rd party developers is finding a way to do
automated testing without installing a file system to test with.  It would
be interesting to find out how the NIO2 file system developers are testing
their providers.

Anyway, his is all probably more than you were asking for, but I hope this
helps.

Cheers,

Mark



On Wed, Oct 9, 2013 at 10:38 AM, Bernd Eckenfels <[hidden email]>wrote:

> Dear [VFS] Developer and Contributors,
>
> Please excuse the spam (bcc to all emails mentioned as developers (8) or
> contributors (6) in the project pom).
>
> The project is currently a bit in sleeping state. I raised a few concerns
> and questions on the commons-dev mailinglist and wanted to direct your
> attention to the list - it would also help me to provide more patches when
> I know how you would prefer to solve things I raised.
>
> Could you maybe get ack to the list and let us know if you have any
> current plans/needs with/for VFS2 and if you had looked at the recent VFS
> discussions on the commons developer list.
>
> Currently I am mostly concerend with concurrency and atomic transactions,
> but also some unclear API meanings, dirty (commented out) code and the unit
> test "suite" system is somewhat confusing to use if you want to write
> providers outside of the main archive (with a -tests.jar dependency only).
>
> I have two new providers, one which allows to project VFS on Blobs in a
> JDBC table and one which simulates a virtual filesystem on top of git-style
> trees. (https://github.com/ecki/**seeburger-vfs2<https://github.com/ecki/seeburger-vfs2>).
> For the former I need to implement some atomicity and concurrency (for the
> content). And for the later some questions around injecting the datasource
> and having additional fs operations come to mind.
>
> Besides that it might be time to think about java.nio.file.spi.**FileSystemProvider
> as well.
>
> If you feel fluent in one of the points mentioned let me know. (if too
> much discussion is coming out of that we can conser moving to an google
> group for those, but for now it is just a dream that that could happen :)
>
> What would be the quickest way for me to get acccess to the VFS wiki?
>
> Greetings
> Bernd
> --
> http://www.zusammenkunft.net
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@commons.**apache.org<[hidden email]>
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VFS] .jar named directories VFS-490

Bernd Eckenfels
Am 09.10.2013, 21:53 Uhr, schrieb Mark Fortner <[hidden email]>:

> In your previous posting about VFS-490 you mentioned not wanting to  
> browse
> JAR files as though they were directories.  It would be nice if that was
> configurable -- I actually use that functionality and find it pretty  
> useful.

Hm... no I do want to browse JAR files (or actually the VFSClassLoader  
wants to), but it tries to browse directories which are named name.jar/  
with the JAR overlay - which does not work. The opening of JAR files as  
filesystems is of course a usefull feature :)

I have seen Gary put some work into it, I have'nt been able to replicate  
his problems, yet.

Greetings
Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

Bernd Eckenfels
In reply to this post by Mark Fortner-3
Am 09.10.2013, 21:53 Uhr, schrieb Mark Fortner <[hidden email]>:

> Hi Bernd,
> I'm a user rather than a developer of VFS.  I have a weekend project to
> create a file browser in JavaFX. Prior to that I wrote an NFS provider  
> for
> a company I was working for.

Thats nice, is it available somewhere?

> Concurrency/thread safety should definitely be a priority.  Part of my
> weekend project uses a threadpool to handle a variety of different types  
> of file operations.
>
> If there was an adapter that could adapt VFS2 calls to NIO2 calls that
> would be nice.  But I think that rewriting all of the current providers  
> to
> make them NIO2 providers would be a bit of a stretch.

Yes, I think one of the first things would be to have an  
FileSystemProvider Adapter which can present a VFS root to NIO.2.


...
> Metadata support would also be useful:
> http://apache-commons.680414.n4.nabble.com/vfs-File-Metadata-td739249.html

I was thinking this could be done with file attributes. Maybe define a  
fixed set of well known attributes like "file.lastaccessed" "file.mode"  
etc. But the discussed approach with own interfaces is also an option.

> Plugin download support would also be useful.  One of the current  
> problems
> that most users face is that the available file systems aren't  
> downloadable
> from a single point.  So if you encounter a URL that requires a different
> provider, you have to hunt around for the provider and manually install  
> it.

Yes, maybe in combination with dependency injection or OSGi...

>  It would be useful VFS could simply check a registry of providers, and
> download the appropriate provider.  It would keep the core download  
> fairly
> small and make it easy for 3rd party providers to be used.

I would not do that in the VFS core, but a nice thing would be to have an  
listener which gets requests for schemes/filetypes and can do the provider  
registering. You can then provide a "RegistryResolver" listener who can be  
used in interactive projects.

>
> The biggest challenge for most 3rd party developers is finding a way to  
> do
> automated testing without installing a file system to test with.  It  
> would
> be interesting to find out how the NIO2 file system developers are  
> testing
> their providers.

With the JDBC provider I have used H2 and Derby, that worked fine (however  
I had to copy the test resources into my project which I want to avoid).  
With the digest archive filesystem I have the problem that the filesystem  
behaves rather untypical so I am not sure if the capability based approach  
to configure the tests to run will be enough - but yes thats a topic I  
will open up a seperate thread later.

> Anyway, his is all probably more than you were asking for, but I hope  
> this
> helps.

Thanks for the feedback :)

Bernd

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

Mark Fortner-3
On Wed, Oct 9, 2013 at 2:05 PM, Bernd Eckenfels <[hidden email]>wrote:

> Am 09.10.2013, 21:53 Uhr, schrieb Mark Fortner <[hidden email]>:
>
>
>  Hi Bernd,
>> I'm a user rather than a developer of VFS.  I have a weekend project to
>> create a file browser in JavaFX. Prior to that I wrote an NFS provider for
>> a company I was working for.
>>
>
> Thats nice, is it available somewhere?


It was proprietary code for the company I worked for at the time.  I don't
have access to the code any longer, but it was based on Sun/Oracle's YANFS
implementation and wasn't that difficult to write.   YANFS uses BSD now (it
used to be CDDL).  I'm not sure if the licenses are compatible.

Part of the reason that I'd like to have some way of resolving 3rd party
providers, is that the user would be able to get the provider, regardless
of the flavor of open source license used.

Cheers,

Mark
Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

Jörg Schaible
Mark Fortner wrote:

[snip]

> It was proprietary code for the company I worked for at the time.  I don't
> have access to the code any longer, but it was based on Sun/Oracle's YANFS
> implementation and wasn't that difficult to write.   YANFS uses BSD now
> (it
> used to be CDDL).  I'm not sure if the licenses are compatible.
>
> Part of the reason that I'd like to have some way of resolving 3rd party
> providers, is that the user would be able to get the provider, regardless
> of the flavor of open source license used.

This has always been possible: http://commons.apache.org/proper/commons-vfs/api.html#Configuring_Commons_VFS

- Jörg


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

Mark Fortner-3
Hi Jörg,
Thanks for the link, but I don't think that quite addresses the issue.

What I'd like to see is the ability to automatically install a VFS provider
that isn't already on your local machine.  For example, if I wanted to see
a directory listing of "sftp://someserver/somedir" and I don't already have
the SFTP provider installed, VFS would download and install it for me.
 This is similar to the way in which Groovy's @Grab annotation
automatically installs dependencies when you attempt to run a script whose
dependencies you don't have.

Cheers,

Mark



On Wed, Oct 9, 2013 at 4:50 PM, Jörg Schaible <[hidden email]> wrote:

> Mark Fortner wrote:
>
> [snip]
>
> > It was proprietary code for the company I worked for at the time.  I
> don't
> > have access to the code any longer, but it was based on Sun/Oracle's
> YANFS
> > implementation and wasn't that difficult to write.   YANFS uses BSD now
> > (it
> > used to be CDDL).  I'm not sure if the licenses are compatible.
> >
> > Part of the reason that I'd like to have some way of resolving 3rd party
> > providers, is that the user would be able to get the provider, regardless
> > of the flavor of open source license used.
>
> This has always been possible:
> http://commons.apache.org/proper/commons-vfs/api.html#Configuring_Commons_VFS
>
> - Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

garydgregory
On Wed, Oct 9, 2013 at 9:40 PM, Mark Fortner <[hidden email]> wrote:

> Hi Jörg,
> Thanks for the link, but I don't think that quite addresses the issue.
>
> What I'd like to see is the ability to automatically install a VFS provider
> that isn't already on your local machine.  For example, if I wanted to see
> a directory listing of "sftp://someserver/somedir" and I don't already have
> the SFTP provider installed, VFS would download and install it for me.
>  This is similar to the way in which Groovy's @Grab annotation
> automatically installs dependencies when you attempt to run a script whose
> dependencies you don't have.

But why bother? Providers are small, unless they depend on a pile of
other jars. Then how would you deal with that? Would VFS then have to
depend on Maven, Ivy or Aether for transitive dependency management.
It sounds out of scope for VFS to me, but it could be an interesting
standalone project that other projects could depend on...

Gary

>
> Cheers,
>
> Mark
>
>
>
> On Wed, Oct 9, 2013 at 4:50 PM, Jörg Schaible <[hidden email]> wrote:
>
>> Mark Fortner wrote:
>>
>> [snip]
>>
>> > It was proprietary code for the company I worked for at the time.  I
>> don't
>> > have access to the code any longer, but it was based on Sun/Oracle's
>> YANFS
>> > implementation and wasn't that difficult to write.   YANFS uses BSD now
>> > (it
>> > used to be CDDL).  I'm not sure if the licenses are compatible.
>> >
>> > Part of the reason that I'd like to have some way of resolving 3rd party
>> > providers, is that the user would be able to get the provider, regardless
>> > of the flavor of open source license used.
>>
>> This has always been possible:
>> http://commons.apache.org/proper/commons-vfs/api.html#Configuring_Commons_VFS
>>
>> - Jörg
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>



--
E-Mail: [hidden email] | [hidden email]
Java Persistence with Hibernate, Second Edition
JUnit in Action, Second Edition
Spring Batch in Action
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

James Carman
You would need metadata like we have in OSGi land in order to piece
everything together reliably.  A dynamically growing OSGi environment would
be cool.

On Wednesday, October 9, 2013, Gary Gregory wrote:

> On Wed, Oct 9, 2013 at 9:40 PM, Mark Fortner <[hidden email]<javascript:;>>
> wrote:
> > Hi Jörg,
> > Thanks for the link, but I don't think that quite addresses the issue.
> >
> > What I'd like to see is the ability to automatically install a VFS
> provider
> > that isn't already on your local machine.  For example, if I wanted to
> see
> > a directory listing of "sftp://someserver/somedir" and I don't already
> have
> > the SFTP provider installed, VFS would download and install it for me.
> >  This is similar to the way in which Groovy's @Grab annotation
> > automatically installs dependencies when you attempt to run a script
> whose
> > dependencies you don't have.
>
> But why bother? Providers are small, unless they depend on a pile of
> other jars. Then how would you deal with that? Would VFS then have to
> depend on Maven, Ivy or Aether for transitive dependency management.
> It sounds out of scope for VFS to me, but it could be an interesting
> standalone project that other projects could depend on...
>
> Gary
>
> >
> > Cheers,
> >
> > Mark
> >
> >
> >
> > On Wed, Oct 9, 2013 at 4:50 PM, Jörg Schaible <[hidden email]<javascript:;>>
> wrote:
> >
> >> Mark Fortner wrote:
> >>
> >> [snip]
> >>
> >> > It was proprietary code for the company I worked for at the time.  I
> >> don't
> >> > have access to the code any longer, but it was based on Sun/Oracle's
> >> YANFS
> >> > implementation and wasn't that difficult to write.   YANFS uses BSD
> now
> >> > (it
> >> > used to be CDDL).  I'm not sure if the licenses are compatible.
> >> >
> >> > Part of the reason that I'd like to have some way of resolving 3rd
> party
> >> > providers, is that the user would be able to get the provider,
> regardless
> >> > of the flavor of open source license used.
> >>
> >> This has always been possible:
> >>
> http://commons.apache.org/proper/commons-vfs/api.html#Configuring_Commons_VFS
> >>
> >> - Jörg
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]<javascript:;>
> >> For additional commands, e-mail: [hidden email]<javascript:;>
> >>
> >>
>
>
>
> --
> E-Mail: [hidden email] <javascript:;> | [hidden email]<javascript:;>
> Java Persistence with Hibernate, Second Edition
> JUnit in Action, Second Edition
> Spring Batch in Action
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email] <javascript:;>
> For additional commands, e-mail: [hidden email]<javascript:;>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: [VFS] .jar named directories VFS-490

Jörg Schaible-3
In reply to this post by Bernd Eckenfels
Bernd Eckenfels wrote:

> Am 09.10.2013, 21:53 Uhr, schrieb Mark Fortner <[hidden email]>:
>
>> In your previous posting about VFS-490 you mentioned not wanting to
>> browse
>> JAR files as though they were directories.  It would be nice if that was
>> configurable -- I actually use that functionality and find it pretty
>> useful.
>
> Hm... no I do want to browse JAR files (or actually the VFSClassLoader
> wants to), but it tries to browse directories which are named name.jar/
> with the JAR overlay - which does not work. The opening of JAR files as
> filesystems is of course a usefull feature :)
>
> I have seen Gary put some work into it, I have'nt been able to replicate
> his problems, yet.

Hehe, I wonder if something similar happens for other schemes following the
pattern name.scheme pattern e.g. ./file.url/ ;-)

- Jörg


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] developer ping - future direction

Ralph Goers
In reply to this post by Bernd Eckenfels
I have been working primarily on Log4j 2 for a while.  I have always planned to come back to start work on VFS 3 that will integrate with Java 7's java.nio.file support.

Ralph


On Oct 9, 2013, at 10:38 AM, Bernd Eckenfels <[hidden email]> wrote:

> Dear [VFS] Developer and Contributors,
>
> Please excuse the spam (bcc to all emails mentioned as developers (8) or contributors (6) in the project pom).
>
> The project is currently a bit in sleeping state. I raised a few concerns and questions on the commons-dev mailinglist and wanted to direct your attention to the list - it would also help me to provide more patches when I know how you would prefer to solve things I raised.
>
> Could you maybe get ack to the list and let us know if you have any current plans/needs with/for VFS2 and if you had looked at the recent VFS discussions on the commons developer list.
>
> Currently I am mostly concerend with concurrency and atomic transactions, but also some unclear API meanings, dirty (commented out) code and the unit test "suite" system is somewhat confusing to use if you want to write providers outside of the main archive (with a -tests.jar dependency only).
>
> I have two new providers, one which allows to project VFS on Blobs in a JDBC table and one which simulates a virtual filesystem on top of git-style trees. (https://github.com/ecki/seeburger-vfs2). For the former I need to implement some atomicity and concurrency (for the content). And for the later some questions around injecting the datasource and having additional fs operations come to mind.
>
> Besides that it might be time to think about java.nio.file.spi.FileSystemProvider as well.
>
> If you feel fluent in one of the points mentioned let me know. (if too much discussion is coming out of that we can conser moving to an google group for those, but for now it is just a dream that that could happen :)
>
> What would be the quickest way for me to get acccess to the VFS wiki?
>
> Greetings
> Bernd
> --
> http://www.zusammenkunft.net
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]