Commons VFS support for Amazon S3?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Commons VFS support for Amazon S3?

Liviu Tudor
Hi everyone,


One of the projects I am to work on in the new year requires amongst other
things storage of certain files in the Amazon S3. VFS sprang to mind as I
knew from browsing the Commons repository that it provides an abstraction
layer for dealing with file system ‹ however, it appears there is no S3
support in VFS. Are there plans to include this in a future release? If so,
is there an estimated release date? And also who are the developers
responsible for this? I'm only asking because I'm thinking/hoping that I
might be able to convince my company to put some effort perhaps into
building S3 support into VFS ‹ and as such would be good to be in direct
contact with the lead developers on this if I manage to pull this off.
(Don't take this as a promise though, it's more of a dream I suppose at this
stage :) But you don't find out till you try it so fingers crossed!)




Liv
 

Liviu Tudor
 
E: [hidden email] <mailto:[hidden email]>
M: +44 (0)7917696626
W: http://about.me/liviutudor <http://about.me/liviutudor>
Skype: liviutudor
 
I'm nobody, nobody's perfect -- therefore I'm perfect!

 



Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

garydgregory
The whole commons community is responsible for VFS but some developers
are more focused on a subset of commons. There are no plans that I
know of for S3. You are welcome to contribute a patch with a JIRA. The
tricky part is unit testing. Usually tests runs with every build but
in this case I am not sure how you can do this without mocking.

Gary

On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:

> Hi everyone,
>
>
> One of the projects I am to work on in the new year requires amongst other
> things storage of certain files in the Amazon S3. VFS sprang to mind as I
> knew from browsing the Commons repository that it provides an abstraction
> layer for dealing with file system ‹ however, it appears there is no S3
> support in VFS. Are there plans to include this in a future release? If so,
> is there an estimated release date? And also who are the developers
> responsible for this? I'm only asking because I'm thinking/hoping that I
> might be able to convince my company to put some effort perhaps into
> building S3 support into VFS ‹ and as such would be good to be in direct
> contact with the lead developers on this if I manage to pull this off.
> (Don't take this as a promise though, it's more of a dream I suppose at this
> stage :) But you don't find out till you try it so fingers crossed!)
>
>
>
>
> Liv
>
>
> Liviu Tudor
>
> E: [hidden email] <mailto:[hidden email]>
> M: +44 (0)7917696626
> W: http://about.me/liviutudor <http://about.me/liviutudor>
> Skype: liviutudor
>
> I'm nobody, nobody's perfect -- therefore I'm perfect!
>
>
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

Liviu Tudor
D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
problem this could be hmmmm food for thought I suppose…
I'll go and talk to my team after the holiday season see if I can pull
some resources towards this.
Thanks for the heads-up!


Liv
 
Liviu Tudor
 
E: [hidden email]
M: +44 (0)7917696626
W: http://about.me/liviutudor
Skype: liviutudor
 
I'm nobody, nobody's perfect -- therefore I'm perfect!

 





On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:

>The whole commons community is responsible for VFS but some developers
>are more focused on a subset of commons. There are no plans that I
>know of for S3. You are welcome to contribute a patch with a JIRA. The
>tricky part is unit testing. Usually tests runs with every build but
>in this case I am not sure how you can do this without mocking.
>
>Gary
>
>On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
>
>> Hi everyone,
>>
>>
>> One of the projects I am to work on in the new year requires amongst
>>other
>> things storage of certain files in the Amazon S3. VFS sprang to mind as
>>I
>> knew from browsing the Commons repository that it provides an
>>abstraction
>> layer for dealing with file system ‹ however, it appears there is no S3
>> support in VFS. Are there plans to include this in a future release? If
>>so,
>> is there an estimated release date? And also who are the developers
>> responsible for this? I'm only asking because I'm thinking/hoping that I
>> might be able to convince my company to put some effort perhaps into
>> building S3 support into VFS ‹ and as such would be good to be in direct
>> contact with the lead developers on this if I manage to pull this off.
>> (Don't take this as a promise though, it's more of a dream I suppose at
>>this
>> stage :) But you don't find out till you try it so fingers crossed!)
>>
>>
>>
>>
>> Liv
>>
>>
>> Liviu Tudor
>>
>> E: [hidden email] <mailto:[hidden email]>
>> M: +44 (0)7917696626
>> W: http://about.me/liviutudor <http://about.me/liviutudor>
>> Skype: liviutudor
>>
>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>
>>
>>
>>
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [hidden email]
>For additional commands, e-mail: [hidden email]
>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

Mark Fortner-3
Looks like there is an S3 plugin already.  I'm not sure what version of VFS
it works with, but it might do the trick.

https://github.com/abashev/vfs-s3

Hope this helps,

Mark




On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]> wrote:

> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
> problem this could be hmmmm food for thought I suppose…
> I'll go and talk to my team after the holiday season see if I can pull
> some resources towards this.
> Thanks for the heads-up!
>
>
> Liv
>
> Liviu Tudor
>
> E: [hidden email]
> M: +44 (0)7917696626
> W: http://about.me/liviutudor
> Skype: liviutudor
>
> I'm nobody, nobody's perfect -- therefore I'm perfect!
>
>
>
>
>
>
>
> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
>
> >The whole commons community is responsible for VFS but some developers
> >are more focused on a subset of commons. There are no plans that I
> >know of for S3. You are welcome to contribute a patch with a JIRA. The
> >tricky part is unit testing. Usually tests runs with every build but
> >in this case I am not sure how you can do this without mocking.
> >
> >Gary
> >
> >On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
> >
> >> Hi everyone,
> >>
> >>
> >> One of the projects I am to work on in the new year requires amongst
> >>other
> >> things storage of certain files in the Amazon S3. VFS sprang to mind as
> >>I
> >> knew from browsing the Commons repository that it provides an
> >>abstraction
> >> layer for dealing with file system ‹ however, it appears there is no S3
> >> support in VFS. Are there plans to include this in a future release? If
> >>so,
> >> is there an estimated release date? And also who are the developers
> >> responsible for this? I'm only asking because I'm thinking/hoping that I
> >> might be able to convince my company to put some effort perhaps into
> >> building S3 support into VFS ‹ and as such would be good to be in direct
> >> contact with the lead developers on this if I manage to pull this off.
> >> (Don't take this as a promise though, it's more of a dream I suppose at
> >>this
> >> stage :) But you don't find out till you try it so fingers crossed!)
> >>
> >>
> >>
> >>
> >> Liv
> >>
> >>
> >> Liviu Tudor
> >>
> >> E: [hidden email] <mailto:[hidden email]>
> >> M: +44 (0)7917696626
> >> W: http://about.me/liviutudor <http://about.me/liviutudor>
> >> Skype: liviutudor
> >>
> >> I'm nobody, nobody's perfect -- therefore I'm perfect!
> >>
> >>
> >>
> >>
> >>
> >
> >---------------------------------------------------------------------
> >To unsubscribe, e-mail: [hidden email]
> >For additional commands, e-mail: [hidden email]
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

Ralph Goers
I recall seeing S3 mentioned on the dev list a while ago, probably in reference to this project and whether we wanted to include it in VFS. It seems to be based on vfs2.  I didn't investigate to see what licenses its dependencies are under, but this project is now Apache licensed.

Ralph

On Dec 23, 2011, at 4:54 PM, Mark Fortner wrote:

> Looks like there is an S3 plugin already.  I'm not sure what version of VFS
> it works with, but it might do the trick.
>
> https://github.com/abashev/vfs-s3
>
> Hope this helps,
>
> Mark
>
>
>
>
> On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]> wrote:
>
>> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
>> problem this could be hmmmm food for thought I suppose…
>> I'll go and talk to my team after the holiday season see if I can pull
>> some resources towards this.
>> Thanks for the heads-up!
>>
>>
>> Liv
>>
>> Liviu Tudor
>>
>> E: [hidden email]
>> M: +44 (0)7917696626
>> W: http://about.me/liviutudor
>> Skype: liviutudor
>>
>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>
>>
>>
>>
>>
>>
>>
>> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
>>
>>> The whole commons community is responsible for VFS but some developers
>>> are more focused on a subset of commons. There are no plans that I
>>> know of for S3. You are welcome to contribute a patch with a JIRA. The
>>> tricky part is unit testing. Usually tests runs with every build but
>>> in this case I am not sure how you can do this without mocking.
>>>
>>> Gary
>>>
>>> On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>>
>>>> One of the projects I am to work on in the new year requires amongst
>>>> other
>>>> things storage of certain files in the Amazon S3. VFS sprang to mind as
>>>> I
>>>> knew from browsing the Commons repository that it provides an
>>>> abstraction
>>>> layer for dealing with file system ‹ however, it appears there is no S3
>>>> support in VFS. Are there plans to include this in a future release? If
>>>> so,
>>>> is there an estimated release date? And also who are the developers
>>>> responsible for this? I'm only asking because I'm thinking/hoping that I
>>>> might be able to convince my company to put some effort perhaps into
>>>> building S3 support into VFS ‹ and as such would be good to be in direct
>>>> contact with the lead developers on this if I manage to pull this off.
>>>> (Don't take this as a promise though, it's more of a dream I suppose at
>>>> this
>>>> stage :) But you don't find out till you try it so fingers crossed!)
>>>>
>>>>
>>>>
>>>>
>>>> Liv
>>>>
>>>>
>>>> Liviu Tudor
>>>>
>>>> E: [hidden email] <mailto:[hidden email]>
>>>> M: +44 (0)7917696626
>>>> W: http://about.me/liviutudor <http://about.me/liviutudor>
>>>> Skype: liviutudor
>>>>
>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

Elijah Zupancic
I would love to help out with this if there is support from the Commons
community on integrating S3 functionality.

-Elijah
On Fri, Dec 23, 2011 at 5:23 PM, Ralph Goers <[hidden email]>wrote:

> I recall seeing S3 mentioned on the dev list a while ago, probably in
> reference to this project and whether we wanted to include it in VFS. It
> seems to be based on vfs2.  I didn't investigate to see what licenses its
> dependencies are under, but this project is now Apache licensed.
>
> Ralph
>
> On Dec 23, 2011, at 4:54 PM, Mark Fortner wrote:
>
> > Looks like there is an S3 plugin already.  I'm not sure what version of
> VFS
> > it works with, but it might do the trick.
> >
> > https://github.com/abashev/vfs-s3
> >
> > Hope this helps,
> >
> > Mark
> >
> >
> >
> >
> > On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]>
> wrote:
> >
> >> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
> >> problem this could be hmmmm food for thought I suppose…
> >> I'll go and talk to my team after the holiday season see if I can pull
> >> some resources towards this.
> >> Thanks for the heads-up!
> >>
> >>
> >> Liv
> >>
> >> Liviu Tudor
> >>
> >> E: [hidden email]
> >> M: +44 (0)7917696626
> >> W: http://about.me/liviutudor
> >> Skype: liviutudor
> >>
> >> I'm nobody, nobody's perfect -- therefore I'm perfect!
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
> >>
> >>> The whole commons community is responsible for VFS but some developers
> >>> are more focused on a subset of commons. There are no plans that I
> >>> know of for S3. You are welcome to contribute a patch with a JIRA. The
> >>> tricky part is unit testing. Usually tests runs with every build but
> >>> in this case I am not sure how you can do this without mocking.
> >>>
> >>> Gary
> >>>
> >>> On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
> >>>
> >>>> Hi everyone,
> >>>>
> >>>>
> >>>> One of the projects I am to work on in the new year requires amongst
> >>>> other
> >>>> things storage of certain files in the Amazon S3. VFS sprang to mind
> as
> >>>> I
> >>>> knew from browsing the Commons repository that it provides an
> >>>> abstraction
> >>>> layer for dealing with file system ‹ however, it appears there is no
> S3
> >>>> support in VFS. Are there plans to include this in a future release?
> If
> >>>> so,
> >>>> is there an estimated release date? And also who are the developers
> >>>> responsible for this? I'm only asking because I'm thinking/hoping
> that I
> >>>> might be able to convince my company to put some effort perhaps into
> >>>> building S3 support into VFS ‹ and as such would be good to be in
> direct
> >>>> contact with the lead developers on this if I manage to pull this off.
> >>>> (Don't take this as a promise though, it's more of a dream I suppose
> at
> >>>> this
> >>>> stage :) But you don't find out till you try it so fingers crossed!)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> Liv
> >>>>
> >>>>
> >>>> Liviu Tudor
> >>>>
> >>>> E: [hidden email] <mailto:[hidden email]>
> >>>> M: +44 (0)7917696626
> >>>> W: http://about.me/liviutudor <http://about.me/liviutudor>
> >>>> Skype: liviutudor
> >>>>
> >>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

garydgregory
Another angle to consider is whether this should be in VFS2 or in a
new Java 7-based FileSystem library. Or both. The biggest hurdle I see
is unit testing.

Gary

On Dec 23, 2011, at 21:34, Elijah Zupancic <[hidden email]> wrote:

> I would love to help out with this if there is support from the Commons
> community on integrating S3 functionality.
>
> -Elijah
> On Fri, Dec 23, 2011 at 5:23 PM, Ralph Goers <[hidden email]>wrote:
>
>> I recall seeing S3 mentioned on the dev list a while ago, probably in
>> reference to this project and whether we wanted to include it in VFS. It
>> seems to be based on vfs2.  I didn't investigate to see what licenses its
>> dependencies are under, but this project is now Apache licensed.
>>
>> Ralph
>>
>> On Dec 23, 2011, at 4:54 PM, Mark Fortner wrote:
>>
>>> Looks like there is an S3 plugin already.  I'm not sure what version of
>> VFS
>>> it works with, but it might do the trick.
>>>
>>> https://github.com/abashev/vfs-s3
>>>
>>> Hope this helps,
>>>
>>> Mark
>>>
>>>
>>>
>>>
>>> On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]>
>> wrote:
>>>
>>>> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
>>>> problem this could be hmmmm food for thought I suppose…
>>>> I'll go and talk to my team after the holiday season see if I can pull
>>>> some resources towards this.
>>>> Thanks for the heads-up!
>>>>
>>>>
>>>> Liv
>>>>
>>>> Liviu Tudor
>>>>
>>>> E: [hidden email]
>>>> M: +44 (0)7917696626
>>>> W: http://about.me/liviutudor
>>>> Skype: liviutudor
>>>>
>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
>>>>
>>>>> The whole commons community is responsible for VFS but some developers
>>>>> are more focused on a subset of commons. There are no plans that I
>>>>> know of for S3. You are welcome to contribute a patch with a JIRA. The
>>>>> tricky part is unit testing. Usually tests runs with every build but
>>>>> in this case I am not sure how you can do this without mocking.
>>>>>
>>>>> Gary
>>>>>
>>>>> On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
>>>>>
>>>>>> Hi everyone,
>>>>>>
>>>>>>
>>>>>> One of the projects I am to work on in the new year requires amongst
>>>>>> other
>>>>>> things storage of certain files in the Amazon S3. VFS sprang to mind
>> as
>>>>>> I
>>>>>> knew from browsing the Commons repository that it provides an
>>>>>> abstraction
>>>>>> layer for dealing with file system ‹ however, it appears there is no
>> S3
>>>>>> support in VFS. Are there plans to include this in a future release?
>> If
>>>>>> so,
>>>>>> is there an estimated release date? And also who are the developers
>>>>>> responsible for this? I'm only asking because I'm thinking/hoping
>> that I
>>>>>> might be able to convince my company to put some effort perhaps into
>>>>>> building S3 support into VFS ‹ and as such would be good to be in
>> direct
>>>>>> contact with the lead developers on this if I manage to pull this off.
>>>>>> (Don't take this as a promise though, it's more of a dream I suppose
>> at
>>>>>> this
>>>>>> stage :) But you don't find out till you try it so fingers crossed!)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Liv
>>>>>>
>>>>>>
>>>>>> Liviu Tudor
>>>>>>
>>>>>> E: [hidden email] <mailto:[hidden email]>
>>>>>> M: +44 (0)7917696626
>>>>>> W: http://about.me/liviutudor <http://about.me/liviutudor>
>>>>>> Skype: liviutudor
>>>>>>
>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

Steve Siebert
Instead of coding directly to S3, perhaps you can code to an
abstraction library such as jclouds
(http://www.jclouds.org/documentation/index)  and let them handle the
integration with the various cloud storage providers.  You would get
simplified testing and multi-provider support instead of an S3
specific implementation.



On Fri, Dec 23, 2011 at 9:37 PM, Gary Gregory <[hidden email]> wrote:

> Another angle to consider is whether this should be in VFS2 or in a
> new Java 7-based FileSystem library. Or both. The biggest hurdle I see
> is unit testing.
>
> Gary
>
> On Dec 23, 2011, at 21:34, Elijah Zupancic <[hidden email]> wrote:
>
>> I would love to help out with this if there is support from the Commons
>> community on integrating S3 functionality.
>>
>> -Elijah
>> On Fri, Dec 23, 2011 at 5:23 PM, Ralph Goers <[hidden email]>wrote:
>>
>>> I recall seeing S3 mentioned on the dev list a while ago, probably in
>>> reference to this project and whether we wanted to include it in VFS. It
>>> seems to be based on vfs2.  I didn't investigate to see what licenses its
>>> dependencies are under, but this project is now Apache licensed.
>>>
>>> Ralph
>>>
>>> On Dec 23, 2011, at 4:54 PM, Mark Fortner wrote:
>>>
>>>> Looks like there is an S3 plugin already.  I'm not sure what version of
>>> VFS
>>>> it works with, but it might do the trick.
>>>>
>>>> https://github.com/abashev/vfs-s3
>>>>
>>>> Hope this helps,
>>>>
>>>> Mark
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]>
>>> wrote:
>>>>
>>>>> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
>>>>> problem this could be hmmmm food for thought I suppose…
>>>>> I'll go and talk to my team after the holiday season see if I can pull
>>>>> some resources towards this.
>>>>> Thanks for the heads-up!
>>>>>
>>>>>
>>>>> Liv
>>>>>
>>>>> Liviu Tudor
>>>>>
>>>>> E: [hidden email]
>>>>> M: +44 (0)7917696626
>>>>> W: http://about.me/liviutudor
>>>>> Skype: liviutudor
>>>>>
>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
>>>>>
>>>>>> The whole commons community is responsible for VFS but some developers
>>>>>> are more focused on a subset of commons. There are no plans that I
>>>>>> know of for S3. You are welcome to contribute a patch with a JIRA. The
>>>>>> tricky part is unit testing. Usually tests runs with every build but
>>>>>> in this case I am not sure how you can do this without mocking.
>>>>>>
>>>>>> Gary
>>>>>>
>>>>>> On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
>>>>>>
>>>>>>> Hi everyone,
>>>>>>>
>>>>>>>
>>>>>>> One of the projects I am to work on in the new year requires amongst
>>>>>>> other
>>>>>>> things storage of certain files in the Amazon S3. VFS sprang to mind
>>> as
>>>>>>> I
>>>>>>> knew from browsing the Commons repository that it provides an
>>>>>>> abstraction
>>>>>>> layer for dealing with file system ‹ however, it appears there is no
>>> S3
>>>>>>> support in VFS. Are there plans to include this in a future release?
>>> If
>>>>>>> so,
>>>>>>> is there an estimated release date? And also who are the developers
>>>>>>> responsible for this? I'm only asking because I'm thinking/hoping
>>> that I
>>>>>>> might be able to convince my company to put some effort perhaps into
>>>>>>> building S3 support into VFS ‹ and as such would be good to be in
>>> direct
>>>>>>> contact with the lead developers on this if I manage to pull this off.
>>>>>>> (Don't take this as a promise though, it's more of a dream I suppose
>>> at
>>>>>>> this
>>>>>>> stage :) But you don't find out till you try it so fingers crossed!)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Liv
>>>>>>>
>>>>>>>
>>>>>>> Liviu Tudor
>>>>>>>
>>>>>>> E: [hidden email] <mailto:[hidden email]>
>>>>>>> M: +44 (0)7917696626
>>>>>>> W: http://about.me/liviutudor <http://about.me/liviutudor>
>>>>>>> Skype: liviutudor
>>>>>>>
>>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

sebb-2-2
On 24 December 2011 02:48, Steven Siebert <[hidden email]> wrote:
> Instead of coding directly to S3, perhaps you can code to an
> abstraction library such as jclouds
> (http://www.jclouds.org/documentation/index)  and let them handle the
> integration with the various cloud storage providers.  You would get
> simplified testing and multi-provider support instead of an S3
> specific implementation.
>

See also http://deltacloud.apache.org/ - any synergy there?

>
> On Fri, Dec 23, 2011 at 9:37 PM, Gary Gregory <[hidden email]> wrote:
>> Another angle to consider is whether this should be in VFS2 or in a
>> new Java 7-based FileSystem library. Or both. The biggest hurdle I see
>> is unit testing.
>>
>> Gary
>>
>> On Dec 23, 2011, at 21:34, Elijah Zupancic <[hidden email]> wrote:
>>
>>> I would love to help out with this if there is support from the Commons
>>> community on integrating S3 functionality.
>>>
>>> -Elijah
>>> On Fri, Dec 23, 2011 at 5:23 PM, Ralph Goers <[hidden email]>wrote:
>>>
>>>> I recall seeing S3 mentioned on the dev list a while ago, probably in
>>>> reference to this project and whether we wanted to include it in VFS. It
>>>> seems to be based on vfs2.  I didn't investigate to see what licenses its
>>>> dependencies are under, but this project is now Apache licensed.
>>>>
>>>> Ralph
>>>>
>>>> On Dec 23, 2011, at 4:54 PM, Mark Fortner wrote:
>>>>
>>>>> Looks like there is an S3 plugin already.  I'm not sure what version of
>>>> VFS
>>>>> it works with, but it might do the trick.
>>>>>
>>>>> https://github.com/abashev/vfs-s3
>>>>>
>>>>> Hope this helps,
>>>>>
>>>>> Mark
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]>
>>>> wrote:
>>>>>
>>>>>> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
>>>>>> problem this could be hmmmm food for thought I suppose…
>>>>>> I'll go and talk to my team after the holiday season see if I can pull
>>>>>> some resources towards this.
>>>>>> Thanks for the heads-up!
>>>>>>
>>>>>>
>>>>>> Liv
>>>>>>
>>>>>> Liviu Tudor
>>>>>>
>>>>>> E: [hidden email]
>>>>>> M: +44 (0)7917696626
>>>>>> W: http://about.me/liviutudor
>>>>>> Skype: liviutudor
>>>>>>
>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
>>>>>>
>>>>>>> The whole commons community is responsible for VFS but some developers
>>>>>>> are more focused on a subset of commons. There are no plans that I
>>>>>>> know of for S3. You are welcome to contribute a patch with a JIRA. The
>>>>>>> tricky part is unit testing. Usually tests runs with every build but
>>>>>>> in this case I am not sure how you can do this without mocking.
>>>>>>>
>>>>>>> Gary
>>>>>>>
>>>>>>> On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
>>>>>>>
>>>>>>>> Hi everyone,
>>>>>>>>
>>>>>>>>
>>>>>>>> One of the projects I am to work on in the new year requires amongst
>>>>>>>> other
>>>>>>>> things storage of certain files in the Amazon S3. VFS sprang to mind
>>>> as
>>>>>>>> I
>>>>>>>> knew from browsing the Commons repository that it provides an
>>>>>>>> abstraction
>>>>>>>> layer for dealing with file system ‹ however, it appears there is no
>>>> S3
>>>>>>>> support in VFS. Are there plans to include this in a future release?
>>>> If
>>>>>>>> so,
>>>>>>>> is there an estimated release date? And also who are the developers
>>>>>>>> responsible for this? I'm only asking because I'm thinking/hoping
>>>> that I
>>>>>>>> might be able to convince my company to put some effort perhaps into
>>>>>>>> building S3 support into VFS ‹ and as such would be good to be in
>>>> direct
>>>>>>>> contact with the lead developers on this if I manage to pull this off.
>>>>>>>> (Don't take this as a promise though, it's more of a dream I suppose
>>>> at
>>>>>>>> this
>>>>>>>> stage :) But you don't find out till you try it so fingers crossed!)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Liv
>>>>>>>>
>>>>>>>>
>>>>>>>> Liviu Tudor
>>>>>>>>
>>>>>>>> E: [hidden email] <mailto:[hidden email]>
>>>>>>>> M: +44 (0)7917696626
>>>>>>>> W: http://about.me/liviutudor <http://about.me/liviutudor>
>>>>>>>> Skype: liviutudor
>>>>>>>>
>>>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>
>>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

garydgregory
On Dec 24, 2011, at 6:20, sebb <[hidden email]> wrote:

> On 24 December 2011 02:48, Steven Siebert <[hidden email]> wrote:
>> Instead of coding directly to S3, perhaps you can code to an
>> abstraction library such as jclouds
>> (http://www.jclouds.org/documentation/index)  and let them handle the
>> integration with the various cloud storage providers.  You would get
>> simplified testing and multi-provider support instead of an S3
>> specific implementation.
>>
>
> See also http://deltacloud.apache.org/ - any synergy there?

Did I miss the java API somewhere?

Gary

>
>>
>> On Fri, Dec 23, 2011 at 9:37 PM, Gary Gregory <[hidden email]> wrote:
>>> Another angle to consider is whether this should be in VFS2 or in a
>>> new Java 7-based FileSystem library. Or both. The biggest hurdle I see
>>> is unit testing.
>>>
>>> Gary
>>>
>>> On Dec 23, 2011, at 21:34, Elijah Zupancic <[hidden email]> wrote:
>>>
>>>> I would love to help out with this if there is support from the Commons
>>>> community on integrating S3 functionality.
>>>>
>>>> -Elijah
>>>> On Fri, Dec 23, 2011 at 5:23 PM, Ralph Goers <[hidden email]>wrote:
>>>>
>>>>> I recall seeing S3 mentioned on the dev list a while ago, probably in
>>>>> reference to this project and whether we wanted to include it in VFS. It
>>>>> seems to be based on vfs2.  I didn't investigate to see what licenses its
>>>>> dependencies are under, but this project is now Apache licensed.
>>>>>
>>>>> Ralph
>>>>>
>>>>> On Dec 23, 2011, at 4:54 PM, Mark Fortner wrote:
>>>>>
>>>>>> Looks like there is an S3 plugin already.  I'm not sure what version of
>>>>> VFS
>>>>>> it works with, but it might do the trick.
>>>>>>
>>>>>> https://github.com/abashev/vfs-s3
>>>>>>
>>>>>> Hope this helps,
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>>> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
>>>>>>> problem this could be hmmmm food for thought I suppose…
>>>>>>> I'll go and talk to my team after the holiday season see if I can pull
>>>>>>> some resources towards this.
>>>>>>> Thanks for the heads-up!
>>>>>>>
>>>>>>>
>>>>>>> Liv
>>>>>>>
>>>>>>> Liviu Tudor
>>>>>>>
>>>>>>> E: [hidden email]
>>>>>>> M: +44 (0)7917696626
>>>>>>> W: http://about.me/liviutudor
>>>>>>> Skype: liviutudor
>>>>>>>
>>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
>>>>>>>
>>>>>>>> The whole commons community is responsible for VFS but some developers
>>>>>>>> are more focused on a subset of commons. There are no plans that I
>>>>>>>> know of for S3. You are welcome to contribute a patch with a JIRA. The
>>>>>>>> tricky part is unit testing. Usually tests runs with every build but
>>>>>>>> in this case I am not sure how you can do this without mocking.
>>>>>>>>
>>>>>>>> Gary
>>>>>>>>
>>>>>>>> On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> One of the projects I am to work on in the new year requires amongst
>>>>>>>>> other
>>>>>>>>> things storage of certain files in the Amazon S3. VFS sprang to mind
>>>>> as
>>>>>>>>> I
>>>>>>>>> knew from browsing the Commons repository that it provides an
>>>>>>>>> abstraction
>>>>>>>>> layer for dealing with file system ‹ however, it appears there is no
>>>>> S3
>>>>>>>>> support in VFS. Are there plans to include this in a future release?
>>>>> If
>>>>>>>>> so,
>>>>>>>>> is there an estimated release date? And also who are the developers
>>>>>>>>> responsible for this? I'm only asking because I'm thinking/hoping
>>>>> that I
>>>>>>>>> might be able to convince my company to put some effort perhaps into
>>>>>>>>> building S3 support into VFS ‹ and as such would be good to be in
>>>>> direct
>>>>>>>>> contact with the lead developers on this if I manage to pull this off.
>>>>>>>>> (Don't take this as a promise though, it's more of a dream I suppose
>>>>> at
>>>>>>>>> this
>>>>>>>>> stage :) But you don't find out till you try it so fingers crossed!)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Liv
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Liviu Tudor
>>>>>>>>>
>>>>>>>>> E: [hidden email] <mailto:[hidden email]>
>>>>>>>>> M: +44 (0)7917696626
>>>>>>>>> W: http://about.me/liviutudor <http://about.me/liviutudor>
>>>>>>>>> Skype: liviutudor
>>>>>>>>>
>>>>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Commons VFS support for Amazon S3?

garydgregory
In reply to this post by sebb-2-2
Good find. That would be a great add to VFS.

Gary

On Dec 24, 2011, at 6:20, sebb <[hidden email]> wrote:

> On 24 December 2011 02:48, Steven Siebert <[hidden email]> wrote:
>> Instead of coding directly to S3, perhaps you can code to an
>> abstraction library such as jclouds
>> (http://www.jclouds.org/documentation/index)  and let them handle the
>> integration with the various cloud storage providers.  You would get
>> simplified testing and multi-provider support instead of an S3
>> specific implementation.
>>
>
> See also http://deltacloud.apache.org/ - any synergy there?
>
>>
>> On Fri, Dec 23, 2011 at 9:37 PM, Gary Gregory <[hidden email]> wrote:
>>> Another angle to consider is whether this should be in VFS2 or in a
>>> new Java 7-based FileSystem library. Or both. The biggest hurdle I see
>>> is unit testing.
>>>
>>> Gary
>>>
>>> On Dec 23, 2011, at 21:34, Elijah Zupancic <[hidden email]> wrote:
>>>
>>>> I would love to help out with this if there is support from the Commons
>>>> community on integrating S3 functionality.
>>>>
>>>> -Elijah
>>>> On Fri, Dec 23, 2011 at 5:23 PM, Ralph Goers <[hidden email]>wrote:
>>>>
>>>>> I recall seeing S3 mentioned on the dev list a while ago, probably in
>>>>> reference to this project and whether we wanted to include it in VFS. It
>>>>> seems to be based on vfs2.  I didn't investigate to see what licenses its
>>>>> dependencies are under, but this project is now Apache licensed.
>>>>>
>>>>> Ralph
>>>>>
>>>>> On Dec 23, 2011, at 4:54 PM, Mark Fortner wrote:
>>>>>
>>>>>> Looks like there is an S3 plugin already.  I'm not sure what version of
>>>>> VFS
>>>>>> it works with, but it might do the trick.
>>>>>>
>>>>>> https://github.com/abashev/vfs-s3
>>>>>>
>>>>>> Hope this helps,
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Dec 23, 2011 at 4:03 PM, Liviu Tudor <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>>> D'oh, good point about unit testing, Gary! Hasn't crossed my mind what a
>>>>>>> problem this could be hmmmm food for thought I suppose…
>>>>>>> I'll go and talk to my team after the holiday season see if I can pull
>>>>>>> some resources towards this.
>>>>>>> Thanks for the heads-up!
>>>>>>>
>>>>>>>
>>>>>>> Liv
>>>>>>>
>>>>>>> Liviu Tudor
>>>>>>>
>>>>>>> E: [hidden email]
>>>>>>> M: +44 (0)7917696626
>>>>>>> W: http://about.me/liviutudor
>>>>>>> Skype: liviutudor
>>>>>>>
>>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 23/12/2011 23:49, "Gary Gregory" <[hidden email]> wrote:
>>>>>>>
>>>>>>>> The whole commons community is responsible for VFS but some developers
>>>>>>>> are more focused on a subset of commons. There are no plans that I
>>>>>>>> know of for S3. You are welcome to contribute a patch with a JIRA. The
>>>>>>>> tricky part is unit testing. Usually tests runs with every build but
>>>>>>>> in this case I am not sure how you can do this without mocking.
>>>>>>>>
>>>>>>>> Gary
>>>>>>>>
>>>>>>>> On Dec 23, 2011, at 18:38, Liviu Tudor <[hidden email]> wrote:
>>>>>>>>
>>>>>>>>> Hi everyone,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> One of the projects I am to work on in the new year requires amongst
>>>>>>>>> other
>>>>>>>>> things storage of certain files in the Amazon S3. VFS sprang to mind
>>>>> as
>>>>>>>>> I
>>>>>>>>> knew from browsing the Commons repository that it provides an
>>>>>>>>> abstraction
>>>>>>>>> layer for dealing with file system ‹ however, it appears there is no
>>>>> S3
>>>>>>>>> support in VFS. Are there plans to include this in a future release?
>>>>> If
>>>>>>>>> so,
>>>>>>>>> is there an estimated release date? And also who are the developers
>>>>>>>>> responsible for this? I'm only asking because I'm thinking/hoping
>>>>> that I
>>>>>>>>> might be able to convince my company to put some effort perhaps into
>>>>>>>>> building S3 support into VFS ‹ and as such would be good to be in
>>>>> direct
>>>>>>>>> contact with the lead developers on this if I manage to pull this off.
>>>>>>>>> (Don't take this as a promise though, it's more of a dream I suppose
>>>>> at
>>>>>>>>> this
>>>>>>>>> stage :) But you don't find out till you try it so fingers crossed!)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Liv
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Liviu Tudor
>>>>>>>>>
>>>>>>>>> E: [hidden email] <mailto:[hidden email]>
>>>>>>>>> M: +44 (0)7917696626
>>>>>>>>> W: http://about.me/liviutudor <http://about.me/liviutudor>
>>>>>>>>> Skype: liviutudor
>>>>>>>>>
>>>>>>>>> I'm nobody, nobody's perfect -- therefore I'm perfect!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>>> For additional commands, e-mail: [hidden email]
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]