[VFS] FTPClientWrapper Broken Pipe

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[VFS] FTPClientWrapper Broken Pipe

Jens Viebig-2
Hello List,
i'm using VFS to regulary check some files on an FTP server. This works fine for a long time, but after a while the check crashes with the following stacktrace:

org.apache.commons.vfs.FileSystemException: Could not determine the type of file ftp://user:[hidden email]/myfile.txt.
        at org.apache.commons.vfs.provider.AbstractFileObject.attach()V(Optimized Method)
        at org.apache.commons.vfs.provider.AbstractFileObject.exists()Z(AbstractFileObject.java:391)
        at my.class.Something.toDo(Unknown Source)
        at java.lang.Thread.run()V(Unknown Source)
        at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown Source)
Caused by: java.net.SocketException: Broken pipe
        at jrockit.net.SocketNativeIO.write(IIIZ)I(Unknown Source)
        at jrockit.net.SocketNativeIO.write(III)I(Unknown Source)
        at jrockit.io.NativeIO.write(Ljava.io.FileDescriptor;II)I(Optimized Method)
        at java.net.AbstractSocketImpl$2.write(II)V(Optimized Method)
        at jrockit.io.NativeIOOutputStream.write([BII)V(Optimized Method)
        at java.io.BufferedOutputStream.flushBuffer()V(Optimized Method)
        at java.io.BufferedOutputStream.write(I)V(Optimized Method)
        at org.apache.commons.net.telnet.Telnet._sendByte(I)V(Optimized Method)
        at org.apache.commons.net.telnet.TelnetOutputStream.write(I)V(Optimized Method)
        at org.apache.commons.net.io.ToNetASCIIOutputStream.write([BII)V(Optimized Method)
        at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes()V(StreamEncoder.java:325)
        at sun.nio.cs.StreamEncoder$CharsetSE.implWrite([CII)V(Optimized Method)
        at sun.nio.cs.StreamEncoder.write([CII)V(StreamEncoder.java:136)
        at java.io.OutputStreamWriter.write([CII)V(OutputStreamWriter.java:191)
        at java.io.BufferedWriter.flushBuffer()V(BufferedWriter.java:107)
        at java.io.BufferedWriter.flush()V(BufferedWriter.java:228)
        at org.apache.commons.net.ftp.FTP.sendCommand(Ljava.lang.String;Ljava.lang.String;)I(Optimized Method)
        at org.apache.commons.net.ftp.FTP.sendCommand(ILjava.lang.String;)I(Optimized Method)
        at org.apache.commons.net.ftp.FTP.port(Ljava.net.InetAddress;I)I(Optimized Method)
        at org.apache.commons.vfs.provider.ftp.FTPClientWrapper.listFiles(Ljava.lang.String;Ljava.lang.String;)[Lorg.apache.commons.net.ftp.FTPFile;(Optimized Method)
        at org.apache.commons.vfs.provider.ftp.FtpFileObject.doGetChildren()V(Optimized Method)
        at org.apache.commons.vfs.provider.ftp.FtpFileObject.getChildFile(Ljava.lang.String;Z)Lorg.apache.commons.net.ftp.FTPFile;(Optimized Method)
        at org.apache.commons.vfs.provider.ftp.FtpFileObject.getInfo(Z)V(Optimized Method)
        at org.apache.commons.vfs.provider.ftp.FtpFileObject.doAttach()V(Optimized Method)
        ... 6 more

When it comes to that error, the filesystem provider never recovers from that crash and all actions on files on the same ftp server fail. The ftp server works fine, i can connect fine from the commandline. it seems that the VFS ftp provider tries to reuse that old damaged connection, i don't see a new attempt to login in my ftp server log. There is no connection from VFS to the ftp server.

I looked in the source code of FTPClientWrapper, and tracked the error down to the listFiles function:

...
    public FTPFile[] listFiles(String key, String relPath) throws IOException
    {
        try
        {
            return getFtpClient().listFiles(key, relPath);
        }
        catch (FTPConnectionClosedException e)
        {
            disconnect();
            return getFtpClient().listFiles(key, relPath);
        }
    }
...

the FTPClient of commons.net throws FTPConnectionClosedException and IOException. if a FTPConnectionClosedException is thrown everything is fine, the old connection is closed and a new one is created. If a more general IOException is thrown the connection is not closed and not set to null, so the FTPClientWrapper tries to use it again. Normally isConnected() should fail then, but it seems that it still returns true und is a weak test. I suggest that the connection should also be disconnected and set to null on more general errors, if there is an IOException there is a big chance that the connection is not usable anymore.

Maybe like this:

    public FTPFile[] listFiles(String key, String relPath) throws IOException
    {
        try
        {
            return getFtpClient().listFiles(key, relPath);
        }
        catch (FTPConnectionClosedException e)
        {
            disconnect();
            return getFtpClient().listFiles(key, relPath);
        }
        catch (IOException ie)
        {
            disconnect();
            throw ie;
        }
    }

This fix should be applied to all functions in FTPClientWrapper

Maybe you have a better approach on fixing this issue ?
Reply | Threaded
Open this post in threaded view
|

Re: [VFS] FTPClientWrapper Broken Pipe

Mario Ivankovits
Hi!
>Caused by: java.net.SocketException: Broken pipe
>  
Do you know why this happens? Some sort of ftp timeout?

>        catch (IOException ie)
>        {
>            disconnect();
>            throw ie;
>        }
>  
I ask as maybe we should retry the operation here too?


>Normally isConnected() should fail then, but it seems that it still returns true
isConnected() in ftp.java is a logical operation if the connection
"should be" established, not a physical check.
There is also a socketIsConnected which tries to check if the
connections is really alive but it works only with jdk1.4 and up.

Which jdk do you use?
If its 1.4 or up, which commons-net version do you use?

---
Mario


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] FTPClientWrapper Broken Pipe

Andrew Elcock
What I did to get round this is to apply a timeout and after a certain
time I "restart" the connection. I hope this works for you to...

Mario Ivankovits wrote:

> Hi!
>
>> Caused by: java.net.SocketException: Broken pipe
>>  
>
> Do you know why this happens? Some sort of ftp timeout?
>
>>        catch (IOException ie)
>>        {
>>            disconnect();
>>            throw ie;
>>        }
>>  
>
> I ask as maybe we should retry the operation here too?
>
>
>> Normally isConnected() should fail then, but it seems that it still
>> returns true
>
> isConnected() in ftp.java is a logical operation if the connection
> "should be" established, not a physical check.
> There is also a socketIsConnected which tries to check if the
> connections is really alive but it works only with jdk1.4 and up.
>
> Which jdk do you use?
> If its 1.4 or up, which commons-net version do you use?
>
> ---
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] FTPClientWrapper Broken Pipe

Jens Viebig-2
Hello, how do you set a timeout on the client side ?
My ftp server times out after 5 minutes, normally the connection
restarts as expected.
My code runs inside a background service that is running 24/7 so there
are a lot of things that can cause an 'unexpected' IOException. A
physical network is never perfect.
Im using bea jrockit jvm that is 1.4 compatible
Im using commons-net 1.4
Yes it seems that isConnected() that is used in FTPClientWrapper does
not do a physical test. I think i will try to replace

all catch blocks

catch (FTPConnectionClosedException e)

with

catch(IOException e)

in FTPClientWrapper

FTPConnectionClosedException is a subclass of  IOException so there  is
no need to have two catch blocks.
Hope this will be fixed in official future releases of commons-VFS

Regards

Andrew Elcock wrote:

> What I did to get round this is to apply a timeout and after a certain
> time I "restart" the connection. I hope this works for you to...
>
> Mario Ivankovits wrote:
>
>> Hi!
>>
>>> Caused by: java.net.SocketException: Broken pipe
>>>  
>>
>>
>> Do you know why this happens? Some sort of ftp timeout?
>>
>>>        catch (IOException ie)
>>>        {
>>>            disconnect();
>>>            throw ie;
>>>        }
>>>  
>>
>>
>> I ask as maybe we should retry the operation here too?
>>
>>
>>> Normally isConnected() should fail then, but it seems that it still
>>> returns true
>>
>>
>> isConnected() in ftp.java is a logical operation if the connection
>> "should be" established, not a physical check.
>> There is also a socketIsConnected which tries to check if the
>> connections is really alive but it works only with jdk1.4 and up.
>>
>> Which jdk do you use?
>> If its 1.4 or up, which commons-net version do you use?
>>
>> ---
>> Mario
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] FTPClientWrapper Broken Pipe

Mario Ivankovits
Jens Viebig wrote:
> Hope this will be fixed in official future releases of commons-VFS
I've changed it now, maybe you can check it.


Thanks!
Mario


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] FTPClientWrapper Broken Pipe

Jens Viebig-2
Thanks for the quick patch. I'm testing the patched FTPClientWrapper with
the sourcecode of commons-VFS-RC2 and it seems to work fine.

With the newest svn checkout i have a new issue. VFS can't create a new
connection to the ftp server, i get an error:

Caused by:
org.apache.commons.vfs.org.apache.commons.vfs.FileSystemException: Could not
connect to FTP server on "10.1.1.42".
        at
org.apache.commons.vfs.provider.ftp.FtpClientFactory.createConnection(Ljava.lang.String;ILjava.lang.String;Ljava.lang.String;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.net.ftp.FTPClient;(FtpClientFactory.java:120)
        at
org.apache.commons.vfs.provider.ftp.FTPClientWrapper.createClient()Lorg.apache.commons.net.ftp.FTPClient;(FTPClientWrapper.java:60)
        at
org.apache.commons.vfs.provider.ftp.FTPClientWrapper.getFtpClient()Lorg.apache.commons.net.ftp.FTPClient;(FTPClientWrapper.java:72)
        at
org.apache.commons.vfs.provider.ftp.FTPClientWrapper.<init>(Lorg.apache.commons.vfs.provider.GenericFileName;Lorg.apache.commons.vfs.FileSystemOptions;)V(FTPClientWrapper.java:43)
        at
org.apache.commons.vfs.provider.ftp.FtpFileProvider.doCreateFileSystem(Lorg.apache.commons.vfs.FileName;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileSystem;(FtpFileProvider.java:74)
        at
org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(Lorg.apache.commons.vfs.FileName;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileObject;(AbstractOriginatingFileProvider.java:76)
        at
org.apache.commons.vfs.provider.AbstractOriginatingFileProvider.findFile(Lorg.apache.commons.vfs.FileObject;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileObject;(AbstractOriginatingFileProvider.java:61)
        at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(Lorg.apache.commons.vfs.FileObject;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.vfs.FileObject;(DefaultFileSystemManager.java:524)
        at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(Lorg.apache.commons.vfs.FileObject;Ljava.lang.String;)Lorg.apache.commons.vfs.FileObject;(DefaultFileSystemManager.java:488)
        at
org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(Ljava.lang.String;)Lorg.apache.commons.vfs.FileObject;(DefaultFileSystemManager.java:458)
        at
com.como.commonwatchfolder.CommonWatchfolderService.run()V(Unknown Source)
        at java.lang.Thread.run()V(Unknown Source)
        at java.lang.Thread.startThreadFromVM(Ljava.lang.Thread;)V(Unknown
Source)
Caused by: org.apache.commons.vfs.FileSystemException: Could not change to
work directory "/".
        at
org.apache.commons.vfs.provider.ftp.FtpClientFactory.createConnection(Ljava.lang.String;ILjava.lang.String;Ljava.lang.String;Ljava.lang.String;Lorg.apache.commons.vfs.FileSystemOptions;)Lorg.apache.commons.net.ftp.FTPClient;(FtpClientFactory.java:97)

On my ftp server log it seems that the client sends an CWD command without a
directory path, which causes the server to send a syntax error

Regards
Jens Viebig

----- Original Message -----
From: "Mario Ivankovits" <[hidden email]>
To: "Jakarta Commons Users List" <[hidden email]>
Sent: Friday, September 02, 2005 7:52 AM
Subject: Re: [VFS] FTPClientWrapper Broken Pipe


> Jens Viebig wrote:
>> Hope this will be fixed in official future releases of commons-VFS
> I've changed it now, maybe you can check it.
>
>
> Thanks!
> Mario
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [VFS] FTPClientWrapper Broken Pipe

Mario Ivankovits
Hi Jens!
> With the newest svn checkout i have a new issue. VFS can't create a
> new connection to the ftp server, i get an error:
> On my ftp server log it seems that the client sends an CWD command
> without a directory path, which causes the server to send a syntax error
Thanks for the detailed report.

Yes, there is some testing code in the current head too, I havent had
the time to run the tests.
Sorry for the inconvenience, it should be fixed now.


Ciao,
Mario


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]