Article delegate-en/4934 of [1-5169] on the server localhost:119
  upper oldest olders older1 this newer1 newers latest
search
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
[Reference:<_A4931@delegate-en.ML_>]
Newsgroups: mail-lists.delegate-en

[DeleGate-En] Re: FTP/SFTP transfer timeouts - Delegate 9.9.7
11 Nov 2010 14:07:31 GMT feedback@delegate.org (Yutaka Sato)
The DeleGate Project


Hi,

In message <_A4931@delegate-en.ML_> on 11/10/10(09:09:51)
you "Trigge, Graham" <ppajabdyi-aj46qzb42lfr.ml@ml.delegate.org> wrote:
 |I have an issue at the moment where a client is using Delegate 9.9.7 as
 |an FTP/SFTP gateway. They have been transferring data fine using this
 |gateway on version 9.7.7 without any issues, but when I upgraded to 9.9.7
 |the transfer of large files is failing (I upgraded as there are other uses
 |of this gateway having timeout issues which were fixed on this version).
 |The file in question is 7Mb in size and I can see that 7Mb of data is
 |being transferred from the client server to the Delegate server, but only
 |around 700-800Kb is being transferred to the SFTP target. The client has
 |sent me the output from their transfer and it seems that when Delegate
 |(during the transfer) responds to the client with " Response code: 226 Ok
 |(upload in progress...)" it is stopping the session.
 |
 |Is there anyway to disable this message for particular clients? I could
 |remove it from the source and recompile, but I think this is the feature
 |that fixed my problem with other clients.

The response with the code 226 is necessary to inform of a client
that the data transfer completed and the client can continue the
next command.  The special 226 response with "(upload in progress...)"
was added in 9.9.2-pre1:
 - assuming an interactive FTP client with a human user
 - to cope with uploading a huge file (like iso image for example)
 - to avoid timeout of a client waiting 226 while uploading a huge file
 - to enable abortion of uploading by the QUIT command or disconnection
 - DeleGate returns 226 with "(upload in progress...)" on slow uploading
 - next commands while uploading result in "450 upload in progress..."

But this mechanism can cause truncation of uploaded data as in your case
where a client finishes immediately after it saw 226 response, and/or the
link with the server is mutch slower than the link with the client, or
uploading to a server takes longer than 5 seconds.
As a workaround, I made the enclosed patch to introduce the "waitput"
option for a MOUNT parameter which make the timeout longer (10 minutes
for example) as follows:

  MOUNT="/* ftp://server/* waitput=600"

Also you need a "datamax" option (as "datamax=1g" for 1G bytes) to
enable uploading data larger than 10MB.

  MOUNT="/* ftp://server/* waitput=3600,datamax=1g"

Cheers,
Yutaka
--
  9 9   Yutaka Sato, CSDP,ITIL-F <y.sato@delegate.org> http://delegate.org/y.sato/
 ( ~ )  National Institute of Advanced Industrial Science and Technology
_<   >_ 1-1-4 Umezono, Tsukuba, Ibaraki, 305-8568 Japan
Do the more with the less -- B. Fuller


*** dist/src/delegate9.9.8-pre18/src/sftp.c	Sun Jun 13 21:03:52 2010
--- ./src/sftp.c	Thu Nov 11 22:35:21 2010
***************
*** 716,731 ****
--- 716,736 ----
  	double St;
  	int nrdy;
  
  	tout = (int)Tout;
  	if( tout <= 0 ) tout = 1;
  	sv1log("--SFTPGW uploading wait=%d\n",tout);
  	tout1 = tout;
  	if( 5 < tout1 ) tout1 = 5;
+ 	if( getMountOpt1(MainConn(),"waitput",AVStr(opt1),sizeof(opt1)) ){
+ 		tout2 = (int)(Scan_period(opt1,'s',0)*1000);
+ 		sv1log("## waitput=%s %d\n",opt1,tout2);
+ 		relay_resp(fs,-1,tout2,BVStr(sresp),com,1);
+ 	}else
  	relay_resp(fs,-1,tout1*1000,BVStr(sresp),com,1);
  	if( strstr(sresp,"sftp>") ){
  		putresp(tc,226,"Ok");
  		return 1;
  	}
  
  	/* resp. to avoid client timeout */
  	putresp(tc,226,"Ok (upload in progress ...)");

  admin search upper oldest olders older1 this newer1 newers latest
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
@_@V