On Wed, 24 May 2006, Yutaka Sato wrote:

Hello,

during my tests, I found three problems, please see below and the 
attached log/conf files.

> In message <_A3236@delegate-en.ML_> on 05/23/06(22:09:12)
> you Steffen Kaiser <p44eqbdyi-qjkxlpk6rja6.ml@delegate.org> wrote:
> |in short it works:
> |  make CC=$CC  CFLAGS="-DSTAT64 -D_LARGEFILE64_SOURCE"
>
> What happens without -D_LARGEFILE64_SOURCE ?

this:

gcc -DSTAT64 -x c++ -DQS -DNONC99 -I../include -DSTAT64 -c setutimes.cc
setutimes.cc: In function `int set_utimes(const char *, int, int)':
setutimes.cc:25: aggregate `struct FileStat st' has incomplete type and 
cannot be initialized
setutimes.cc:28: type `stat64' is not yet defined
make[4]: *** [setutimes.o] Error 1
make[4]: Leaving directory `/mnt/install/linux/delegate9.2.2-pre2a/rary'
make[3]: *** [libx] Error 2
make[3]: Leaving directory `/mnt/install/linux/delegate9.2.2-pre2a/rary'
mkmake: ERROR LOG is left at 
/mnt/install/linux/delegate9.2.2-pre2a/rary/mkmake.err
mkmake: ERROR LOG is left at 
/mnt/install/linux/delegate9.2.2-pre2a/rary/mkmake.err
make[2]: *** [../lib/library.a] Error 2
make[2]: Leaving directory `/mnt/install/linux/delegate9.2.2-pre2a/src'
make[1]: *** [start0] Error 2
make[1]: Leaving directory `/mnt/install/linux/delegate9.2.2-pre2a/src'
mkmake: ERROR LOG is left at 
/mnt/install/linux/delegate9.2.2-pre2a/src/mkmake.err
mkmake: ERROR LOG is left at 
/mnt/install/linux/delegate9.2.2-pre2a/src/mkmake.err
make: *** [all] Error 2

> escape the problem with the enclosed patch.

This worked - sort of.

Problem 1) Downloaded file is longer than original one.

ncftp /pub/debian-iso/sarge > dir
-rw-r--r--  1 root   4686706688   Jun 13  2005 
debian-31r0a-i386-binary-1.iso
-rw-r--r--  1 root   4428851200   Jun 13  2005 
debian-31r0a-i386-binary-2.iso
-rw-r--r--  1 root    113541120   Jun 13  2005 
debian-31r0a-i386-netinst.iso
ncftp /pub/debian-iso/sarge > get debian-31r0a-i386-binary-1.iso
debian-31r0a-i386-binary-1.iso:    ETA:   0:00  785.63/373.59 MB   14.79 
MB/s

However: The returned size is wrong - however, ncftp proceeds downloading 
the file.

I'm not sure, how ncftp tries to get the file size, this one command is 
logged:

05/24 08:58:27.87 [22518] 2+0/14: Unknown request: MLST 
debian-31r0a-i386-binary-1.iso^M

but I'm assume ncftp uses the SIZE command. (Same size is reported by 
SIZE).

BUT the downloaded file differs in size, it is 756 bytes longer than the 
original one.

So I tested it further,

dd if=/dev/zero of=test bs=1024 count=175
perl  ~/ftp-test2.pl ftp://127.0.0.1:2121/pub/test >test2 && ls -al test*
-rw-r--r--    1 root     root       179200 May 24 15:14 test
-rw-r--r--    1 root     root       179201 May 24 15:14 test2

cmp -l test*
  65537   0 277
cmp: EOF on test

dd if=/dev/zero of=test bs=1024 count=176
perl  ~/ftp-test2.pl ftp://127.0.0.1:2121/pub/test >test2 && ls -al test*
-rw-r--r--    1 root     root       180224 May 24 15:15 test
-rw-r--r--    1 root     root       180226 May 24 15:15 test2

((test2 is one byte longer!))

cmp -l test*
  65537   0 277
cmp: EOF on test

When I use /dev/random except /dev/zero, the data is shifted by one byte 
from 65537, the 0xFF is inserted.

With count=240, the downloaded file is three bytes longer.
It sort of looks like that there is a problem each 64KB.

I used attached ftp-test2.pl script and explicitly set TYPE I and dumped 
the stuff through STDOUT, so there is no filesystem related stuff. Also, 
the test file is filled with \0's, hence, no CR/LF transformation should 
be done.

Problem 2) The local file is cached.

OK, the host I tried is a Woody box, which is fairly old and I tried to 
check if the downloading stuff is interferred by the old software and used 
another box for download, which is Debian Sarge (current stable). The 
utilities (like cmp) on Sarge do accept 4GB files, those of Woody don't.

The situation is this:

Workstation -> Delegate FTP relay 10.20.10.6:2121 -> Delegate MASTER 
ux-2s04:11011 -> Delegate FTP ux-2s04:2121 -> file:///mnt/mirror/pub

I have a chain of two Delegate-Servers, a generalist (ux-2s04) that DOES 
CACHING and is to tunnel all requests into the world. On the same host, 
the Debian DVD image are located and are served via Delegate FTP.
On an relay some application level proxies forward the requests to the 
generalist. None of them performs caching.

I configured the FTP relay with "MOUNT=/pub/* ftp://ux-2s04:2121/pub/*", 
but through the MASTER=ux-2s04:11011. I guess I have to re-think this 
Master stuff, in order to avoid this problem.

Anyway: The file is downloaded, and the Master caches the file, and when 
it reaches the 2GB boundary, the server drops the connection.

Problem 3) cd works exactly one time

OK, I do this:

ftp relay:2121

cd /pub/debian-iso/sarge
get file
--> OK

BUT when I do:

ftp relay:2121
cd /pub
cd debian-iso/sarge

I get: Could not chdir to sarge/: server said: 
/pub/debian-iso/pub/debian-iso/sarge: No such directory.

This is also true, when I terminate the connection and re-connect again 
too soon, from the logfile I guess that the connection is still hold, once 
dropped, the first CD is successful.

In the master's 2121 logfile this is printed:

05/24 14:01:57.59 [22928] 1+0/8: local echo for PWD: 
ftp://:0//mnt/mirror/pub/debian-iso/
05/24 14:01:57.59 [22928] 1+0/8: I-SAY: 257 "/pub/debian-iso" is current 
directory.^M
05/24 14:01:57.59 [22928] 1+0/10: *** 
/pub/debian-iso/pub/debian-iso/sarge/ => 
file://localhost/mnt/mirror/pub/debian-iso/pu
b/debian-iso/sarge/ ***
05/24 14:01:57.59 [22928] 1+0/10: MOUNTED-TO-LOCAL#UNKNOWN: 
file://localhost/mnt/mirror/pub/debian-iso/pub/debian-iso/sarge/
05/24 14:01:57.59 [22928] 1+0/11: *** /pub/debian-iso/ => 
file://localhost/mnt/mirror/pub/debian-iso/ ***
05/24 14:01:57.59 [22928] 1+0/11: MOUNTED-TO-LOCAL: 
file://localhost/mnt/mirror/pub/debian-iso/
05/24 14:01:57.59 [22928] 1+0/12: local echo for PWD: 
ftp://:0//mnt/mirror/pub/debian-iso/
05/24 14:01:57.59 [22928] 1+0/12: I-SAY: 257 "/pub/debian-iso" is current 
directory.^M
05/24 14:01:57.59 [22928] 1+0/14: *** 
/pub/debian-iso/pub/debian-iso/sarge/ => 
file://localhost/mnt/mirror/pub/debian-iso/pu
b/debian-iso/sarge/ ***

The CD seems to append the absolute path to the CWD.


Bye,

-- 

Steffen Kaiser