Article freyasx-en/5 of [1-11] on the server localhost:7119
  upper oldest olders older1 this newer1 newers latest
search
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
[Reference:<6e74fad0510050604v4d4ce6cao8873bd4107096b10@mail.gmail.com>]
Newsgroups: mail-lists.freyasx-en

[FreyaSX-En] Re: FreyaSX problem with files > 2GB / compiling on AMD64
07 Oct 2005 09:22:54 GMT ysato@delegate.org (Yutaka Sato)
The DeleGate Project

Hi,

In message <6e74fad0510050604v4d4ce6cao8873bd4107096b10@mail.gmail.com> on 10/05/05(22:04:47)
you Stefan Grothkopp <tum_SF7DUWH.ml@delegate.org> wrote:
 |it worked just fine this time.
 |Thank you very much for your great work,

It's nice.

 |>  |I'm trying to build an index of around 1.2 million web pages.
 |>  |The Problem is, that fmerge seems not to be able to write index files
 |>  |greater than 2 GB.

I'm curious about the size of each files in your case.
Possibly it could be, I suppose, 10GB for .fdif (and the source data),
6GB for .ctx, 4GB for .idx and 2GB for .lex, and so on.

I erased limitations of file size caused by "signed int" in the program
code of FreyaSX, but there are still several limitations due to the data
representation (offset or index of 32bits) stored in index files.
For example, .ctx file cannot be larger than 16GB because each unit data
in it (in 4 bytes) is pointed from .map file with offset data of 32bits.

 |> The revised version has been uploaded as
 |> ftp://ftp.delegate.org/pub/freyasx/freyasx-0.99.12-pre1.tar.gz

I got rid of another limitation for .ctx file and uploaded the version
as the official release "freyasx-0.99.12"

Cheers,
Yutaka
--
  D G   Yutaka Sato <pfqcabdyi-mxhgu472ih3w.ml@delegate.org> http://delegate.org/y.sato/
 ( - )  National Institute of Advanced Industrial Science and Technology
_<   >_ 1-1-4 Umezono, Tsukuba, Ibaraki, 305-8568 Japan
Do the more with the less -- B. Fuller

  admin search upper oldest olders older1 this newer1 newers latest
[Top/Up] [oldest] - [Older+chunk] - [Newer+chunk] - [newest + Check]
@_@V