--- jim_fougeron <jfoug@...> wrote:
> OpenPFGW (win32) uses FILE structures, which under VC (and MinGW),
> uses a 32 bit max sized file. I would assume that 70m values in the
> pfgw.log file pushed over a 4GB level.
>
> Your only alternative, is to cut the file up.

I agree with the conclusion (and of course, move each pfgw.log aside before
starting the next slice), but the figures don't quite add up:

> --- In openpfgw@yahoogroups.com, mikeoakes2@... wrote:
> > 33319550988:P:0:2:65
> > 2 71
> > 3 71
> > ...
> > This file is 4 Gb in size, and has 180,259,222 lines.

> > The pfgw.log file should have about 100M lines.

Each pfgw.log line will be 4 characters longer than the above.
"a b" -> "a*2^b+1"

The target pfgw.log size therefore would be
(5/9)*(4G+720M) = 2.6G

If the 4G was really 4.3G (decimal multipliers, of course),
then the above becomes 2.8G

> > But the pfgw.log file stops growing at all after about 70M lines,

(70/100)*2.6G = 1.8G
(70/100)*2.8G = 1.95G

Which points more towards a 2^31 limit than a 2^32 one.
However, both fall short of even 2^31.

One possible culpret would be this:
long ftell(FILE *stream);
int fseek(FILE *stream, long offset, int whence);

But even if you're not using the above, another could be:
struct stat {
...
off_t st_size; /* total size, in bytes */

typedef __off_t off_t;
__STD_TYPE __OFF_T_TYPE __off_t;
#define __OFF_T_TYPE __SLONGWORD_TYPE
#define __SLONGWORD_TYPE long int

I assume windows has an equally 1980s set of types.

So basically anything you want to do with files that involves knowing its size
(such as seeking to the end, to append) hits a brick wall at 2^31.

Good for controlling washing machines, that's all.

Phil

() ASCII ribbon campaign () Hopeless ribbon campaign
/\ against HTML mail /\ against gratuitous bloodshed

[stolen with permission from Daniel B. Cristofani]

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com