Search the web
Sign In
New User? Sign Up
openpfgw · Co-ordination of the OpenPFGW project
  • Members Only
  • Post
  • Files
  • Photos
  • Links
  • Database
  • Polls
  • Members
  • Calendar
  • Promote
  • Groups Labs (Beta)
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Want to share photos of your group with the world? Add a group photo to Flickr.

Best of Y! Groups

   Check them out and nominate your group.
Having problems with message search? Fill out this form to ensure your group is one of the first to be migrated to the new message search system.

Messages

   Messages Help
Message #
Search:
Advanced
Bug in processing huge input files from NewPGen   Message List  
Reply | Forward Message #1830 of 2070 |
I believe there is a problem with writing exceptionally long pfgw.log files.
 
I am using the command line
pfgw -b107 xxx.inp
with an input file coming from NewPGen, and starting:-
33319550988:P:0:2:65
2 71
3 71
...
This file is 4 Gb in size, and has 180,259,222 lines.
 
The pfgw.log file should have about 100M lines.
But the pfgw.log file stops growing at all after about 70M lines, although PFGW continues to run.
 
Furthermore, on killing the task, the file pfgw.log is in a very bad state.
Attempting to delete it with
del pfgw.log
at the command prompt hangs the operating system.
 
There is no problem with the same job on files 1/10 the size.
 
PFGW version:-
The executable file pfgw.exe is date-stamped 30 Jan 2005, and says "Version 1.2.0 for Windows".
 
Operating system:-
Windows 2000 Professional.
 
Can anyone shed light on this - and even better correct the problem?
Thanks.
 
-MIke Oakes
Tue Jun 6, 2006 1:15 pm

Show Message Option
mikeoakes2
Offline Offline
Send Email Send Email

Forward 		Message #1830 of 2070 |
Expand Messages Author Sort by Date

I believe there is a problem with writing exceptionally long pfgw.log files. I am using the command line pfgw -b107 xxx.inp with an input file coming from... 		mikeoakes2@...
mikeoakes2
Offline Send Email 		Jun 6, 2006
1:22 pm

OpenPFGW (win32) uses FILE structures, which under VC (and MinGW), uses a 32 bit max sized file. I would assume that 70m values in the pfgw.log file pushed...
jim_fougeron
Offline Send Email 		Jun 6, 2006
4:11 pm

Thanks Jim. I understand that there are several ways round this. However... This problem came upon me unawares, and could do the same to anyone at any time. A...
mikeoakes2@...
mikeoakes2
Offline Send Email 		Jun 6, 2006
7:40 pm

Hello Mike, ... What do you mean by "quarantineing" the files? What kind of filesystem do you use on your disk, by the way (FAT32? NTFS?) ? Have you tried to...
Peter Kosinar
pkosinar
Offline Send Email 		Jun 6, 2006
10:45 pm

... The HD file system is NTFS. After a reboot, you can /rename/ the file pfgw.log, but any attempt to delete it causes the command prompt to go to 99% cpu...
mikeoakes2@...
mikeoakes2
Offline Send Email 		Jun 7, 2006
8:22 am

... That looks like 2^30 bytes minus just under 1 MB of unflushed cached writes. ... This sounds like a case for backing up your data - writable DVDs, CDRs, ...
Phil Carmody
thefatphil
Offline Send Email 		Jun 7, 2006
4:00 pm

... PFGW doesn't write any files. It asks the OS to write data to the filesystem. That's what OSes are for. ... This OS bug. ... Same company - same mindset. ...
Phil Carmody
thefatphil
Offline Send Email 		Jun 7, 2006
7:55 am

... From: Phil Carmody <thefatphil@...> To: openpfgw@yahoogroups.com Sent: Wed, 7 Jun 2006 00:53:57 -0700 (PDT) Subject: Re: [openpfgw] Re: Bug in...
mikeoakes2@...
mikeoakes2
Offline Send Email 		Jun 7, 2006
9:06 am

... You assume I am presuming. I'm not. I _know_ what an OS's job is, I don't have to make guesses. ... 1 fopen, many fprintfs/fputss, 1 fclose or many {...
Phil Carmody
thefatphil
Offline Send Email 		Jun 7, 2006
1:12 pm

This "bug" is most likely compounded due to the fseek in PFGW. However, step back for a second. First off, why are you using PFGW to PRP test a*71#+1 ?? PFGW...
jim_fougeron
Offline Send Email 		Jun 7, 2006
2:58 pm

... It's done in a very simple manner (don't have the actual sources by hand; checking somewhat older version but I really doubt it has changed much since...
Peter Kosinar
pkosinar
Offline Send Email 		Jun 7, 2006
3:57 pm

hello, I have been following this thread about the log file of pfgw being too large. I use cygwin to run pfgw and I can manipulate files that are larger than...
Simon Plouffe
badaboum0
Offline Send Email 		Jun 6, 2006
7:53 pm

... I agree with the conclusion (and of course, move each pfgw.log aside before ... Each pfgw.log line will be 4 characters longer than the above. "a b" ->...
Phil Carmody
thefatphil
Offline Send Email 		Jun 7, 2006
7:03 am

No-one seems to have pinpointed code which could explain where misbehaviour at c. 1 Gb o/p file size is coming from. [BTW NewPGen, which Paul J tells me also...
mikeoakes2@...
mikeoakes2
Offline Send Email 		Jun 8, 2006
8:54 am

... Paul found it - it's either or both of the "t" in the fopen mode, and/or the unnecessary fseek. As DOS users like their carriage returns, I think even ANSI...
Phil Carmody
thefatphil
Offline Send Email 		Jun 8, 2006
11:34 am

One month later, and still no response at all on this pfgw Winxxx problem, or even for what Winxxx variant(s) it is :-( (I don't count responses which says...
mikeoakes2@...
mikeoakes2
Offline Send Email 		Jul 10, 2006
8:37 am
< Prev Topic  |  Next Topic >
Message #
Search:
Advanced
Copyright © 2009 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines - Help