[Bioclusters] FW: [PBS-USERS] max file size

[Goran Ceric]

I accidentaly sent this to George Jones instead to the list because it came
from PBS-Users, not Bioclusters list. Here it is again. I remember using
same kind of flags on some older versions of tcsh and it didn't work. It
could have been my mistake, it doesn't matter anyway. I created some tcsh
6.11 rpms at http://www.genetics.wustl.edu/eddy/people/goran/tcsh.php Maybe
someone will find them useful. I also posted spec file in case someone wants
to add to it to create more complicated builds.

Goran Ceric
System Administrator
Washington University, St. Louis
Department of Genetics, Eddy Lab
goran@genetics.wustl.edu
http://www.genetics.wustl.edu/eddy


-----Original Message-----
From: owner-pbs-users@PBSpro.com [mailto:owner-pbs-users@PBSpro.com]On
Behalf Of Jones George
Sent: Monday, May 13, 2002 6:40 AM
To: pbs-users@PBSpro.com; gwinn.allan@towerautomotive.com
Subject: Re: [PBS-USERS] max file size


This issue came up in the BioClusters discussion group.  Here is how one
person solved the issue with tcsh that is shipped with redhat.  Hope it
helps..

Folks:

  Ivo and Goran are correct about the tcsh shipped with RedHat.  I will
send a quick note to RedHat tomorrow with this suggestion.

  The "fix" to tcsh to handle large files works like this:

1) pull down tcsh tarball (6.11.0 is the latest, I grabbed it from
ftp://ftp.gw.com/pub/unix/tcsh/)

2) untar-gzip into a directory.

3) run ./configure (alter any options you want)

4) now edit the Makefile, and add the following to DFLAGS= line
(something like line 139 or so).

	 -D__USE_FILE_OFFSET64 -D__USE_LARGEFILE64

5) run make

6) sanity check.

    (original RedHat tcsh)

    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    6 >ls -alF dbEST.fsa
    -rw-r--r--    1 landman  landman  4490972019 Aug 15  2001 dbEST.fsa
    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    7 >df -m .
    Filesystem           1M-blocks      Used Available Use% Mounted on
    /dev/md0                 62494     13628     48866  22% /scratch
    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    8 >cp dbEST.fsa > d
    cp: missing destination file
    Try `cp --help' for more information.
    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    9 >cp dbEST.fsa  d
    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    10 >ls -alF d
    -rw-rw-r--    1 landman  landman  4490972019 May  9 00:23 d
    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    11 >ls -alF dbEST.fsa
    -rw-r--r--    1 landman  landman  4490972019 Aug 15  2001 dbEST.fsa
    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    12 >cat dbEST.fsa >> d
    Filesize limit exceeded (core dumped)

    (with the compiled tcsh)

    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    4 >cat dbEST.fsa >> d
    [landman@genome.dtw.macsch.com:/scratch/bio/biotest]
    5 >ls -alF d
    -rw-rw-r--    1 landman  landman  8981944038 May  9 00:36 d

And it seems to work.

Joe

On Thu, 2002-05-09 at 00:07, Ivo Grosse wrote:
> Goran Ceric <goran@genetics.wustl.edu> wrote on Wed, 8 May 2002:
>
> > I cannot for example do cat file1 >> file2 into a file > 2 GB using tcsh
>
> Neither can I ... I apologize for my previous email ... and now I would
> like to repeat your question ... with twice the volume: why can't tcsh
> handle large files?
>
> Ivo

>>> "Allan Gwinn" <gwinn.allan@towerautomotive.com> 05/10/02 03:42PM >>>
Hello all,

We are running msc linux on a small cluster.
It seems that we run into a 2GB filesize limit when jobs are submitted via
pbs.
When they are not submitted via pbs there is no such problem.

In the pbs.conf file I tried setting resources_max.file=something larger
than 2GB
and this did not seem to work.

Can anyone shed some light on this?

Thanks,

    -Allan

__________________________________________________________________________
To unsubscribe: email majordomo@OpenPBS.org with body "unsubscribe
pbs-users"
For message archives: http://www.OpenPBS.org/UserArea/pbs-users.html
    -    -    -    -    -    -    -    -    -    -    -    -    -    -
Academic Site? Use PBS Pro free, see: http://www.PBSpro.com/academia.html
OpenPBS and the pbs-users mailing list are sponsored by Veridian.
__________________________________________________________________________