[Biodevelopers] XML for huge DB?

[Dan Bolser]

Sorry, I have found the answer... I am an XML 
beginner....

Thanks,
Dan.


On Thu, 31 Jul 2003, Dan Bolser wrote:

> No, the problem is that a big results file can grab 50% of the 4GB
> memory on the system. When I run 4 processes (and a file of this
> size takes about 1 hour to process with XML::Simple) then as soon
> as more that one process encounters a big file I am skuppered.
> 
> I am looking for a memory lite way of parsing the blast results
> files from XML, I.E. one HST at a time with a print event
> for each, rather than whole file at a time processing from
> XML::Simple....
> 
> Dan.
> 
> Michael Gruenberger wrote:
> 
> >Hello,
> >
> >if you are on Unix, does 'nice' do what you are asking?!
> >See:
> >http://www.phys.ksu.edu/~esry/Computing/Nicing.html
> >And: 
> >man nice
> >
> >I don't know if it affects memory usage, but you can give your parsing
> >process a lower priority so it shouldn't take your whole system down...
> >
> >Cheers,
> >
> >Michael
> >On Thu, 2003-07-31 at 16:02, Dan Bolser wrote:
> >  
> >
> >>Hello,
> >>
> >>How can I use XML efficiently to parse multiple blast results
> >>files?
> >>
> >>I want to parse them on a multi processor environment, without
> >>hitting the system memory limit.
> >>
> >>This is likely to happen, as big files take the most time, so the
> >>processes tend to work on big files at the same time, leading
> >>to a system memory outage....
> >>
> >>Cheers,
> >>Dan.
> >>
> >>_______________________________________________
> >>Biodevelopers mailing list
> >>Biodevelopers at bioinformatics.org
> >>https://bioinformatics.org/mailman/listinfo/biodevelopers
> >>    
> >>
> 
> 
> _______________________________________________
> Biodevelopers mailing list
> Biodevelopers at bioinformatics.org
> https://bioinformatics.org/mailman/listinfo/biodevelopers
>