Error on CVS commit: cvs [commit aborted]: out of memory; can not allocate 441235120 bytes

Discussion:

2017-08-24 16:50:30 UTC

I am trying to commit a file but am receiving errors as follows:
cvs [commit aborted]: out of memory; can not allocate 441235120 bytes
I have a feeling the file must be too large in the cvs repository. the size is below:
11269769 1025804 -r--r--r-- 1 cvs cvs 1049389801 Aug 23 19:29 /repository/cvsroot/udn/netForensics/buildinputs/vms/SunJRELinux-x64.vm,v

I would like the history, but more importantly I need to commit the file with this name. What can I do?If I have to lose the history I will but how can I make this work?
I know it is old as well but cvs is running on a 32 bit linx OS, "Red Hat Enterprise Linux Server release 5 (Tikanga)".
KM

Thorsten Glaser

2017-08-24 20:30:01 UTC

Permalink

Post by KM
cvs [commit aborted]: out of memory; can not allocate 441235120 bytes
I have a feeling the file must be too large in the cvs repository. the siz=

e is below:=C2=A0

Post by KM
11269769 1025804 -r--r--r--=C2=A0=C2=A0 1 cvs=C2=A0=C2=A0=C2=A0=C2=A0=C2=

=A0 cvs=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1049389801 Aug 23 19:29 /repository/c=
vsroot/udn/netForensics/buildinputs/vms/SunJRELinux-x64.vm,v

Yes, the file is over 1 GiB large. It likely also has too many lines.

You could try raising =E2=80=9Culimit -d=E2=80=9D or use an LP64 build of C=
VS on
the server.

bye,
//mirabilos
--=20
[...] if maybe ext3fs wasn't a better pick, or jfs, or maybe reiserfs, oh b=
ut
what about xfs, and if only i had waited until reiser4 was ready... in the =
be-
ginning, there was ffs, and in the middle, there was ffs, and at the end, t=
here
was still ffs, and the sys admins knew it was good. :) -- Ted Unangst =C3=
=BCber *fs

Arthur Barrett

2017-08-25 02:55:33 UTC

Permalink

The per process (virtual) memory of RHEL5 x32 is limited to 3GB:
https://access.redhat.com/articles/rhel-limits

CVS generally needs at least 2x the size of the file for (virtual) memory, and probably more for committing a revision (particularly on a branch).

So *in theory* you should be able to get this to work - but I'm not sure if in practice you will achieve it. Checking rlimit (ulimit) as Thorsten Glaser suggested is worthwhile, as well as checking actual page space / virtual memory allocated to the server etc.

We did a whole heap of work with CVSNT on this error - mostly for windows, though some of the improvements we made could help CVSNT on Linux too.
http://customer.march-hare.com/webtools/bugzilla/ttshow_bug.cgi?id=6317&tt=1

Other things you can try:
- if the CVS server and the client is on the same PC:
+ if using pserver/ssh then try :local: instead
+ if using :local: try :pserver: instead
+ use a different client (or server) so that the memory is spread across the two physical machines

But I don't think CVS is really the best tool for storing files > 100MB or so - you are better off using a backup/archiving system instead.

Regards,

Arthur

-----Original Message-----
From: info-cvs
org] On Behalf Of KM
Sent: 25 August 2017 02:51
To: Cvs-user-list
Subject: Error on CVS commit: cvs [commit aborted]: out of
memory; can notallocate 441235120 bytes
cvs [commit aborted]: out of memory; can not allocate 441235120 bytes
I have a feeling the file must be too large in the cvs
repository. the size is below:
11269769 1025804 -r--r--r-- 1 cvs cvs 1049389801
Aug 23 19:29
/repository/cvsroot/udn/netForensics/buildinputs/vms/SunJRELin
ux-x64.vm,v
I would like the history, but more importantly I need to
commit the file with this name. What can I do?If I have to
lose the history I will but how can I make this work?
I know it is old as well but cvs is running on a 32 bit linx
OS, "Red Hat Enterprise Linux Server release 5 (Tikanga)".
KM

2017-08-25 14:58:32 UTC

Permalink

Thanks for the feedback. I added the file with a new name for now, so it's smaller. it of course as you mentioned doesn't solve the fundamental issue. ... checking in these large data files. the ulimit -v returned unlimited.
I have changed my build script to use the new file instead of the original.
I was wondering if it is possible/recommended to remove the file completely from CVS (the actual repository) so I can re-use the name if I want. I am sure in time this will happen with a few other files of this type. I am always very nervous when doing such things, but I wanted to ask.
thanks in advancekm

From: Arthur Barrett <***@march-hare.com>
To: KM <***@yahoo.com>
Cc: Cvs-user-list <info-***@nongnu.org>
Sent: Thursday, August 24, 2017 10:55 PM
Subject: RE: Error on CVS commit: cvs [commit aborted]: out of memory; can notallocate 441235120 bytes

The per process (virtual) memory of RHEL5 x32 is limited to 3GB:
https://access.redhat.com/articles/rhel-limits

CVS generally needs at least 2x the size of the file for (virtual) memory, and probably more for committing a revision (particularly on a branch).

So *in theory* you should be able to get this to work - but I'm not sure if in practice you will achieve it. Checking rlimit (ulimit) as Thorsten Glaser suggested is worthwhile, as well as checking actual page space / virtual memory allocated to the server etc.

We did a whole heap of work with CVSNT on this error - mostly for windows, though some of the improvements we made could help CVSNT on Linux too.
http://customer.march-hare.com/webtools/bugzilla/ttshow_bug.cgi?id=6317&tt=1

Other things you can try:
- if the CVS server and the client is on the same PC:
+ if using pserver/ssh then try :local: instead
+ if using :local: try :pserver: instead
+ use a different client (or server) so that the memory is spread across the two physical machines

But I don't think CVS is really the best tool for storing files > 100MB or so - you are better off using a backup/archiving system instead.

Regards,

Arthur