about imports and cvs update

[Harry Putnam]

I don't do many imports... maybe 5-5 in the last 10 yrs..
And never did one that was not the start of a new repository.

Is it a misuse of import to use it to start a new directory hierarchy
beneath some exiting hierarchy in CVSROOT?

I mean like this:

Existing CVSROOT
CVSROOT/base/
subdir/
subdir/
subdir/

CVSROOT after the import:
CVSROOT/base/
subdir/
NEW_IMPORT/ (also subdir of base)
subdir/
subdir/

If that is NOT a misuse:
Can someone tell me if this behavior is normal.

I have a checked out module. Its the whole of CVSROOT/base.
(not much on my home single user machine)

module name is `base'
So I have `base' checked out at ~/base

On a remote machine (on home lan) I've imported a directory hierarchy
like so:

cvs import -m "some blab" base/NEW_IMPORT blah start

Now, back on the local host which is also the cvs server for the home
lan, I go to the checked out module, which hasn't been used since I
did the import.

I run `cvs -n update -dP'.. just checking.

It shows no output.

Shouldn't cvs be telling me about the newly imported directory
hierarchy under `base', and be listing files needing to be updated?

[Michael Haggerty]

Post by Harry Putnam
I don't do many imports... maybe 5-5 in the last 10 yrs..
And never did one that was not the start of a new repository.
Is it a misuse of import to use it to start a new directory hierarchy
beneath some exiting hierarchy in CVSROOT?

IMHO "cvs import" should only be used to create a vendor branch, which
is a branch that is used to actively track sources from a third party.
There is no need to use "cvs import" to get your own source code into
CVS, as it does lots of extra stuff that is useless when not tracking
third-party sources. There are also several aspects of vendor branches
that are not recorded carefully by CVS, so the (ab)use of vendor
branches can make it harder to migrate away from CVS when that
inevitable day comes.

To add a new directory hierarchy within a CVS repository, simply check
out a working copy, copy the new directories to the right place in the
directory hierarchy, then use "cvs add" and "cvs commit" to put them
under version control.

Michael

[Harry Putnam]

Post by Michael Haggerty

Post by Harry Putnam
I don't do many imports... maybe 5-5 in the last 10 yrs..
And never did one that was not the start of a new repository.
Is it a misuse of import to use it to start a new directory hierarchy
beneath some exiting hierarchy in CVSROOT?

IMHO "cvs import" should only be used to create a vendor branch, which
is a branch that is used to actively track sources from a third party.
There is no need to use "cvs import" to get your own source code into
CVS, as it does lots of extra stuff that is useless when not tracking
third-party sources. There are also several aspects of vendor branches
that are not recorded carefully by CVS, so the (ab)use of vendor
branches can make it harder to migrate away from CVS when that
inevitable day comes.
To add a new directory hierarchy within a CVS repository, simply check
out a working copy, copy the new directories to the right place in the
directory hierarchy, then use "cvs add" and "cvs commit" to put them
under version control.

Well, that is true, but that method requires a lot more hand work if
the added sub hierarchy is more than 1 directory and a few files.

So does cvs not offer any method to add a whole chunk of directories
and files en masse.

Once an import has been done, as in this case.. is there any
continuing harm?

What I did to get back on an even keel was delete the checked out
module that shows no updates needed, and re-checkout.. this time the
import is present.

So will I have continuing problems with that sub hierarchy?

" . . . . . . . . . . . . . . . . . . so the (ab)use of vendor

Post by Michael Haggerty
branches can make it harder to migrate away from CVS when that
inevitable day comes."

Sounds like you think cvs is on its last legs... I had hoped to
continue using it for a good while yet. Is it likely to fall out of
development?

To me, some of the alternatives like subversion or git, seem really
confusing, at least in there documentation, (I've used neither) with
there emphasis on revisions depending on the whole repo snapshot.

[Spiro Trikaliotis]

Hello Harry,

Post by Harry Putnam

Post by Michael Haggerty
To add a new directory hierarchy within a CVS repository, simply check
out a working copy, copy the new directories to the right place in the
directory hierarchy, then use "cvs add" and "cvs commit" to put them
under version control.

Well, that is true, but that method requires a lot more hand work if
the added sub hierarchy is more than 1 directory and a few files.

It can be automated with two calls to find(1).

Post by Harry Putnam
Once an import has been done, as in this case.. is there any
continuing harm?

Note that Michael is working on cvs2svn, a conversion tool to convert
from CVS to Subversion (SVN). I am sure he knows what he is speaking
about. ;)

I converted some CVS repositories to SVN with cvs2svn myself. Every
time, a vendor branch generated some problems which had to be solved by
hand, especially if the vendor branch was used in a way it was not meant
to be used. But, please don't ask me of the details, I don't remember.

Post by Harry Putnam
Sounds like you think cvs is on its last legs... I had hoped to
continue using it for a good while yet. Is it likely to fall out of
development?

I don't think so. There are some alternatives, yes, but every one has
its pros and cons. What I like most is the ability of CVS to have a kind
of "view", that is, a checked out sandbox which combines parts of
different branches. With tagging in the right way, this can be totally
automated. Neither SVN nor git (the two you mentioned) have this ability
- at least, they did not the time I checked them.

Thus, I think every tool has its advantages and disadvantages. Use the
one which suits your needs best.

Best regards,
Spiro.

[Harry Putnam]

Post by Spiro Trikaliotis

Post by Harry Putnam
Well, that is true, but that method requires a lot more hand work if
the added sub hierarchy is more than 1 directory and a few files.

It can be automated with two calls to find(1).

Thanks for this and the other input I snipped.

Sorry to be a pest but working with cvs only as a single user on home
setup hasn't taught me a lot about better ways of using it.

I'd be very interested to see any kind of automating or scripting
people have developed from more serious use than I'll ever have.

Do you mind illustrating what you mean about the calls to `find'?

An outline with no details of any other methods you use with cvs that
makes things easier would be well appreciated.

I don't mean necessarily `NO' details... but only as much as is handy
for you.

I'm just asking for anything easily passed along with a minimum of
effort on your part.

[Spiro Trikaliotis]

Hello Harry,

Post by Harry Putnam
Do you mind illustrating what you mean about the calls to `find'?

Ok. I assume there are no file names that are insecure to use on the
command line! The same holds true for directory names.

Then you can add all directories by using

$ find . -type d | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --

The "find" searches for every directory ("-type d") in the current
directory (".") and every sub-directory. The "grep" ensures we do not
handle CVS directories, if these exist. (Note: The "\/CVS$" filters out
("-v") every directory that ends with /CVS.)

Then, I use the file names as parameters for a "cvs add". This add all
directories. The "-n 10" ensures that no more than 10 directories are
used for every "cvs add" command.

The "tr \\n \\0" makes sure that we replace every newline (\n) with an
ASCII NUL (\0). The helps us for xargs, as we use the -0 option. This
way, if a directory contains a space, it is handled correctly.


Having done this, the directory structure was generated. Now, we can add
all the files:

$ find . -type f | grep -v \/CVS\/ | tr \\n \\0 | xargs -0 -n 10 cvs add --

We search files ("-type f"), leave out everything that is in a "CVS"
directory (grep -v \/CVS\/), then we replace every newline with NUL
again and give the output to a "cvs add" command.


I am doing this in two steps (first the directories, then the files), as
I am not sure if find actually guarantees that directories are output
before the files that are in them. Additonally, I can give different
options to "cvs add" for files and directories.

Otherwise, you might want to use something like:

$ find . | grep -v \/CVS\/ | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --


For all of this, there is one simple rule: Use at your own risk!

HTH,
Spiro.

[Harry Putnam]

Post by Spiro Trikaliotis
$ find . | grep -v \/CVS\/ | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --

Nice.. and thanks for the detailed exposition. I've used unix like
OS for many yrs, and still often am slightly awed by the power of its
supposed simple tools in the hands of someone who knows how to use them.

[Kenneth Wolcott]

Hi Harry;

Post by Spiro Trikaliotis

Post by Spiro Trikaliotis
$ find . | grep -v \/CVS\/ | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10

cvs add --
Nice.. and thanks for the detailed exposition. I've used unix like
OS for many yrs, and still often am slightly awed by the power of its
supposed simple tools in the hands of someone who knows how to use them.

I strongly agree with your comment here. It is *WONDERFUL* to see some
well-written UNIX code with very well-written exposition to go with it. It
is definitely amazing the POWER that UNIX can have in the right hands.

Ken Wolcott

[Spiro Trikaliotis]

Hello,

Post by Harry Putnam

Post by Spiro Trikaliotis
$ find . | grep -v \/CVS\/ | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --

Nice.. and thanks for the detailed exposition. I've used unix like
OS for many yrs, and still often am slightly awed by the power of its
supposed simple tools in the hands of someone who knows how to use them.

... but this surely does *not* apply to my code above, nor to me. ;)

Best regards,
Spiro.

[Harry Putnam]

Post by Spiro Trikaliotis
Hello,

Post by Harry Putnam

Post by Spiro Trikaliotis
$ find . | grep -v \/CVS\/ | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --

Nice.. and thanks for the detailed exposition. I've used unix like
OS for many yrs, and still often am slightly awed by the power of its
supposed simple tools in the hands of someone who knows how to use them.

... but this surely does *not* apply to my code above, nor to me. ;)

OK, maybe the double use of grep -v could have been combined (but then
wouldn't be quite so apparent what its doing). But
otherwise it is a nice display of how to use unix tools...

[Spiro Trikaliotis]

Hello again,
[...]

Post by Harry Putnam

Post by Spiro Trikaliotis
$ find . | grep -v \/CVS\/ | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --

OK, maybe the double use of grep -v could have been combined

... but this looks *really* ugly:

$ find . | grep -v \/CVS\\\(\/\\\|$\\\) | tr \\n \\0 | xargs -0 -n 10 cvs add --

or, a little bit better:

$ find . | grep -v "\/CVS\(\/\|$\)" | tr \\n \\0 | xargs -0 -n 10 cvs add --

Either way, I prefer understandability over efficiency here. It is not
that such a skript is running 24/7 on my machines. ;)


Anyhow, I am sure there are better ways to achieve the intended result
than what I have written here.

Best regards,
Spiro.

[Michael Haggerty]

Post by Spiro Trikaliotis
Ok. I assume there are no file names that are insecure to use on the
command line! The same holds true for directory names.
Then you can add all directories by using
$ find . -type d | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --

The find command is flexible enough to do most of the work itself:

$ find . -type d \( -name CVS -prune -o -print0 \) | xargs -0 -n 10 cvs
add --

and this variation should be safe regardless of what crazy characters
are present in filenames.

Post by Spiro Trikaliotis
Having done this, the directory structure was generated. Now, we can add
$ find . -type f | grep -v \/CVS\/ | tr \\n \\0 | xargs -0 -n 10 cvs add --

Similarly,

$ find . -type d -name CVS -prune -o -type f -print0 | xargs -0 -n 10
cvs add --

Post by Spiro Trikaliotis
For all of this, there is one simple rule: Use at your own risk!

Ditto.

Michael

[Spiro Trikaliotis]

Hello Michael,
[...]

Post by Spiro Trikaliotis
$ find . -type d | grep -v \/CVS$ | tr \\n \\0 | xargs -0 -n 10 cvs add --

$ find . -type d \( -name CVS -prune -o -print0 \) | xargs -0 -n 10 cvs add --
and this variation should be safe regardless of what crazy characters
are present in filenames.

Thank you!

I already thought it must be easier than grepping through it myself.
However, I did not find the right places in the manpage of find in order
to solve this.

I already knew about -print0 - in fact, I am using it regularly.
However, this would have interferred with my grep, thus, I avoided it.

Again, thank you for giving me the starting point to really understand
find.

Best regards,
Spiro.

[Harry Putnam]

Post by Spiro Trikaliotis

$ find . -type d \( -name CVS -prune -o -print0 \) | xargs -0 -n 10 cvs add --
and this variation should be safe regardless of what crazy characters
are present in filenames.

Thank you!

I like it but:
Now you've stepped away from readability a slight bit in my opinion. I
suspect there are many cvs users even with unix backgrounds who would
have to ponder the -prune and -print0 flags a bit.

I think maybe the double grep -v is a bit more obvious. I know its not
really considered kosher by old unix hands...

I use find very frequently and have for many yrs, but would have had
to pound on `man find' a while to come up with that.

I've used both -prune and -print0 but don't often find need of it. In
fact, only rarely.

[Spiro Trikaliotis]

Hello Harry,

Post by Harry Putnam

Post by Spiro Trikaliotis

$ find . -type d \( -name CVS -prune -o -print0 \) | xargs -0 -n 10 cvs add --
and this variation should be safe regardless of what crazy characters
are present in filenames.

Thank you!

Now you've stepped away from readability a slight bit in my opinion.

This might be, but I am sure that the version with -print0 is "safer"
than mimiking the same with the "tr \\n \\0" command. The latter tries
to insert the null bytes after they have been generated. I am not
completely sure if I did not overlook any case where we could already
have an enter in a file name (is this possible on any platform)?

Post by Harry Putnam
I think maybe the double grep -v is a bit more obvious. I know its not
really considered kosher by old unix hands...

The double grep was especially needed as I was not completely sure how
many backslashes I needed to quote, and I was too lazy to try out. ;)

Post by Harry Putnam
I've used both -prune and -print0 but don't often find need of it. In
fact, only rarely.

I use -print0 almost always with I pipe it through xargs (with -0), and
I would advise to do so whenever possible.

Best regards,
Spiro.

[Michael Haggerty]

Post by Harry Putnam

Post by Michael Haggerty
To add a new directory hierarchy within a CVS repository, simply check
out a working copy, copy the new directories to the right place in the
directory hierarchy, then use "cvs add" and "cvs commit" to put them
under version control.

Well, that is true, but that method requires a lot more hand work if
the added sub hierarchy is more than 1 directory and a few files.
So does cvs not offer any method to add a whole chunk of directories
and files en masse.

"cvs add" is not recursive, but it's trivial anyway with a shell command
or two; something like

find new-subdir -name CVS -prune -o -print | xargs cvs add

should do the trick. Anyway, you might want a little human interaction
anyway to make sure, for example, that binary files are marked as such
and unnecessary junk does not get added to the repository.

Post by Harry Putnam
Once an import has been done, as in this case.. is there any
continuing harm?

Nothing serious, but the CVS history remains unnecessarily complicated.

Post by Harry Putnam
" . . . . . . . . . . . . . . . . . . so the (ab)use of vendor

Post by Michael Haggerty
branches can make it harder to migrate away from CVS when that
inevitable day comes."

Sounds like you think cvs is on its last legs... I had hoped to
continue using it for a good while yet. Is it likely to fall out of
development?

Although the popularity of CVS relative to that of other VCSs definitely
seems to be in decline, I certainly did not mean to imply that CVS will
stop existing anytime soon. But this is not the right forum for a
debate about the relative merits of other VCSs relative to CVS. There
are certainly enough comparisons on the internet for anybody who is
interested.

Michael

[Harry Putnam]

Post by Michael Haggerty
"cvs add" is not recursive, but it's trivial anyway with a shell command
or two; something like
find new-subdir -name CVS -prune -o -print | xargs cvs add
should do the trick. Anyway, you might want a little human interaction
anyway to make sure, for example, that binary files are marked as such
and unnecessary junk does not get added to the repository.

Thanks... that looks helpful.

[Larry Jones]

Post by Harry Putnam
I don't do many imports... maybe 5-5 in the last 10 yrs..
And never did one that was not the start of a new repository.
Is it a misuse of import to use it to start a new directory hierarchy
beneath some exiting hierarchy in CVSROOT?

Not at all -- CVS really doesn't care about the directory structure.

Post by Harry Putnam
On a remote machine (on home lan) I've imported a directory hierarchy
cvs import -m "some blab" base/NEW_IMPORT blah start
Now, back on the local host which is also the cvs server for the home
lan, I go to the checked out module, which hasn't been used since I
did the import.
I run `cvs -n update -dP'.. just checking.
It shows no output.

Yes, that's normal.

Post by Harry Putnam
Shouldn't cvs be telling me about the newly imported directory
hierarchy under `base', and be listing files needing to be updated?

Yes it should, but the way the CVS code is structured prevents update
with -n from doing everything that it does without the -n. Actually
running the update (without the -n) will do the right thing.

[Harry Putnam]

Post by Larry Jones

Post by Harry Putnam
Shouldn't cvs be telling me about the newly imported directory
hierarchy under `base', and be listing files needing to be updated?

Yes it should, but the way the CVS code is structured prevents update
with -n from doing everything that it does without the -n. Actually
running the update (without the -n) will do the right thing.

OK, good to know I wasn't just daydreaming. And thanks for the
details. Always very helpful to here from experienced people.