I've always believed that, with respect to Kernel objects, reference
counting meant that the Kernel kept track of the number of handles
(unique 32-bit numbers) that it issued for each Kernel object. In most
cases, when that handle comes back to the Kernel in a call to
CloseHandle() or similar, the count is decremented and, if now zero, the
Kernel object is destroyed.

A recent experience seems to indicate that this belief is incorrect.

Please observe the following code. It's part of the preamble to a call
to CreateProcess() using pipes to receive the output of a command line
program. Two pipes are created and the handles of the input ends are
duplicated to make them inheritable. DUPLICATE_CLOSE_SOURCE should cause
the original non-inheritable handle to be closed.
CreatePipe(&hpipeStdOut, &hPipe, NULL, BUFFER_SIZE);
DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdOutput,
0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
);
assert(!CloseHandle(hPipe));
CreatePipe(&hpipeStdErr, &hPipe, NULL, BUFFER_SIZE);
DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdError,
0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
);
assert(!CloseHandle(hPipe));

Originally, it was written without the assert() statements and, as far
as I can tell, worked correctly.

I added the assert() statements as part of an effort to assure myself
that the program was no leaking handles. (The application of which this
a small part will pass through this code many times during its execution.)

I was surprised to find that the assertions failed, meaning that
CloseHandle() was returning a non-zero value indicating success--it was
able to the handles being passed and that in turn meant that the
original handles were _not_ closed by DuplicateHandle().

Further, I found that when my application attempted to read from the
output end of the pipes the ReadFile() failed with ERROR_BROKEN_PIPE
indicating that the input handle had been closed even though the command
line program had not had a chance to terminate. It looks like
CloseHandle() with the original handle closes both the original handle
and the duplicated handle.

I moved the assert() statement to a point after the call to
CreateProcess() and after the calls to CloseHandle() that close the two
startupInfo handles. (I don't need them anymore since they've been
inherited into the command line program by this point.) Now I get a
first chance exception from the CloseHandle() in the assert() and the
assertion fails; assert() pops up a message box.

Here's what I now believe to be happening: the Kernel is not counting
the number of unique handles that have been passed out, but is instead
counting the number of processes to which those handles have been
passed. The Kernel treats any handle owned by a process as equivalent,
at least in the context of CloseHandle(). (Any handle passed to
CloseHandle() closes all handles to that Kernel object that are owned by
the calling process.) I may, if I can find time, do some more
investigation to see whether all handles are treated as equivalent with
respect to access and inheritance.

Any thoughts on this? Is this behavior documented somewhere?

Norm

--
--
To reply, change domain to an adult feline.

> Any handle passed to
> CloseHandle() closes all handles to that Kernel object that are owned by
> the calling process.

I doubt very much that this could possibly be true.

It seems to me that you are jumping into conclusions. For example, I can
offer an alternative explanation. (Note, it is purely theoretical, not based
on any knowledge of kernel internals, and I didn't check anything of it in
practice, so it is just a speculation.) What could happen is that when you
tell the kernel that you won't be needing the old handle (by setting
DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
In that case the handle produced by DuplicateHandle (the one put into
startupinfo) would have the same value as the original handle (but it
wouldn't be the same handle though). Then your call to CloseHandle would
close the new (duplicated) handle.

Does this make any sense to you? Of course, I'm sure one can think of more
possible explanations for what you are observing.
--
Regards,
Sergei.

> Any handle passed to
> CloseHandle() closes all handles to that Kernel object that are owned by
> the calling process.

I doubt very much that this could possibly be true.

It seems to me that you are jumping into conclusions. For example, I can
offer an alternative explanation. (Note, it is purely theoretical, not based
on any knowledge of kernel internals, and I didn't check anything of it in
practice, so it is just a speculation.) What could happen is that when you
tell the kernel that you won't be needing the old handle (by setting
DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
In that case the handle produced by DuplicateHandle (the one put into
startupinfo) would have the same value as the original handle (but it
wouldn't be the same handle though). Then your call to CloseHandle would
close the new (duplicated) handle.

Does this make any sense to you? Of course, I'm sure one can think of more
possible explanations for what you are observing.
--
Regards,
Sergei.

> Any handle passed to
> CloseHandle() closes all handles to that Kernel object that are owned by
> the calling process.

I doubt very much that this could possibly be true.

It seems to me that you are jumping into conclusions. For example, I can
offer an alternative explanation. (Note, it is purely theoretical, not based
on any knowledge of kernel internals, and I didn't check anything of it in
practice, so it is just a speculation.) What could happen is that when you
tell the kernel that you won't be needing the old handle (by setting
DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
In that case the handle produced by DuplicateHandle (the one put into
startupinfo) would have the same value as the original handle (but it
wouldn't be the same handle though). Then your call to CloseHandle would
close the new (duplicated) handle.

Does this make any sense to you? Of course, I'm sure one can think of more
possible explanations for what you are observing.
--
Regards,
Sergei.

"Norman Bullen" <norm@BlackKittenAssociates.com.INVALID> wrote in message
news:DzMOf.2188$Bj7.1401@newsread2.news.pas.earthlink.net...
> I've always believed that, with respect to Kernel objects, reference
> counting meant that the Kernel kept track of the number of handles (unique
> 32-bit numbers) that it issued for each Kernel object. In most cases, when
> that handle comes back to the Kernel in a call to CloseHandle() or
> similar, the count is decremented and, if now zero, the Kernel object is
> destroyed.

Not quite, kernel objects can also be referenced by kernel mode modules,
this also increases the reference count. It is not necessarily true that
NtCloseHandle will cause the object to be destroyed, but it will remove it
from the process handle table for the process context in which you called
NtCloseHandle.

Carly


> A recent experience seems to indicate that this belief is incorrect.
>
> Please observe the following code. It's part of the preamble to a call to
> CreateProcess() using pipes to receive the output of a command line
> program. Two pipes are created and the handles of the input ends are
> duplicated to make them inheritable. DUPLICATE_CLOSE_SOURCE should cause
> the original non-inheritable handle to be closed.
> CreatePipe(&hpipeStdOut, &hPipe, NULL, BUFFER_SIZE);
> DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdOutput,
> 0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
> );
> assert(!CloseHandle(hPipe));
> CreatePipe(&hpipeStdErr, &hPipe, NULL, BUFFER_SIZE);
> DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdError,
> 0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
> );
> assert(!CloseHandle(hPipe));
>
> Originally, it was written without the assert() statements and, as far as
> I can tell, worked correctly.
>
> I added the assert() statements as part of an effort to assure myself that
> the program was no leaking handles. (The application of which this a small
> part will pass through this code many times during its execution.)
>
> I was surprised to find that the assertions failed, meaning that
> CloseHandle() was returning a non-zero value indicating success--it was
> able to the handles being passed and that in turn meant that the original
> handles were _not_ closed by DuplicateHandle().
>
> Further, I found that when my application attempted to read from the
> output end of the pipes the ReadFile() failed with ERROR_BROKEN_PIPE
> indicating that the input handle had been closed even though the command
> line program had not had a chance to terminate. It looks like
> CloseHandle() with the original handle closes both the original handle and
> the duplicated handle.
>
> I moved the assert() statement to a point after the call to
> CreateProcess() and after the calls to CloseHandle() that close the two
> startupInfo handles. (I don't need them anymore since they've been
> inherited into the command line program by this point.) Now I get a first
> chance exception from the CloseHandle() in the assert() and the assertion
> fails; assert() pops up a message box.
>
> Here's what I now believe to be happening: the Kernel is not counting the
> number of unique handles that have been passed out, but is instead
> counting the number of processes to which those handles have been passed.
> The Kernel treats any handle owned by a process as equivalent, at least in
> the context of CloseHandle(). (Any handle passed to CloseHandle() closes
> all handles to that Kernel object that are owned by the calling process.)
> I may, if I can find time, do some more investigation to see whether all
> handles are treated as equivalent with respect to access and inheritance.
>
> Any thoughts on this? Is this behavior documented somewhere?
>
> Norm
>
> --
> --
> To reply, change domain to an adult feline.
>

"Norman Bullen" <norm@BlackKittenAssociates.com.INVALID> wrote in message
news:DzMOf.2188$Bj7.1401@newsread2.news.pas.earthlink.net...
> I've always believed that, with respect to Kernel objects, reference
> counting meant that the Kernel kept track of the number of handles (unique
> 32-bit numbers) that it issued for each Kernel object. In most cases, when
> that handle comes back to the Kernel in a call to CloseHandle() or
> similar, the count is decremented and, if now zero, the Kernel object is
> destroyed.

Not quite, kernel objects can also be referenced by kernel mode modules,
this also increases the reference count. It is not necessarily true that
NtCloseHandle will cause the object to be destroyed, but it will remove it
from the process handle table for the process context in which you called
NtCloseHandle.

Carly


> A recent experience seems to indicate that this belief is incorrect.
>
> Please observe the following code. It's part of the preamble to a call to
> CreateProcess() using pipes to receive the output of a command line
> program. Two pipes are created and the handles of the input ends are
> duplicated to make them inheritable. DUPLICATE_CLOSE_SOURCE should cause
> the original non-inheritable handle to be closed.
> CreatePipe(&hpipeStdOut, &hPipe, NULL, BUFFER_SIZE);
> DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdOutput,
> 0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
> );
> assert(!CloseHandle(hPipe));
> CreatePipe(&hpipeStdErr, &hPipe, NULL, BUFFER_SIZE);
> DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdError,
> 0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
> );
> assert(!CloseHandle(hPipe));
>
> Originally, it was written without the assert() statements and, as far as
> I can tell, worked correctly.
>
> I added the assert() statements as part of an effort to assure myself that
> the program was no leaking handles. (The application of which this a small
> part will pass through this code many times during its execution.)
>
> I was surprised to find that the assertions failed, meaning that
> CloseHandle() was returning a non-zero value indicating success--it was
> able to the handles being passed and that in turn meant that the original
> handles were _not_ closed by DuplicateHandle().
>
> Further, I found that when my application attempted to read from the
> output end of the pipes the ReadFile() failed with ERROR_BROKEN_PIPE
> indicating that the input handle had been closed even though the command
> line program had not had a chance to terminate. It looks like
> CloseHandle() with the original handle closes both the original handle and
> the duplicated handle.
>
> I moved the assert() statement to a point after the call to
> CreateProcess() and after the calls to CloseHandle() that close the two
> startupInfo handles. (I don't need them anymore since they've been
> inherited into the command line program by this point.) Now I get a first
> chance exception from the CloseHandle() in the assert() and the assertion
> fails; assert() pops up a message box.
>
> Here's what I now believe to be happening: the Kernel is not counting the
> number of unique handles that have been passed out, but is instead
> counting the number of processes to which those handles have been passed.
> The Kernel treats any handle owned by a process as equivalent, at least in
> the context of CloseHandle(). (Any handle passed to CloseHandle() closes
> all handles to that Kernel object that are owned by the calling process.)
> I may, if I can find time, do some more investigation to see whether all
> handles are treated as equivalent with respect to access and inheritance.
>
> Any thoughts on this? Is this behavior documented somewhere?
>
> Norm
>
> --
> --
> To reply, change domain to an adult feline.
>

"Norman Bullen" <norm@BlackKittenAssociates.com.INVALID> wrote in message
news:DzMOf.2188$Bj7.1401@newsread2.news.pas.earthlink.net...
> I've always believed that, with respect to Kernel objects, reference
> counting meant that the Kernel kept track of the number of handles (unique
> 32-bit numbers) that it issued for each Kernel object. In most cases, when
> that handle comes back to the Kernel in a call to CloseHandle() or
> similar, the count is decremented and, if now zero, the Kernel object is
> destroyed.

Not quite, kernel objects can also be referenced by kernel mode modules,
this also increases the reference count. It is not necessarily true that
NtCloseHandle will cause the object to be destroyed, but it will remove it
from the process handle table for the process context in which you called
NtCloseHandle.

Carly


> A recent experience seems to indicate that this belief is incorrect.
>
> Please observe the following code. It's part of the preamble to a call to
> CreateProcess() using pipes to receive the output of a command line
> program. Two pipes are created and the handles of the input ends are
> duplicated to make them inheritable. DUPLICATE_CLOSE_SOURCE should cause
> the original non-inheritable handle to be closed.
> CreatePipe(&hpipeStdOut, &hPipe, NULL, BUFFER_SIZE);
> DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdOutput,
> 0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
> );
> assert(!CloseHandle(hPipe));
> CreatePipe(&hpipeStdErr, &hPipe, NULL, BUFFER_SIZE);
> DuplicateHandle(hProcess, hPipe, hProcess, &startupInfo.hStdError,
> 0, TRUE, DUPLICATE_CLOSE_SOURCE|DUPLICATE_SAME_ACCESS
> );
> assert(!CloseHandle(hPipe));
>
> Originally, it was written without the assert() statements and, as far as
> I can tell, worked correctly.
>
> I added the assert() statements as part of an effort to assure myself that
> the program was no leaking handles. (The application of which this a small
> part will pass through this code many times during its execution.)
>
> I was surprised to find that the assertions failed, meaning that
> CloseHandle() was returning a non-zero value indicating success--it was
> able to the handles being passed and that in turn meant that the original
> handles were _not_ closed by DuplicateHandle().
>
> Further, I found that when my application attempted to read from the
> output end of the pipes the ReadFile() failed with ERROR_BROKEN_PIPE
> indicating that the input handle had been closed even though the command
> line program had not had a chance to terminate. It looks like
> CloseHandle() with the original handle closes both the original handle and
> the duplicated handle.
>
> I moved the assert() statement to a point after the call to
> CreateProcess() and after the calls to CloseHandle() that close the two
> startupInfo handles. (I don't need them anymore since they've been
> inherited into the command line program by this point.) Now I get a first
> chance exception from the CloseHandle() in the assert() and the assertion
> fails; assert() pops up a message box.
>
> Here's what I now believe to be happening: the Kernel is not counting the
> number of unique handles that have been passed out, but is instead
> counting the number of processes to which those handles have been passed.
> The Kernel treats any handle owned by a process as equivalent, at least in
> the context of CloseHandle(). (Any handle passed to CloseHandle() closes
> all handles to that Kernel object that are owned by the calling process.)
> I may, if I can find time, do some more investigation to see whether all
> handles are treated as equivalent with respect to access and inheritance.
>
> Any thoughts on this? Is this behavior documented somewhere?
>
> Norm
>
> --
> --
> To reply, change domain to an adult feline.
>

Sergei Zhirikov wrote:
>>Any handle passed to
>>CloseHandle() closes all handles to that Kernel object that are owned by
>>the calling process.
>
>
> I doubt very much that this could possibly be true.
>
> It seems to me that you are jumping into conclusions. For example, I can
> offer an alternative explanation. (Note, it is purely theoretical, not based
> on any knowledge of kernel internals, and I didn't check anything of it in
> practice, so it is just a speculation.) What could happen is that when you
> tell the kernel that you won't be needing the old handle (by setting
> DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
> In that case the handle produced by DuplicateHandle (the one put into
> startupinfo) would have the same value as the original handle (but it
> wouldn't be the same handle though). Then your call to CloseHandle would
> close the new (duplicated) handle.
>
> Does this make any sense to you? Of course, I'm sure one can think of more
> possible explanations for what you are observing.
> --
> Regards,
> Sergei.
>
>
I had an opportunity today to re-run this in the debugger and found that
Sergei is correct; DuplicateHandle() is returning the same numeric value
for the new handle as was passed for the original handle that was to be
copied and closed.

So when I added the the assert(!CloseHandel(hPipe)) statement it was
actually closing the new handle (and thus the kernel object) before it
could be inherited by the new process.

Sorry for the confusion.

Norm

--
--
To reply, change domain to an adult feline.

Sergei Zhirikov wrote:
>>Any handle passed to
>>CloseHandle() closes all handles to that Kernel object that are owned by
>>the calling process.
>
>
> I doubt very much that this could possibly be true.
>
> It seems to me that you are jumping into conclusions. For example, I can
> offer an alternative explanation. (Note, it is purely theoretical, not based
> on any knowledge of kernel internals, and I didn't check anything of it in
> practice, so it is just a speculation.) What could happen is that when you
> tell the kernel that you won't be needing the old handle (by setting
> DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
> In that case the handle produced by DuplicateHandle (the one put into
> startupinfo) would have the same value as the original handle (but it
> wouldn't be the same handle though). Then your call to CloseHandle would
> close the new (duplicated) handle.
>
> Does this make any sense to you? Of course, I'm sure one can think of more
> possible explanations for what you are observing.
> --
> Regards,
> Sergei.
>
>
I had an opportunity today to re-run this in the debugger and found that
Sergei is correct; DuplicateHandle() is returning the same numeric value
for the new handle as was passed for the original handle that was to be
copied and closed.

So when I added the the assert(!CloseHandel(hPipe)) statement it was
actually closing the new handle (and thus the kernel object) before it
could be inherited by the new process.

Sorry for the confusion.

Norm

--
--
To reply, change domain to an adult feline.

Sergei Zhirikov wrote:
>>Any handle passed to
>>CloseHandle() closes all handles to that Kernel object that are owned by
>>the calling process.
>
>
> I doubt very much that this could possibly be true.
>
> It seems to me that you are jumping into conclusions. For example, I can
> offer an alternative explanation. (Note, it is purely theoretical, not based
> on any knowledge of kernel internals, and I didn't check anything of it in
> practice, so it is just a speculation.) What could happen is that when you
> tell the kernel that you won't be needing the old handle (by setting
> DUPLICATE_CLOSE_SOURCE) the kernel might decide to reuse the handle value.
> In that case the handle produced by DuplicateHandle (the one put into
> startupinfo) would have the same value as the original handle (but it
> wouldn't be the same handle though). Then your call to CloseHandle would
> close the new (duplicated) handle.
>
> Does this make any sense to you? Of course, I'm sure one can think of more
> possible explanations for what you are observing.
> --
> Regards,
> Sergei.
>
>
I had an opportunity today to re-run this in the debugger and found that
Sergei is correct; DuplicateHandle() is returning the same numeric value
for the new handle as was passed for the original handle that was to be
copied and closed.

So when I added the the assert(!CloseHandel(hPipe)) statement it was
actually closing the new handle (and thus the kernel object) before it
could be inherited by the new process.

Sorry for the confusion.

Norm

--
--
To reply, change domain to an adult feline.

Norman Bullen <norm@BlackKittenAssociates.com.INVALID> wrote:
>So when I added the the assert(!CloseHandel(hPipe)) statement it was
>actually closing the new handle (and thus the kernel object) before it
>could be inherited by the new process.

Just as an aside, I think it's VERY bad practice to have side-effects
inside an assert expression!

--
Lucian

Norman Bullen <norm@BlackKittenAssociates.com.INVALID> wrote:
>So when I added the the assert(!CloseHandel(hPipe)) statement it was
>actually closing the new handle (and thus the kernel object) before it
>could be inherited by the new process.

Just as an aside, I think it's VERY bad practice to have side-effects
inside an assert expression!

--
Lucian

Norman Bullen <norm@BlackKittenAssociates.com.INVALID> wrote:
>So when I added the the assert(!CloseHandel(hPipe)) statement it was
>actually closing the new handle (and thus the kernel object) before it
>could be inherited by the new process.

Just as an aside, I think it's VERY bad practice to have side-effects
inside an assert expression!

--
Lucian

In article <4vav02teaqg5458cth15veg5mc1ob11en8@4ax.com>, Lucian Wischik
<lu.nn@wischik.com> wrote:
>Norman Bullen <norm@BlackKittenAssociates.com.INVALID> wrote:
>>So when I added the the assert(!CloseHandel(hPipe)) statement it was
>>actually closing the new handle (and thus the kernel object) before it
>>could be inherited by the new process.
>
>Just as an aside, I think it's VERY bad practice to have side-effects
>inside an assert expression!

Agreed, and here's why:

#ifdef NDEBUG
#define assert(exp) ((void)0)
#else

So, in the debug version, you close the handle, and in the release version,
you don't close the handle. Makes you wonder why you'd want to test and debug
a version of the program that has different behaviours from the one you give
to your customers!

Alun.
~~~~

[Please don't email posters, if a Usenet response is appropriate.]
--
Texas Imperial Software | Find us at http://www.wftpd.com or email
23921 57th Ave SE | alun@wftpd.com.
Washington WA 98072-8661 | WFTPD, WFTPD Pro are Windows FTP servers.
Fax/Voice +1(425)807-1787 | Try our NEW client software, WFTPD Explorer.

In article <4vav02teaqg5458cth15veg5mc1ob11en8@4ax.com>, Lucian Wischik
<lu.nn@wischik.com> wrote:
>Norman Bullen <norm@BlackKittenAssociates.com.INVALID> wrote:
>>So when I added the the assert(!CloseHandel(hPipe)) statement it was
>>actually closing the new handle (and thus the kernel object) before it
>>could be inherited by the new process.
>
>Just as an aside, I think it's VERY bad practice to have side-effects
>inside an assert expression!

Agreed, and here's why:

#ifdef NDEBUG
#define assert(exp) ((void)0)
#else

So, in the debug version, you close the handle, and in the release version,
you don't close the handle. Makes you wonder why you'd want to test and debug
a version of the program that has different behaviours from the one you give
to your customers!

Alun.
~~~~

[Please don't email posters, if a Usenet response is appropriate.]
--
Texas Imperial Software | Find us at http://www.wftpd.com or email
23921 57th Ave SE | alun@wftpd.com.
Washington WA 98072-8661 | WFTPD, WFTPD Pro are Windows FTP servers.
Fax/Voice +1(425)807-1787 | Try our NEW client software, WFTPD Explorer.

In article <4vav02teaqg5458cth15veg5mc1ob11en8@4ax.com>, Lucian Wischik
<lu.nn@wischik.com> wrote:
>Norman Bullen <norm@BlackKittenAssociates.com.INVALID> wrote:
>>So when I added the the assert(!CloseHandel(hPipe)) statement it was
>>actually closing the new handle (and thus the kernel object) before it
>>could be inherited by the new process.
>
>Just as an aside, I think it's VERY bad practice to have side-effects
>inside an assert expression!

Agreed, and here's why:

#ifdef NDEBUG
#define assert(exp) ((void)0)
#else

So, in the debug version, you close the handle, and in the release version,
you don't close the handle. Makes you wonder why you'd want to test and debug
a version of the program that has different behaviours from the one you give
to your customers!

Alun.
~~~~

[Please don't email posters, if a Usenet response is appropriate.]
--
Texas Imperial Software | Find us at http://www.wftpd.com or email
23921 57th Ave SE | alun@wftpd.com.
Washington WA 98072-8661 | WFTPD, WFTPD Pro are Windows FTP servers.
Fax/Voice +1(425)807-1787 | Try our NEW client software, WFTPD Explorer.